CoreOS updated its key value store, etcd, on Friday. This was the first full release for the Raft-based key-value store in the new year. The update focused on performance, reliability and scalability improvements.
For many users, etcd is better known as the configuration storage system used by Google’s Kubernetes container-management platform. Etcd 3.1 included features aimed at improving the consistent storage systems performance at scale.
Some of the improvements in etcd 3.1 focus on reducing chatter between and around servers as much as possible. The more members a cluster of etcd servers has, the slower the whole cluster moves due to the greater demand for replication between nodes. This is due to a high rate of between-node chatter.
In etcd 3.1, CoreOS introduces gRPC proxy, an attempt to help developers increase performance around this bottleneck. Anthony Romano, software engineer at CoreOS, wrote in a blog entry that “The gRPC proxy includes a cache of recently accessed keys. The cache serves serialized key fetches, which don’t need to go through consensus, that would otherwise be handled by a cluster member. The advantage is the proxy absorbs serialized key request spam from misbehaving or misconfigured clients.”
According to Romano, “Etcd provides both serialized and linearized consistency models for reading keys. A serialized read is fast, no consensus is necessary, but unsuitable for many applications because it may return stale data. A linearized read returns the most recent keys by going through etcd’s underlying raft protocol, and therefore carries greater overhead. While etcd 3.0 processes linearized reads through direct raft proposals, chewing precious disk bandwidth and incurring the corresponding latency penalty, linearized reads in 3.1 issue idempotent, fsync-free linearized raft index requests. These index requests are also batched; concurrent linearized reads coalesce into a single index request. The upshot is linearized reads have lower latency and better throughput.”
This release of etcd also includes new APIs. These add features to better manage leases, handle keys by revision, and to reduce total round-trips to the server overall. These will make it easier to find all resources attached to a session, and other changes will make it easier overall to deal with updating and tracking the lifespan of keys stored in etcd.