Highly Available Distributed Configuration Stores (Consul, etcd, zookeeper)¶

This page is mostly based on a short presentation with two of my friends. You can get the slides here.

What are Configuration Stores?¶

Configuration Stores are, as the name suggests, used to store the configuration of your application. This can range from simple settings like the hostname or port, up to complex constructs that change how your program behaves. The main difference between configuration stores and more trivial configuration files (e.g. .ini, .config) is that configuration stores can notify your program that the configuration has changed. If your program requires having up-to-date information at any given time, it gets the information like a push-notification, instead of having to pull the configuration file frequently.

By using a configuration store, you introduce a single point of failure. When your application can't reach the configuration store due to a crash, it has to run with stale data or might not even function at all. Thus, it is important to deploy the configuration store with high availability, for example by using replication across multiple servers. However, now you have the problem of potentially serving inconsistent or stale data (If you're interested in this, Consul and etcd use the Raft Consensus Algorithm).

What can they be used for?¶

Configuration Stores can be used to:

• Store your usual application configuration
• Implement service discovery (on which hosts can my application reach a service)
• Coordinate distributed algorithms (e.g. with locks)

Details about Consul, etcd, Zookeeper¶

For more information about the three systems, please look at the slides.

Which implementation should you use?¶

• Consul: If you need an "all inclusive" service discovery framework (with service-to-service encryption, health data, ...)
• etcd: If you need a fast, distributed key-value store
• ZooKeeper: If you like legacy systems