Install etcd Distributed Key-Value Store on VPS
Home › Install etcd Distributed Key-Value Store on VPS

Install etcd Distributed Key-Value Store on VPS

June 2026 · 8 min read

etcd is an open-source distributed key-value store that uses the Raft consensus protocol to manage data across multiple nodes. It is widely adopted by Kubernetes for storing cluster configuration, state information, and metadata. Installing etcd on your VPS enables you to build highly available database clusters and understand how distributed systems work in practice.

What is etcd and Why Use It

etcd (easy to command) is a distributed key-value database designed to store critical information such as configuration settings, state information, and metadata for distributed systems. Key characteristics of etcd include:

Kubernetes relies on etcd to store the entire cluster state including pod configurations, services, secrets, configmaps, and other critical data. A single etcd cluster failure can take down your entire Kubernetes deployment.

Install etcd on Ubuntu/Debian

The simplest way to install etcd is to download the binary from the official GitHub releases, as the version in apt repositories may be outdated.

# Download the latest version (check GitHub releases for current version)
ETCD_VER=v3.5.13
DOWNLOAD_URL=https://github.com/etcd-io/etcd/releases/download/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz

cd /tmp
wget -q --show-progress "$DOWNLOAD_URL"
tar xzf etcd-${ETCD_VER}-linux-amd64.tar.gz
cd etcd-${ETCD_VER}-linux-amd64

# Make binaries available system-wide
sudo mv etcd etcdctl /usr/local/bin/

# Verify installation
etcd --version
etcdctl version

Alternatively, you can use the apt package manager if available:

sudo apt-get update
sudo apt-get install -y etcd-client
# For etcd server, binary download is recommended

Configure Single Node etcd

For testing or development environments, you can start etcd with default settings immediately:

# Start etcd with default configuration
etcd

# In another terminal, test with etcdctl
etcdctl put key1 "Hello etcd"
etcdctl get key1

For production environments, create a systemd service with proper configuration:

# Create etcd user
sudo useradd -r -s /bin/false etcd

# Create data directory
sudo mkdir -p /var/lib/etcd
sudo chown -R etcd:etcd /var/lib/etcd

# Create configuration file
sudo mkdir -p /etc/etcd
sudo tee /etc/etcd/etcd.conf.yml > /dev/null < /dev/null <

Configure etcd 3-Node Cluster

Setting up a cluster is fundamental to etcd because Raft consensus requires a quorum (majority of nodes) to make decisions. A 3-node cluster can tolerate one node failure while maintaining data consistency. This is the recommended minimum for production deployments.

# Node 1 (IP: 192.168.1.10) - etcd-node1
sudo tee /etc/etcd/etcd.conf.yml > /dev/null < /dev/null < /dev/null <

Critical: Initialize the cluster on all three nodes at approximately the same time (within 1-2 minutes). If the initialization takes too long or is staggered, the cluster may fail to form correctly. All nodes should have matching initial-cluster configuration and token.

Using etcdctl CLI

etcdctl is the command-line client for interacting with etcd. Set environment variables to point to your cluster endpoints:

# Set cluster endpoints
export ETCDCTL_ENDPOINTS=192.168.1.10:2379,192.168.1.11:2379,192.168.1.12:2379

# Test connectivity
etcdctl endpoint health
# Output:
# 192.168.1.10:2379, healthy, took = 3.123ms
# 192.168.1.11:2379, healthy, took = 2.850ms
# 192.168.1.12:2379, healthy, took = 3.456ms

# Store key-value pairs
etcdctl put /app/config "production"
etcdctl put /app/version "1.2.3"
etcdctl put /database/host "db.example.com"

# Retrieve values
etcdctl get /app/config
# /app/config
# production

# Get all keys with prefix
etcdctl get /app/ --prefix
# /app/config
# production
# /app/version
# 1.2.3

# Delete a key
etcdctl del /app/version

# Watch for changes (blocks until a change occurs)
etcdctl watch /app/config &
# Change the value in another terminal:
etcdctl put /app/config "staging"
# Watch will show:
# PUT
# /app/config
# staging

# Get cluster statistics
etcdctl member list
etcdctl member status

etcd Backup and Restore

Creating backups is critical since etcd stores your cluster state. Data loss can have severe consequences for your entire infrastructure.

# Create snapshot backup (avoid writes during backup)
etcdctl --endpoints=192.168.1.10:2379 snapshot save backup.db

# Verify snapshot integrity
etcdctl snapshot status backup.db

# Restore to a new node or test environment
sudo systemctl stop etcd
sudo rm -rf /var/lib/etcd/*

# Restore from snapshot
etcdctl snapshot restore backup.db \
  --name etcd-node1 \
  --initial-cluster etcd-node1=http://192.168.1.10:2380 \
  --initial-cluster-token etcd-cluster \
  --initial-advertise-peer-urls http://192.168.1.10:2380 \
  --data-dir=/var/lib/etcd

# Fix ownership
sudo chown -R etcd:etcd /var/lib/etcd

# Start etcd again
sudo systemctl start etcd
Tip: For multi-node cluster restore, configure each node as a single-node cluster first, then use member add to join nodes back into the cluster.

Verify etcd is Working Correctly

Use these commands to verify that etcd is operating correctly:

# Check service status
sudo systemctl status etcd

# Check cluster health
etcdctl endpoint health --endpoints=192.168.1.10:2379,192.168.1.11:2379,192.168.1.12:2379

# Test API access
curl -s http://localhost:2379/version | jq .
# Output:
# {
#   "etcdserver": "3.5.13",
#   "etcdcluster": "3.5.0"
# }

# Test write and read operations
etcdctl put test-key "test-value"
etcdctl get test-key
# test-key
# test-value

# List cluster members
etcdctl member list --endpoints=192.168.1.10:2379
# 1234567890abcdef, started, etcd-node1, http://192.168.1.10:2380, http://192.168.1.10:2379, false
# 2345678901bcdef0, started, etcd-node2, http://192.168.1.11:2380, http://192.168.1.11:2379, false
# 3456789012cdef01, started, etcd-node3, http://192.168.1.12:2380, http://192.168.1.12:2379, false
Tip: Check logs with sudo journalctl -u etcd -n 50 -f if you encounter issues. Logs often provide helpful error messages and suggestions.

Troubleshooting Common Issues

Error: "connection refused" when connecting to etcd

This occurs when the etcd daemon is not running or the endpoint is incorrect.

# Check if etcd is running
sudo systemctl status etcd

# Start etcd if stopped
sudo systemctl start etcd

# Verify port 2379 is listening
sudo netstat -tlnp | grep 2379

# Allow firewall access
sudo ufw allow 2379/tcp
sudo ufw allow 2380/tcp

Error: "election timeout" or "no leaders"

This indicates network connectivity issues between cluster nodes or an incomplete cluster formation.

# Test network connectivity between nodes
ping 192.168.1.10
ping 192.168.1.11
ping 192.168.1.12

# Test peer-to-peer communication (port 2380)
nc -zv 192.168.1.10 2380
nc -zv 192.168.1.11 2380
nc -zv 192.168.1.12 2380

# Check logs for root cause
sudo journalctl -u etcd --no-pager | grep -i "election\|leader\|timeout"

# Restart etcd on all nodes if network is normal
sudo systemctl restart etcd

Error: "database corruption" or "bad magic"

This occurs when the etcd process crashes unexpectedly or disk errors corrupt data.

# Check if etcd process is still running
ps aux | grep etcd

# Stop etcd gracefully
sudo systemctl stop etcd

# Force kill if graceful stop fails
sudo pkill -9 etcd

# Remove stale lock file
sudo rm -f /var/lib/etcd/lock

# Start etcd again
sudo systemctl start etcd

Error: "failed to parse initial-cluster members"

Configuration file syntax errors prevent etcd from starting.

# Check configuration file
cat /etc/etcd/etcd.conf.yml

# Ensure:
# 1. initial-cluster contains all three nodes
# 2. YAML syntax is correct
# 3. IP addresses and ports are accurate

# Correct format example:
# initial-cluster: 'etcd-node1=http://192.168.1.10:2380,etcd-node2=http://192.168.1.11:2380,etcd-node3=http://192.168.1.12:2380'

# Check error details
sudo journalctl -u etcd -n 100

Frequently Asked Questions

Q: Is a 3-node cluster always required?

No. A 3-node cluster is considered best practice because it can tolerate one node failure while maintaining quorum (2 nodes) for consensus decisions. Raft requires a quorum, so odd-numbered clusters are preferred (1, 3, 5, 7...). A 5-node cluster can tolerate two failures. Single-node etcd (1 node) works for development but is not recommended for production use due to lack of high availability.

Q: Should I use gRPC or HTTP API?

etcdctl uses gRPC by default since version 3. gRPC offers better performance, lower latency, and multiplexing compared to HTTP. However, HTTP API is still available for curl commands and debugging. For most purposes, etcdctl provides the best experience and is the recommended interface.

Q: Why does etcd require persistent storage?

etcd must use persistent storage (disk) to ensure data survives node restarts. The data-dir (typically /var/lib/etcd) stores all key-value pairs and cluster state on disk. When nodes restart, they recover state from disk. Additionally, etcd writes logs for every commit, so sufficient disk space and I/O performance are important. Snapshot backups should also be stored on separate, reliable storage.

RELATED ARTICLES

  • Set Up Consul Service Discovery on VPS