Backup & Restore data in Kubernetes

Backing up your kubernetes cluster data by backuping up etcd is of crucial importance. etcd is the backend storage solution for the deployed Kubernetes cluster. All the K8s objects, applications & configuration are stored in etcd.

  1. Backing up ‘etcd’ data is done using etcd command line tool: etcdctl
ETCDCTL_API=3 etcdctl --endpoints $ENDPOINT snapshot save <filename>

2. Restoring etcd can be done from a backup using the etcdctl snapshot restore command:

ETCDCTL_API=3 etcdctl snapshot restore <filename>

Lets have a handson demo session on the above operations. Steps are as follows:

1) Execute etcdctl command to check Cluster name. Lookup for the in the etcd cluster:

ETCDCTL_API=3 etcdctl get \
> --endpoints= \
> --cacert=/home/cloud_user/etcd-certs/etcd-ca.pem \
> --cert=/home/cloud_user/etcd-certs/etcd-server.crt \
> --key=/home/cloud_user/etcd-certs/etcd-server.key
The returned value for the is beebox

–endpoint -> instructing etcdctl how to reach out to etcd server
–cacert -> public certificate for certificate authority
–cert -> the client certificate
–key -> certificate key

2) Snapshot save will save the backup with the name supplied as argument:

ETCDCTL_API=3 etcdctl snapshot save /home/cloud_user/etcd_backup.db \
> --endpoints= \
> --cacert=/home/cloud_user/etcd-certs/etcd-ca.pem \
> --cert=/home/cloud_user/etcd-certs/etcd-server.crt \
> --key=/home/cloud_user/etcd-certs/etcd-server.key

3) Now, stop etcd, Delete etcd data dir. Reset etcd by removing all the existing etcd data:

4) Snapshot restore on the etcd data will restore the data back as a temporary etcd cluster. This command will spinup a temp etcd cluster, saving the data from the backup file to a new data dir in the same location where the previous data dir was:

sudo ETCDCTL_API=3 etcdctl snapshot restore /home/cloud_user/etcd_backup.db \
> --initial-cluster etcd-restore= \
> --initial-advertise-peer-urls \
> --name etcd-restore \
> --data-dir /var/lib/etcd

Cluster with etcd restore
Required settings for temp cluster
Temp name as ‘etcd-restore’
Data directory to be used

5) Since I executed the operation as Root, I will have to make etcd as the user.
Set ownership on the new directory & start etcd:

sudo chown -R etcd:etcd /var/lib/etcd
sudo systemctl start etcd

6) Verify the restored data by looking up the value for the key again:

ETCDCTL_API=3 etcdctl get \
> --endpoints= \
> --cacert=/home/cloud_user/etcd-certs/etcd-ca.pem \
> --cert=/home/cloud_user/etcd-certs/etcd-server.crt \
> --key=/home/cloud_user/etcd-certs/etcd-server.key

At this point, I can say with confidence that our etcd database is properly restored.
With this demonstration, we come to the end of topic: “High availability in Kubernetes”.

Cloud, DevOps Enthusiast :)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store