KUBERNETES: How to Backup Kubernetes Certificates?

In Kubernetes, certificates are by default installed in below directories.

/etc/kubernetes/pki/

/var/lib/kubelet/pki/

Below is the tar command to backup above Kubernetes certificate directories.

sudo tar cvzf ~/`hostname -s`_cert_backup.tgz /etc/kubernetes/pki/ /var/lib/kubelet/pki/

Above command will create backup tgz file under user home directory with short hostname of the Node where the command is executed.

Read more "KUBERNETES: How to Backup Kubernetes Certificates?"

KUBERNETES (K8S): ETCDCTL Command To Validate The Status of ETCD Database Backup

To Validate the status of etcd database that was backed up and make sure you are able to read the stats from snapshot backup, run below ETCDCTL command. Change /backup directory name in below command to directory where you want to backup etcd database.

            sudo ETCDCTL_API=3 etcdctl --write-out=table snapshot status /backup/etcd.db

        Below is sample output of above command.

            sudo ETCDCTL_API=3 etcdctl --write-out=table snapshot status /home/opc/etcd.db

            +----------+----------+------------+------------+

            |   HASH   | REVISION | TOTAL KEYS | TOTAL SIZE |

            +----------+----------+------------+------------+

            | 56414efa |   940057 |       2001 |     7.0 MB |

            +----------+----------+------------+------------+

Read more "KUBERNETES (K8S): ETCDCTL Command To Validate The Status of ETCD Database Backup"

KUBERNETES (K8S): ETCDCTL Command To Backup ETCD Database (Snapshot Backup)

Below is etcdctl command that has to be run on control.master node to backup etcd database. Backup will be snapshot backup. Change /backup direcotry name in below command to directory where you want to backup etcd database.

            ETCDCTL_API=3 etcdctl \

              --endpoints=https://127.0.0.1:2379 \

              --cacert=/etc/kubernetes/pki/etcd/ca.crt \

              --cert=/etc/kubernetes/pki/etcd/server.crt \

              --key=/etc/kubernetes/pki/etcd/server.key \

              snapshot save /backup/etcd.db

After successful execution of above command, you should see output as follows:

            Snapshot saved at /backup/etcd.db

Above certs will be the default cert names and location. If the cert names/directories are different for etcd, you can gather those details by looking at /etc/kubernetes/manifests/etcd.yaml etcd yaml file.

Below are the important entry lines in etcd yaml file which shows the etcd URL/port, cert fetials and etcd image version being used.

            kubeadm.kubernetes.io/etcd.advertise-client-urls: https://10.XX.XX.219:2379

            - --advertise-client-urls=https://10.XX.XX.219:2379

            - --cert-file=/etc/kubernetes/pki/etcd/server.crt

            - --initial-advertise-peer-urls=https://10.XX.XX.219:2380

            - --key-file=/etc/kubernetes/pki/etcd/server.key

            - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt

            - --peer-key-file=/etc/kubernetes/pki/etcd/peer.key

            - --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

            - --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt

            image: container-registry.oracle.com/olcne/etcd:3.5.6

Read more "KUBERNETES (K8S): ETCDCTL Command To Backup ETCD Database (Snapshot Backup)"