Geeking in technology since 1985, with IBM Development, focused upon Docker and Kubernetes on the IBM Z LinuxONE platform In the words of Dr Cathy Ryan, "If you don't write it down, it never happened". To paraphrase one of my clients, "Every day is a school day". I do, I learn, I share. The postings on this site are my own and don’t necessarily represent IBM’s positions, strategies or opinions. Remember, YMMV https://infosec.exchange/@davehay

Whilst building a new "vanilla" Kubernetes 1.25.4 cluster, I'd started the kubelet service via: -

systemctl start kubelet.service

and then decided to check how it was doing: -

systemctl status kubelet.service

● kubelet.service - kubelet: The Kubernetes Node Agent

Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)

Drop-In: /etc/systemd/system/kubelet.service.d

└─10-kubeadm.conf

Active: activating (auto-restart) (Result: exit-code) since Thu 2022-11-24 01:04:45 PST; 9s ago

Docs: https://kubernetes.io/docs/home/

Process: 19526 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS >

Main PID: 19526 (code=exited, status=1/FAILURE)

which was slightly worrying ....

I checked the system logs ( this is an Ubuntu 20.04.5 LTS box ) : -

cat /var/log/syslog

which, in part, reported: -

Nov 24 01:11:04 acarids2 systemd[1]: Started kubelet: The Kubernetes Node Agent.

Nov 24 01:11:04 acarids2 kubelet[20446]: E1124 01:11:04.390575   20446 run.go:74] "command failed" err="failed to load kubelet config file, error: failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file \"/var/lib/kubelet/config.yaml\", error: open /var/lib/kubelet/config.yaml: no such file or directory, path: /var/lib/kubelet/config.yaml"

Nov 24 01:11:04 acarids2 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE

Nov 24 01:11:04 acarids2 systemd[1]: kubelet.service: Failed with result 'exit-code'.

Nov 24 01:11:14 acarids2 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 67.

Nov 24 01:11:14 acarids2 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.

specifically this: -

open /var/lib/kubelet/config.yaml: no such file or directory, path: /var/lib/kubelet/config.yaml"

At that point, common sense prevailed ....

This was very early on in the build process, and I'd NOT yet initialised the K8s API Server ( on the Control Plane node ) and, therefore, NOT yet joined the Compute Node to the yet-to-be-started API Server.

Therefore, until I finished creating the cluster, and joining the Compute Node, what did I expect ?

Once I ran kubeadm init on the Control Plane node, and kubeadm join , on the Compute Node, all was well: -

systemctl status kubelet.service

which looks happier: -

● kubelet.service - kubelet: The Kubernetes Node Agent

Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)

Drop-In: /etc/systemd/system/kubelet.service.d

└─10-kubeadm.conf

Active: active (running) since Thu 2022-11-24 01:08:29 PST; 8min ago

Docs: https://kubernetes.io/docs/home/

Main PID: 21417 (kubelet)

Tasks: 16 (limit: 9442)

Memory: 40.7M

CGroup: /system.slice/kubelet.service

└─21417 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubele>