After a recent power outage, my Kubernetes cluster failed to come back online. I cannot connect to kube-apiserver through kubectl anymore. Upon checking the controller node, I can see these errors with kubelet service.
Feb 03 00:43:06 tom-lab1 kubelet[3475]: E0203 00:43:06.555186 3475 kubelet.go:2422] "Error getting node" err="node \"tom-lab1\" not found" Feb 03 00:43:06 tom-lab1 kubelet[3475]: E0203 00:43:06.656323 3475 kubelet.go:2422] "Error getting node" err="node \"tom-lab1\" not found" In kube-apiserver logs /var/log/container/kube-apiserver-tom-lab1_kube-system_kube-apiserver-2bec70209c1231c69a6501aea951f4ff5bed1996174028cd7e0396b2c4dc34e0.log I can see the errors below.
Metrics-server is one of the most common service deployed in their Kubernetes clusters. It is designed to be used for autoscaling purposes. In my case, I simply want to have a way to easily check my nodes status with kubectl top node.
In this post I will walk you through the steps I took to get metrics-server deployed on a local Kubernetes cluster. There are a few interesting issues I bumped into along the way.
In this post I will show how I deployed Pihole onto my local Kubernetes cluster. In compare with some other online tutorials, this deployment saves Pihole configuration onto persistent volumes, so you won’t need to reconfigure everything after pod recreation. By using LoadBalancer, original client IPs are captured correctly in my Pihole instead of seeing everything was from the Cluster IP.
If you want to learn how to build your own local Kubernetes cluster, read this post.
Problem You got a pod left in NotReady status due to failed helm chart installation or resource deployment.
➜ ✗ kubectl get pods NAME READY STATUS RESTARTS AGE release-0.11-kube-promethe-admission-create-7j7fz 1/2 NotReady 2 8m1s Tried to remove the pod with kubectl delete pod will just cause the pod to be recreated.
Solutions Uninstall helm chart Run helm ls -A to list all current installed charts.
I have been following the COVID situation in Shanghai lately. Intentionally or not, it is surprisingly hard to find a daily chart to reflect the proper case numbers overthere. To solve this problem, I decided to build a webapp just does exactly that: a simple chart the displays Shanghai daily COVID cases that includes both symptomatic and asymptomatic cases.
The overall design includes a backend API that provides daily numbers in JSON format, and a front-end page that presents the data in the form of an area chart.
Tom Tech Blog