k8s自签证书过期x509: certificate has expired or is not yet valid报错

一、 问题表现

使用kubelet get node后报错,x509: certificate has expired or is not yet valid,提示证书过期。

[root@master ~]# kubectl get node
Unable to connect to the server: x509: certificate has expired or is not yet valid: current time 2023-02-17T09:56:22+08:00 is after 2023-01-12T10:42:07Z

二、 问题排查

集群是由kubeadm创建。但是它创建的apiserver、controller-manager等证书默认只有一年的有效期，同时kubelet 证书也只有一年有效期，一年之后kubernetes将停止服务。

官方文档:
https://kubernetes.io/zh-cn/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/
你可以使用 check-expiration 子命令来检查证书何时过期

kubeadm certs check-expiration

输出类似于以下内容：

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Dec 28, 2023 05:54 UTC   <invalid>                               no      
apiserver                  Dec 28, 2023 05:54 UTC   <invalid>       ca                      no      
apiserver-etcd-client      Dec 28, 2023 05:54 UTC   <invalid>       etcd-ca                 no      
apiserver-kubelet-client   Dec 28, 2023 05:54 UTC   <invalid>       ca                      no      
controller-manager.conf    Dec 28, 2023 05:54 UTC   <invalid>                               no      
etcd-healthcheck-client    Dec 28, 2023 05:54 UTC   <invalid>       etcd-ca                 no      
etcd-peer                  Dec 28, 2023 05:54 UTC   <invalid>       etcd-ca                 no      
etcd-server                Dec 28, 2023 05:54 UTC   <invalid>       etcd-ca                 no      
front-proxy-client         Dec 28, 2023 05:54 UTC   <invalid>       front-proxy-ca          no      
scheduler.conf             Dec 28, 2023 05:54 UTC   <invalid>                               no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Dec 25, 2032 05:54 UTC   8y              no      
etcd-ca                 Dec 25, 2032 05:54 UTC   8y              no      
front-proxy-ca          Dec 25, 2032 05:54 UTC   8y              no    

三、 问题解决

1. 查看证书到期时间

# 查看证书到期时间
kubeadm certs check-expiration

2. 更新自签证书

#更新自签证书
kubeadm certs renew all

3. 查看最新时间

#查看最新时间
[root@pkm-04 kubernetes]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Dec 27, 2024 10:52 UTC   364d                                    no      
apiserver                  Dec 27, 2024 10:52 UTC   364d            ca                      no      
apiserver-etcd-client      Dec 27, 2024 10:52 UTC   364d            etcd-ca                 no      
apiserver-kubelet-client   Dec 27, 2024 10:52 UTC   364d            ca                      no      
controller-manager.conf    Dec 27, 2024 10:52 UTC   364d                                    no      
etcd-healthcheck-client    Dec 27, 2024 10:52 UTC   364d            etcd-ca                 no      
etcd-peer                  Dec 27, 2024 10:52 UTC   364d            etcd-ca                 no      
etcd-server                Dec 27, 2024 10:52 UTC   364d            etcd-ca                 no      
front-proxy-client         Dec 27, 2024 10:52 UTC   364d            front-proxy-ca          no      
scheduler.conf             Dec 27, 2024 10:52 UTC   364d                                    no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Dec 25, 2032 05:54 UTC   8y              no      
etcd-ca                 Dec 25, 2032 05:54 UTC   8y              no      
front-proxy-ca          Dec 25, 2032 05:54 UTC   8y              no   

4.复制配置

#复制配置
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

5.重启kubelet,docker（master与node都要重启）

#重启kubelet,docker（master与node都要重启）
systemctl restart docker
systemctl restart kubelet

参考资料：
https://www.cnblogs.com/cerberus43/p/17130266.html
https://kubernetes.io/zh-cn/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/