K8s集群重启与恢复-Node节点启停
2022/3/1 22:51:22
本文主要是介绍K8s集群重启与恢复-Node节点启停,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!
1 应用场景
场景:在实际工作中,可能某个 Worker 节点需要维护,迁移,我们需要平滑的停止、启动该节点,应尽量减少启停中对集群、业务造成的影响
注意:剔除 Worker 节点操作中,该 Worker 节点上的工作负载(Pod)将被驱逐到其他节点上,请确保集群资源充足
2 操作步骤
2.1 停止 Worker 节点调度
# 查看信息 root@sh-gpu091:~# kubectl get node NAME STATUS ROLES AGE VERSION 172.19.13.31 Ready node 403d v1.14.1 sh-cpu032.aibee.cn Ready node 403d v1.14.1 sh-cpu033.aibee.cn Ready node 403d v1.14.1 sh-gpu091.aibee.cn Ready node 403d v1.14.1 sh-gpu101.aibee.cn Ready node 403d v1.14.1 # 停止 Worker 节点调度 root@sh-gpu091:~# kubectl cordon sh-gpu101.aibee.cn node/sh-gpu101.aibee.cn cordoned
查看节点状态
root@sh-gpu091:~# kubectl get node NAME STATUS ROLES AGE VERSION 172.19.13.31 Ready node 403d v1.14.1 sh-cpu032.aibee.cn Ready node 403d v1.14.1 sh-cpu033.aibee.cn Ready node 403d v1.14.1 sh-gpu091.aibee.cn Ready node 403d v1.14.1 sh-gpu101.aibee.cn Ready,SchedulingDisabled node 403d v1.14.1
2.2 驱逐 Worker 节点上的工作负载
# --ignore-daemonsets 驱逐 pod 时忽略 daemonset # --delete-local-data 驱逐 pod 时删除 pod 的临时数据,该参数不会删除持久化数据 root@sh-gpu091:~# kubectl drain sh-gpu101.aibee.cn --delete-local-data --ignore-daemonsets --force node/sh-gpu101.aibee.cn already cordoned WARNING: ignoring DaemonSet-managed Pods: cattle-system/cattle-node-agent-8wcvs, kube-system/kube-flannel-ds-kqzhc, kube-system/nvidia-device-plugin-daemonset-rr2lf, monitoring/prometheus-node-exporter-xtbxp evicting pod "model-server-0" evicting pod "singleview-proxy-client-pbdownloader-0" evicting pod "singleview-proxy-service-0" pod/singleview-proxy-client-pbdownloader-0 evicted pod/singleview-proxy-service-0 evicted pod/model-server-0 evicted node/sh-gpu101.aibee.cn evicted
2.3 停止 Docker、Kubelet 等服务
systemctl stop kubelet systemctl stop docker
查看节点上是否还有业务 pod
kubectl get pod -A -o wide |grep sh-gpu101.aibee.cn
如果不需要恢复了,可以删除节点并确认节点信息
root@sh-gpu091:~# kubectl delete node sh-gpu101.aibee.cn node "sh-gpu101.aibee.cn" deleted root@sh-gpu091:~# kubectl get node NAME STATUS ROLES AGE VERSION 172.19.13.31 Ready node 403d v1.14.1 sh-cpu032.aibee.cn Ready node 403d v1.14.1 sh-cpu033.aibee.cn Ready node 403d v1.14.1 sh-gpu091.aibee.cn Ready node 403d v1.14.1 root@sh-gpu091:~#
2.4 恢复 Worker 节点
systemctl start docker systemctl status docker systemctl start kubelet systemctl status kubelet
2.5 允许 Worker 节点调度
# 取消不可调度 kubectl uncordon sh-gpu101.aibee.cn
这篇关于K8s集群重启与恢复-Node节点启停的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!
- 2024-05-29一文搞定 KubeKey 3.1.1 离线部署 KubeSphere 3.4.1 和 Kubernetes v1.28
- 2024-05-29云原生周刊:K8s 上的 gRPC 名称解析和负载平衡
- 2024-05-08云原生周刊:Kubernetes v1.30 发布 | 2024.4.22
- 2024-05-08云原生周刊:K8s 中的服务和网络 | 2024.4.29
- 2024-03-29kubernetes crashloopbackoff
- 2024-03-13ubuntu install kubernetes
- 2024-03-07k8s coredns containercreating
- 2024-03-04k8s for dummies
- 2024-03-01pulumi kubernetes operator
- 2024-02-06云原生周刊:K8s 1.26 到 1.29 版本的更新 | 2024.1.29