[Kubernetes]node节点pod无法启动/节点删除网络重置
程序员文章站
2022-07-13 22:46:10
...
node1之前反复添加过,添加之前需要清除下网络
[email protected]:/var/lib/kubelet# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
nginx-8586cf59-6zw9k 1/1 Running 0 9m 10.244.3.3 node2
nginx-8586cf59-jk5pc 0/1 ContainerCreating 0 9m <none> node1
nginx-8586cf59-vm9h4 0/1 ContainerCreating 0 9m <none> node1
nginx-8586cf59-zjb84 1/1 Running 0 9m 10.244.3.2 node2
[email protected]:~# journalctl -u kubelet
failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod "nginx-8586cf59-rm4sh_default" network: failed to set bridge addr: "cni0" already has an IP address different from 10.244.2.1/24
12252 cni.go:227] Error while adding to cni network: failed to set bridge addr: "cni0" already
排查节点日志：journalctl -u kubelet
Nov 15 09:03:15 mvxl0851 kubelet[783]: E1115 09:03:15.577421 783 cni.go:259] Error adding network: failed to set bridge addr: "cni0" already has an IP address different from 10.16.2.1/24
Nov 15 09:03:15 mvxl0851 kubelet[783]: E1115 09:03:15.577465 783 cni.go:227] Error while adding to cni network: failed to set bridge addr: "cni0" already has an IP address different from 10.16.2.1/24
Nov 15 09:03:15 mvxl0851 kubelet[783]: E1115 09:03:15.652288 783 remote_runtime.go:92] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod "htt
Nov 15 09:03:15 mvxl0851 kubelet[783]: E1115 09:03:15.652386 783 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "httpd-app-5fbccd7c6c-khnj2_default(8a6c7915-e871-11e8-af2a-0050568a39f2)" failed: rpc er
Nov 15 09:03:15 mvxl0851 kubelet[783]: E1115 09:03:15.652411 783 kuberuntime_manager.go:647] createPodSandbox for pod "httpd-app-5fbccd7c6c-khnj2_default(8a6c7915-e871-11e8-af2a-0050568a39f2)" failed: rpc e
Nov 15 09:03:15 mvxl0851 kubelet[783]: E1115 09:03:15.652516 783 pod_workers.go:186] Error syncing pod 8a6c7915-e871-11e8-af2a-0050568a39f2 ("httpd-app-5fbccd7c6c-khnj2_default(8a6c7915-e871-11e8-af2a-00505
Nov 15 09:03:16 mvxl0851 kubelet[783]: W1115 09:03:16.266688 783 pod_container_deletor.go:77] Container "2af4863fef146c5be51a704837c7943c3beab015fbd5cc2c705caee7216f5a6f" not found in pod's containers
Nov 15 09:03:18 mvxl0851 kubelet[783]: E1115 09:03:18.147904 783 cni.go:259] Error adding network: failed to set bridge addr: "cni0" already has an IP address different from 10.16.2.1/24
Nov 15 09:03:18 mvxl0851 kubelet[783]: E1115 09:03:18.147970 783 cni.go:227] Error while adding to cni network: failed to set bridge addr: "cni0" already has an IP address different from 10.16.2.1/24
Nov 15 09:03:18 mvxl0851 kubelet[783]: E1115 09:03:18.202769 783 remote_runtime.go:92] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod "htt
Nov 15 09:03:18 mvxl0851 kubelet[783]: E1115 09:03:18.202868 783 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "httpd-app-5fbccd7c6c-khnj2_default(8a6c7915-e871-11e8-af2a-0050568a39f2)" failed: rpc er
Nov 15 09:03:18 mvxl0851 kubelet[783]: E1115 09:03:18.202896 783 kuberuntime_manager.go:647] createPodSandbox for pod "httpd-app-5fbccd7c6c-khnj2_default(8a6c7915-e871-11e8-af2a-0050568a39f2)" failed: rpc e
Nov 15 09:03:18 mvxl0851 kubelet[783]: E1115 09:03:18.203047 783 pod_workers.go:186] Error syncing pod 8a6c7915-e871-11e8-af2a-0050568a39f2 ("httpd-app-5fbccd7c6c-khnj2_default(8a6c7915-e871-11e8-af2a-00505
Nov 15 09:03:18 mvxl0851 kubelet[783]: W1115 09:03:18.332581 783 pod_container_deletor.go:77] Container "780e7ab6407c660cb6e4b4b0696fb24a9ba3b605b0c5e005ea4f51679f7eaf28" not found in pod's containers
Nov 15 09:03:19 mvxl0851 kubelet[783]: I1115 09:03:19.301582 783 reconciler.go:191] operationExecutor.UnmountVolume started for volume "default-token-qwrr2" (UniqueName: "kubernetes.io/secret/8a6c7915-e871-
Nov 15 09:03:19 mvxl0851 kubelet[783]: I1115 09:03:19.305596 783 operation_generator.go:633] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a6c7915-e871-11e8-af2a-0050568a39f2-default-to
Nov 15 09:03:19 mvxl0851 kubelet[783]: I1115 09:03:19.402052 783 reconciler.go:297] Volume detached for volume "default-token-qwrr2" (UniqueName: "kubernetes.io/secret/8a6c7915-e871-11e8-af2a-0050568a39f2-d
Nov 15 09:03:20 mvxl0851 kubelet[783]: E1115 09:03:20.188278 783 cni.go:259] Error adding network: failed to set bridge addr: "cni0" already has an IP address different from 10.16.2.1/24
Nov 15 09:03:20 mvxl0851 kubelet[783]: E1115 09:03:20.188325 783 cni.go:227] Error while adding to cni network: failed to set bridge addr: "cni0" already has an IP address different from 10.16.2.1/24
Nov 15 09:03:20 mvxl0851 kubelet[783]: E1115 09:03:20.254502 783 remote_runtime.go:92] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod "htt
Nov 15 09:03:20 mvxl0851 kubelet[783]: E1115 09:03:20.254585 783 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "httpd-app-5fbccd7c6c-khnj2_default(8a6c7915-e871-11e8-af2a-0050568a39f2)" failed: rpc er
Nov 15 09:03:20 mvxl0851 kubelet[783]: E1115 09:03:20.254612 783 kuberuntime_manager.go:647] createPodSandbox for pod "httpd-app-5fbccd7c6c-khnj2_default(8a6c7915-e871-11e8-af2a-0050568a39f2)" failed: rpc e
Nov 15 09:03:20 mvxl0851 kubelet[783]: E1115 09:03:20.254712 783 pod_workers.go:186] Error syncing pod 8a6c7915-e871-11e8-af2a-0050568a39f2 ("httpd-app-5fbccd7c6c-khnj2_default(8a6c7915-e871-11e8-af2a-00505
Nov 15 09:03:20 mvxl0851 kubelet[783]: W1115 09:03:20.410453 783 pod_container_deletor.go:77] Container "73e22f0f076e0472b31cfc83ed6f855d50048993e8bbd8508d2886ad3ea45b48" not found in pod's containers
Nov 15 09:03:27 mvxl0851 kubelet[783]: E1115 09:03:27.860718 783 file.go:76] Unable to read manifest path "/etc/kubernetes/manifests": path does not exist, ignoring重点关注：
cni.go:259] Error adding network: failed to set bridge addr: "cni0" already has an IP address different from 10.16.2.1/24重置kubernetes服务，重置网络。删除网络配置，link
kubeadm reset
systemctl stop kubelet
systemctl stop docker
rm -rf /var/lib/cni/
rm -rf /var/lib/kubelet/*
rm -rf /etc/cni/
ifconfig cni0 down
ifconfig flannel.1 down
ifconfig docker0 down
ip link delete cni0
ip link delete flannel.1
systemctl start docker
加入节点
systemctl start docker
kubeadm join --token 55c2c6.2a4bde1bc73a6562 192.168.1.144:6443 --discovery-token-ca-cert-hash sha256:0fdf8cfc6fecc18fded38649a4d9a81d043bf0e4bf57341239250dcc62d2c832