欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

kubernetes 中遇见的一些坑(持续更新)

程序员文章站 2022-07-14 09:17:11
...

一、官网镜像无法下载

解决方法:需要FQ

配置dockerFQ机:

cat /usr/lib/systemd/system/docker.service
 
[Service]
Environment="HTTP_PROXY=http://10.53.16.201:1080/"
 
重启docker服务 

 

 

二、pause k8s镜像下载失败

pod启动失败,查看pod详情(kubectl describe pods podname)

    Events:
      FirstSeen LastSeen    Count   From            SubobjectPath   Type        Reason      Message
      --------- --------    -----   ----            -------------   --------    ------      -------
      56s       56s     1   {default-scheduler }            Normal      Scheduled   Successfully assigned nfs-rc-fc2w8 to duni-node1
      11s       11s     1   {kubelet duni-node1}            Warning     FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "image pull failed for gcr.io/google_containers/pause-amd64:3.0, this may be because there are no credentials on this request.  details: (Get https://gcr.io/v1/_ping: dial tcp 74.125.203.82:443: i/o timeout)"

 

解决:

找到FQ机器 下载镜像,上传到自己的私有仓库:xxx.xxxx.xxxx/pause-amd64:3.0

gcr.io/google_containers/pause-amd64:3.0

 

修改kubelet 配置:

--pod-infra-container-image=xxx.xxxx.xxxx/pause-amd64:3.0 \

 

重启: kubelet

 

三、权限问题

通过rc配置文件起pod,rc中配置了privileged为true,发现pod状态一直Running不起来,查看pod详情发现

    [root@docker tmp]# kubectl describe pods nfs-rc-acbo1
    Name:       nfs-rc-acbo1
    Namespace:  default
    Node:       duni-node2
    Labels:     role=nfs-server
    Status:     Pending
    IP:     
    Controllers:    ReplicationController/nfs-rc
    Containers:
      nfs-server:
        Image:          192.168.100.90:5000/nfs-data
        Port:           2049/TCP
        Volume Mounts:      <none>
        Environment Variables:  <none>
    Conditions:
      Type      Status
      PodScheduled  True 
    No volumes.
    QoS Class:  BestEffort
    Tolerations:    <none>
    Events:
      FirstSeen LastSeen    Count   From            SubobjectPath   Type        Reason          Message
      --------- --------    -----   ----            -------------   --------    ------          -------
      27s       27s     1   {default-scheduler }            Normal      Scheduled       Successfully assigned nfs-rc-acbo1 to duni-node2
      27s       27s     1   {kubelet duni-node2}            Warning     FailedValidation    Error validating pod nfs-rc-acbo1.default from api, ignoring: spec.containers[0].securityContext.privileged: Forbidden: disallowed by policy

 

解决:

修改所有node以及master节点的k8s配置文件 vim /etc/kubernetes/config

$ KUBE_ALLOW_PRIV="--allow-privileged=true"
$ systemctl restart kube-apiserver


四、getsockopt: connection timed out’问题 
如果安装的docker版本为1.13及以上,并且网络畅通,flannel、etcd都正常,但还是会出现getsockopt: connection timed out'的错误,则可能是iptables配置问题。具体问题:
Error: 'dial tcp 10.233.50.3:8443: getsockopt: connection timed out
  • 1
docker从1.13版本开始,可能将iptables FORWARD chain的默认策略设置为DROP,从而导致ping其他Node上的Pod IP失败,遇到这种问题时,需要手动设置策略为ACCEPT:
sudo iptables -P FORWARD ACCEPT
  • 1
使用iptables -nL命令查看,发现Forward的策略还是drop,可是我们明明执行了iptables -P FORWARD ACCEPT。原来,docker是在这句话执行之后启动的,需要每次在docker之后再执行这句话。。。这么做有点太麻烦了,所以我们修改下docker的启动脚本:
vi /usr/lib/systemd/system/docker.service
[Service] Type=notify # the default is not to use systemd for cgroups because the delegate issues still # exists and systemd currently does not support the cgroup feature set required # for containers run by docker
ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS $DOCKER_OPTS $DOCKER_DNS_OPTIONS # 添加这行操作,在每次重启docker之前都会设置iptables策略为ACCEPT
ExecStartPost=/sbin/iptables -I FORWARD -s 0.0.0.0/0 -j ACCEPT
ExecReload=/bin/kill -s HUP $MAINPID
 
 
五、部署 Ingress Controller 的问题

部署完了后端就得把最重要的组件 Nginx+Ingres Controller(官方统一称为 Ingress Controller) 部署上
➜ ~ kubectl create -f nginx-ingress-controller.yaml daemonset "nginx-ingress-lb" created
注意:官方的 Ingress Controller 有个坑,至少我看了 DaemonSet 方式部署的有这个问题:没有绑定到宿主机 80 端口,也就是说前端 Nginx 没有监听宿主机 80 端口(这还玩个卵啊);所以需要把配置搞下来自己加一下 hostNetwork,截图如下
 
 
 
hostNetwork: true ##这行
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0
 

六、问题6
kube-proxy: E0103 18:30:04.060770 40611 proxier.go:977] conntrack return with error: error looking for path of conntrack: exec: "conntrack": executable file not found in $PATH
 
yum install conntrack-tools -y
 
 
七、api-server 多台api
uthentication.go:58] Unable to authenticate the request due to an error: [invalid bearer token, [invalid bearer token, [invalid bearer token, crypto/rsa: verification error, invalid bearer token
 
update-ca-trust
 
八、容器状态为Terminating
强制删除 状态STATUS为Terminating:
kubectl delete pod kubernetes-dashboard-5c7d5fc568-gq7lj --grace-period=0 --force -n kube-system


九、访问dashboard的问题
Unauthorized问题
{ "kind": "Status",
"apiVersion": "v1",
"metadata": { },
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401 }
 
解决方法: 
新建basic_auth_file文件,并在其中添加:
admin,admin,1002
  • 1
文件内容格式:password,username,uid
然后在api-server配置文件(即上面的配置文件)中添加--=/etc/kubernetes/basic_auth_file \
保存重启kube-apiserver: