010.Kubernetes二进制部署kube-controller-manager
程序员文章站
2022-08-20 08:11:16
一 部署高可用kube-controller-manager 1.1 高可用kube-controller-manager介绍 本实验部署一个三实例 kube-controller-manager 的集群,启动后将通过竞争选举机制产生一个 leader 节点,其它节点为阻塞状态。当 leader 节 ......
一 部署高可用kube-controller-manager
1.1 高可用kube-controller-manager介绍
本实验部署一个三实例 kube-controller-manager 的集群,启动后将通过竞争选举机制产生一个 leader 节点,其它节点为阻塞状态。当 leader 节点不可用时,阻塞的节点将再次进行选举产生新的 leader 节点,从而保证服务的可用性。
为保证通信安全,本文档先生成 x509 证书和私钥,kube-controller-manager 在如下两种情况下使用该证书:
- 与 kube-apiserver 的安全端口通信;
- 在安全端口(https,10252) 输出 prometheus 格式的 metrics。
1.2 创建kube-controller-manager证书和私钥
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# cat > kube-controller-manager-csr.json <<eof 3 { 4 "cn": "system:kube-controller-manager", 5 "hosts": [ 6 "127.0.0.1", 7 "172.24.8.71", 8 "172.24.8.72", 9 "172.24.8.73" 10 ], 11 "key": { 12 "algo": "rsa", 13 "size": 2048 14 }, 15 "names": [ 16 { 17 "c": "cn", 18 "st": "shanghai", 19 "l": "shanghai", 20 "o": "system:kube-controller-manager", 21 "ou": "system" 22 } 23 ] 24 } 25 eof 26 #创建kube-controller-manager的ca证书请求文件
解释:
hosts 列表包含所有 kube-controller-manager 节点 ip;
cn 和 o 均为 system:kube-controller-manager,kubernetes 内置的 clusterrolebindings system:kube-controller-manager 赋予 kube-controller-manager 工作所需的权限。
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem \ 3 -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json \ 4 -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager #生成ca密钥(ca-key.pem)和证书(ca.pem)
1.3 分发证书和私钥
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for master_ip in ${master_ips[@]} 4 do 5 echo ">>> ${master_ip}" 6 scp kube-controller-manager*.pem root@${master_ip}:/etc/kubernetes/cert/ 7 done
1.4 创建和分发kubeconfig
kube-controller-manager 使用 kubeconfig 文件访问 apiserver,该文件提供了 apiserver 地址、嵌入的 ca 证书和 kube-controller-manager 证书:
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# kubectl config set-cluster kubernetes \ 4 --certificate-authority=/opt/k8s/work/ca.pem \ 5 --embed-certs=true \ 6 --server=${kube_apiserver} \ 7 --kubeconfig=kube-controller-manager.kubeconfig 8 9 [root@k8smaster01 work]# kubectl config set-credentials system:kube-controller-manager \ 10 --client-certificate=kube-controller-manager.pem \ 11 --client-key=kube-controller-manager-key.pem \ 12 --embed-certs=true \ 13 --kubeconfig=kube-controller-manager.kubeconfig 14 15 [root@k8smaster01 work]# kubectl config set-context system:kube-controller-manager \ 16 --cluster=kubernetes \ 17 --user=system:kube-controller-manager \ 18 --kubeconfig=kube-controller-manager.kubeconfig 19 20 [root@k8smaster01 work]# kubectl config use-context system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig 21 22 [root@k8smaster01 ~]# cd /opt/k8s/work 23 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 24 [root@k8smaster01 work]# for master_ip in ${master_ips[@]} 25 do 26 echo ">>> ${master_ip}" 27 scp kube-controller-manager.kubeconfig root@${master_ip}:/etc/kubernetes/ 28 done
1.5 创建kube-controller-manager的systemd
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# cat > kube-controller-manager.service.template <<eof 4 [unit] 5 description=kubernetes controller manager 6 documentation=https://github.com/googlecloudplatform/kubernetes 7 8 [service] 9 workingdirectory=${k8s_dir}/kube-controller-manager 10 execstart=/opt/k8s/bin/kube-controller-manager \\ 11 --profiling \\ 12 --cluster-name=kubernetes \\ 13 --controllers=*,bootstrapsigner,tokencleaner \\ 14 --kube-api-qps=1000 \\ 15 --kube-api-burst=2000 \\ 16 --leader-elect \\ 17 --use-service-account-credentials\\ 18 --concurrent-service-syncs=2 \\ 19 --bind-address=##master_ip## \\ 20 --secure-port=10252 \\ 21 --tls-cert-file=/etc/kubernetes/cert/kube-controller-manager.pem \\ 22 --tls-private-key-file=/etc/kubernetes/cert/kube-controller-manager-key.pem \\ 23 --port=0 \\ 24 --authentication-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\ 25 --client-ca-file=/etc/kubernetes/cert/ca.pem \\ 26 --requestheader-allowed-names="" \\ 27 --requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem \\ 28 --requestheader-extra-headers-prefix="x-remote-extra-" \\ 29 --requestheader-group-headers=x-remote-group \\ 30 --requestheader-username-headers=x-remote-user \\ 31 --authorization-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\ 32 --cluster-signing-cert-file=/etc/kubernetes/cert/ca.pem \\ 33 --cluster-signing-key-file=/etc/kubernetes/cert/ca-key.pem \\ 34 --experimental-cluster-signing-duration=8760h \\ 35 --horizontal-pod-autoscaler-sync-period=10s \\ 36 --concurrent-deployment-syncs=10 \\ 37 --concurrent-gc-syncs=30 \\ 38 --node-cidr-mask-size=24 \\ 39 --service-cluster-ip-range=${service_cidr} \\ 40 --pod-eviction-timeout=6m \\ 41 --terminated-pod-gc-threshold=10000 \\ 42 --root-ca-file=/etc/kubernetes/cert/ca.pem \\ 43 --service-account-private-key-file=/etc/kubernetes/cert/ca-key.pem \\ 44 --kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\ 45 --logtostderr=true \\ 46 --v=2 47 restart=on-failure 48 restartsec=5 49 50 [install] 51 wantedby=multi-user.target 52 eof
1.6 分发systemd
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for (( i=0; i < 3; i++ )) 4 do 5 sed -e "s/##master_name##/${master_names[i]}/" -e "s/##master_ip##/${master_ips[i]}/" kube-controller-manager.service.template > kube-controller-manager-${master_ips[i]}.service 6 done #修正相应ip 7 [root@k8smaster01 work]# ls kube-controller-manager*.service 8 [root@k8smaster01 ~]# cd /opt/k8s/work 9 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 10 [root@k8smaster01 work]# for master_ip in ${master_ips[@]} 11 do 12 echo ">>> ${master_ip}" 13 scp kube-controller-manager-${master_ip}.service root@${master_ip}:/etc/systemd/system/kube-controller-manager.service 14 done #分发system
二 启动并验证
2.1 启动kube-controller-manager 服务
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for master_ip in ${master_ips[@]} 4 do 5 echo ">>> ${master_ip}" 6 ssh root@${master_ip} "mkdir -p ${k8s_dir}/kube-controller-manager" 7 ssh root@${master_ip} "systemctl daemon-reload && systemctl enable kube-controller-manager && systemctl restart kube-controller-manager" 8 done
2.2 检查kube-controller-manager 服务
1 [root@k8smaster01 ~]# source /opt/k8s/bin/environment.sh 2 [root@k8smaster01 ~]# for master_ip in ${master_ips[@]} 3 do 4 echo ">>> ${master_ip}" 5 ssh root@${master_ip} "systemctl status kube-controller-manager|grep active" 6 done
2.3 查看输出的 metrics
1 [root@k8smaster01 ~]# curl -s --cacert /opt/k8s/work/ca.pem --cert /opt/k8s/work/admin.pem --key /opt/k8s/work/admin-key.pem https://172.24.8.71:10252/metrics |head
注意:以上命令在 kube-controller-manager 节点上执行。
2.4 查看权限
1 [root@k8smaster01 ~]# kubectl describe clusterrole system:kube-controller-manager
clusterole system:kube-controller-manager 的权限很小,只能创建 secret、serviceaccount 等资源对象,各 controller 的权限分散到 clusterrole system:controller:xxx 中。
当在 kube-controller-manager 的启动参数中添加 --use-service-account-credentials=true 参数,这样 main controller 会为各 controller 创建对应的 serviceaccount xxx-controller。内置的 clusterrolebinding system:controller:xxx 将赋予各 xxx-controller serviceaccount 对应的 clusterrole system:controller:xxx 权限。
1 [root@k8smaster01 ~]# kubectl get clusterrole|grep controller
如deployment controller:
1 [root@k8smaster01 ~]# kubectl describe clusterrole system:controller:deployment-controller
2.5 查看当前leader
1 [root@k8smaster01 ~]# kubectl get endpoints kube-controller-manager --namespace=kube-system -o yaml
kubelet 认证和授权:https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authorization
上一篇: 数据挖掘
下一篇: Redis必备面试题《难点篇》
推荐阅读
-
Centos7.5安装mysql5.7.24二进制包方式部署
-
009.Kubernetes二进制部署kube-apiserver
-
012.Kubernetes二进制部署worker节点Flannel
-
003.Kubernetes二进制部署准备
-
004.Kubernetes二进制部署创建证书
-
006.Kubernetes二进制部署ETCD
-
005.Kubernetes二进制部署kubectl
-
010.Kubernetes二进制部署kube-controller-manager
-
014.Kubernetes二进制部署docker
-
013.Kubernetes二进制部署worker节点Nginx实现高可用