kubernetes 1.14.1二进制离线安装高可用生产环境部署手册

程序员文章站 2024-03-12 12:09:20

...

2019独角兽企业重金招聘Python工程师标准>>> kubernetes 1.14.1二进制离线安装高可用生产环境部署手册

零、准备阶段

0. 安装机器列表

主机名	主机地址
master01	10.70.68.122
master02	10.70.67.149
master03	10.70.67.150
node01	10.70.160.21
node02	10.70.160.210
node03	10.70.160.211

/etc/hosts

10.70.68.122 master01
10.70.67.149 master02
10.70.67.150 master03
10.70.160.21 node01
10.70.160.210 node02
10.70.160.211 node03
10.72.46.165 img.cn

0.1 软件下载

docker
- 文档地址 https://docs.docker.com/
- Docker的release notes https://docs.docker.com/engine/release-notes
- Docker下载地址 https://docs.docker.com/v18.03/install/linux/docker-ce/centos/
- Centos版本可以下载rpm二进制安装包 https://download.docker.com/linux/centos/7/x86_64/stable/Packages/
etcd
- 下载地址 https://github.com/etcd-io/etcd/releases
- 配置信息请参考官方文档 https://github.com/etcd-io/etcd/tree/master/Documentation
kubernetes
- 在CHANGELOG-version.md中可以得到二进制文件的下载地址
- https://github.com/kubernetes/kubernetes/releases
- kube-apiserver,kube-controller-manager,kube-scheduler等组件的详细配置文件请参考官方网站
  - https://kubernetes.io/zh/docs/reference/command-line-tools-reference/kube-proxy/
  - https://kubernetes.io/zh/docs/admin/kube-apiserver/
flannel
- 下载地址 https://github.com/coreos/flannel/releases
- 配置文档 https://github.com/coreos/flannel

0.2 配置免**登录

ssh-******
# 将公钥复制到各服务节点
ssh-copy-id [email protected]
scp ~/.ssh/id_rsa* [email protected]:~/.ssh/

1. 系统设置

在所有master和node节点上执行以下脚本 sh KubernetesInstall-01-nodePrepare.sh

#!/bin/bash
# Initialize the machine. This needs to be executed on every machine.
# Add host domain name.
cat>>/etc/hosts<<EOF
10.70.68.122 master01
10.70.67.149 master02
10.70.67.150 master01
10.70.160.21 node01
10.70.160.210 node02
10.70.160.211 node03
10.72.46.165 img.cn
EOF
# Modify related kernel parameters.
cat>/etc/sysctl.d/kubernetes.conf<<EOF
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl -p /etc/sysctl.d/kubernetes.conf>&/dev/null
# Turn off and disable the firewalld.
systemctl stop firewalld
systemctl disable firewalld
# Disable the SELinux.
sed -i.bak 's/=enforcing/=disabled/' /etc/selinux/config
# Disable the swap .
sed -i.bak 's/^.*swap/#&/g' /etc/fstab
# Reboot the machine.
reboot

2. 部署etcd

2.1 创建etcd证书

将以下文件放到和脚本相同的目录中

cfssl-certinfo_linux-amd64
cfssljson_linux-amd64
cfssl_linux-amd64

再执行sh KubernetesInstall-02-etcdCAPrepare.sh

#!/bin/bash
\cp cfssl* /usr/local/bin/
chmod +x /usr/local/bin/cfssl*
ETCD_SSL=/etc/etcd/ssl
mkdir -p $ETCD_SSL
# Create some CA certificates for etcd cluster.
cat<<EOF>$ETCD_SSL/ca-config.json
{
  "signing": {
    "default": {
      "expiry": "876000h"
    },
    "profiles": {
      "www": {
         "expiry": "876000h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
EOF
cat<<EOF>$ETCD_SSL/ca-csr.json
{
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Chengdu",
            "ST": "Chengdu"
        }
    ]
}
EOF
cat<<EOF>$ETCD_SSL/server-csr.json
{
    "CN": "etcd",
    "hosts": [
    "10.70.68.122",
    "10.70.67.149",
    "10.70.67.150"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Chengdu",
            "ST": "Chengdu"
        }
    ]
}
EOF
cd $ETCD_SSL
cfssl_linux-amd64 gencert -initca ca-csr.json | cfssljson_linux-amd64 -bare ca -
cfssl_linux-amd64 gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson_linux-amd64 -bare server
ls $ETCD_SSL/*.pem

2.1.1 将证书文件复制到其它节点

ssh master02 "mkdir -p /etc/etcd/ssl"
scp /etc/etcd/ssl/*.pem master02:/etc/etcd/ssl/

2.2 安装

将etcd的安装程序etcd-v3.3.12-linux-amd64.tar.gz和脚本放在同一目录下， sh KubernetesInstall-03-etcdInstall.sh etcd-v3.3.12-linux-amd64 10.70.68.122 etcd-01 "etcd-01=https://10.70.68.122:2380,etcd-02=https://10.70.67.149:2380,etcd-03=https://10.70.67.150:2380"

etcd-v3.3.12-linux-amd64 etcd软件安装包的前缀
10.70.68.122 当前结点的ip地址
etcd-01 etcd结点的名称
"etcd-01=https://10.70.68.122:2380,etcd-02=https://10.70.67.149:2380,etcd-03=https://10.70.67.150:2380" 集群中所有结点组成的端点信息

#!/bin/bash
# Deploy and configurate etcd service on the master node.
#etcd-v3.3.12-linux-amd64
ETCD_INATLL_FILE=$1
#10.70.68.122
IP=$2

NAME=$3
#etcd-01=https://10.70.68.122:2380,etcd-02=https://10.70.67.149:2380,etcd-03=https://10.70.67.150:2380
ENDPOINTS=$4
ETCD_CONF=/etc/etcd/etcd.conf
ETCD_SSL=/etc/etcd/ssl
ETCD_SERVICE=/usr/lib/systemd/system/etcd.service
tar -xzf $ETCD_INATLL_FILE.tar.gz
cp -p $ETCD_INATLL_FILE/etc* /usr/local/bin/

mkdir -p /var/lib/etcd/default.etcd
# The etcd configuration file. 
cat>$ETCD_CONF<<EOF
#[Member]
ETCD_NAME="$NAME"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://$IP:2380"
ETCD_LISTEN_CLIENT_URLS="https://$IP:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://$IP:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://$IP:2379"
ETCD_INITIAL_CLUSTER="$ENDPOINTS"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

# The etcd servcie configuration file.
cat>$ETCD_SERVICE<<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=$ETCD_CONF
ExecStart=/usr/local/bin/etcd \
--name=\${ETCD_NAME} \
--data-dir=\${ETCD_DATA_DIR} \
--listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
--advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--initial-cluster=\${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} \
--initial-cluster-state=new \
--cert-file=/etc/etcd/ssl/server.pem \
--key-file=/etc/etcd/ssl/server-key.pem \
--peer-cert-file=/etc/etcd/ssl/server.pem \
--peer-key-file=/etc/etcd/ssl/server-key.pem \
--trusted-ca-file=/etc/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/etc/etcd/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable etcd.service --now
systemctl status etcd

2.2.1 在其它结点安装(需要修改IP地址和etcd的name)

sh KubernetesInstall-04-etcdPeerInstall.sh master01 etcd-01 etcd-02 10.70.68.122 10.70.67.149

master01 etcd主结点的名称，需要从这里复制文件
etcd-01 etcd主结点的名称，需要将这个名称替换成其它的名称
etcd-02 新结点的名称
10.70.68.122 etcd主结点的ip地址，需要被替换成新的地址
10.70.67.149 etcd当前结点的地址

#!/bin/bash
# Deploy etcd on the node1.
node=$1
name=$2
newName=$3
source=$4
dest=$5
ETCD_SSL=/etc/etcd/ssl
mkdir -p $ETCD_SSL
scp $node:$ETCD_SSL/{ca*pem,server*pem} $ETCD_SSL/
scp $node:/etc/etcd/etcd.conf /etc/etcd/
scp $node:/usr/lib/systemd/system/etcd.service /usr/lib/systemd/system/
scp $node:/usr/local/bin/etc* /usr/local/bin/
sed -i "/ETCD_NAME/{s/$name/$newName/g}" /etc/etcd/etcd.conf #修改IP和etcd名称
sed -i "/ETCD_LISTEN_PEER_URLS/{s/$source/$dest/g}" /etc/etcd/etcd.conf
sed -i "/ETCD_LISTEN_CLIENT_URLS/{s/$source/$dest/g}" /etc/etcd/etcd.conf
sed -i "/ETCD_INITIAL_ADVERTISE_PEER_URLS/{s/$source/$dest2/g}" /etc/etcd/etcd.conf
sed -i "/ETCD_ADVERTISE_CLIENT_URLS/{s/$source/$dest/g}" /etc/etcd/etcd.conf
systemctl daemon-reload
systemctl enable etcd.service --now
systemctl status etcd

2.3 在Node结点安装docker

如果使用离线方式安装可以将docker.rpm文件上传到服务器，运行yum install docker.rpm进行安装。

2.4 验证

etcdctl \
--ca-file=/etc/etcd/ssl/ca.pem \
--cert-file=/etc/etcd/ssl/server.pem \
--key-file=/etc/etcd/ssl/server-key.pem \
--endpoints="https://10.70.68.122:2379,etcd-02=https://10.70.67.149:2379,etcd-03=https://10.70.67.150:2379" cluster-health

3. 部署`Flannel`网络插件

3.1 在`etcd`中添加配置数据

在master节点上执行以下脚本

ENDPOINTS=https://10.70.68.122:2379,etcd-02=https://10.70.67.149:2379,etcd-03=https://10.70.67.150:2379
# Check whether etcd cluster is healthy.
etcdctl \
--ca-file=/etc/etcd/ssl/ca.pem \
--cert-file=/etc/etcd/ssl/server.pem \
--key-file=/etc/etcd/ssl/server-key.pem \
--endpoints="$ENDPOINTS" cluster-health

# Writing into a predetermined subnetwork.
cd /etc/etcd/ssl
etcdctl \
--ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \
--endpoints="$ENDPOINTS" \
set /coreos.com/network/config  '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'
cd ~

3.2 复制`etcd`证书到各node节点

首先在各节点上创建etcd的配置文件存放目录

ssh node01 " mkdir -p /etc/etcd/ssl"
ssh node02 " mkdir -p /etc/etcd/ssl"
ssh node03 " mkdir -p /etc/etcd/ssl"

再将证书文件复制到各node节点

scp  /etc/etcd/ssl/*.pem node01:/etc/etcd/ssl/

3.3 在各`node`节点上执行以下脚本

将脚本与flannel-v0.11.0-linux-amd64.tar.gz文件放在同一目录下，

sh KubernetesInstall-05-flannel.sh "https://10.70.68.122:2379,https://10.70.67.149:2379,https://10.70.67.150:2379" flannel-v0.11.0-linux-amd64.tar.gz

"etcd-01=https://10.70.68.122:2379,etcd-02=https://10.70.67.149:2379,etcd-03=https://10.70.67.150:2379" etcd端点列表
flannel-v0.11.0-linux-amd64.tar.gz 安装文件路径

#!/bin/bash
ENDPOINTS=$1
INSTALL_FILE=$2
KUBE_CONF=/etc/kubernetes
FLANNEL_CONF=$KUBE_CONF/flannel.conf
mkdir -p $KUBE_CONF
tar -xvzf $INSTALL_FILE
mv {flanneld,mk-docker-opts.sh} /usr/local/bin/


# Configuration the flannel service.
cat>$FLANNEL_CONF<<EOF
FLANNEL_ETCD_KEY="/coreos.com/network"
FLANNEL_OPTIONS="--etcd-endpoints=$ENDPOINTS -etcd-cafile=/etc/etcd/ssl/ca.pem -etcd-certfile=/etc/etcd/ssl/server.pem -etcd-keyfile=/etc/etcd/ssl/server-key.pem"
EOF
cat>/usr/lib/systemd/system/flanneld.service<<EOF
[Unit]
Description=Flanneld overlay address etcd agent
After=network-online.target network.target
Before=docker.service

[Service]
Type=notify
EnvironmentFile=$FLANNEL_CONF
ExecStart=/usr/local/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
ExecStartPost=/usr/local/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

# Modify the docker service.
sed -i.bak -e '/ExecStart/i EnvironmentFile=\/run\/flannel\/subnet.env' -e 's/ExecStart=\/usr\/bin\/dockerd/ExecStart=\/usr\/bin\/dockerd $DOCKER_NETWORK_OPTIONS/g' /usr/lib/systemd/system/docker.service

# Start or restart related services.
systemctl daemon-reload
systemctl enable flanneld --now
systemctl restart docker
systemctl status flanneld
systemctl status docker
ip address show

3.4 etcd配置详细参考

https://github.com/coreos/flannel/blob/master/Documentation/configuration.md

一、部署master节点

1. 创建CA证书

sh KubernetesInstall-06-CA.sh

#!/bin/bash
# Deploy the master node.
KUBE_SSL=/etc/kubernetes/ssl
mkdir -p $KUBE_SSL

# Create CA.
cat>$KUBE_SSL/ca-config.json<<EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "kubernetes": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
EOF
cat>$KUBE_SSL/ca-csr.json<<EOF
{
    "CN": "kubernetes",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Chengdu",
            "ST": "Chengdu",
            "O": "
			ir",
            "OU": "IMG"
        }
    ]
}
EOF
cat>$KUBE_SSL/server-csr.json<<EOF
{
    "CN": "kubernetes",
    "hosts": [
      "10.0.0.1",
      "192.168.1.1",
      "127.0.0.1",
      "10.70.160.21",
      "10.70.160.210",
      "10.70.160.211",
      "10.70.68.122",
      "10.70.67.149",
      "10.70.67.150",
      "localhost",
      "apiclient",
      "kubernetes",
      "kubernetes.default",
      "kubernetes.default.svc",
      "kubernetes.default.svc.cluster",
      "kubernetes.default.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Chengdu",
            "ST": "Chengdu",
            "O": "eone",
            "OU": "IMG"
        }
    ]
}
EOF
cd $KUBE_SSL
cfssl_linux-amd64 gencert -initca ca-csr.json | cfssljson_linux-amd64 -bare ca -
cfssl_linux-amd64 gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson_linux-amd64 -bare server

# Create kube-proxy CA.
cat>$KUBE_SSL/kube-proxy-csr.json<<EOF
{
  "CN": "system:kube-proxy",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
        "C": "CN",
        "L": "Chengdu",
        "ST": "Chengdu",
        "O": "eonezhang",
        "OU": "IMG"
    }
  ]
}
EOF
cfssl_linux-amd64 gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson_linux-amd64 -bare kube-proxy
ls *.pem
cd ~

2. 安装api-server

将安装程序kubernetes-server-linux-amd64.tar.gz和脚本KubernetesInstall-07-apiServer.sh放在当前用户home目录下

sh KubernetesInstall-07-apiServer.sh 10.70.160.21 "https://10.70.68.122:2379,https://10.70.67.149:2379,https://10.70.67.150:2379"

10.70.160.21 当前节点IP
"https://10.70.68.122:2379,https://10.70.67.149:2379,https://10.70.67.150:2379" etcd节点地址

#!/bin/bash
IP=$1
ETCD_ENDPOINTS=$2
KUBE_ETC=/etc/kubernetes
KUBE_API_CONF=/etc/kubernetes/apiserver.conf
tar zxf kubernetes-server-linux-amd64.tar.gz
\cp kubernetes/server/bin/{kube-apiserver,kube-scheduler,kube-controller-manager} /usr/local/bin/

mkdir -p $KUBE_ETC

# Create a token file.
cat>$KUBE_ETC/token.csv<<EOF
$(head -c 16 /dev/urandom | od -An -t x | tr -d ' '),kubelet-bootstrap,10001,"system:kubelet-bootstrap"
EOF

# Create a kube-apiserver configuration file.
cat >$KUBE_API_CONF<<EOF
KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--etcd-servers=$ETCD_ENDPOINTS \
--bind-address=$IP \
--secure-port=6443 \
--advertise-address=$IP \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--enable-bootstrap-token-auth \
--token-auth-file=$KUBE_ETC/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=$KUBE_ETC/ssl/server.pem  \
--tls-private-key-file=$KUBE_ETC/ssl/server-key.pem \
--client-ca-file=$KUBE_ETC/ssl/ca.pem \
--service-account-key-file=$KUBE_ETC/ssl/ca-key.pem \
--etcd-cafile=/etc/etcd/ssl/ca.pem \
--etcd-certfile=/etc/etcd/ssl/server.pem \
--etcd-keyfile=/etc/etcd/ssl/server-key.pem"
EOF

# Create the kube-apiserver service.
cat>/usr/lib/systemd/system/kube-apiserver.service<<EOF
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=etcd.service
Wants=etcd.service

[Service]
EnvironmentFile=-$KUBE_API_CONF
ExecStart=/usr/local/bin/kube-apiserver \$KUBE_APISERVER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable kube-apiserver.service --now
systemctl status kube-apiserver.service

3. 安装api-controller-manager

sh KubernetesInstall-08-controllerManager.sh

#!/bin/bash
# Deploy the controller-manager service.
KUBE_CONTROLLER_CONF=/etc/kubernetes/kube-controller-manager.conf

cat>$KUBE_CONTROLLER_CONF<<EOF
KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=true \
--v=4 \
--master=127.0.0.1:8080 \
--leader-elect=true \
--address=127.0.0.1 \
--service-cluster-ip-range=10.0.0.0/24 \
--cluster-name=kubernetes \
--cluster-signing-cert-file=/etc/kubernetes/ssl/ca.pem \
--cluster-signing-key-file=/etc/kubernetes/ssl/ca-key.pem  \
--root-ca-file=/etc/kubernetes/ssl/ca.pem \
--service-account-private-key-file=/etc/kubernetes/ssl/ca-key.pem"
EOF

cat>/usr/lib/systemd/system/kube-controller-manager.service<<EOF
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes

[Service]
EnvironmentFile=-$KUBE_CONTROLLER_CONF
ExecStart=/usr/local/bin/kube-controller-manager \$KUBE_CONTROLLER_MANAGER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable kube-controller-manager.service --now
sleep 20
systemctl status kube-controller-manager.service

4. 安装scheduler

sh KubernetesInstall-09-scheduler.sh

#!/bin/bash
# Deploy the scheduler service.
KUBE_ETC=/etc/kubernetes
KUBE_SCHEDULER_CONF=$KUBE_ETC/kube-scheduler.conf
cat>$KUBE_SCHEDULER_CONF<<EOF
KUBE_SCHEDULER_OPTS="--logtostderr=true \
--v=4 \
--master=127.0.0.1:8080 \
--leader-elect"
EOF

cat>/usr/lib/systemd/system/kube-scheduler.service<<EOF
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes

[Service]
EnvironmentFile=-$KUBE_SCHEDULER_CONF
ExecStart=/usr/local/bin/kube-scheduler \$KUBE_SCHEDULER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable kube-scheduler.service --now
sleep 20
systemctl status kube-scheduler.service

5. 查看集群状态

mv /opt/apps/k8s/bin/kubectl /usr/local/bin
kubectl get cs

二、部署Node节点

1. 创建bootstrap和kube-proxy的kubeconfig文件

在master01节点上执行下面的脚本创建bootstrap.kubeconfig和kube-proxy.kubeconfig文件

sh KubernetesInstall-10-nodePrepare.sh

#!/bin/bash
BOOTSTRAP_TOKEN=$(awk -F "," '{print $1}' /etc/kubernetes/token.csv)
KUBE_SSL=/etc/kubernetes/ssl/
KUBE_APISERVER="https://10.70.68.122:6443"

cd $KUBE_SSL
# Set cluster parameters.
kubectl config set-cluster kubernetes \
  --certificate-authority=./ca.pem \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=bootstrap.kubeconfig

# Set client parameters.
kubectl config set-credentials kubelet-bootstrap \
  --token=${BOOTSTRAP_TOKEN} \
  --kubeconfig=bootstrap.kubeconfig

# Set context parameters. 
kubectl config set-context default \
  --cluster=kubernetes \
  --user=kubelet-bootstrap \
  --kubeconfig=bootstrap.kubeconfig

# Set context.
kubectl config use-context default --kubeconfig=bootstrap.kubeconfig

# Create kube-proxy kubeconfig file. 
kubectl config set-cluster kubernetes \
  --certificate-authority=./ca.pem \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=kube-proxy.kubeconfig

kubectl config set-credentials kube-proxy \
  --client-certificate=./kube-proxy.pem \
  --client-key=./kube-proxy-key.pem \
  --embed-certs=true \
  --kubeconfig=kube-proxy.kubeconfig

kubectl config set-context default \
  --cluster=kubernetes \
  --user=kube-proxy \
  --kubeconfig=kube-proxy.kubeconfig

kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
cd ~

# Bind kubelet-bootstrap user to system cluster roles.
kubectl create clusterrolebinding kubelet-bootstrap \
  --clusterrole=system:node-bootstrapper \
  --user=kubelet-bootstrap

2. 配置kube-proxy和kubelet服务

在各kubernetes node节点上执行以下脚本

sh KubernetesInstall-11-nodeProxyKubelet.sh master01 10.70.160.21

master01 主节点名称
10.70.160.21 当前节点IP地址

#!/bin/bash
KUBE_CONF=/etc/kubernetes
KUBE_SSL=$KUBE_CONF/ssl
mkdir -p $KUBE_SSL
MASTER_HOST=$1
IP=$2
mkdir $KUBE_SSL
scp $MASTER_HOST:~/kubernetes/server/bin/{kube-proxy,kubelet} /usr/local/bin/
scp $MASTER_HOST:$KUBE_CONF/ssl/{bootstrap.kubeconfig,kube-proxy.kubeconfig} $KUBE_CONF
cat>$KUBE_CONF/kube-proxy.conf<<EOF
KUBE_PROXY_OPTS="--logtostderr=true \
--v=4 \
--hostname-override=$IP \
--cluster-cidr=10.0.0.0/24 \
--kubeconfig=$KUBE_CONF/kube-proxy.kubeconfig"
EOF
cat>/usr/lib/systemd/system/kube-proxy.service<<EOF
[Unit]
Description=Kubernetes Proxy
After=network.target

[Service]
EnvironmentFile=-$KUBE_CONF/kube-proxy.conf
ExecStart=/usr/local/bin/kube-proxy \$KUBE_PROXY_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable kube-proxy.service --now
sleep 20
systemctl status kube-proxy.service -l
cat>$KUBE_CONF/kubelet.yaml<<EOF
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: $IP
port: 10250
readOnlyPort: 10255
cgroupDriver: cgroupfs
clusterDNS: ["10.0.0.2"]
clusterDomain: cluster.local.
failSwapOn: false
authentication:
  anonymous:
    enabled: true
EOF
cat>$KUBE_CONF/kubelet.conf<<EOF
KUBELET_OPTS="--logtostderr=true \
--v=4 \
--hostname-override=$IP \
--kubeconfig=$KUBE_CONF/kubelet.kubeconfig \
--bootstrap-kubeconfig=$KUBE_CONF/bootstrap.kubeconfig \
--config=$KUBE_CONF/kubelet.yaml \
--cert-dir=$KUBE_SSL \
--pod-infra-container-image=img.cn/google-containers/pause-amd64:3.0"
EOF
cat>/usr/lib/systemd/system/kubelet.service<<EOF
[Unit]
Description=Kubernetes Kubelet
After=docker.service
Requires=docker.service

[Service]
EnvironmentFile=$KUBE_CONF/kubelet.conf
ExecStart=/usr/local/bin/kubelet \$KUBELET_OPTS
Restart=on-failure
KillMode=process

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable kubelet.service --now
sleep 20
systemctl status kubelet.service -l

三、 Approve kubelet CSR请求

在master节点上执行sh KubernetesInstall-masterApprove.sh

#!/bin/bash
CSRS=$(kubectl get csr | awk '{if(NR>1) print $1}')
for csr in $CSRS;
    do
        kubectl certificate approve $csr;
    done

查看集群状态

kubectl get cs
kubectl get node

运行测试

kubectl run nginx --image=nginx --replicas=3
kubectl expose deployment nginx --port=88 --target-port=80 --type=NodePort
kubectl get pods
kubectl get svc

转载于:https://my.oschina.net/eonezhang/blog/3038910

上一篇： Eclipse使用maven搭建spring mvc图文教程

下一篇： java基础_构造方法与setget方法