kubernetes 初始化

说明

kubernetes master, compute node, rook node 部署前， 参考下面步骤
docker 软件安装
kubetnetes 软件安装
自定义 registry
镜像获取

安装前准备

docker 安装

安装系统常用软件

yum install -y yum-utils device-mapper-persistent-data lvm2 policycoreutils-python libcgroup libtool-ltdl

安装 docker CE 软件源

yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

启用软件源

yum-config-manager --enable docker-ce-nightly  
yum-config-manager --disable docker-ce-test  

docker 软件安装

yum install docker-ce docker-ce-cli containerd.io

kubetnetes 安装

安装前准备

主机名定义

命令行参考

hostnamectl set-hostname xxxxxx

把集群中所有主机名都写入 /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.189.20.xx ns-yun-020065.vclound.com
10.189.20.xx ns-yun-020066.vclound.com
10.189.20.xx ns-yun-020067.vclound.com
10.189.20.xxx  ns-storage-020100.vclound.com
10.189.20.xxx  ns-storage-020101.vclound.com
10.189.20.xxx  ns-storage-020102.vclound.com
10.189.20.xxx  ns-storage-020104.vclound.com

关闭 selinux

cat /etc/selinux/config
SELINUX=disabled

关闭 NetworkManager

systemctl stop NetworkManager
systemctl disable Networkmanager

清空防火墙规则

iptables -F
iptables -t nat -F

netfilter 管理

netfilter bridge 模块加载

modprobe br_netfilter

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1
EOF

sysctl -p

关闭 swap 分区

swapoff -a
free
sed -i /swap/s/^/#/ /etc/fstab

更新 kubernetes yum 源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF 

安装 kubernetes 软件

yum -y install kubelet kubeadm kubectl 

cgroupfs 管理

假如使用了 cgroupfs 那么修改配置

/etc/systemd/system/kubelet.service.d/10-kubeadm.conf (添加下面行)
Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=cgroupfs"

服务启动

systemctl daemon-reload
systemctl restart docker && systemctl enable docker
systemctl restart kubelet && systemctl enable kubelet

镜像获取

由于国内无法直接下载 kubernetes docker images 可以把 registry 指向 registry.cn-hangzhou.aliyuncs.com
参考阿里云帮助

1. 到 https://account.aliyun.com/register/register.htm 注册合法账号
2. k8s master 上命令行执行 docker login registry.cn-hangzhou.aliyuncs.com
3. 输入用户名，密码

[[email protected] tmp]# docker login registry.cn-hangzhou.aliyuncs.com
Username: your user name
Password: password
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store	
Login Succeeded

登录 registry 信息会自动保存到 /root/.docker/config.json 中

4. 下载下面 images

registry.aliyuncs.com/google_containers/kube-apiserver:v1.13.3
registry.aliyuncs.com/google_containers/kube-proxy:v1.13.3
registry.aliyuncs.com/google_containers/kube-controller-manager:v1.13.3
registry.aliyuncs.com/google_containers/kube-scheduler:v1.13.3
registry.aliyuncs.com/google_containers/coredns:1.2.6
registry.aliyuncs.com/google_containers/etcd:3.2.24
registry.aliyuncs.com/google_containers/pause:3.1
registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1

ex:
[[email protected] ~]# docker pull registry.aliyuncs.com/google_containers/kube-apiserver:v1.13.3
v1.13.3: Pulling from google_containers/kube-apiserver
73e3e9d78c61: Already exists
d261e2f8ca5b: Pull complete
Digest: sha256:d274dc290247bb761dac4a7aa088198a76b0abcb7427ce78ec3b24a9d8773782
Status: Downloaded newer image for registry.aliyuncs.com/google_containers/kube-apiserver:v1.13.3

5. 镜像标签改名 (因为 kubernetes 指定了镜像名字 k8s.grc.io/xxxxx )

docker tag registry.aliyuncs.com/google_containers/kube-apiserver:v1.13.3 k8s.gcr.io/kube-apiserver:v1.13.3  
docker tag registry.aliyuncs.com/google_containers/kube-proxy:v1.13.3 k8s.gcr.io/kube-proxy:v1.13.3 
docker tag registry.aliyuncs.com/google_containers/kube-controller-manager:v1.13.3 k8s.gcr.io/kube-controller-manager:v1.13.3  
docker tag registry.aliyuncs.com/google_containers/kube-scheduler:v1.13.3 k8s.gcr.io/kube-scheduler:v1.13.3 
docker tag registry.aliyuncs.com/google_containers/coredns:1.2.6 k8s.gcr.io/coredns:1.2.6 
docker tag registry.aliyuncs.com/google_containers/etcd:3.2.24 k8s.gcr.io/etcd:3.2.24 
docker tag registry.aliyuncs.com/google_containers/pause:3.1 k8s.gcr.io/pause:3.1

6. 最后，所需的镜像列表如下

k8s.gcr.io/kube-apiserver:v1.13.3  
k8s.gcr.io/kube-controller-manager:v1.13.3  
k8s.gcr.io/kube-scheduler:v1.13.3  
k8s.gcr.io/kube-proxy:v1.13.3  
k8s.gcr.io/pause:3.1  
k8s.gcr.io/etcd:3.2.24  
k8s.gcr.io/coredns:1.2.6  

查询镜像列表

[[email protected] ceph]# kubeadm config images list
k8s.gcr.io/kube-apiserver:v1.13.3
k8s.gcr.io/kube-controller-manager:v1.13.3
k8s.gcr.io/kube-scheduler:v1.13.3
k8s.gcr.io/kube-proxy:v1.13.3
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.2.24
k8s.gcr.io/coredns:1.2.6