docker三剑客docker-swarm：认识docker-swarm和单间docker-swarm集群

一、认识docker-swarm

二、搭建docker-swarm集群

[[e-mail] ~/compose]#docker-compose down
Stopping compose_web2_1    ... done
Stopping compose_web1_1    ... done
Stopping compose_haproxy_1 ... done
Removing compose_web2_1    ... done
Removing compose_web1_1    ... done
Removing compose_haproxy_1 ... done
Removing network compose_haproxy-net

[[e-mail] ~]#docker volume rm compose_webdata 
compose_webdata

2. 升级docker-ce

yum update docker-ce
yum update docker-ce-cli
systemctl restart docker

3. swarm初始化

[[e-mail] ~]#docker swarm init  # 默认server1是管理节点

[[e-mail] ~]#docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
434iwcmh9gsil4bs3rjm0xtnq *   server1             Ready               Active              Leader              19.03.8

将server2和server3加入管理节点

[[e-mail] ~]# docker swarm join --token SWMTKN-1-3ze4ydrb4p9w8la5tgnf3h6ezqnfcfc2b1b158thrsy3p19viy-00buur49j925804uc6vyne1gl 172.25.60.1:2377
[[e-mail] ~]# docker swarm join --token SWMTKN-1-3ze4ydrb4p9w8la5tgnf3h6ezqnfcfc2b1b158thrsy3p19viy-00buur49j925804uc6vyne1gl 172.25.60.1:2377

[[e-mail] ~]#docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
434iwcmh9gsil4bs3rjm0xtnq *   server1             Ready               Active              Leader              19.03.8
6lb9ftc8qm2mntup69ix8jcno     server2             Ready               Active                                  19.03.8
tkm4di4em3degw6muvq46jmi2     server3             Ready               Active                                  19.03.8

[[e-mail] ~]#cd /etc/docker/
[[e-mail] /etc/docker]#ls
certs.d  daemon.json  key.json
[[e-mail] /etc/docker]#cat daemon.json 
{
  "registry-mirrors": ["https://qe6d82ah.mirror.aliyuncs.com"]
}
[[e-mail] /etc/docker]#scp daemon.json server2:/etc/docker
daemon.json                                                                                                     100%   67    88.8KB/s   00:00    
[[e-mail] /etc/docker]#scp daemon.json server3:/etc/docker
daemon.json               

[[e-mail] ~]# cd /etc/docker/
[[e-mail] docker]# systemctl daemon-reload
[[e-mail] docker]# systemctl restart docker

[[e-mail] ~]# cd /etc/docker/
[[e-mail] docker]# systemctl daemon-reload 
[[e-mail] docker]# systemctl restart docker

[[e-mail] docker]# yum install bash-* # 补齐指令

创建覆盖性网络

[[e-mail] ~]#docker network create -d overlay mynet
8nohe3jqn8k3se56febj41tsu

创建服务

[[e-mail] ~]#docker service create --name my_cluster --network mynet --replicas 3 -p 80:80 ikubernetes/myapp:v1


[[e-mail] docker]# docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
ikubernetes/myapp       <none>              d4a5e0eaa84f        2 years ago         15.5MB

[[e-mail] docker]# docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
ikubernetes/myapp       <none>              d4a5e0eaa84f        2 years ago         15.5MB

[[e-mail] ~]#docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE                  PORTS
xpe6pn85pn5l        my_cluster          replicated          3/3                 ikubernetes/myapp:v1   *:80->80/tcp
[[e-mail] ~]#docker service ps my_cluster
ID                  NAME                IMAGE                  NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
27fiyv55u1t6        my_cluster.1        ikubernetes/myapp:v1   server3             Running             Running 2 minutes ago                       
t4e5vmo8gpf2        my_cluster.2        ikubernetes/myapp:v1   server1             Running             Running 3 minutes ago                       
drz58grz109i        my_cluster.3        ikubernetes/myapp:v1   server2             Running             Running 3 minutes ago 

监控

[[e-mail] ~]#docker pull dockersamples/visualizer

[[e-mail] ~]#docker service create \
>   --name=viz \
>   --publish=8080:8080/tcp \
>   --constraint=node.role==manager \
>   --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
>   dockersamples/visualizer
ersi0ixi4ezh4qtegwfwd9t78
overall progress: 1 out of 1 tasks 
1/1: running   [==================================================>] 
verify: Service converged 

容器的拉升

[[e-mail] ~]#docker service scale my_cluster=6
my_cluster scaled to 6
overall progress: 6 out of 6 tasks 
1/6: running   [==================================================>] 
2/6: running   [==================================================>] 
3/6: running   [==================================================>] 
4/6: running   [==================================================>] 
5/6: running   [==================================================>] 
6/6: running   [==================================================>] 
verify: Service converged 

缩减

[[e-mail] ~]#docker service scale my_cluster=3
my_cluster scaled to 3
overall progress: 3 out of 3 tasks 
1/3: running   [==================================================>] 
2/3: running   [==================================================>] 
3/3: running   [==================================================>] 
verify: Service converged 

滚动更新

[[e-mail] ~]#docker service update --image ikubernetes/myapp:v2 --update-parallelism 3 --update-delay 5s my_cluster  # 每次更新3个间隔5秒

数据卷挂载

[[e-mail] ~/compose]#docker service rm my_cluster 
my_cluster
[[e-mail] ~]#cd compose/

[[e-mail] ~/compose]#cat docker-compose.yml 
version: "3.7"
services:

  myapp:
    image: ikubernetes/myapp:v1
    networks:
      - mynet
    ports:
      - "80:80"
    deploy:
      replicas: 3
      update_config:
        parallelism: 2
        delay: 5s
      restart_policy:
        condition: on-failure

networks:
  mynet:

[[e-mail] ~/compose]#docker stack deploy -c docker-compose.yml my_cluster
Creating network my_cluster_mynet
Creating service my_cluster_myapp

[[e-mail] ~/compose]#docker stack ps my_cluster 
ID                  NAME                 IMAGE                  NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
1r7usd9qq6er        my_cluster_myapp.1   ikubernetes/myapp:v1   server2             Running             Running 2 minutes ago                       
yvqvmpph4he7        my_cluster_myapp.2   ikubernetes/myapp:v1   server3             Running             Running 2 minutes ago                       
2utj8bfphga0        my_cluster_myapp.3   ikubernetes/myapp:v1   server1             Running             Running 2 minutes ago 

挂载卷

[[e-mail] ~/compose]#cat docker-compose.yml 
version: "3.7"
services:

  myapp:
    image: ikubernetes/myapp:v1
    networks:
      - mynet
    ports:
      - "80:80"
    volumes:
      - webdata:/usr/share/nginx/html
    deploy:
      replicas: 3
      update_config:
        parallelism: 2
        delay: 5s
      restart_policy:
        condition: on-failure

networks:
  mynet:

volumes: 
  webdata:

[[e-mail] ~/compose]#docker stack deploy -c docker-compose.yml my_cluster

查看数据卷是否生效

[[e-mail] ~/compose]#docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED              STATUS                 PORTS               NAMES
2e724ff415aa        ikubernetes/myapp:v1              "nginx -g 'daemon of…"   About a minute ago   Up About a minute      80/tcp              my_cluster_myapp.2.cethe55513uuvin7irlkqznwn

[[e-mail] ~/compose]#docker inspect 2e724ff415aa
"Mounts": [
            {
                "Type": "volume",
                "Name": "my_cluster_webdata",
                "Source": "/var/lib/docker/volumes/my_cluster_webdata/_data",
                "Destination": "/usr/share/nginx/html",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            }
        ],

资源限制

[[e-mail] ~/compose]#cat docker-compose.yml 
version: "3.7"
services:

  myapp:
    image: ikubernetes/myapp:v1
    networks:
      - mynet
    ports:
      - "80:80"
    volumes:
      - webdata:/usr/share/nginx/html
    deploy:
      replicas: 3
      update_config:
        parallelism: 2
        delay: 5s
      restart_policy:
        condition: on-failure
      resources:
        limits:
          cpus: '0.50'
          memory: 50M
        reservations:
          cpus: '0.25'
          memory: 20M

networks:
  mynet:

volumes: 
  webdata:

[[e-mail] ~/compose]#docker stack deploy -c docker-compose.yml my_cluster

启动仓库

[[e-mail] ~/harbor]#docker stack rm my_cluster 
Removing service my_cluster_myapp
Removing network my_cluster_mynet
[[e-mail] ~/harbor]#docker-compose up -d
[[e-mail] ~/harbor]#docker-compose ps
      Name                     Command                  State                                      Ports                               
---------------------------------------------------------------------------------------------------------------------------------------
chartmuseum         ./docker-entrypoint.sh           Up (healthy)   9999/tcp                                                           
clair               ./docker-entrypoint.sh           Up (healthy)   6060/tcp, 6061/tcp                                                 
clair-adapter       /clair-adapter/clair-adapter     Up (healthy)   8080/tcp                                                           
harbor-core         /harbor/harbor_core              Up (healthy)                                                                      
harbor-db           /docker-entrypoint.sh            Up (healthy)   5432/tcp                                                           
harbor-jobservice   /harbor/harbor_jobservice  ...   Up (healthy)                                                                      
harbor-log          /bin/sh -c /usr/local/bin/ ...   Up (healthy)   127.0.0.1:1514->10514/tcp                                          
harbor-portal       nginx -g daemon off;             Up (healthy)   8080/tcp                                                           
nginx               nginx -g daemon off;             Up (healthy)   0.0.0.0:4443->4443/tcp, 0.0.0.0:80->8080/tcp, 0.0.0.0:443->8443/tcp
notary-server       /bin/sh -c migrate-patch - ...   Up                                                                                
notary-signer       /bin/sh -c migrate-patch - ...   Up                                                                                
redis               redis-server /etc/redis.conf     Up (healthy)   6379/tcp                                                           
registry            /home/harbor/entrypoint.sh       Up (healthy)   5000/tcp                                                           
registryctl         /home/harbor/start.sh            Up (healthy)

将myapp：v1和myapp：v2和监控上传到仓库
server1:

docker push reg.westos.org/library/myapp:v1
docker push reg.westos.org/library/myapp:v2
docker push reg.westos.org/library/visualizer
docker push reg.westos.org/library/haproxy:latest

server2：

docker rmi d4a5e0eaa84f  # 删除myapp1
docker rmi 54202d3f0f35  # 删除myapp2

server3：

docker rmi d4a5e0eaa84f  # 删除myapp1
docker rmi 54202d3f0f35  # 删除myapp2

修改server1\server2\server3的私有仓库

[[e-mail] docker]# cat daemon.json 
{
  "registry-mirrors": ["https://reg.westos.org"]
}
[[e-mail] /etc/docker]#systemctl daemon-reload 
[[e-mail] /etc/docker]#systemctl restart docker
[[e-mail] docker]# cat daemon.json 
{
  "registry-mirrors": ["https://reg.westos.org"]
}
[[e-mail] docker]# systemctl daemon-reload 
[[e-mail] docker]# systemctl restart docker

[[e-mail] docker]# vim daemon.json 
[[e-mail] docker]# systemctl daemon-reload 
[[e-mail] docker]# systemctl restart docker

同步key

[[e-mail] /etc/docker]#ls
certs.d  daemon.json  key.json
[[e-mail] /etc/docker]#scp -r certs.d/ server3:/etc/docker/
ca.crt  

添加解析

[[e-mail] docker]# cat /etc/hosts
172.25.60.1 server1 reg.westos.org

[[e-mail] ~/compose]#cat docker-compose.yml 
version: "3.7"
services:

  myapp:
    image: myapp:v1
    networks:
      - mynet
    ports:
      - "8000:80"
    deploy:
      replicas: 3
      update_config:
        parallelism: 2
        delay: 5s
      restart_policy:
        condition: on-failure
      resources:
        limits:
          cpus: '0.50'
          memory: 50M
        reservations:
          cpus: '0.25'
          memory: 20M

networks:
  mynet:

[[e-mail] ~/compose]#docker stack deploy -c docker-compose.yml my_cluster
Creating network my_cluster_mynet
Creating service my_cluster_myapp
[[e-mail] ~/compose]#docker stack ls
NAME                SERVICES            ORCHESTRATOR
my_cluster          1                   Swarm
[[e-mail] ~/compose]#docker stack ps my_cluster 
ID                  NAME                 IMAGE               NODE                DESIRED STATE       CURRENT STATE            ERROR               PORTS
aerh36m9tg6v        my_cluster_myapp.1   myapp:v1            server1             Running             Running 12 seconds ago                       
byrsvscevafz        my_cluster_myapp.2   myapp:v1            server2             Running             Running 9 seconds ago                        
7uw99017uaoh        my_cluster_myapp.3   myapp:v1            server3             Running             Running 14 seconds ago 

部署portainer监控，可是化界面
下载portainer-agent.tar和portainer.tar的安装包，加载到镜像中

[[e-mail] ~]#docker load -i portainer.tar
[[e-mail] ~]#docker load -i portainer-agent.tar

新建portainer仓库

[[e-mail] ~/portainer]#docker images|grep portainer
portainer/portainer                  latest                           19d07168491a        13 months ago       74.1MB
reg.westos.org/portainer/portainer   latest                           19d07168491a        13 months ago       74.1MB
portainer/agent                      latest                           9335796fedf9        15 months ago       12.4MB
reg.westos.org/portainer/agent       latest                           9335796fedf9        15 months ago       12.4MB

上传到portainer仓库

[[e-mail] ~/portainer]#docker push reg.westos.org/portainer/portainer:latest 
[[e-mail] ~/portainer]#docker push reg.westos.org/portainer/agent:latest 

[[e-mail] ~/portainer]#docker stack deploy -c portainer-agent-stack.yml portainer
Creating network portainer_agent_network
Creating service portainer_agent
Creating service portainer_portainer

[[e-mail] ~/portainer]#docker stack ls
NAME                SERVICES            ORCHESTRATOR
my_cluster          1                   Swarm
portainer           2                   Swarm
[[e-mail] ~/portainer]#docker stack rm my_cluster
Removing service my_cluster_myapp
Removing network my_cluster_mynet