docker swarm错误: Error response from daemon: rpc error: code = 4 desc = context deadline exceeded

docker swarm: Error response from daemon: rpc error: code = 4 desc = context deadline exceeded

碰到一个问题，记录在此。

错误内容：

所有docker swarm/service命令都不能执行，返回如下错误：
包括 $ docker node ls等swarm相关的命令：

$ docker swarm join-token manager
Error response from daemon: rpc error: code = 4 desc = context deadline exceeded

但是docker命令都正常， 例如docker ps/docker images等等，只是docker swarm的命令无法执行。

原因分析：

查看docker的log
$ sudo tail -f /var/log/messages | grep docker
注意不同的Linux发行版本，docker log位置不一致；上述是Redhat的办法：

dockerd: time="2019-04-18T11:58:16.371148755+08:00" level=debug msg="miss notification: dest IP 10.0.0.4, dest MAC "
dockerd: time="2019-04-18T11:58:16.373118682+08:00" level=debug msg="miss notification: dest IP 10.0.0.8, dest MAC "
dockerd: time="2019-04-18T11:58:16.780678187+08:00" level=error msg="agent: session failed" error="session initiation timed out" module="node/agent" node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:16.780713226+08:00" level=debug msg="agent: rebuild session" module="node/agent" node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:17.371118179+08:00" level=debug msg="miss notification: dest IP 10.0.0.6, dest MAC "
dockerd: time="2019-04-18T11:58:17.373148037+08:00" level=debug msg="miss notification: dest IP 10.0.0.4, dest MAC "
dockerd: time="2019-04-18T11:58:18.371017818+08:00" level=debug msg="miss notification: dest IP 10.0.0.8, dest MAC "
dockerd: time="2019-04-18T11:58:18.373146453+08:00" level=debug msg="miss notification: dest IP 10.0.0.6, dest MAC "
dockerd: time="2019-04-18T11:58:18.643814757+08:00" level=info msg="3a81dc09ea33b88e is starting a new election at term 812" module=raft node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:18.643848592+08:00" level=info msg="3a81dc09ea33b88e became candidate at term 813" module=raft node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:18.643865419+08:00" level=info msg="3a81dc09ea33b88e received MsgVoteResp from 3a81dc09ea33b88e at term 813" module=raft node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:18.643899515+08:00" level=info msg="3a81dc09ea33b88e [logterm: 2, index: 141] sent MsgVote request to 8be3f17aae7108e at term 813" module=raft node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:18.645083884+08:00" level=debug msg="failed to send message MsgVote" error="rpc error: code = 13 desc = connection error: desc = \"transport: x509: certificate has expired or is not yet valid\"" peer_id=8be3f17aae7108e
dockerd: time="2019-04-18T11:58:19.371110322+08:00" level=debug msg="miss notification: dest IP 10.0.0.8, dest MAC "
dockerd: time="2019-04-18T11:58:19.371435727+08:00" level=debug msg="miss notification: dest IP 10.0.0.4, dest MAC "

说明swarm cluster环境通信遇到了问题。

解决办法：

重新初始化cluster环境即可：

$ docker swarm init --force-new-cluster

然后就一切正常了。