PHP Swoole长连接常见问题

连接失效问题
例子
其中，redis常见的报错就是：

配置项：timeout
报错信息：error while reading line from the server
redis可以配置如果客户端经过多少秒还不给redis服务器发送数据，那么就会把连接close掉。

mysql常见的报错：

配置项：wait_timeout & interactive_timeout
报错信息：has gone away
和redis服务器一样，mysql也会定时的去清理掉没用的连接。

如何解决
1、用的时候进行重连

2、定时发送心跳维持连接

用的时候进行重连
优点是简单，缺点是面临短连接的问题。

定时发送心跳维持连接
推荐。

如何维持长连接

tcp协议中实现的tcp_keepalive

 

操作系统底层提供了一组tcp的keepalive配置：

 1 tcp_keepalive_time (integer; default: 7200; since linux 2.2)
 2 the number of seconds a connection needs to be idle before tcp
 3 begins sending out keep-alive probes. keep-alives are sent only
 4 when the so_keepalive socket option is enabled. the default
 5 value is 7200 seconds (2 hours). an idle connection is
 6 terminated after approximately an additional 11 minutes (9
 7 probes an interval of 75 seconds apart) when keep-alive is
 8 enabled.
 9  
10 note that underlying connection tracking mechanisms and
11 application timeouts may be much shorter.
12  
13 tcp_keepalive_intvl (integer; default: 75; since linux 2.4)
14 the number of seconds between tcp keep-alive probes.
15  
16 tcp_keepalive_probes (integer; default: 9; since linux 2.2)
17 the maximum number of tcp keep-alive probes to send before
18 giving up and killing the connection if no response is obtained
19 from the other end.
20 8

swoole底层把这些配置开放出来了，例如：

 1 ?php
 2  
 3 $server = new \swoole\server('127.0.0.1', 6666, swoole_process);
 4  
 5 $server->set([
 6 'worker_num' => 1,
 7 'open_tcp_keepalive' => 1,
 8 'tcp_keepidle' => 4, // 对应tcp_keepalive_time
 9 'tcp_keepinterval' => 1, // 对应tcp_keepalive_intvl
10 'tcp_keepcount' => 5, // 对应tcp_keepalive_probes
11 ]);

其中：

1 'open_tcp_keepalive' => 1, // 总开关，用来开启tcp_keepalive
2 'tcp_keepidle' => 4, // 4s没有数据传输就进行检测
3 // 检测的策略如下：
4 'tcp_keepinterval' => 1, // 1s探测一次，即每隔1s给客户端发一个包（然后客户端可能会回一个ack的包，如果服务端收到了这个ack包，那么说明这个连接是活着的）
5 'tcp_keepcount' => 5, // 探测的次数，超过5次后客户端还没有回ack包，那么close此连接

 

我们来实战测试体验一下，服务端脚本如下：

 1 <?php
 2  
 3 $server = new \swoole\server('127.0.0.1', 6666, swoole_process);
 4  
 5 $server->set([
 6 'worker_num' => 1,
 7 'open_tcp_keepalive' => 1, // 开启tcp_keepalive
 8 'tcp_keepidle' => 4, // 4s没有数据传输就进行检测
 9 'tcp_keepinterval' => 1, // 1s探测一次
10 'tcp_keepcount' => 5, // 探测的次数，超过5次后还没有回包close此连接
11 ]);
12  
13 $server->on('connect', function ($server, $fd) {
14 var_dump("client: connect $fd");
15 });
16  
17 $server->on('receive', function ($server, $fd, $reactor_id, $data) {
18 var_dump($data);
19 });
20  
21 $server->on('close', function ($server, $fd) {
22 var_dump("close fd $fd");
23 });
24  
25 $server->start();

我们启动这个服务器：

1 ~/codedir/phpcode/hyperf-skeleton # php server.php

然后通过tcpdump进行抓包：

~/codedir/phpcode/hyperf-skeleton # tcpdump -i lo port 6666
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type en10mb (ethernet), capture size 262144 bytes

我们此时正在监听lo上的6666端口的数据包。

然后我们用客户端去连接它：

1 ~/codedir/phpcode/hyperf-skeleton # nc 127.0.0.1 6666

此时服务端会打印出消息：

~/codedir/phpcode/hyperf-skeleton # php server.php
string(17) "client: connect 1"

tcpdump的输出信息如下：

1 01:48:40.178439 ip localhost.33933 > localhost.6666: flags [s], seq 43162537, win 43690, options [mss 65495,sackok,ts val 9833698 ecr 0,nop,wscale 7], length 0
2 01:48:40.178484 ip localhost.6666 > localhost.33933: flags [s.], seq 1327460565, ack 43162538, win 43690, options [mss 65495,sackok,ts val 9833698 ecr 9833698,nop,wscale 7], length 0
3 01:48:40.178519 ip localhost.33933 > localhost.6666: flags [.], ack 1, win 342, options [nop,nop,ts val 9833698 ecr 9833698], length 0
4 01:48:44.229926 ip localhost.6666 > localhost.33933: flags [.], ack 1, win 342, options [nop,nop,ts val 9834104 ecr 9833698], length 0
5 01:48:44.229951 ip localhost.33933 > localhost.6666: flags [.], ack 1, win 342, options [nop,nop,ts val 9834104 ecr 9833698], length 0
6 01:48:44.229926 ip localhost.6666 > localhost.33933: flags [.], ack 1, win 342, options [nop,nop,ts val 9834104 ecr 9833698], length 0
7 01:48:44.229951 ip localhost.33933 > localhost.6666: flags [.], ack 1, win 342, options [nop,nop,ts val 9834104 ecr 9833698], length 0
8 01:48:44.229926 ip localhost.6666 > localhost.33933: flags [.], ack 1, win 342, options [nop,nop,ts val 9834104 ecr 9833698], length 0
9 // 省略了其他的输出

我们会发现最开始的时候，会打印三次握手的包：

01:48:40.178439 ip localhost.33933 > localhost.6666: flags [s], seq 43162537, win 43690, options [mss 65495,sackok,ts val 9833698 ecr 0,nop,wscale 7], length 0
01:48:40.178484 ip localhost.6666 > localhost.33933: flags [s.], seq 1327460565, ack 43162538, win 43690, options [mss 65495,sackok,ts val 9833698 ecr 9833698,nop,wscale 7], length 0
01:48:40.178519 ip localhost.33933 > localhost.6666: flags [.], ack 1, win 342, options [nop,nop,ts val 9833698 ecr 9833698], length 0

然后，停留了4s没有任何包的输出。

之后，每隔1s左右就会打印出一组：

1 01:52:54.359341 ip localhost.6666 > localhost.43101: flags [.], ack 1, win 342, options [nop,nop,ts val 9859144 ecr 9858736], length 0
2 01:52:54.359377 ip localhost.43101 > localhost.6666: flags [.], ack 1, win 342, options [nop,nop,ts val 9859144 ecr 9855887], length 0

其实这就是我们配置的策略：

1 'tcp_keepinterval' => 1, // 1s探测一次
2 'tcp_keepcount' => 5, // 探测的次数，超过5次后还没有回包close此连接

因为我们操作系统底层会自动的给客户端回ack，所以这个连接不会在5次探测后被关闭。操作系统底层会持续不断的发送这样的一组包：

1 01:52:54.359341 ip localhost.6666 > localhost.43101: flags [.], ack 1, win 342, options [nop,nop,ts val 9859144 ecr 9858736], length 0
2 01:52:54.359377 ip localhost.43101 > localhost.6666: flags [.], ack 1, win 342, options [nop,nop,ts val 9859144 ecr 9855887], length 0

如果我们要测试5次探测后关闭这个连接，可以禁掉6666端口的包：

1 ~/codedir/phpcode/hyperf-skeleton # iptables -a input -p tcp --dport 6666 -j drop

这样会把所有从6666端口进来的包给禁掉，自然，服务器就接收不到从客户端那一边发来的ack包了。

然后服务器过5秒就会打印出close（服务端主动的调用了close方法，给客户端发送了fin包）：

1 ~/codedir/phpcode/hyperf-skeleton # php server.php
2 string(17) "client: connect 1"
3 string(10) "close fd 1"

我们恢复一下iptables的规则：

1 ~/codedir/phpcode # iptables -d input -p tcp -m tcp --dport 6666 -j drop

即把我们设置的规则给删除了。

通过tcp_keepalive的方式实现心跳的功能，优点是简单，不要写代码就可以完成这个功能，并且发送的心跳包小。缺点是依赖于系统的网络环境，必须保证服务器和客户端都实现了这样的功能，需要客户端配合发心跳包。还有一个更为严重的缺点是如果客户端和服务器不是直连的，而是通过代理来进行连接的，例如socks5代理，它只会转发应用层的包，不会转发更为底层的tcp探测包，那这个心跳功能就失效了。

所以，swoole就提供了其他的解决方案，一组检测死连接的配置。

1 'heartbeat_check_interval' => 1, // 1s探测一次
2 'heartbeat_idle_time' => 5, // 5s未发送数据包就close此连接

swoole实现的heartbeat

我们来测试一下：

 1 <?php
 2  
 3 $server = new \swoole\server('127.0.0.1', 6666, swoole_process);
 4  
 5 $server->set([
 6 'worker_num' => 1,
 7 'heartbeat_check_interval' => 1, // 1s探测一次
 8 'heartbeat_idle_time' => 5, // 5s未发送数据包就close此连接
 9 ]);
10  
11 $server->on('connect', function ($server, $fd) {
12 var_dump("client: connect $fd");
13 });
14  
15 $server->on('receive', function ($server, $fd, $reactor_id, $data) {
16 var_dump($data);
17 });
18  
19 $server->on('close', function ($server, $fd) {
20 var_dump("close fd $fd");
21 });
22  
23 $server->start();

然后启动服务器：

1 ~/codedir/phpcode/hyperf-skeleton # php server.php

然后启动tcpdump：

 

1 ~/codedir/phpcode # tcpdump -i lo port 6666
2 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
3 listening on lo, link-type en10mb (ethernet), capture size 262144 bytes

 

然后再启动客户端：

1 ~/codedir/phpcode/hyperf-skeleton # nc 127.0.0.1 6666

 

此时服务器端打印：

1 ~/codedir/phpcode/hyperf-skeleton # php server.php
2 string(17) "client: connect 1"

 

然后tcpdump打印：

 

1 02:48:32.516093 ip localhost.42123 > localhost.6666: flags [s], seq 1088388248, win 43690, options [mss 65495,sackok,ts val 10193342 ecr 0,nop,wscale 7], length 0
2 02:48:32.516133 ip localhost.6666 > localhost.42123: flags [s.], seq 80508236, ack 1088388249, win 43690, options [mss 65495,sackok,ts val 10193342 ecr 10193342,nop,wscale 7], length 0
3 02:48:32.516156 ip localhost.42123 > localhost.6666: flags [.], ack 1, win 342, options [nop,nop,ts val 10193342 ecr 10193342], length 0

 

这是三次握手信息。

然后过了5s后，tcpdump会打印出：

1 02:48:36.985027 ip localhost.6666 > localhost.42123: flags [f.], seq 1, ack 1, win 342, options [nop,nop,ts val 10193789 ecr 10193342], length 0
2 02:48:36.992172 ip localhost.42123 > localhost.6666: flags [.], ack 2, win 342, options [nop,nop,ts val 10193790 ecr 10193789], length 0

也就是服务端发送了fin包。因为客户端没有发送数据，所以swoole关闭了连接。

然后服务器端会打印：

1 ~/codedir/phpcode/hyperf-skeleton # php server.php
2 string(17) "client: connect 1"
3 string(10) "close fd 1"

 

所以，heartbeat和tcp keepalive还是有一定的区别的，tcp keepalive有保活连接的功能，但是heartbeat存粹是检测没有数据的连接，然后关闭它，并且只可以在服务端这边配置，如果需要保活，也可以让客户端配合发送心跳。

如果我们不想让服务端close掉连接，那么就得在应用层里面不断的发送数据包来进行保活，例如我在nc客户端里面不断的发送包：

 1 ~/codedir/phpcode/hyperf-skeleton # nc 127.0.0.1 6666
 2 ping
 3 ping
 4 ping
 5 ping
 6 ping
 7 ping
 8 ping
 9 ping
10 ping

 

我发送了9个ping包给服务器，tcpdump的输出如下：

 1 // 省略了三次握手的包
 2 02:57:53.697363 ip localhost.44195 > localhost.6666: flags [p.], seq 1:6, ack 1, win 342, options [nop,nop,ts val 10249525 ecr 10249307], length 5
 3 02:57:53.697390 ip localhost.6666 > localhost.44195: flags [.], ack 6, win 342, options [nop,nop,ts val 10249525 ecr 10249525], length 0
 4 02:57:55.309532 ip localhost.44195 > localhost.6666: flags [p.], seq 6:11, ack 1, win 342, options [nop,nop,ts val 10249686 ecr 10249525], length 5
 5 02:57:55.309576 ip localhost.6666 > localhost.44195: flags [.], ack 11, win 342, options [nop,nop,ts val 10249686 ecr 10249686], length 0
 6 02:57:58.395206 ip localhost.44195 > localhost.6666: flags [p.], seq 11:16, ack 1, win 342, options [nop,nop,ts val 10249994 ecr 10249686], length 5
 7 02:57:58.395239 ip localhost.6666 > localhost.44195: flags [.], ack 16, win 342, options [nop,nop,ts val 10249994 ecr 10249994], length 0
 8 02:58:01.858094 ip localhost.44195 > localhost.6666: flags [p.], seq 16:21, ack 1, win 342, options [nop,nop,ts val 10250341 ecr 10249994], length 5
 9 02:58:01.858126 ip localhost.6666 > localhost.44195: flags [.], ack 21, win 342, options [nop,nop,ts val 10250341 ecr 10250341], length 0
10 02:58:04.132584 ip localhost.44195 > localhost.6666: flags [p.], seq 21:26, ack 1, win 342, options [nop,nop,ts val 10250568 ecr 10250341], length 5
11 02:58:04.132609 ip localhost.6666 > localhost.44195: flags [.], ack 26, win 342, options [nop,nop,ts val 10250568 ecr 10250568], length 0
12 02:58:05.895704 ip localhost.44195 > localhost.6666: flags [p.], seq 26:31, ack 1, win 342, options [nop,nop,ts val 10250744 ecr 10250568], length 5
13 02:58:05.895728 ip localhost.6666 > localhost.44195: flags [.], ack 31, win 342, options [nop,nop,ts val 10250744 ecr 10250744], length 0
14 02:58:07.150265 ip localhost.44195 > localhost.6666: flags [p.], seq 31:36, ack 1, win 342, options [nop,nop,ts val 10250870 ecr 10250744], length 5
15 02:58:07.150288 ip localhost.6666 > localhost.44195: flags [.], ack 36, win 342, options [nop,nop,ts val 10250870 ecr 10250870], length 0
16 02:58:08.349124 ip localhost.44195 > localhost.6666: flags [p.], seq 36:41, ack 1, win 342, options [nop,nop,ts val 10250990 ecr 10250870], length 5
17 02:58:08.349156 ip localhost.6666 > localhost.44195: flags [.], ack 41, win 342, options [nop,nop,ts val 10250990 ecr 10250990], length 0
18 02:58:09.906223 ip localhost.44195 > localhost.6666: flags [p.], seq 41:46, ack 1, win 342, options [nop,nop,ts val 10251145 ecr 10250990], length 5
19 02:58:09.906247 ip localhost.6666 > localhost.44195: flags [.], ack 46, win 342, options [nop,nop,ts val 10251145 ecr 10251145], length 0

 

有9组数据包的发送。（这里的flags [p.]代表push的含义）

此时服务器还没有close掉连接，实现了客户端保活连接的功能。然后我们停止发送ping，过了5秒后tcpdump就会输出一组：

02:58:14.811761 ip localhost.6666 > localhost.44195: flags [f.], seq 1, ack 46, win 342, options [nop,nop,ts val 10251636 ecr 10251145], length 0
02:58:14.816420 ip localhost.44195 > localhost.6666: flags [.], ack 2, win 342, options [nop,nop,ts val 10251637 ecr 10251636], length 0
服务端那边发送了fin包，说明服务端close掉了连接。服务端的输出如下：

 1 ~/codedir/phpcode/hyperf-skeleton # php server.php
 2 string(17) "client: connect 1"
 3 string(5) "ping
 4 "
 5 string(5) "ping
 6 "
 7 string(5) "ping
 8 "
 9 string(5) "ping
10 "
11 string(5) "ping
12 "
13 string(5) "ping
14 "
15 string(5) "ping
16 "
17 string(5) "ping
18 "
19 string(5) "ping
20 "
21 string(10) "close fd 1"

 

然后我们在客户端那边ctrl + c来关闭连接：

 1 ~/codedir/phpcode/hyperf-skeleton # nc 127.0.0.1 6666
 2 ping
 3 ping
 4 ping
 5 ping
 6 ping
 7 ping
 8 ping
 9 ping
10 ping
11 ^cpunt!
12 
13 ~/codedir/phpcode/hyperf-skeleton #

 

此时，tcpdump的输出如下：

1 03:03:02.257667 ip localhost.44195 > localhost.6666: flags [f.], seq 46, ack 2, win 342, options [nop,nop,ts val 10280414 ecr 10251636], length 0
2 03:03:02.257734 ip localhost.6666 > localhost.44195: flags [r], seq 2678621620, win 0, length 0

 

应用层心跳

1、制定ping/pong协议（mysql等自带ping协议）
2、客户端灵活的发送ping心跳包
3、服务端onrecive检查可用性回复pong
例如：

 1 $server->on('receive', function (\swoole\server $server, $fd, $reactor_id, $data)
 2 {
 3 if ($data == 'ping')
 4 {
 5 checkdb();
 6 checkservicea();
 7 checkredis();
 8 $server->send('pong');
 9 }
10 });

 

结论
1、tcp的keepalive最简单，但是有兼容性问题，不够灵活
2、swoole提供的keepalive最实用，但是需要客户端配合，复杂度适中
3、应用层的keepalive最灵活但是最麻烦