11gR2 RAC启用iptables导致节点宕机问题处理

程序员文章站 2022-06-11 12:47:14

...

在安装数据库时，绝大多数都是要求把selinux及iptables关闭，然后再进行安装的。但是在运营商的系统中，很多安全的因素，需要将现

通常，在安装数据库时，绝大多数都是要求把selinux及iptables关闭，然后再进行安装的。但是在运营商的系统中，很多安全的因素，需要将现网的数据库主机上的iptables开启的。

在开启iptables时就要注意了，比如一RAC中的hosts配置如下：
192.168.142.115 subsdb1
192.168.142.117 subsdb1-vip
10.0.0.115 subsdb1-priv
192.168.142.116 subsdb2
192.168.142.118 subsdb2-vip
10.0.0.116 subsdb2-priv
192.168.142.32 db-scan

那么理所当然的要将上面的IP都要放通的。但是在实际操作中，，已经放通了上面的IP，结果数据库一的个实例宕掉了。

看看数据库的alert日志：

Tue Aug 20 00:29:40 2013
IPC Send timeout detected. Sender: ospid 8284 [Oracle@subsdb2 (LMD0)]
Receiver: inst 1 binc 1740332689 ospid 15851
IPC Send timeout to 1.0 inc 10 for msg type 65521 from opid 12
Tue Aug 20 00:29:48 2013
IPC Send timeout detected. Sender: ospid 8276 [oracle@subsdb2 (PING)]
Receiver: inst 2 binc 1801834534 ospid 8276
Tue Aug 20 00:29:52 2013
Detected an inconsistent instance membership by instance 2
Errors in file /oracle/app/oracle/diag/rdbms/gdordb/GDORDB2/trace/GDORDB2_lmon_8282.trc (incident=784092):
ORA-29740: evicted by instance number 2, group incarnation 12
Incident details in: /oracle/app/oracle/diag/rdbms/gdordb/GDORDB2/incident/incdir_784092/GDORDB2_lmon_8282_i784092.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /oracle/app/oracle/diag/rdbms/gdordb/GDORDB2/trace/GDORDB2_lmon_8282.trc:
ORA-29740: evicted by instance number 2, group incarnation 12
LMON (ospid: 8282): terminating the instance due to error 29740
Tue Aug 20 00:29:54 2013
ORA-1092 : opitsk aborting process
Tue Aug 20 00:29:54 2013
License high water mark = 29
Tue Aug 20 00:29:57 2013
System state dump requested by (instance=2, osid=8282 (LMON)), summary=[abnormal instance termination].
System State dumped to trace file /oracle/app/oracle/diag/rdbms/gdordb/GDORDB2/trace/GDORDB2_diag_8272.trc
Instance terminated by LMON, pid = 8282
USER (ospid: 31106): terminating the instance
Instance terminated by USER, pid = 31106

单纯从上面来看，初步可以断定是内部通信有问题，但是如何解决？
但再从数据库的alert和ASM实例的alert日志中都有这样的信息：
Private Interface 'bond2:1' configured from GPnP for use as a private interconnect.
[name='bond2:1', type=1, ip=169.254.148.209, mac=00-25-b5-00-00-67, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'bond0' configured from GPnP for use as a public interface.
[name='bond0', type=1, ip=192.168.142.116, mac=00-25-b5-00-01-cb, net=192.168.142.0/24, mask=255.255.255.0, use=public/1]
Picked latch-free SCN scheme 3

从这个信息来看，RAC的内部通信还要用到net=169.254.0.0/16的IP，再从MOS Doc ID 1383737.1也有这样的说明，最后用ifconfig查到了RAC的两个节点中使用到的169网段的IP为：
169.254.122.59
169.254.148.209
在iptables中放通了这两个IP后，集群正常。