Oracle 10g RAC 在Linux下安装经验总结

      在Linux 系统上如red hat as 5 上安装Oracle 10g RAC 时，我们经常会碰到这样或那样的错误。有的是新环境缺少这样那样的配置，有的是老环境有这样那些的不合适设置，有时候还能碰到一些bug 。

(miki西游 @mikixiyou 文档，原文链接: http://mikixiyou.iteye.com/blog/1555489 )

我从2005 年开始安装10g RAC 到现在，遇见了各种各样的问题。现在将这些安装过程中注意的要点和出现的问题做一个总结。

一、安装操作系统补丁包

 

建议使用 system-config-packages命令在vnc下打开图形化安装工具，将开发包全部安装上，免得烦人。

否则，您需要检查和安装下列的补丁包。

rpm -ivh setarch-2.0-1.1.x86_64.rpm

rpm -ivh make-3.81-3.el5.x86_64.rpm

rpm -ivh glibc-2.5-24.*

rpm -ivh libaio-0.3.106-3.2.*

rpm -ivh compat-libstdc++-*

rpm -ivh compat-gcc-34-3.4.6-4.x86_64.rpm

rpm -ivh compat-gcc-34-c++-3.4.6-4.x86_64.rpm

rpm -ivh gcc-4.1.2-42.el5.x86_64.rpm

rpm -ivh libXp-1.0.0-8.1.el5.*

rpm -ivh openmotif-2.3.0-0.5.el5.*

rpm -ivh compat-db-4.2.52-5.1.*

如果没有安装完整，可能会有一面一些错误会遇到。

错误一：

  /tmp/OraInstall2007-12-30_02-16-11PM/jre/1.4.2/lib/i386/libawt.so: libXp.so.6: cannot open shared object file: No such file or directory occurred..

 

原因是：缺少支持打印的图形化动态链接库libXp.so.6

这个错误是由于缺少系统安装包，在RHEL5 以前的版本，可以安装xorg-x11-deprecated-libs 包即可，这个安装包可以在系统光盘第三张盘中找到（对于redhat as4.2 以前，redhat as4.4 是在第四张安装光盘）

xorg-x11-deprecated-libs-6.8.2-31.i386.rpm

 

错误二：

Exception :java.lang.UnsatisfiedLinkError:/usr/java/j2sdk1.4.2_06/jre/lib/i386/libawt.so: libXt.so.6: cannot open shared object file: No such file or directory

这个错误是也是由于缺少系统安装包造成的，只需安装XFree86-libs 包即可，这个安装包可以在系统光盘中找到。

  在上面两个错误如果出现在redhat as5 版本中，解决如下：

在RHEL5 中上面的包被libXp 所取代了，因此rpm -ivh libXp-1.0.0-8.i386.rpm 将包打上后问题即可解决。

  错误三：

error while loading shared libraries: libstdc++-libc6.1-1.so.2

只用执行如下操作ln -s libstdc++-libc6.2-2.so.3 libstdc++-libc6.1-1.so.2

最后安装“rpm -ivh libXp-1.0.0-8.i386.rpm ”问题解决。

  错误四：在安装数据库的时候出现

“uError in invoking target 'all_no_orcl ihsodbc' of  makefile'/u01/app/oracle/oracle/product/10.2.0/db_1/rdbms/lib/ins_rdbms.mk'.See '/u01/app/oracle/oraInventory/logs/installActions2006-10-16_06-41-37PM.log'for details ”的错误。

经过分析缺少一个包：compat-libstdc++-33.i386.rpm 的包

  [root@dg1 Server]# rpm -ivh compat-libstdc++-*.rpm

warning: compat-libstdc++-296-2.96-138.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186

Preparing...                ########################################### [100%]

   1:compat-libstdc++-33    ########################################### [ 50%]

   2:compat-libstdc++-296   ########################################### [100%]

 

安装后，问题解决。

 

二、操作系统配置

1 、/etc/hosts 配置

 

新集群配置：

192.168.100.14       rac1

192.168.100.15       rac1-vip

192.168.212.14       rac1-priv

 

192.168.100.18       rac2

192.168.100.19       rac2-vip

192.168.212.18       rac2-priv

 

2 、核心参数配置

/etc/sysctl.conf

#////////////////////////////////////////////////oracle 调优后参数

#ORACLE

#kernel.shmall = 3279547

#kernel.shmmax = 8294967295

kernel.shmmni = 4096

#semaphores: semmsl, semmns, semopm, semmni

kernel.sem = 256 32000 100 142

fs.file-max = 327679

net.ipv4.ip_local_port_range = 1024 65000

kernel.msgmni = 2878

kernel.msgmax = 8192

kernel.msgmnb = 65535

net.core.rmem_default=262144

net.core.rmem_max=262144

net.core.wmem_default=262144

net.core.wmem_max=262144

 

vm.min_free_kbytes=409600

 

3 、调整限制

  /etc/security/limits.conf

oracle    soft   nproc      4096

oracle    hard   nproc      16384

oracle    soft   nofile     16384

oracle    hard   nofile     65536

 

4 、

修改 /etc/selinux/config 配置文件，把 SELLINUX 参数改为：

SELINUX=disabled

 

三、创建oracle 用户和组

/usr/sbin/groupadd oinstall     -g    201 

/usr/sbin/groupadd dba          -g      202

/usr/sbin/groupadd oper         -g      203

/usr/sbin/useradd -m -u 1101 -g oinstall -G dba,oper -d /u01/home/oracle -s /bin/bash -c "Oracle Software Owner" oracle

 

设置.bash_profile

# .bash_profile

####

export ORACLE_BASE=/u01/app/oracle

export ORACLE_HOME=/u01/app/oracle/product/10.2.0/db

export ORA_CRS_HOME=/u01/app/oracle/product/10.2.0/crs

export PATH=/sbin:$ORA_CRS_HOME/bin:$ORACLE_HOME/bin:$ORACLE_HOME/Apache/Apache/bin:$PATH

export ORACLE_OWNER=oracle

export ORACLE_SID=xxdb2

#export ORACLE_TERM=vt100

export THREADS_FLAG=native

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib

export LD_LIBRARY_PATH

export PATH=$PATH:$ORACLE_HOME/bin

export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK

export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data

CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib

export CLASSPATH

 

export LANG=en_US

PS1="\${PWD}@`hostname`=>\${ORACLE_SID}\$"

umask 022

 

 

mkdir -p /u01/app/oracle

chown -R oracle:oinstall /u01/app/oracle

chmod -R 775 /u01/app/oracle

 

四、配置SSH 互信

在rac3 上以oracle user run:

rac3-> mkdir ~/.ssh

rac3-> chmod 700 ~/.ssh

rac3-> ssh-keygen -t rsa

rac3-> ssh-keygen -t dsa

 

在rac1 上以oracle user run:

rac1-> ssh rac3 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

rac1-> ssh rac3 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

oracle@rac3'spassword:

rac1-> scp ~/.ssh/authorized_keys rac3:~/.ssh/authorized_keys

oracle@rac3'spassword:

authorized_keys                           100% 1716     1.7KB/s   00:00

rac1-> scp ~/.ssh/authorized_keys rac3:~/.ssh/authorized_keys

authorized_keys                            100% 1716     1.7KB/s   00:00

 

将authorized_keys copy 到rac2 主机, 建立与rac2 的ssh 信任

rac2-> scp ~/.ssh/authorized_keys rac2:~/.ssh/authorized_keys

authorized_keys                           100% 1716     1.7KB/s   00:00

 

 

四、存储设置

The /dev/dm-N devices are used internally by device-mapper-multipath and are non-persistent across reboot, so should not be used. The /dev/mpath/ devices are created for multipath devices to be visible together, however, may not be available during early stages of boot, so, again, should not be used. However, /dev/mapper/ devices are persistent and created sufficiently early during boot - use only these devices to access and interact with multipathed devices.

在redhat 5.4 存储设置:

[root@rac2 ~]# more /etc/udev/rules.d/60-raw.rules

# Enter raw device bindings here.

ACTION=="add", KERNEL=="/dev/mapper/mpath6", RUN+="/bin/raw /dev/raw/raw1 %N"

ACTION=="add", KERNEL=="/dev/mapper/mpath7", RUN+="/bin/raw /dev/raw/raw2 %N"

 

ACTION=="add", KERNEL=="raw1", OWNER="root", GROUP="oinstall", MODE="777"

ACTION=="add", KERNEL=="raw2", OWNER="oracle", GROUP="oinstall", MODE="777"

 

[root@rac2 ~]# more /etc/rc.d/rc.local

#!/bin/sh

#

# This script will be executed *after* all the other init scripts.

# You can put your own initialization stuff in here if you don't

# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

 

/bin/raw /dev/raw/raw1 /dev/mapper/mpath6

/bin/raw /dev/raw/raw2 /dev/mapper/mpath7

[root@rac2 ~]#

 

 

 

[root@rac1 ~]# chown -R oracle:oinstall /dev/mapper/mpath[1-5]

[root@rac1 ~]# chmod -R 755 /dev/mapper/mpath[1-5]

[root@rac1 ~]#

 

 

 

//////

 

ACTION=="add", KERNEL=="/dev/sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"

ACTION=="add", KERNEL=="/dev/sdb2", RUN+="/bin/raw /dev/raw/raw2 %N"

ACTION=="add", KERNEL=="/dev/sdb3", RUN+="/bin/raw /dev/raw/raw3 %N"

ACTION=="add", KERNEL=="/dev/sdb4", RUN+="/bin/raw /dev/raw/raw4 %N"

KERNEL=="raw1", OWNER="root",   GROUP="oinstall", MODE="640"

KERNEL=="raw2", OWNER="oracle", GROUP="oinstall", MODE="640"

KERNEL=="raw3", OWNER="oracle", GROUP="dba", MODE="660"

KERNEL=="raw4", OWNER="oracle", GROUP="dba", MODE="660"

 

/etc/udev/rule.d/60-raw.rules

这种方法的配置如下：

  [root@rac2 rules.d]# cat 60-raw.rules

# Enter raw device bindings here.

#

# An example would be:

#   ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"

# to bind /dev/raw/raw1 to /dev/sda, or

#   ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"

# to bind /dev/raw/raw2 to the device with major 8, minor 1.

ACTION=="add", KERNEL=="dm-7", RUN+="/bin/raw /dev/raw/raw1 %N"

ACTION=="add", KERNEL=="dm-8", RUN+="/bin/raw /dev/raw/raw2 %N"

 

ACTION=="add", KERNEL=="raw1", OWNER="root", GROUP="oinstall", MODE="660"

ACTION=="add", KERNEL=="raw2", OWNER="oracle", GROUP="oinstall", MODE="660"

  -- 这个是对raw 设备进行权限设置的

  [root@rac2 rules.d]# cat /etc/rc.d/rc.local

#!/bin/sh

#

# This script will be executed *after* all the other init scripts.

# You can put your own initialization stuff in here if you don't

# want to do the full Sys V style init stuff.

 

touch /var/lock/subsys/local

 

/bin/raw /dev/raw/raw1 /dev/dm-7

/bin/raw /dev/raw/raw2 /dev/dm-8

 

-- 系统启动时挂载raw 设备

 

五、安装CRS

/runInstaller -ignoreSysPrereqs &

 

 

执行root.sh 失败，报错如下：

Failed to upgrade Oracle Cluster Registry configuration

原因是未删除干净，原来已安装有 Oracle RAC 软件，

如果你在装crs 的时候运行了root.sh 后提示上面的英文错误，再也无法运行了，可以尝试清理crs 残留信息，删除执行crs 目录结构后的最重要一步，执行下面的命令。

dd if=/dev/zero of=/dev/sdb1 bs=8192 count=2000

dd if=/dev/zero of=/dev/sdc1 bs=8192 count=2000

将/dev/raw/raw1-->/dev/sdb1 的设备清空 orc disk

将/dev/raw/raw2-->/dev/sdc1 的设备清空 vote disk

 

Oracle 10g RAC 在Linux 上手工清理的方法如下：

      rm -f /etc/init.d/init.cssd

      rm -f /etc/init.d/init.crs

      rm -f /etc/init.d/init.crsd

      rm -f /etc/init.d/init.evmd

      rm -f /etc/rc2.d/K96init.crs

      rm -f /etc/rc2.d/S96init.crs

      rm -f /etc/rc3.d/K96init.crs

      rm -f /etc/rc3.d/S96init.crs

      rm -f /etc/rc5.d/K96init.crs

      rm -f /etc/rc5.d/S96init.crs

      rm -Rf /etc/oracle/scls_scr

      rm -f /etc/inittab.crs

      cp /etc/inittab.orig /etc/inittab

 

dd if=/dev/zero of=/dev/raw/raw1 bs=8192K count=100

dd if=/dev/zero of=/dev/raw/raw2 bs=8192K count=100

 

 

如果root.sh 报错如下：

Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 Oracle.  All rights reserved.

2010-12-28 00:35:36.935: [ OCRCONF][1325856320]ocrconfig starts...

2010-12-28 00:35:36.935: [ OCRCONF][1325856320]Upgrading OCR data

2010-12-28 00:35:36.941: [ OCRCONF][1325856320]OCR already in current version.

2010-12-28 00:35:36.947: [ OCRCONF][1325856320]Failed to call clsssinit (21)

2010-12-28 00:35:36.947: [ OCRCONF][1325856320]Failed to make a backup copy of OCR

2010-12-28 00:35:36.947: [ OCRCONF][1325856320]Exiting [status=failed]...

[root@rac1 lost+found]#

这是Bug.4679769 FAILED TO FORMAT OCR DISK USING CLSFMT

 

综上，在oracle rac中使用raw device还是asmlib看情况而定，我一直使用raw device。在重装时，务必清理干净。bash_profile的配置大家可以参考。

这是10gRAC的，我还有一个关于11g RAC的安装总结，等有空整理出来，再放上来。希望得到您的支持。（虚荣心啊，人性的弱点吗？）