ORA-12537 TNS-12518 Process m000 died

最近客户邮件描述无法从客户端连接到数据库，其错误号为ORA-12537: TNS:connection closed，连接被关闭。直接通过tnsping没有任何

最近客户邮件描述无法从客户端连接到数据库，其错误号为ORA-12537: TNS:connection closed，连接被关闭。直接通过tnsping没有任何问题。listener日志中出现TNS-12518: TNS:listener could not hand off client connection。即Listener无法分发客户端连接。下面是具体的错误信息、分析与解决的过程。

1、前端错误信息

C:\Program Files\VMware\VMware vSphere CLI>sqlplus robinson@sywgmr

SQL*Plus: Release 10.2.0.1.0 - Production on Thu Mar 27 10:49:00 2014

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Enter password:
ERROR:
ORA-12537: TNS:connection closed

2、故障分析

oracle@SZDB:/users/oracle/OraHome10g/network/log> tail listener_sywgmr.log
TNS-12547: TNS:lost contact
TNS-12560: TNS:protocol adapter error
TNS-00517: Lost contact
Linux Error: 32: Broken pipe
27-MAR-2014 11:02:38 * (CONNECT_DATA=(SID=SYWGMR)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))) * (ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=63561)) * establish * SYWGMR * 12518
TNS-12518: TNS:listener could not hand off client connection
TNS-12547: TNS:lost contact
TNS-12560: TNS:protocol adapter error
TNS-00517: Lost contact
Linux Error: 32: Broken pipe

oracle@SZDB:~> oerr ora 12537
12537, 00000, "TNS:connection closed"
// *Cause: "End of file" condition has been reached; partner has disconnected.
// *Action: None needed; this is an information message.

oracle@SZDB:~> oerr ora 12518
12518, 00000, "TNS:listener could not hand off client connection"
// *Cause: The process of handing off a client connection to another process
// failed.
// *Action: Turn on listener tracing and re-execute the operation. Verify
// that the listener and database instance are properly configured for
// direct handoff. If problem persists, call Oracle Support.
// *Comment: The problem can be worked around by configuring dispatcher(s)
// to specifically handle the desired presentation(s), and connecting
// directly to the dispatcher, bypassing the listener.
#根据上面的2个error no，没有获得太多的有用信息
#ora-12518建议作一个trace.其comment说明可以通过配置disaptcher来解决，而当前我们的数据库使用的是dedicate方式
#关于如何配置监听器trace，可参考:
#此处不做trace，，笔者尝试reload以及restart 监听，故障依旧

#下面查看alert log 日志
#下面的日志里表明无法创建m000进程，m000进程是SMON进程的奴隶进程
oracle@SZDB:/users/oracle> tail /u02/database/SYWGMR/bdump/alert_SYWGMR.log
Thu Mar 27 11:00:28 2014
ksvcreate: Process(m000) creation failed
Thu Mar 27 11:01:29 2014
Process m000 died, see its trace file
Thu Mar 27 11:01:29 2014
ksvcreate: Process(m000) creation failed
Thu Mar 27 11:02:30 2014
Process m000 died, see its trace file
Thu Mar 27 11:02:30 2014
ksvcreate: Process(m000) creation failed

#在06:19:02 2014时有一个Error，无法衍生job 奴隶进程
Thu Mar 27 06:19:02 2014
Process J000 died, see its trace file
Thu Mar 27 06:19:02 2014
kkjcre1p: unable to spawn jobq slave process
Thu Mar 27 06:19:02 2014
Errors in file /u02/database/SYWGMR/bdump/sywgmr_cjq0_7780.trc:

#从上面的alert log可以看出实例无法创建新的进程，推断有可能是由于超出了实例设定进程的最大值
oracle@SZDB:~> export ORACLE_SID=SYWGMR
oracle@SZDB:~> sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Thu Mar 27 10:56:27 2014

Copyright (c) 1982, 2011, Oracle. All rights reserved.

Connected to:
Oracle Database 10g Release 10.2.0.3.0 - 64bit Production

-- Author : Leshami
-- Blog :

SQL> col RESOURCE_NAME for a20
SQL> col LIMIT_VALUE for a20
SQL> select resource_name,MAX_UTILIZATION,LIMIT_VALUE from v$resource_limit where resource_name in ('processes','sessions');

RESOURCE_NAME MAX_UTILIZATION LIMIT_VALUE
-------------------- --------------- --------------------
processes 1000 1000
sessions 1004 1105

#上面的查询结果果然如此

3、故障解决

#找到了引起故障的原因，就好处理了。下面是关于processes的描述
PROCESSES specifies the maximum number of operating system user processes that can simultaneously connect to Oracle.
Its value should allow for all background processes such as locks, job queue processes, and parallel execution processes.

The default values of the SESSIONS and TRANSACTIONS parameters are derived from this parameter.
Therefore, if you change the value of PROCESSES, you should evaluate whether to adjust the values of those derived parameters.

#修改实例的processes参数，如下，不支持memory的修改，因此需要加scope
sys> alter system set processes=1500;
alter system set processes=1500
*
ERROR at line 1:
ORA-02095: specified initialization parameter cannot be modified

sys> alter system set processes=1500 scope=spfile;

System altered.

#下面是修改并重启后的结果，如果你使用的是pfile参数，则直接修改pfile中processes的值再重启即可。
sys> show parameter processes