Oracle 11g 单实例使用+asm修改主机名导致ORA-29701 故障分析

解决

把服务器名修改为原来的，重启服务器。

故障

建表空间失败。
在这里插入图片描述

分析

查看告警日志

ORA-1119 signalled during: create tablespace splex datafile ‘+DATA’ size 2000M…
Tue May 20 18:04:28 2025
create tablespace splex datafile ‘+DATA/option/datafile/splex01.dbf’ size 2G
WARNING: ASM communication error: op 11 state 0x50 (3113)
ERROR: slave communication error with ASM
ORA-1119 signalled during: create tablespace splex datafile ‘+DATA/option/datafile/splex01.dbf’ size 2G…

往前查，有其他ORA报错，最早发生在Mar 19.

Wed Mar 19 11:25:50 2025
NOTE: ASMB terminating
Errors in file /u01/app/oracle/diag/rdbms/option/Option/trace/Option_asmb_3250.trc:
ORA-15064: communication failure with ASM instance
ORA-03113: end-of-file on communication channel
Process ID:
Session ID: 1135 Serial number: 27
Errors in file /u01/app/oracle/diag/rdbms/option/Option/trace/Option_asmb_3250.trc:
ORA-15064: communication failure with ASM instance
ORA-03113: end-of-file on communication channel
Process ID:
Session ID: 1135 Serial number: 27
ASMB (ospid: 3250): terminating the instance due to error 15064
Wed Mar 19 11:25:51 2025
System state dump requested by (instance=1, osid=3250 (ASMB)), summary=[abnormal instance termination].
System State dumped to trace file /u01/app/oracle/diag/rdbms/option/Option/trace/Option_diag_3224_20250319112551.trc
Dumping diagnostic data in directory=[cdmp_20250319112551], requested by (instance=1, osid=3250 (ASMB)), summary=[abnormal instance termination].
Instance terminated by ASMB, pid = 3250

侦听看不到asm实例

lsnrctl status

正常情况可以看到asm实例和数据库实例。但是看不到asm实例。判断
asm运行故障。

怀疑存储故障

查看硬盘没有故障灯。

查MOS错误代码ORA-29701

Preliminary Checks When “ERROR: unrecoverable error ORA-29701 raised in ASM I/O path” in Oracle Restart Setup (Doc ID 1473701.1)

提到的oracle,grid的权限，group等，经检查，都没有问题。

沟通

跟客户沟通，确认客户前段时间修改了服务器名，从option修改为 db-option。

尝试重启数据库和has

关闭数据库时报错，直接kill -9杀死数据库和asm的pmon进程

手动启动has报错

[root@db-option init.d]# /u01/app/11.2.0/grid/bin/crsctl start has
CLSU-00100: Operating System function: opendir failed with error data: 2
CLSU-00101: Operating System error message: No such file or directory
CLSU-00103: error location: scrsearch1
CLSU-00104: additional error information: cant open scr home dir scls_scr_getval
CRS-4000: Command Start failed, or completed with errors

chatgpt分析：
ls -ld /etc/oracle/scls_scr 这个目录有问题

检查发现目录名字是原来的主机名，启动时读取这个而目录失败。

所以考虑改名会解决。

服务器改名并重启

#hostnamectl set-hostname option

重启服务器。

asm和数据库自动启动正常。

建表空间正常。

总结

单实例+ASM数据库，管理和RAC类似，对OS的IP, 主机名的修改不像文件系统那么简单。

一般来说，在部署前应该确定服务器名，数据库名，IP等。修改要慎重。如果要修改，参考文档

How to change Hostname / IP for a Grid Infrastructure Oracle Restart
Standalone Configuration (SIHA) 11.2 and Later (Doc ID 1552810.1)

1) Configure the CSS & OHAS services as root user as follows:
# <Grid Infrastructure Oracle Home>/crs/install/roothas.pl -deconfig -force
Please note that from 12.1.0.2 onwards need to run roothas.sh instead of roothas.pl
2) Now Change the hostname
3) Reconfigure the cluster
# cd <11.2 Grid infrastructure Oracle Home>
# ./root.sh
For 12.1.0.2 and later releases ,
Please run <Grid infrastructure Oracle Home>/crs/install/roothas.sh
4) Go to the grid home bin directory.
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-state=eon8x73rq_280&id=1552810.1
2/8Use the srvctl add database command to add the database in an Oracle Restart configuration. If you have
role separation then use the DB home owner for this.
5) Please perform the next steps as oracle or grid OS user (as the Grid Infrastructure OS owner):
5/21/25, 11:00 AM
Document 1552810.1
$> <Grid Infrastructure Oracle Home>/bin/crsctl modify resource "ora.cssd" -
init -attr "AUTO_START=1" -unsupported
NOTE: "-unsupported" is not required for 11.2 version
6) Restart the OHAS stack as grid or oracle OS user (as the Grid Infrastructure OS owner):
$> <Grid Infrastructure Oracle Home>/bin/crsctl stop has
$> <Grid Infrastructure Oracle Home>/bin/crsctl start has
7) Check the CSS & OHAS state as grid or oracle OS user (as the Grid Infrastructure OS owner):
$> <Grid Infrastructure Oracle Home>/bin/crsctl check has
$> <Grid Infrastructure Oracle Home>/bin/crsctl check css
$> <Grid Infrastructure Oracle Home>/bin/ crsctl stat resource
$> <Grid Infrastructure Oracle Home>/bin/crsctl stat res -t
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-state=eon8x73rq_280&id=1552810.1
3/85/21/25, 11:00 AM
Document 1552810.1
Note: If the CSS & OHAS service did NOT start, then you will need to reboot the Linux/unix box and check
them again.
8) Recreate the default listener (LISTENER) using port 1521 (or using your desired port), thru the NETCA GUI
located on the new Grid Infrastructure Oracle Home (or manually if you do not have graphical access) as grid
or oracle OS user (as the Grid Infrastructure OS owner):
$> srvctl add listener
$> srvctl start listener
9) Please create the init+ASM.ora file on the <Grid Infrastructure Oracle Home>/dbs directory with the next
parameters:
asm_diskgroups= <list of diskgroups>
asm_diskstring= '/dev/oracleasm/disks/*' ◄= use the existing asm_diskstring
value. This can be found with "asmcmd dsget" command.
instance_type='asm'
large_pool_size=12M
10) Add the ASM instance as grid or oracle OS user (as the Grid Infrastructure OS owner):
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-state=eon8x73rq_280&id=1552810.1
4/85/21/25, 11:00 AM
Document 1552810.1
$> <Grid Infrastructure Oracle Home>/bin/srvctl add asm
$> <12.1 Grid Infrastructure Oracle Home>/bin/srvctl add asm
11) Enable ASM instance Auto Start as grid or oracle OS user (as the Grid Infrastructure OS owner) as
follow:
$> <Grid Infrastructure Oracle Home>/bin/crsctl modify resource "ora.asm" -init
-attr "AUTO_START=1" -unsupported
NOTE: "-unsupported" is not required for 11.2 version
12) Make sure the disks are discovered by kfod as grid or oracle OS user (as the Grid Infrastructure OS
owner) as follow:
Example:
$> <Grid Infrastructure Oracle Home>/bin/kfod asm_diskstring='ORCL:*' disks=all
◄= if using ASMlib, otherwise use below.
Or
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-state=eon8x73rq_280&id=1552810.1
5/85/21/25, 11:00 AM
Document 1552810.1
$> <Grid Infrastructure Oracle Home>/bin/kfod asm_diskstring='<full path ASM
disks location>/*' disks=all
13) If so, then startup the ASM instance as grid or oracle OS user (as the Grid Infrastructure OS owner) as
follow:
$> export ORACLE_SID=+ASM
$> <Grid Infrastructure Oracle Home>/bin/sqlplus “/as sysasm”
SQL> startup pfile=init+ASM.ora --#init file from point #6
SQL> show parameter asm
14) Validate that the candidate disks are being discovered:
SQL> select path from v$asm_disk;
15) Create a new ASM instance spfile:
SQL> create spfile from pfile;
16) Add the new ASM spfile and listener to the new ASM instance resource:
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-state=eon8x73rq_280&id=1552810.1
6/85/21/25, 11:00 AM
Document 1552810.1
$> <Grid Infrastructure Oracle Home>/bin/srvctl modify asm -p <spfile full
path>
$> <Grid Infrastructure Oracle Home>/bin/srvctl modify asm -l LISTENER
17) Validate the OHAS (Oracle Restart) services start as follows:
$> <Grid Infrastructure Oracle Home>/bin/crsctl stop has
$> <Grid Infrastructure Oracle Home>/bin/crsctl start has
$> <Grid Infrastructure Oracle Home>/bin/crsctl stat res
$> <Grid Infrastructure Oracle Home>/bin/crsctl stat res -t