网络设备重启造成rac监听故障连接数据库报ora-12170:TNS:连接超时的错误

由于网络管理员把所有网络设备做了重启造成rac的虚拟IP发生漂移造成监听程序失效应用连接数据库
时报ora-12170:TNS:连接超时的错误
从下面的信息可以看到两个节点的vip漂移到另一个节点上使监听offline了

[root@hnzdb2 bin]# ./crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....SM1.asm application    ONLINE    ONLINE    hnzdb1
ora....ER.lsnr application    ONLINE    OFFLINE
ora....B1.lsnr application    ONLINE    OFFLINE
ora.hnzdb1.gsd application    ONLINE    ONLINE    hnzdb1
ora.hnzdb1.ons application    ONLINE    ONLINE    hnzdb1
ora.hnzdb1.vip application    ONLINE    ONLINE    hnzdb2
ora....SM2.asm application    ONLINE    ONLINE    hnzdb2
ora....ER.lsnr application    ONLINE    OFFLINE
ora....B2.lsnr application    ONLINE    OFFLINE
ora.hnzdb2.gsd application    ONLINE    ONLINE    hnzdb2
ora.hnzdb2.ons application    ONLINE    ONLINE    hnzdb2
ora.hnzdb2.vip application    ONLINE    ONLINE    hnzdb1
ora.orcl.db    application    ONLINE    ONLINE    hnzdb1
ora....l1.inst application    ONLINE    ONLINE    hnzdb1
ora....l2.inst application    ONLINE    ONLINE    hnzdb2

尝试重启所有服务报虚拟ip漂移的错误信息

[root@hnzdb2 bin]# ./crs_start -all
hnzdb1 : CRS-1019: Resource ora.hnzdb2.LISTENER.lsnr (application) cannot run on
 hnzdb1
hnzdb2 : CRS-1018: Resource ora.hnzdb2.vip (application) is already running on h
nzdb1

hnzdb1 : CRS-1018: Resource ora.hnzdb1.vip (application) is already running on h
nzdb2
hnzdb2 : CRS-1019: Resource ora.hnzdb1.LISTENER.lsnr (application) cannot run on
 hnzdb2

hnzdb1 : CRS-1019: Resource ora.hnzdb2.LISTENER_HNZDB2.lsnr (application) cannot
 run on hnzdb1
hnzdb2 : CRS-1018: Resource ora.hnzdb2.vip (application) is already running on h
nzdb1

hnzdb1 : CRS-1018: Resource ora.hnzdb1.vip (application) is already running on h
nzdb2
hnzdb2 : CRS-1019: Resource ora.hnzdb1.LISTENER_HNZDB1.lsnr (application) cannot
 run on hnzdb2

CRS-0223: Resource 'ora.hnzdb1.LISTENER.lsnr' has placement error.

CRS-0223: Resource 'ora.hnzdb1.LISTENER_HNZDB1.lsnr' has placement error.

CRS-0223: Resource 'ora.hnzdb2.LISTENER.lsnr' has placement error.

CRS-0223: Resource 'ora.hnzdb2.LISTENER_HNZDB2.lsnr' has placement error.

先停止所有服务再来重启

[root@hnzdb2 bin]# ./crs_stop -all
Attempting to stop `ora.hnzdb2.gsd` on member `hnzdb2`
Attempting to stop `ora.hnzdb2.ons` on member `hnzdb2`
Stop of `ora.hnzdb2.gsd` on member `hnzdb2` succeeded.
Attempting to stop `ora.hnzdb1.gsd` on member `hnzdb1`
Attempting to stop `ora.orcl.db` on member `hnzdb1`
Attempting to stop `ora.hnzdb1.ons` on member `hnzdb1`
Stop of `ora.hnzdb2.ons` on member `hnzdb2` succeeded.
Stop of `ora.hnzdb1.gsd` on member `hnzdb1` succeeded.
Stop of `ora.hnzdb1.ons` on member `hnzdb1` succeeded.
^[Stop of `ora.orcl.db` on member `hnzdb1` succeeded.
`ora.orcl.orcl1.inst` is already OFFLINE.
`ora.orcl.orcl2.inst` is already OFFLINE.
Attempting to stop `ora.hnzdb2.ASM2.asm` on member `hnzdb2`
Target set to OFFLINE for `ora.hnzdb2.LISTENER_HNZDB2.lsnr`
Target set to OFFLINE for `ora.hnzdb1.LISTENER_HNZDB1.lsnr`
Attempting to stop `ora.hnzdb1.ASM1.asm` on member `hnzdb1`
Target set to OFFLINE for `ora.hnzdb2.LISTENER.lsnr`
Attempting to stop `ora.hnzdb2.vip` on member `hnzdb1`
Target set to OFFLINE for `ora.hnzdb1.LISTENER.lsnr`
Attempting to stop `ora.hnzdb1.vip` on member `hnzdb2`
Stop of `ora.hnzdb2.vip` on member `hnzdb1` succeeded.
Stop of `ora.hnzdb1.vip` on member `hnzdb2` succeeded.

[root@hnzdb2 bin]# ./crs_start -all