ORA-00349|激活 ADG 備庫時遇到的問題及處理方法
近日有一套實時同步的 ASM 管理的單機 11204 ADG 備庫，由於業務需要，想要脫離主庫的約束，想激活拉成讀寫庫直接升級成 ASM 管理的 19C，閃回快照模式無法滿足要求，隻能 ALTER DATABASE ACTIVATE STANDBY DATABASE 強制切成可讀寫的主庫。說幹就幹，先將其切成主庫，升級過程等下次在一起討論。
--主庫 --主庫設置為 defer, 取消備庫日志應用，關庫啟動到 mount 狀態進行。 show parameter log_archive_dest_state_2 alter system set log_archive_dest_state_2=defer scope=both sid='*'; --備庫 alter database recover managed standby database cancel; shu immediate startup mount --強制拉成主庫，很遺憾報錯 ORA-00349 SQL> ALTER DATABASE ACTIVATE STANDBY DATABASE; ALTER DATABASE ACTIVATE STANDBY DATABASE * ERROR at line 1: ORA-00349: failure obtaining block size for '+JIEKE_DATA' ORA-15001: diskgroup "JIEKE_DATA" does not exist or is not mounted ORA-15001: diskgroup "JIEKE_DATA" does not exist or is not mounted
使用 ACTIVATE 命令想強制拉成主庫，很遺憾如下圖報錯 ORA-00349。alert 日志中發現有很多清理 redo log 的報錯，“ORA-00313: open failed…”無法打開日志組 5、6、23，於是查看日志組成員確實發現 redolog 創建的有問題，member 成員顯示的為不存在的磁盤組 “+JIEKE_DATA” 而不是具體路徑，真是存在的磁盤組“+JIEKER_DATA”。這就是問題所在，redolog 創建錯誤，切成主庫時 redolog 又是必須的，故報錯瞭，那麼現在就是將這個錯誤的 redolog 重建，問題就會得到解決。但實際上不是這樣的，折騰瞭好久也沒解決，繼續往下看。
GROUP# Member
---------- ---------------------------------------------------------------------------------------------------
5 +JIEKE_DATA
5 +JIEKE_DATA
6 +JIEKE_DATA
6 +JIEKE_DATA
23 +JIEKE_DATA
23 +JIEKE_DATA
11 +JIEKER_DATA/jiekexu/onlinelog/group_11.1621.1065127343
11 +JIEKER_ARCH/jiekexu/onlinelog/group_11.389.1065127355
12 +JIEKER_DATA/jiekexu/onlinelog/group_12.1620.1065127363
12 +JIEKER_ARCH/jiekexu/onlinelog/group_12.395.1065127371
13 +JIEKER_DATA/jiekexu/onlinelog/group_13.1619.1065127381
SELECT to_char(f.group#) groupno, to_char(l.thread#) thread, f.member member, f.type redo_file_type, l.status group_status, f.status member_status, l.bytes/1024/1024 bytes, l.archived archived
FROM v$logfile f, v$log l WHERE f.group# = l.grSQL> oup# ORDER BY f.group#, f.member;
SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> 2
Group Thread Member Redo Type Group Status Member Status Size(M) Archived
------ ------ -------------------- ---------- ------------ --------------- -------- ----------
5 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
6 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
23 1 +JIEKE_DATA ONLINE CLEARING_CUR INVALID 4,096 YES
RENT
1 +JIEKE_DATA ONLINE CLEARING_CUR INVALID 4,096 YES
RENT
27 2 +JIEKER_DATA/jiekexu/ ONLINE UNUSED 4,096 YES
onlinelog/group_27.1
741.1065129955
28 2 +JIEKER_DATA/jiekexu/ ONLINE UNUSED 4,096 YES
onlinelog/group_28.1
742.1065129973
alert 日志如下：
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 5 of thread 1
Clearing online redo logfile 5 +JIEKE_DATA
Clearing online log 5 of thread 1 sequence number 4751
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 5 of thread 1
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 5 of thread 1
Clearing online redo logfile 5 complete
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 6 of thread 2
Clearing online redo logfile 6 +JIEKE_DATA
Clearing online log 6 of thread 2 sequence number 2592
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 6 of thread 2
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 6 of thread 2
Clearing online redo logfile 6 complete
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 23 of thread 1
Clearing online redo logfile 23 +JIEKE_DATA
Clearing online log 23 of thread 1 sequence number 4752
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 23 of thread 1
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 23 of thread 1
Clearing online redo logfile 23 complete
Resetting resetlogs activation ID 2008461997 (0x77b6b2ad)
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 5 of thread 1
Errors in file /app/oracle/diag/rdbms/jiekexu/jiekexu/trace/jiekexu_ora_30872.trc:
ORA-00313: open failed for members of log group 5 of thread 1
ORA-349 signalled during: ALTER DATABASE ACTIVATE PHYSICAL STANDBY DATABASE…
Tue Mar 02 22:42:30 2021
alter database drop logfile group 23
ORA-1623 signalled during: alter database drop logfile group 23…
Tue Mar 02 22:45:07 2021
RFS[7]: Assigned to RFS process 10180
RFS[7]: Opened log for thread 2 sequence 2592 dbid 1797812601 branch 1063804222
Archived Log entry 1100 added for thread 2 sequence 2592 rlc 1063804222 ID 0x77b6b2ad dest 2:
Tue Mar 02 22:45:08 2021
RFS[8]: Assigned to RFS process 10277
RFS[8]: Selected log 17 for thread 2 sequence 2593 dbid 1797812601 branch 1063804222
Tue Mar 02 22:45:08 2021
Primary database is in MAXIMUM PERFORMANCE mode
重建備庫 redolog
那麼，這裡將上演的是重建備庫 redolog 的相關操作步驟。
SQL> alter database drop logfile group 27; alter database drop logfile group 27 * ERROR at line 1: ORA-01156: recovery or flashback in progress may need access to files --由於開啟瞭日志應用進程，直接刪除會報錯，故需要停止日志應用，修改參數 standby_file_management 為手動。 SQL> alter database recover managed standby database cancel; Database altered. SQL> alter system set standby_file_management='manual' scope=both sid='*'; Database altered. SQL> show parameter standby_file_management NAME TYPE VALUE ------------------------------------ ---------------------- ------------------------------ standby_file_management string manual SQL> alter database drop logfile group 27; Database altered. --由於日志組不能少於 2 個，故 日志組 28 不不能夠刪除。 SQL> alter database drop logfile group 28; alter database drop logfile group 28 * ERROR at line 1: ORA-01567: dropping log 28 would leave less than 2 log files for instance JIEKEXU2 (thread 2) ORA-00312: online log 28 thread 2: '+JIEKER_DATA/jiekexu/onlinelog/group_28.1742.1065129973'
下面則通過繼續應用日志、重啟、主庫切日志、重命名等各種手段繼續嘗試刪除這三個有問題的日志組。
--重啟備庫刪除日志組 6、23，由於是當前日志組無法刪除，慶幸日志組 5 成功刪除瞭。 SQL> alter database drop logfile group 6; alter database drop logfile group 6 * ERROR at line 1: ORA-01623: log 6 is current log for instance JIEKEXU2 (thread 2) - cannot drop ORA-00312: online log 6 thread 2: '+JIEKE_DATA' ORA-00312: online log 6 thread 2: '+JIEKE_DATA' SQL> alter database drop logfile group 5; Database altered. SQL> alter database drop logfile group 23; alter database drop logfile group 23 * ERROR at line 1: ORA-01623: log 23 is current log for instance jiekexu (thread 1) - cannot drop ORA-00312: online log 23 thread 1: '+JIEKE_DATA' ORA-00312: online log 23 thread 1: '+JIEKE_DATA' --隻剩兩組日志組也是當前日志組，則當前日志組無法刪除，嘗試進行 rename 操作，但也是無效或者缺失命令。 SQL> alter database rename '+JIEKE_DATA' to '+JIEKER_DATA'; alter database rename '+JIEKE_DATA' to '+JIEKER_DATA' * ERROR at line 1: ORA-02231: missing or invalid option to ALTER DATABASE SQL> alter database rename '+JIEKE_DATA' to '+JIEKER_DATA/jiekexu/onlinelog/group_6.dbf'; alter database rename '+JIEKE_DATA' to '+JIEKER_DATA/jiekexu/onlinelog/group_6.dbf' * ERROR at line 1: ORA-02231: missing or invalid option to ALTER DATABASE --當然繼續激活為主庫肯定也是報錯。那就繼續開啟日志同步模式，先保持備庫同步吧。 SQL> ALTER DATABASE ACTIVATE STANDBY DATABASE; ALTER DATABASE ACTIVATE STANDBY DATABASE * ERROR at line 1: ORA-00349: failure obtaining block size for '+JIEKE_DATA' ORA-15001: diskgroup "JIEKE_DATA" does not exist or is not mounted ORA-15001: diskgroup "JIEKE_DATA" does not exist or is not mounted SQL> alter database recover managed standby database using current logfile disconnect from session; Database altered.
重建備庫控制文件解決
第二日早晨，睡醒之後頭腦清晰想到備庫既然無法刪除，那主庫肯定是可以刪除的。通過主庫刪除日志組 6、23 之後，再重建一個備庫的控制文件就可以解決。這樣也很簡單，主庫刪除備庫有問題的兩組日志組後使用 rman 備份一個備庫的 控制文件，然後 scp 到備庫，備庫重啟到 nomount 恢復控制文件，啟動到 mount 就好瞭。
--主庫： SQL> alter database drop logfile group 6; Database altered. SQL> alter database drop logfile group 23; Database altered. rman target / RMAN> backup current controlfile for standby format '/home/oracle/backup20210303%d_%I_%s_%p.ctl'; scp /home/oracle/backup20210303%d_%I_%s_%p.ctl jiekeadg:/home/oracle/ --備庫 SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL; Database altered. SQL> shu immediate ORA-01109: database not open Database dismounted. ORACLE instance shut down. SQL> startup nomount ORACLE instance started. Total System Global Area 1.0689E+11 bytes Fixed Size 2265864 bytes Variable Size 4.2144E+10 bytes Database Buffers 6.4425E+10 bytes Redo Buffers 323678208 bytes SQL> exit jiekeadg:/home/oracle(jiekexu)>rman target / Recovery Manager: Release 11.2.0.4.0 - Production on Wed Mar 3 10:07:34 2021 Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved. connected to target database: JIEKEXU (not mounted) RMAN> restore standby controlfile from '/home/oracle/backup20210303JIEKEXU_1797812601_106_1.ctl'; Starting restore at 2021-03-03 10:08:03 using target database control file instead of recovery catalog allocated channel: ORA_DISK_1 channel ORA_DISK_1: SID=3009 device type=DISK channel ORA_DISK_1: restoring control file channel ORA_DISK_1: restore complete, elapsed time: 00:00:01 output file name=+JIEKER_DATA/jiekexu/controlfile/current.1739.1065125909 output file name=+JIEKER_ARCH/jiekexu/controlfile/current.323.1065125911 Finished restore at 2021-03-03 10:08:04 RMAN> sql'alter database mount'; sql statement: alter database mount released channel: ORA_DISK_1 RMAN> exit -- SQLPLUS 裡應用 MRP0 進程同步數據。 jiekeadg:/home/oracle(jiekexu)>sqlplus / as sysdba SQL> alter database recover managed standby database using current logfile disconnect from session; Database altered. SQL> / NAME VALUE UNIT TIME_COMPUTED ------------- -------------------- ------------------------------ ------------------------------ transport lag +00 00:00:00 day(2) to second(0) interval 03/03/2021 10:19:08 apply lag +00 00:00:00 day(2) to second(0) interval 03/03/2021 10:19:08
重建控制文件後恢復的備庫中就沒有瞭錯誤的磁盤組，但這樣時間長一些則備庫日志應用會延遲，因為 standby_log 日志狀態全部為 UNASSIGNED 的。需要重建備庫 standby redolog 日志組後它的狀態才會變成 ACTIVE。
取消日志應用，修改參數為 manual 刪除原有日志組，重建日志組改回參數應用日志同步進程即可。
SELECT to_char(f.group#) groupno, to_char(l.thread#) thread, f.member member, f.type redo_file_type, l.status group_status, f.status member_status, l.bytes/1024/1024 bytes, l.archived archived
FROM v$logfile f, v$log l WHERE f.group# = l.group# ORDER BY f.group#, f.member;SQL>
SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> 2
Group Thread Member Redo Type Group Status Member Status Size(M) Archived
------ ------ -------------------- ---------- ------------ --------------- -------- ----------
1 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
2 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
3 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
4 1 +JIEKE_DATA ONLINE CURRENT INVALID 4,096 YES
1 +JIEKE_DATA ONLINE CURRENT INVALID 4,096 YES
5 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
7 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
8 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
9 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
10 2 +JIEKE_DATA ONLINE CURRENT INVALID 4,096 YES
2 +JIEKE_DATA ONLINE CURRENT INVALID 4,096 YES
24 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
Database altered.
SQL> show parameter standby_file_management
NAME TYPE VALUE
------------------------------------ ---------------------- ------------------------------standby_file_management string manual
SQL>
SQL> select inst_id,GROUP#,THREAD#,SEQUENCE#,USED,ARCHIVED,STATUS from gv$standby_log;
INST_ID GROUP# THREAD# SEQUENCE# USED Archived STATUS
---------- ---------- ---------- ---------- ---------- ---------- --------------------
1 11 1 0 0 YES UNASSIGNED
1 12 1 0 0 YES UNASSIGNED
1 13 1 0 0 YES UNASSIGNED
1 14 1 0 0 YES UNASSIGNED
1 15 1 0 0 YES UNASSIGNED
alter database drop logfile group 11;
alter database drop logfile group 12;
alter database drop logfile group 13;
alter database drop logfile group 14;
alter database drop logfile group 15;
ALTER DATABASE ADD standby LOGFILE THREAD 1 ('+JIEKER_DATA','+JIEKER_ARCH') SIZE 4G;
ALTER DATABASE ADD standby LOGFILE THREAD 1 ('+JIEKER_DATA','+JIEKER_ARCH') SIZE 4G;
ALTER DATABASE ADD standby LOGFILE THREAD 1 ('+JIEKER_DATA','+JIEKER_ARCH') SIZE 4G;
ALTER DATABASE ADD standby LOGFILE THREAD 1 ('+JIEKER_DATA','+JIEKER_ARCH') SIZE 4G;
ALTER DATABASE ADD standby LOGFILE THREAD 1 ('+JIEKER_DATA','+JIEKER_ARCH') SIZE 4G;
SQL> select inst_id,GROUP#,THREAD#,SEQUENCE#,USED,ARCHIVED,STATUS from gv$standby_log;
INST_ID GROUP# THREAD# SEQUENCE# USED Archived STATUS
---------- ---------- ---------- ---------- ---------- ---------- --------------------
1 6 1 4797 3026358272 YES ACTIVE
1 11 1 0 0 YES UNASSIGNED
1 12 1 0 0 YES UNASSIGNED
1 13 1 0 0 YES UNASSIGNED
1 14 1 0 0 YES UNASSIGNED
SQL> alter system set standby_file_management='AUTO' scope=both sid='*';
SQL> alter database open;
alter database recover managed standby database using current logfile disconnect from session;
Database altered.
找到最終問題所在
完成重建後，本以為已經萬事大吉瞭，但一開庫應用日志卻發現後臺日志中所有 redolog 日志組都被清理瞭。這才意識到問題的根源所在，立馬查看瞭路徑相關的參數發現 db_create_online_log_dest 參數設置錯誤，導致創建出瞭錯誤的磁盤組。
jiekeadg:/home/oracle(jiekexu)>sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Wed Mar 3 10:52:04 2021 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> show parameter db_create_online_log_dest_ NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ db_create_online_log_dest_1 string +JIEKE_DATA db_create_online_log_dest_2 string +JIEKE_DATA db_create_online_log_dest_3 string db_create_online_log_dest_4 string db_create_online_log_dest_5 string SQL> alter system set db_create_online_log_dest_1='+JIEKER_DATA' scope=spfile; System altered. SQL> alter system set db_create_online_log_dest_2='+JIEKER_DATA' scope=spfile; System altered.
但是現在修改完參數所有的日志組成員也都出現在錯誤的磁盤組瞭，因為有瞭當前日志組 4、10 占用瞭，重建日志組也行不通，故隻能再次重建備庫控制文件瞭。以上主庫備份控制文件傳到備庫恢復的過程再來一遍即可。
SELECT to_char(f.group#) groupno, to_char(l.thread#) thread, f.member member, f.type redo_file_type, l.status group_status, f.status member_status, l.bytes/1024/1024 bytes, l.archived archived FROM v$logfile f, v$log l WHERE f.group# = l.group# ORDER BY f.group#, f.member;SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> SQL> 2 Group Thread Member Redo Type Group Status Member Status Size(M) Archived ------ ------ -------------------- ---------- ------------ --------------- -------- ---------- 1 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 2 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 3 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 4 1 +JIEKE_DATA ONLINE CURRENT INVALID 4,096 YES 1 +JIEKE_DATA ONLINE CURRENT INVALID 4,096 YES 5 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 1 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 7 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 8 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 9 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 10 2 +JIEKE_DATA ONLINE CURRENT INVALID 4,096 YES 2 +JIEKE_DATA ONLINE CURRENT INVALID 4,096 YES 24 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 2 +JIEKE_DATA ONLINE CLEARING INVALID 4,096 YES 20 rows selected.
最終激活備庫
再一次重建後正常恢復同步，然後關閉實例啟動到 mount 狀態，激活 ADG 備庫，重啟驗證即可。
SQL> shu immediate Database closed. Database dismounted. ORACLE instance shut down. SQL> SQL> startup mount ORACLE instance started. Total System Global Area 1.0689E+11 bytes Fixed Size 2265864 bytes Variable Size 4.2144E+10 bytes Database Buffers 6.4425E+10 bytes Redo Buffers 323678208 bytes Database mounted. SQL> SQL> ALTER DATABASE ACTIVATE STANDBY DATABASE; Database altered. SQL> alter database open; Database altered. SQL> select DATABASE_ROLE from v$database; DATABASE_ROLE -------------------------------- PRIMARY SQL> shu immediate Database closed. Database dismounted. ORACLE instance shut down. SQL> startup ORACLE instance started. Total System Global Area 1.0689E+11 bytes Fixed Size 2265864 bytes Variable Size 4.2144E+10 bytes Database Buffers 6.4425E+10 bytes Redo Buffers 323678208 bytes Database mounted. Database opened. SQL> exit
到此這篇關於ORA-00349|激活 ADG 備庫時遇到的問題及處理方法的文章就介紹到這瞭,更多相關ORA-00349內容請搜索WalkonNet以前的文章或繼續瀏覽下面的相關文章希望大傢以後多多支持WalkonNet！
推薦閱讀：
- DBCA命令行搭建Oracle ADG的流程
- Oracle數據庫的備份與恢復案例詳解
- Oracle縮表空間的完整解決實例
- Oracle查看表空間使用率以及爆滿解決方案詳解
- 關於oracle數據庫表空間擴容的問題