模拟redo丢失

对数据库的一个pdb模拟事务操作，然后abort库，并且删除所有redo，模拟生产环境redo丢失的case

[oracle@ora1221 oradata]$ ss

SQL*Plus: Release 12.2.0.0.3 Production on Wed Jun 15 10:13:20 2016

Copyright (c) 1982, 2016, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup

ORACLE instance started.

Total System Global Area 2516582400 bytes

Fixed Size 8260048 bytes

Variable Size 671090224 bytes

Database Buffers 1828716544 bytes

Redo Buffers 8515584 bytes

Database mounted.

Database opened.

SQL>

SQL>

SQL> set pages 100

SQL> show pdbs;

CON_ID CON_NAME OPEN MODE RESTRICTED

---------- ------------------------------ ---------- ----------

2 PDB$SEED READ ONLY NO

3 PDB1 MOUNTED

4 PDB2 MOUNTED

SQL> select con_id,file#,checkpoint_change# from v$datafile_header order by 1;

CON_ID FILE# CHECKPOINT_CHANGE#

---------- ---------- ------------------

1 1 1500157

1 3 1500157

1 4 1500157

1 7 1500157

2 5 1371280

2 6 1371280

2 8 1371280

3 9 1499902

3 12 1499902

3 11 1499902

3 10 1499902

4 15 1499903

4 14 1499903

4 13 1499903

4 16 1499903

15 rows selected.

SQL> alter PLUGGABLE database pdb1 open;

Pluggable database altered.

SQL> show pdbs;

CON_ID CON_NAME OPEN MODE RESTRICTED

---------- ------------------------------ ---------- ----------

2 PDB$SEED READ ONLY NO

3 PDB1 READ WRITE NO

4 PDB2 MOUNTED

SQL> alter session set container=pdb1;

Session altered.

SQL> create user chf identified by oracle;

User created.

SQL> grant dba to chf;

Grant succeeded.

SQL> create table chf.t_pdb1_xifenfei as select * from dba_objects;

Table created.

SQL> delete from chf.t_pdb1_xifenfei;

72426 rows deleted.

--另外一个节点

[oracle@ora1221 ~]$ ss

SQL*Plus: Release 12.2.0.0.3 Production on Wed Jun 15 10:19:21 2016

Copyright (c) 1982, 2016, Oracle. All rights reserved.

Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.0.3 - 64bit Production

SQL> shutdown abort

ORACLE instance shut down.

[oracle@ora1221 orcl12c2]$ ls redo*

redo01.log redo02.log redo03.log

[oracle@ora1221 orcl12c2]$ rm redo0*

[oracle@ora1221 orcl12c2]$ ls -l redo*

ls: cannot access redo*: No such file or directory

尝试启动数据库

[oracle@ora1221 orcl12c2]$ ss

SQL*Plus: Release 12.2.0.0.3 Production on Wed Jun 15 10:26:20 2016

Copyright (c) 1982, 2016, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup mount;

ORACLE instance started.

Total System Global Area 2516582400 bytes

Fixed Size 8260048 bytes

Variable Size 671090224 bytes

Database Buffers 1828716544 bytes

Redo Buffers 8515584 bytes

Database mounted.

SQL> alter database open;

alter database open

*

ERROR at line 1:

ORA-00313: open failed for members of log group 1 of thread 1

ORA-00312: online log 1 thread 1: '/u01/app/oracle/oradata/orcl12c2/redo01.log'

ORA-27037: unable to obtain file status

Linux-x86_64 Error: 2: No such file or directory

Additional information: 7

使用隐含参数启动

----pfile里面增加

_allow_error_simulation=TRUE

_allow_resetlogs_corruption=true

~

SQL> shutdown immediate;

ORA-01109: database not open

Database dismounted.

ORACLE instance shut down

SQL> startup pfile='/tmp/pfile' mount

ORACLE instance started.

Total System Global Area 2516582400 bytes

Fixed Size 8260048 bytes

Variable Size 671090224 bytes

Database Buffers 1828716544 bytes

Redo Buffers 8515584 bytes

Database mounted.

SQL> alter database open resetlogs;

alter database open resetlogs

*

ERROR at line 1:

ORA-00603: ORACLE server session terminated by fatal error

ORA-01092: ORACLE instance terminated. Disconnection forced

ORA-00600: internal error code, arguments: [kcbzib_kcrsds_1], [], [], [], [],

[], [], [], [], [], [], []

Process ID: 36797

Session ID: 16 Serial number: 24277

继续重启库

ORA-600 kcbzib_kcrsds_1错误尝试重启数据库,如果不行,考虑使用bbed修改文件头信息

SQL> startup mount pfile='/tmp/pfile'

ORACLE instance started.

Total System Global Area 2516582400 bytes

Fixed Size 8260048 bytes

Variable Size 671090224 bytes

Database Buffers 1828716544 bytes

Redo Buffers 8515584 bytes

Database mounted.

SQL> recover database until cancel;

ORA-00283: recovery session canceled due to errors

ORA-16433: The database or pluggable database must be opened in read/write

mode.

SQL> alter database backup controlfile to trace as '/tmp/ctl';

alter database backup controlfile to trace as '/tmp/ctl'

*

ERROR at line 1:

ORA-16433: The database or pluggable database must be opened in read/write

mode.

重建控制文件

SQL> startup nomount pfile='/tmp/pfile'

ORACLE instance started.

Total System Global Area 2516582400 bytes

Fixed Size 8260048 bytes

Variable Size 671090224 bytes

Database Buffers 1828716544 bytes

Redo Buffers 8515584 bytes

SQL> CREATE CONTROLFILE REUSE DATABASE "orcl12c2" RESETLOGS NOARCHIVELOG

2 MAXLOGFILES 50

3 MAXLOGMEMBERS 5

4 MAXDATAFILES 100

5 MAXINSTANCES 1

6 MAXLOGHISTORY 226

7 LOGFILE

8 GROUP 1 '/u01/app/oracle/oradata/orcl12c2/redo01.log' SIZE 200M,

9 GROUP 2 '/u01/app/oracle/oradata/orcl12c2/redo02.log' SIZE 200M,

10 GROUP 3 '/u01/app/oracle/oradata/orcl12c2/redo03.log' SIZE 200M

11 DATAFILE

12 '/u01/app/oracle/oradata/orcl12c2/system01.dbf',

13 '/u01/app/oracle/oradata/orcl12c2/sysaux01.dbf',

14 '/u01/app/oracle/oradata/orcl12c2/undotbs01.dbf',

15 '/u01/app/oracle/oradata/orcl12c2/pdbseed/system01.dbf',

16 '/u01/app/oracle/oradata/orcl12c2/pdbseed/sysaux01.dbf',

17 '/u01/app/oracle/oradata/orcl12c2/users01.dbf',

18 '/u01/app/oracle/oradata/orcl12c2/pdbseed/undotbs01.dbf',

19 '/u01/app/oracle/oradata/orcl12c2/pdb1/system01.dbf',

20 '/u01/app/oracle/oradata/orcl12c2/pdb1/sysaux01.dbf',

21 '/u01/app/oracle/oradata/orcl12c2/pdb1/undotbs01.dbf',

22 '/u01/app/oracle/oradata/orcl12c2/pdb1/users01.dbf',

23 '/u01/app/oracle/oradata/orcl12c2/pdb2/system01.dbf',

24 '/u01/app/oracle/oradata/orcl12c2/pdb2/sysaux01.dbf',

25 '/u01/app/oracle/oradata/orcl12c2/pdb2/undotbs01.dbf',

26 '/u01/app/oracle/oradata/orcl12c2/pdb2/users01.dbf'

27 CHARACTER SET AL32UTF8

28 ;

Control file created.

SQL> recover database until cancel;

ORA-00283: recovery session canceled due to errors

ORA-01610: recovery using the BACKUP CONTROLFILE option must be done

SQL> recover database until cancel using backup controlfile;

ORA-00279: change 1500161 generated at 06/15/2016 10:40:42 needed for thread 1

ORA-00289: suggestion :

/u01/app/oracle/product/12.2.0/db_2/dbs/arch1_1_914582438.dbf

ORA-00280: change 1500161 for thread 1 is in sequence #1

Specify log: {=suggested | filename | AUTO | CANCEL}

/u01/app/oracle/oradata/orcl12c2/redo01.log

Log applied.

Media recovery complete.

SQL> alter database open resetlogs;

Database altered.

1

open过程alert日志

1

open pdb1

1

SQL> alter PLUGGABLE database pdb1 open;

Pluggable database altered.

pdb1 open alert日志

2016-06-15T11:13:39.423057+08:00

alter PLUGGABLE database pdb1 open

PDB1(3):Autotune of undo retention is turned on.

2016-06-15T11:13:39.495559+08:00

PDB1(3):Endian type of dictionary set to little

PDB1(3):[40547] Successfully onlined Undo Tablespace 2.

PDB1(3):Undo initialization finished serial:0 start:371149831 end:371149872 diff:41 ms (0.0 seconds)

PDB1(3):Database Characterset for PDB1 is AL32UTF8

PDB1(3):*********************************************************************

PDB1(3):WARNING: The following temporary tablespaces in container(PDB1)

PDB1(3): contain no files.

PDB1(3): This condition can occur when a backup controlfile has

PDB1(3): been restored. It may be necessary to add files to these

PDB1(3): tablespaces. That can be done using the SQL statement:

PDB1(3):

PDB1(3): ALTER TABLESPACE ADD TEMPFILE

PDB1(3):

PDB1(3): Alternatively, if these temporary tablespaces are no longer

PDB1(3): needed, then they can be dropped.

PDB1(3): Empty temporary tablespace: TEMP

PDB1(3):*********************************************************************

PDB1(3):Opatch validation is skipped for PDB PDB1 (con_id=0)

PDB1(3):Opening pdb with no Resource Manager plan active

Pluggable database PDB1 opened read write

Completed: alter PLUGGABLE database pdb1 open

open pdb2

SQL> alter PLUGGABLE database pdb2 open;

alter PLUGGABLE database pdb2 open

*

ERROR at line 1:

ORA-00600: internal error code, arguments: [17090], [], [], [], [], [], [], [],

[], [], [], []

分析alert日志和trace文件

--alert日志部分

PDB1(3):alter PLUGGABLE database pdb2 open

PDB1(3):ORA-65118 signalled during: alter PLUGGABLE database pdb2 open...

2016-06-15T11:28:57.439963+08:00

PDB1(3):Unified Audit: Audit record write to table failed due to ORA-25153.

Writing the audit record to OS spillover file. Please grep in the trace files for ORA-25153 for more diagnostic information.

Errors in file /u01/app/oracle/diag/rdbms/orcl12c2/orcl12c2/trace/orcl12c2_ora_40547.trc (incident=29073) (PDBNAME=PDB1):

ORA-00600: internal error code, arguments: [17090], [], [], [], [], [], [], [], [], [], [], []

PDB1(3):Incident details in: /u01/app/oracle/diag/rdbms/orcl12c2/orcl12c2/incident/incdir_29073/orcl12c2_ora_40547_i29073.trc

PDB1(3):*****************************************************************

PDB1(3):An internal routine has requested a dump of selected redo.

PDB1(3):This usually happens following a specific internal error, when

PDB1(3):analysis of the redo logs will help Oracle Support with the

PDB1(3):diagnosis.

PDB1(3):It is recommended that you retain all the redo logs generated (by

PDB1(3):all the instances) during the past 12 hours, in case additional

PDB1(3):redo dumps are required to help with the diagnosis.

PDB1(3):*****************************************************************

2016-06-15T11:28:59.123041+08:00

PDB1(3):Use ADRCI or Support Workbench to package the incident.

See Note 411.1 at My Oracle Support for error and packaging details.

2016-06-15T11:28:59.945667+08:00

Dumping diagnostic data in directory=[cdmp_20160615112859], requested by (instance=1, osid=40547), summary=[incident=29073].

2016-06-15T11:35:59.987419+08:00

PDB1(3): alter PLUGGABLE database pdb2 open

PDB1(3):ORA-65118 signalled during: alter PLUGGABLE database pdb2 open...

--trace部分

PARSING IN CURSOR #0x7f051a3d7650 len=118 dep=1 uid=0 oct=3 lid=0 tim=372490287736 hv=1128335472 ad='0x6ca82f00' sqlid='gu930gd1n223h'

select tablespace_name, tablespace_size, allocated_space, free_space, con_id from cdb_temp_free_space order by con_id

END OF STMT

EXEC #0x7f051a3d7650:c=0,e=144,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=4,plh=2538033465,tim=372490287732

FETCH #0x7f051a3d7650:c=0,e=290,p=0,cr=14,cu=0,mis=0,r=0,dep=1,og=4,plh=2538033465,tim=372490288109

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 52 FileOperation=2 fileno=0 filetype=36 obj#=402 tim=372490288373

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 17 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490288577

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 3 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490288655

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 690 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490289365

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 6 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490289470

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 445 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490289934

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 3 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490289983

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 375 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490290374

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 6 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490290453

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 367 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490290839

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 3 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490290882

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 355 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291252

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 4 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291298

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 276 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291590

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 1 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291614

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 256 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291879

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 2 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490291903

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 261 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490292172

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 30 FileOperation=3 fileno=0 filetype=36 obj#=402 tim=372490292225

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 934 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293171

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 3 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293208

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 245 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293465

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 262 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293755

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 1 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490293780

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 250 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490294039

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 256 FileOperation=8 fileno=1 filetype=36 obj#=402 tim=372490294323

WAIT #0x7f051bd5f870: nam='Disk file operations I/O' ela= 8 FileOperation=5 fileno=0 filetype=36 obj#=402 tim=372490294359

2016-06-15T11:36:00.055196+08:00

Incident 29074 created, dump file: /u01/app/oracle/diag/rdbms/orcl12c2/orcl12c2/incident/incdir_29074/orcl12c2_ora_40547_i29074.trc

ORA-00600: internal error code, arguments: [17090], [], [], [], [], [], [], [], [], [], [], []

从中可以判断出来是由于CDB$ROOT的未增加tempfile导致

SQL> alter tablespace temp add tempfile '/u01/app/oracle/oradata/orcl12c2/temp01.dbf' reuse;

Tablespace altered.

SQL> alter PLUGGABLE database pdb2 open;

Pluggable database altered.

查看数据库恢复情况

SQL> show pdbs;

CON_ID CON_NAME OPEN MODE RESTRICTED

---------- ------------------------------ ---------- ----------

2 PDB$SEED READ ONLY NO

3 PDB1 READ WRITE NO

4 PDB2 READ WRITE NO

SQL> select con_id,file#,checkpoint_change#,resetlogs_change# from v$datafile_header;

CON_ID FILE# CHECKPOINT_CHANGE# RESETLOGS_CHANGE#

---------- ---------- ------------------ -----------------

1 1 2500167 1500164

1 3 2500167 1500164

1 4 2500167 1500164

2 5 1371280 1341067

2 6 1371280 1341067

1 7 2500167 1500164

2 8 1371280 1341067

3 9 2501017 1500164

3 10 2501017 1500164

3 11 2501017 1500164

3 12 2501017 1500164

4 13 2502748 1500164

4 14 2502748 1500164

4 15 2502748 1500164

4 16 2502748 1500164

15 rows selected.

至此基本上测试完成在在cdb环境中丢失redo的恢复。在生产中，需要把temp加全，并且建议重建数据库