一:Voting Disk

Voting Disk 这个文件主要用于记录节点成员状态，在出现脑裂时，决定那个Partion获得控制权，其他的Partion必须从集群中剔除。Voting disk使用的是一种“多数可用算法”，如果有多个Voting disk，则必须一半以上的Votedisk同时使用，Clusterware才能正常使用。 比如配置了4个Votedisk，坏一个Votedisk，集群可以正常工作，如果坏了2个，则不能满足半数以上，集群会立即宕掉，

所有节点立即重启，所以如果添加Votedisk，尽量不要只添加一个，而应该添加2个。这点和OCR 不一样。OCR 只需配置一个。

1.1查看votedisk的位置：

[root@jy1 ~]# cd u01/app/oracle/product/10.2.0/crs/bin

[root@jy1 bin]# ./crsctl query css votedisk

0. 0 /dev/raw/raw2

located 1 votedisk(s).

1.2备份votedisk盘：

[root@jy1 bin]# dd if=/dev/raw/raw2 of=/home/oracle/votedisk.bak

6291456+0 records in

6291456+0 records out

3221225472 bytes (3.2 GB) copied, 201.63 seconds, 16 MB/s

1.3通过Strings 命令来查看 voting disk 的内容

[root@jy1 bin]# strings /home/oracle/votedisk.bak | sort -u

fSLC

ssLckcoT

SslcLlik

sSlcrEp0

}|{z

1.4恢复votedisk盘：

[root@jy1 bin]# dd if=/home/oracle/votedisk.bak of=/dev/raw/raw2

6291456+0 records in

6291456+0 records out

3221225472 bytes (3.2 GB) copied, 201.63 seconds, 16 MB/s

二 :OCR

Oracle Clusterware把整个集群的配置信息放在共享存储上,这些信息包括了集群节点的列表,集群数据库实例到节点的映射以及CRS应用程序资源信息。存放的位置就在OCR Disk上. 在整个集群中，只有一个节点能对OCR Disk 进行读写操作，这个节点叫作Master Node，所有节点都会在内存中保留一份OCR的拷贝，同时有一个OCR Process 从这个内存中读取内容。 OCR 内容发生改变时，由Master Node的OCR Process负责同步到其他节点的OCR Process。

Oracle 每4个小时对其做一次备份，并且保留最后的3个备份，以及前一天，前一周的最后一个备份。 这个备份由Master Node CRSD进程完成，备份的默认位置是$CRS_HOME/crs/cdata/目录下，可以通过ocrconfig -backuploc 命令修改到新的目录。 每次备份后，备份文件名自动更改，以反应备份时间顺序，最近一次的备份

叫作backup00.ocr。这些备份文件除了保存在本地，DBA还应该在其他存储设备上保留一份，以防止意外的存储故障。

[root@jy1 crs]# pwd

/u01/app/oracle/product/10.2.0/crs/cdata/crs

[root@jy1 crs]# ls -lrt

total 12396

-rw-r--r-- 1 root root 4227072 Nov 17 14:45 backup00.ocr

-rw-r--r-- 1 root root 4227072 Nov 17 14:45 week.ocr

-rw-r--r-- 1 root root 4227072 Nov 17 14:45 day.ocr

在安装clusterware过程中，如果选择External Redundancy冗余方式，则只能输入一个OCR磁盘位置。 但是Oracle允许配置两个OCR 磁盘互为镜像，以防止OCR 磁盘的单点故障。 OCR 磁盘和Votedisk磁盘不一样，OCR磁盘最多只能有两个，一个Primary OCR 和一个Mirror OCR。

Oracle 推荐在对集群做调整时，比如增加，删除节点之前，修改RAC IP之前，对OCR做一个备份，可以使用export 备份到指定文件，如果做了replace或者restore 等操作，Oracle 建议使用 cluvfy comp ocr -n all 命令来做一次全面的检查。对OCR的备份与恢复，我们可以使用ocrconfig 命令。

[root@jy1 bin]# ./ocrconfig --help

Name:

ocrconfig - Configuration tool for Oracle Cluster Registry.

Synopsis:

ocrconfig [option]

option:

-export [-s online]

- Export cluster register contents to a file

-import - Import cluster registry contents from a file

-upgrade [ []]

- Upgrade cluster registry from previous version

-downgrade [-version ]

- Downgrade cluster registry to the specified version

-backuploc - Configure periodic backup location

-showbackup - Show backup information

-restore - Restore from physical backup

-replace ocr|ocrmirror [] - Add/replace/remove a OCR device/file

-overwrite - Overwrite OCR configuration on disk

-repair ocr|ocrmirror - Repair local OCR configuration

-help - Print out this help information

Note:

A log file will be created in

$ORACLE_HOME/log//client/ocrconfig_.log. Please ensure

you have file creation privileges in the above directory before

running this tool.

1. 用导出导入备份恢复OCR

1.1首先关闭所有节点的CRS

[root@jy1 bin]# ./crsctl stop crs

Stopping resources.

Successfully stopped CRS resources

Stopping CSSD.

Shutting down CSS daemon.

Shutdown request successfully issued.

[root@jy2 bin]# ./crsctl stop crs

Stopping resources.

Successfully stopped CRS resources

Stopping CSSD.

Shutting down CSS daemon.

Shutdown request successfully issued.

1.2用root 用户导出OCR内容

[root@jy1 bin]# ./ocrconfig -export /u01/ocrbak.exp

[root@jy1 bin]# ls -lrt /u01

total 96

drwxr-xr-x 3 root root 4096 Nov 10 23:12 app

drwxrwxrwx 6 root root 4096 Nov 11 11:54 tmp

-rw-r--r-- 1 root root 84375 Nov 17 16:52 ocrbak.exp

1.3重启CRS

[root@jy1 bin]# ./crsctl start crs

Attempting to start CRS stack

The CRS stack will be started shortly

[root@jy2 bin]# ./crsctl start crs

Attempting to start CRS stack

The CRS stack will be started shortly

1.4检查CRS 状态

Cannot communicate with EVM

[root@jy1 bin]# ./crsctl check crs

CSS appears healthy

CRS appears healthy

EVM appears healthy

[root@jy2 bin]# ./crsctl check crs

CSS appears healthy

CRS appears healthy

EVM appears healthy

[root@jy1 bin]# ./crs_stat -t

Name Type Target State Host

------------------------------------------------------------

ora....SM1.asm application ONLINE ONLINE jy1

ora....Y1.lsnr application ONLINE ONLINE jy1

ora.jy1.gsd application ONLINE ONLINE jy1

ora.jy1.ons application ONLINE ONLINE jy1

ora.jy1.vip application ONLINE ONLINE jy1

ora....SM2.asm application ONLINE ONLINE jy2

ora....Y2.lsnr application ONLINE ONLINE jy2

ora.jy2.gsd application ONLINE ONLINE jy2

ora.jy2.ons application ONLINE ONLINE jy2

ora.jy2.vip application ONLINE ONLINE jy2

ora.jyrac.db application ONLINE ONLINE jy2

ora....c1.inst application ONLINE ONLINE jy1

ora....c2.inst application ONLINE ONLINE jy2

1.5 检查OCR一致性

[root@jy1 bin]# ./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 2

Total space (kbytes) : 3145640

Used space (kbytes) : 3816

Available space (kbytes) : 3141824

ID : 1032702449

Device/File Name : /dev/raw/raw1

Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

1.6破坏OCR内容

[root@jy1 bin]# dd if=/dev/zero of=/dev/raw/raw1 bs=8192 count=1000

1000+0 records in

1000+0 records out

8192000 bytes (8.2 MB) copied, 0.355733 seconds, 23.0 MB/s

1.7再次检查OCR一致性

[root@jy1 bin]# ./ocrcheck

PROT-601: Failed to initialize ocrcheck

再来执行crs_stat -t命令就会发现crs已经终止了

[root@jy1 bin]# ./crs_stat -t

CRS-0184: Cannot communicate with the CRS daemon.

1.8使用cluvfy 工具检查一致性

[root@jy1 cluvfy]# su - oracle

[oracle@jy1 ~]$ cd /soft/clusterware/cluvfy

[oracle@jy1 ~]$ ./runcluvfy.sh comp ocr -n all

Verifying OCR integrity

Unable to retrieve nodelist from Oracle clusterware.

Verification cannot proceed.

1.9使用Import 恢复OCR 内容(使用restore选项只能导入OCR自动产生的物理备份， import选项只能导入通过export选项导出的的逻辑备份)

[root@jy1 bin]# ./ocrconfig -import /u01/ocrbak.exp

1.10 再次检查OCR

[root@jy1 bin]# ./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 2

Total space (kbytes) : 3145640

Used space (kbytes) : 3816

Available space (kbytes) : 3141824

ID : 1032702449

Device/File Name : /dev/raw/raw1

Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

1.11 使用cluvfy工具检查

[root@jy1 cluvfy]# su - oracle

[oracle@jy1 ~]$ cd /soft/clusterware/cluvfy

[oracle@jy1 cluvfy]$ ./runcluvfy.sh comp ocr -n all

Verifying OCR integrity

Checking OCR integrity...

Checking the absence of a non-clustered configuration...

All nodes free of non-clustered, local-only configurations.

Uniqueness check for OCR device passed.

Checking the version of OCR...

OCR of correct Version "2" exists.

Checking data integrity of OCR...

Data integrity check for OCR passed.

OCR integrity check passed.

Verification of OCR integrity was successful.

2使用自动备份恢复OCR

2.1关闭运行在集群数据库的所有节点上的CRS服务程序(在Oracle 11gR2 中已经没有了init.crs 命令了。 只能通过crsctl stop crs命令来关闭CRS.)

/etc/init.d/init.crs stop 或者crsctl stop crs

2.2 通过ocrconfig 的showbackup选项查看最近的备份

[root@jy1 bin]# /etc/init.d/init.crs stop

Shutting down Oracle Cluster Ready Services (CRS):

Stopping resources.

Successfully stopped CRS resources

Stopping CSSD.

Shutting down CSS daemon.

Shutdown request successfully issued.

Shutdown has begun. The daemons should exit soon.

[root@jy2 bin]# /etc/init.d/init.crs stop

Shutting down Oracle Cluster Ready Services (CRS):

Stopping resources.

Successfully stopped CRS resources

Stopping CSSD.

Shutting down CSS daemon.

Shutdown request successfully issued.

Shutdown has begun. The daemons should exit soon.

2.2通过ocrconfig 的showbackup选项查看最近的备份

[root@jy1 bin]# ./ocrconfig -showbackup

jy1 2014/11/17 14:45:54 /u01/app/oracle/product/10.2.0/crs/cdata/crs

jy1 2014/11/17 14:45:54 /u01/app/oracle/product/10.2.0/crs/cdata/crs

jy1 2014/11/17 14:45:54 /u01/app/oracle/product/10.2.0/crs/cdata/crs

[root@jy1 bin]# ls -lrt /u01/app/oracle/product/10.2.0/crs/cdata/crs

total 12396

-rw-r--r-- 1 root root 4227072 Nov 17 14:45 backup00.ocr

-rw-r--r-- 1 root root 4227072 Nov 17 14:45 week.ocr

-rw-r--r-- 1 root root 4227072 Nov 17 14:45 day.ocr

2.3破坏OCR内容

[root@jy1 bin]# dd if=/dev/zero of=/dev/raw/raw1 bs=8192 count=1000

1000+0 records in

1000+0 records out

8192000 bytes (8.2 MB) copied, 0.355733 seconds, 23.0 MB/s

2.4再次检查OCR一致性

[root@jy1 bin]# ./ocrcheck

PROT-601: Failed to initialize ocrcheck

再来执行crs_stat -t命令就会发现crs已经终止了

[root@jy1 bin]# ./crs_stat -t

CRS-0184: Cannot communicate with the CRS daemon.

2.5使用cluvfy 工具检查一致性

[root@jy1 cluvfy]# su - oracle

[oracle@jy1 ~]$ cd /soft/clusterware/cluvfy

[oracle@jy1 ~]$ ./runcluvfy.sh comp ocr -n all

Verifying OCR integrity

Unable to retrieve nodelist from Oracle clusterware.

Verification cannot proceed.

2.6通过ocrconfig的restore或import选项导入OCR数据(使用restore选项只能导入OCR自动产生的物理备份,import选项只能导入通过export选项导出的的逻辑备份)

ocrconfig -restore filename_location

[root@jy1 bin]# ./ocrconfig -restore /u01/app/oracle/product/10.2.0/crs/cdata/crs/backup00.ocr

2.7 检查CRS

[root@jy1 bin]# ./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 2

Total space (kbytes) : 3145640

Used space (kbytes) : 3816

Available space (kbytes) : 3141824

ID : 1387716561

Device/File Name : /dev/raw/raw1

Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

2.8 使用cluvfy工具检查

[root@jy1 cluvfy]# su - oracle

[oracle@jy1 ~]$ cd /soft/clusterware/cluvfy

[oracle@jy1 cluvfy]$ ./runcluvfy.sh comp ocr -n all

Verifying OCR integrity

Checking OCR integrity...

Checking the absence of a non-clustered configuration...

All nodes free of non-clustered, local-only configurations.

Uniqueness check for OCR device passed.

Checking the version of OCR...

OCR of correct Version "2" exists.

Checking data integrity of OCR...

Data integrity check for OCR passed.

OCR integrity check passed.

Verification of OCR integrity was successful.

2.9 在所有节点上重新启动CRS

/etc/init.d/init.crs start 而在Oracle 11gR2使用:crsctl start crs 命令来启动CRS.

[root@jy1 bin]# /etc/init.d/init.crs start

Startup will be queued to init within 90 seconds.

[root@jy2 bin]# /etc/init.d/init.crs start

Startup will be queued to init within 90 seconds.

[root@jy1 bin]# ./crs_stat -t

Name Type Target State Host

------------------------------------------------------------

ora....SM1.asm application ONLINE ONLINE jy1

ora....Y1.lsnr application ONLINE ONLINE jy1

ora.jy1.gsd application ONLINE ONLINE jy1

ora.jy1.ons application ONLINE ONLINE jy1

ora.jy1.vip application ONLINE ONLINE jy1

ora....SM2.asm application ONLINE ONLINE jy2

ora....Y2.lsnr application ONLINE ONLINE jy2

ora.jy2.gsd application ONLINE ONLINE jy2

ora.jy2.ons application ONLINE ONLINE jy2

ora.jy2.vip application ONLINE ONLINE jy2

ora.jyrac.db application ONLINE ONLINE jy1

ora....c1.inst application ONLINE ONLINE jy1

ora....c2.inst application ONLINE ONLINE jy2