环境：Oracle RAC(GI 11.2.0.4 + DB 11.2.0.4)
本文应用补丁信息：
Patch 23615403 - Combo of OJVM Component 11.2.0.4.160719 DB PSU + GI PSU 11.2.0.4.160719 (Jul2016)

本文主要演示使用opatch auto自动应用补丁的过程。

• 1.更新OPatch版本
• 2.配置ocm.rsp文件
• 3.使用opatch auto应用补丁
• 4.数据库修改

1.更新OPatch版本

在所有节点的GI HOME和DB HOME下，都更新为符合应用补丁需求的OPatch版本。
将之前OPatch备份，解压新版本的OPatch到$ORACLE_HOME目录下，最后检查OPatch版本。

--更新OPatch版本 @all nodes including GI HOME and DB HOME
mv OPatch OPatch_bak
unzip p6880880_112000_Linux-x86-64.zip -d $ORACLE_HOME
$ORACLE_HOME/OPatch/opatch version

2.配置ocm.rsp文件

在各个节点的grid用户下OPatch下，配置ocm.rsp文件。

--生成ocm响应文件 @all nodes
[grid@jyrac2 ~]$ cd /opt/app/11.2.0/grid/OPatch/ocm/bin/
[grid@jyrac2 bin]$ ls
emocmrsp
[grid@jyrac2 bin]$ ./emocmrsp
OCM Installation Response Generator 10.3.7.0.0 - Production
Copyright (c) 2005, 2012, Oracle and/or its affiliates.  All rights reserved.Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name: You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  y
The OCM configuration response file (ocm.rsp) was successfully created.
[grid@jyrac2 bin]$ ls
emocmrsp  ocm.rsp
[grid@jyrac2 bin]$ pwd
/opt/app/11.2.0/grid/OPatch/ocm/bin

3.使用opatch auto应用补丁

使用opatch auto应用补丁，具体需要参照补丁包中附带的readme来操作。
根据readme中描述，对于GI HOME和DB HOME都不共享的情况下，使用root用户在每个节点执行opatch auto的具体命令即可实现自动应用补丁。

Case 1: GI Home and the Database Homes that are not shared and ACFS file system is not configured.
As root user, execute the following command on each node of the cluster:
opatch auto UNZIPPED_PATCH_LOCATION/23615403 -ocmrf ocm response file

先确认补丁包的解压目录（各节点保持一致，方便操作）：

[grid@jyrac2 bin]$ cd /opt/app/media/
[grid@jyrac2 media]$ ls
23615403  p23615403_112040_Linux-x86-64.zip  PatchSearch.xml

然后使用opatch auto应用补丁，
--opatch auto @all nodes
[root@jyrac1 23615403]# /opt/app/11.2.0/grid/OPatch/opatch auto /opt/app/media/23615403 -ocmrf /opt/app/11.2.0/grid/OPatch/ocm/bin/ocm.rsp
正常应用补丁的输出类似下面这样：

[root@jyrac1 23615403]# /opt/app/11.2.0/grid/OPatch/opatch auto /opt/app/media/23615403 -ocmrf /opt/app/11.2.0/grid/OPatch/ocm/bin/ocm.rsp
Executing /opt/app/11.2.0/grid/perl/bin/perl /opt/app/11.2.0/grid/OPatch/crs/patch11203.pl -patchdir /opt/app/media -patchn 23615403 -ocmrf /opt/app/11.2.0/grid/OPatch/ocm/bin/ocm.rsp -paramfile /opt/app/11.2.0/grid/crs/install/crsconfig_paramsThis is the main log file: /opt/app/11.2.0/grid/cfgtoollogs/opatchauto2017-06-28_04-40-15.logThis file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/opt/app/11.2.0/grid/cfgtoollogs/opatchauto2017-06-28_04-40-15.report.log2017-06-28 04:40:15: Starting Clusterware Patch Setup
Using configuration parameter file: /opt/app/11.2.0/grid/crs/install/crsconfig_paramsStopping RAC /opt/app/oracle/product/11.2.0/dbhome_1 ...
Stopped RAC /opt/app/oracle/product/11.2.0/dbhome_1 successfullypatch /opt/app/media/23615403/23054359  apply successful for home  /opt/app/oracle/product/11.2.0/dbhome_1
patch /opt/app/media/23615403/23054319/custom/server/23054319  apply successful for home  /opt/app/oracle/product/11.2.0/dbhome_1
patch /opt/app/media/23615403/23177551  apply successful for home  /opt/app/oracle/product/11.2.0/dbhome_1 Stopping CRS...
Stopped CRS successfullypatch /opt/app/media/23615403/23054359  apply successful for home  /opt/app/11.2.0/grid
patch /opt/app/media/23615403/23054319  apply successful for home  /opt/app/11.2.0/grid
patch /opt/app/media/23615403/22502505  apply successful for home  /opt/app/11.2.0/grid Starting CRS...
Installing Trace File Analyzer
CRS-4123: Oracle High Availability Services has been started.Starting RAC /opt/app/oracle/product/11.2.0/dbhome_1 ...
Started RAC /opt/app/oracle/product/11.2.0/dbhome_1 successfullyopatch auto succeeded.

这个opatch auto的过程会自动关闭数据库和集群，打完补丁之后自动启动集群和数据库。
同样，第一个节点成功后，在第二个节点应用补丁，输出基本一样，不再演示。

4.数据库修改

最后就是根据readme具体说明，在数据库执行：

cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle.sql psu apply
SQL> QUITcd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> @utlrp.sqlCheck the following log files in $ORACLE_BASE/cfgtoollogs/catbundle for any errors:catbundle_PSU_<database SID>_APPLY_<TIMESTAMP>.log
catbundle_PSU_<database SID>_GENERATE_<TIMESTAMP>.logSQL > @dbmsjdev.sql
SQL > exec dbms_java_dev.disable

最后，可以通过 select * from dba_registry_history; 查询一下补丁应用的情况：

SQL> set lines 1000
SQL> select * from dba_registry_history;ACTION_TIME                                                                 ACTION                         NAMESPACE                      VERSION                                ID BUNDLE_SERIES                  COMMENTS
--------------------------------------------------------------------------- ------------------------------ ------------------------------ ------------------------------ ---------- ------------------------------ ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
24-AUG-13 12.03.45.119862 PM                                                APPLY                          SERVER                         11.2.0.4                                0 PSU                            Patchset 11.2.0.2.0
16-AUG-16 03.26.07.674544 AM                                                APPLY                          SERVER                         11.2.0.4                                0 PSU                            Patchset 11.2.0.2.0
28-JUN-17 07.54.40.436948 AM                                                jvmpsu.sql                     SERVER                         11.2.0.4.160719OJVMPSU                  0                                RAN jvmpsu.sql
28-JUN-17 07.55.44.682881 AM                                                APPLY                          SERVER                         11.2.0.4                           160719 PSU                            PSU 11.2.0.4.1607194 rows selected.

总的来说，这个opatch auto不出问题的情况下，操作上要比手工打补丁简单的多。
若是想更可控的手工打补丁的话，可以参考之前的文章：《Oracle 11g RAC 应用补丁简明版》