案例：ORA-04031 12.1.0.2 on exadata x7
2024-04-15 05:36:20
点击蓝字 关注我们
某银行客户近期频繁出现ORA-04031，报错如下:
[TOC00000]
Jump to table of contents
Dump continued from file: /u01/app/oracle/diag/rdbms/coredb/coredb1/trace/coredb1_ora_291223.trc
[TOC00001]
ORA-04031: unable to allocate 920 bytes of shared memory ("shared pool","SELECTACC...","SQLA^559c65a1","qeeRwo: qeeCreateRwo")
[TOC00001-END]此次报错的subheap为SQLA^559c65a1，通过trace查看该子堆与父堆的信息。
******************************************************
HEAP DUMP heap name="sga heap" desc=0x600013d0extent sz=0x4d3a8 alt=272 het=32767 rec=9 flg=130 opc=0parent=(nil) owner=(nil) nex=(nil) xsz=0x1 heap=(nil)fl2=0x60, nex=(nil) pdb id=0ds for latch 1: 0x600884c8 ds for latch 2: 0x60092218 ds for latch 3: 0x6009bf68 ds for latch 4: 0x600a5cb8 ds for latch 5: 0x600afa08 ds for latch 6: 0x600b9758 ds for latch 7: 0x600c34a8 reserved granule count 0 (granule size 33554432)
******************************************************
HEAP DUMP heap name="SQLA^559c65a1" desc=0x192d7c9b00extent sz=0xfe8 alt=32767 het=456 rec=0 flg=2 opc=2parent=0x600013d0 owner=0x192d7c99b8 nex=(nil) xsz=0x2000000 heap=(nil)fl2=0x27, nex=(nil), dsxvers=1, dsxflg=0x0dsx first ext=0x1a0df95090dsx empty ext bytes=0 subheap rc link=0x1a0df95100,0x1a0df95100pdb id=0Subheap has 85360 bytes of memory allocated可以看到该子堆已经分配了85360 bytes，extent大小为4072。
根据chunk的分配原则，首先尝试在子堆中分配，如果子堆free list无法分配，则向父堆申请。这时申请的chunk大小将会是extent size，并作为一个新的extent挂在子堆下。
查看子堆的free list如下：
HEAP DUMP heap name="SQLA^559c65a1" desc=0x192d7c9b00extent sz=0xfe8 alt=32767 het=456 rec=0 flg=2 opc=2parent=0x600013d0 owner=0x192d7c99b8 nex=(nil) xsz=0x2000000 heap=(nil)fl2=0x27, nex=(nil), dsxvers=1, dsxflg=0x0dsx first ext=0x1a0df95090dsx empty ext bytes=0 subheap rc link=0x1a0df95100,0x1a0df95100pdb id=0
EXTENT 0 addr=0x1933b5dfc8Chunk 1933b5dfd8 sz= 704 free " "
Dump of memory from 0x0000001933B5DFD8 to 0x0000001933B5E298
1933B5DFD0 000002C1 C0B38F00 [........]
1933B5DFE0 00000000 00000000 2D7C9BD8 00000019 [..........|-....]
1933B5DFF0 2D7C9BD8 00000019 01010101 00000000 [..|-............]
1933B5E000 00000000 00000000 0EB447B8 00000000 [.........G......]
1933B5E010 00000000 00000070 00000001 C0B38F00 [....p...........]
1933B5E020 00000000 00000000 33B5E948 00000019 [........H..3....]
1933B5E030 2F36F320 00000019 00000001 00000000 [ .6/............]
1933B5E040 33B5DFE0 00000019 00000000 00000000 [...3............]
1933B5E050 33B5E050 00000019 33B5E050 00000019 [P..3....P..3....]
1933B5E060 000008D9 40B38F00 33B5DFF0 00000019 [.......@...3....]
1933B5E070 33B5DFF0 00000019 0EBBECFC 00000000 [...3............]
1933B5E080 000002A0 00000328 9DF614B8 00000019 [....(...........]
1933B5E090 33B5E200 00000019 00000000 00000000 [...3............]
1933B5E0A0 2F36F258 00000019 00000000 00000000 [X.6/............]
1933B5E0B0 00000000 00000000 00000000 00000000 [................]
1933B5E0C0 33B5E150 00000019 00000000 00000000 [P..3............]
1933B5E0D0 00000000 00000000 00000000 00000000 [................]Repeat 7 times
1933B5E150 33B5E300 00000019 00000000 00000000 [...3............]
1933B5E160 00000000 00040340 00000000 00000000 [....@...........]
1933B5E170 00000000 00000000 33B5E088 00000019 [...........3....]
1933B5E180 00000000 00000000 00000004 00000000 [................]
1933B5E190 00000000 00000000 D4881520 0000001A [........ .......]
1933B5E1A0 00000006 00000010 00000000 00000000 [................]
1933B5E1B0 00000000 00000000 00000000 00000000 [................]
1933B5E1C0 000018CA FFFFFFFF 00000000 00000000 [................]
1933B5E1D0 00000000 00000000 00000000 00000000 [................]Repeat 1 times
1933B5E1F0 8938A048 161BEF84 ECDBBD88 2D39B55E [H.8.........^.9-]
1933B5E200 33B5E2A0 00000019 00000010 00000000 [...3............]
1933B5E210 00000004 00000000 00000000 00000000 [................]
1933B5E220 00000000 00000000 00000000 00000000 [................]Repeat 6 times
1933B5E290 00000000 00000000 [........] Chunk 1933b5e298 sz= 928 freeable "qeeRwo: qeeCrea"
...
FREE LISTS:Bucket 0 size=24Chunk 1a0df950c8 sz= 0 kghdsxBucket 1 size=88Bucket 2 size=152Bucket 3 size=280Bucket 4 size=536Chunk 1933b5dfd8 sz= 704 free " "
Dump of memory from 0x0000001933B5DFD8 to 0x0000001933B5E298
1933B5DFD0 000002C1 C0B38F00 [........]
1933B5DFE0 00000000 00000000 2D7C9BD8 00000019 [..........|-....]
1933B5DFF0 2D7C9BD8 00000019 01010101 00000000 [..|-............]
1933B5E000 00000000 00000000 0EB447B8 00000000 [.........G......]
1933B5E010 00000000 00000070 00000001 C0B38F00 [....p...........]
1933B5E020 00000000 00000000 33B5E948 00000019 [........H..3....]
1933B5E030 2F36F320 00000019 00000001 00000000 [ .6/............]
1933B5E040 33B5DFE0 00000019 00000000 00000000 [...3............]
1933B5E050 33B5E050 00000019 33B5E050 00000019 [P..3....P..3....]
1933B5E060 000008D9 40B38F00 33B5DFF0 00000019 [.......@...3....]
1933B5E070 33B5DFF0 00000019 0EBBECFC 00000000 [...3............]
1933B5E080 000002A0 00000328 9DF614B8 00000019 [....(...........]
1933B5E090 33B5E200 00000019 00000000 00000000 [...3............]
1933B5E0A0 2F36F258 00000019 00000000 00000000 [X.6/............]
1933B5E0B0 00000000 00000000 00000000 00000000 [................]
1933B5E0C0 33B5E150 00000019 00000000 00000000 [P..3............]
1933B5E0D0 00000000 00000000 00000000 00000000 [................]Repeat 7 times
1933B5E150 33B5E300 00000019 00000000 00000000 [...3............]
1933B5E160 00000000 00040340 00000000 00000000 [....@...........]
1933B5E170 00000000 00000000 33B5E088 00000019 [...........3....]
1933B5E180 00000000 00000000 00000004 00000000 [................]
1933B5E190 00000000 00000000 D4881520 0000001A [........ .......]
1933B5E1A0 00000006 00000010 00000000 00000000 [................]
1933B5E1B0 00000000 00000000 00000000 00000000 [................]
1933B5E1C0 000018CA FFFFFFFF 00000000 00000000 [................]
1933B5E1D0 00000000 00000000 00000000 00000000 [................]Repeat 1 times
1933B5E1F0 8938A048 161BEF84 ECDBBD88 2D39B55E [H.8.........^.9-]
1933B5E200 33B5E2A0 00000019 00000010 00000000 [...3............]
1933B5E210 00000004 00000000 00000000 00000000 [................]
1933B5E220 00000000 00000000 00000000 00000000 [................]Repeat 6 times
1933B5E290 00000000 00000000 [........] Bucket 5 size=1048Bucket 6 size=2072Bucket 7 size=4120Bucket 8 size=4144Bucket 9 size=4168Bucket 10 size=4192Bucket 11 size=4216
Total free space = 704请求940字节，但是子堆free list只有704字节，所以这时会向父堆申请一个子堆extent size的chunk，即4072字节，但是trace中并未发现父堆的详细信息。
Did Not dump SGA Heap desc=0x600013d0. Current time: 10/26/2021 04:13:50Last SGA heap dump at 10/26/2021 04:13:38SGA Heap Dump Count = 2查看subpool的top组件：
==============================================
TOP 10 MEMORY USES FOR SGA HEAP SUB POOL 1
----------------------------------------------
"ksfqpn " 430 MB 30%
"gcs resources " 375 MB 26%
"gcs shadows " 207 MB 14%
"db_block_hash_buckets " 100 MB 7%
"free memory " 93 MB 6%
"init_heap_kfsg " 48 MB 3%
"dbktb: trace buffer " 15 MB 1%
"ASH buffers " 14 MB 1%
"object queue hash buckets " 14 MB 1%
"Checkpoint queue " 13 MB 1%
==============================================
TOP 10 MAXIMUM MEMORY USES FOR SGA HEAP SUB POOL 1
----------------------------------------------
"ksfqpn " 430 MB 26%
"gcs resources " 375 MB 23%
"gcs shadows " 207 MB 13%
"SQLA " 132 MB 8%
"free memory " 114 MB 7%
"db_block_hash_buckets " 100 MB 6%
"init_heap_kfsg " 48 MB 3%
"KGLH0 " 46 MB 3%
"dbktb: trace buffer " 15 MB 1%
"ASH buffers " 14 MB 1%-----------------------------------------
free memory 207 MB
memory alloc. 2880 MB
Sub total 3086 MB
==============================================
TOP 10 MEMORY USES FOR SGA HEAP SUB POOL 2
----------------------------------------------
"ksfqpn " 430 MB 29%
"gcs resources " 375 MB 25%
"gcs shadows " 207 MB 14%
"db_block_hash_buckets " 102 MB 7%
"free memory " 92 MB 6%
"SQLA " 36 MB 2%
"FileOpenBlock " 24 MB 2%
"KGLH0 " 20 MB 1%
"gcs res hash bucket " 16 MB 1%
"dbktb: trace buffer " 15 MB 1%
==============================================
TOP 10 MAXIMUM MEMORY USES FOR SGA HEAP SUB POOL 2
----------------------------------------------
"ksfqpn " 430 MB 26%
"gcs resources " 375 MB 22%
"gcs shadows " 207 MB 12%
"SQLA " 168 MB 10%
"free memory " 114 MB 7%
"db_block_hash_buckets " 102 MB 6%
"KGLH0 " 52 MB 3%
"FileOpenBlock " 24 MB 1%
"gcs res hash bucket " 16 MB 1%
"dbktb: trace buffer " 15 MB 1%-----------------------------------------
free memory 206 MB
memory alloc. 2944 MB
Sub total 3150 MB
==============================================
TOP 10 MEMORY USES FOR SGA HEAP SUB POOL 3
----------------------------------------------
"ksfqpn " 430 MB 29%
"gcs resources " 375 MB 25%
"gcs shadows " 209 MB 14%
"free memory " 109 MB 7%
"db_block_hash_buckets " 101 MB 7%
"SQLA " 20 MB 1%
"gc name table " 18 MB 1%
"gcs res hash bucket " 16 MB 1%
"dbktb: trace buffer " 15 MB 1%
"KGLH0 " 14 MB 1%
==============================================
TOP 10 MAXIMUM MEMORY USES FOR SGA HEAP SUB POOL 3
----------------------------------------------
"ksfqpn " 430 MB 26%
"gcs resources " 375 MB 22%
"gcs shadows " 209 MB 12%
"free memory " 136 MB 8%
"SQLA " 133 MB 8%
"db_block_hash_buckets " 101 MB 6%
"KGLH0 " 47 MB 3%
"KQR L PO " 23 MB 1%
"gc name table " 18 MB 1%
"gcs res hash bucket " 16 MB 1%-----------------------------------------
free memory 245 MB
memory alloc. 2908 MB
Sub total 3153 MB
==============================================
TOP 10 MEMORY USES FOR SGA HEAP SUB POOL 4
----------------------------------------------
"ksfqpn " 430 MB 29%
"gcs resources " 375 MB 25%
"gcs shadows " 210 MB 14%
"free memory " 111 MB 7%
"db_block_hash_buckets " 100 MB 7%
"SQLA " 44 MB 3%
"KGLH0 " 22 MB 1%
"gcs res hash bucket " 16 MB 1%
"dbktb: trace buffer " 15 MB 1%
"ASH buffers " 14 MB 1%
==============================================
TOP 10 MAXIMUM MEMORY USES FOR SGA HEAP SUB POOL 4
----------------------------------------------
"ksfqpn " 430 MB 25%
"gcs resources " 375 MB 22%
"gcs shadows " 210 MB 12%
"SQLA " 176 MB 10%
"free memory " 131 MB 8%
"db_block_hash_buckets " 100 MB 6%
"KGLH0 " 56 MB 3%
"gcs res hash bucket " 16 MB 1%
"dbktb: trace buffer " 15 MB 1%
"ASH buffers " 14 MB 1%-----------------------------------------
free memory 242 MB
memory alloc. 2971 MB
Sub total 3213 MB
==============================================
TOP 10 MEMORY USES FOR SGA HEAP SUB POOL 5
----------------------------------------------
"ksfqpn " 430 MB 30%
"gcs resources " 375 MB 26%
"gcs shadows " 210 MB 15%
"db_block_hash_buckets " 101 MB 7%
"free memory " 96 MB 7%
"SQLA " 17 MB 1%
"KGLH0 " 15 MB 1%
"KQR X SO " 15 MB 1%
"dbktb: trace buffer " 15 MB 1%
"ASH buffers " 14 MB 1%
==============================================
TOP 10 MAXIMUM MEMORY USES FOR SGA HEAP SUB POOL 5
----------------------------------------------
"ksfqpn " 430 MB 26%
"gcs resources " 375 MB 23%
"gcs shadows " 210 MB 13%
"SQLA " 129 MB 8%
"free memory " 115 MB 7%
"db_block_hash_buckets " 101 MB 6%
"KGLH0 " 46 MB 3%
"KQR X SO " 38 MB 2%
"KQR X PO " 18 MB 1%
"dbktb: trace buffer " 15 MB 1%-----------------------------------------
free memory 211 MB
memory alloc. 2879 MB
Sub total 3090 MB
==============================================
TOP 10 MEMORY USES FOR SGA HEAP SUB POOL 6
----------------------------------------------
"ksfqpn " 430 MB 30%
"gcs resources " 375 MB 26%
"gcs shadows " 208 MB 14%
"db_block_hash_buckets " 100 MB 7%
"free memory " 92 MB 6%
"keomg: entry list " 36 MB 3%
"ges big msg buffers " 30 MB 2%
"dbktb: trace buffer " 15 MB 1%
"ASH buffers " 14 MB 1%
"object queue hash buckets " 14 MB 1%
==============================================
TOP 10 MAXIMUM MEMORY USES FOR SGA HEAP SUB POOL 6
----------------------------------------------
"ksfqpn " 430 MB 23%
"gcs resources " 375 MB 20%
"free memory " 233 MB 12%
"gcs shadows " 208 MB 11%
"PMR sga heap " 134 MB 7%
"SQLA " 128 MB 7%
"db_block_hash_buckets " 100 MB 5%
"KGLH0 " 47 MB 2%
"keomg: entry list " 36 MB 2%
"ges big msg buffers " 30 MB 2%-----------------------------------------
free memory 325 MB
memory alloc. 3021 MB
Sub total 3346 MB
==============================================
TOP 10 MEMORY USES FOR SGA HEAP SUB POOL 7
----------------------------------------------
"ksfqpn " 430 MB 29%
"gcs resources " 375 MB 25%
"gcs shadows " 207 MB 14%
"db_block_hash_buckets " 101 MB 7%
"free memory " 94 MB 6%
"FileOpenBlock " 24 MB 2%
"KQR L PO " 18 MB 1%
"SQLA " 16 MB 1%
"dbktb: trace buffer " 15 MB 1%
"ASH buffers " 14 MB 1%
==============================================
TOP 10 MAXIMUM MEMORY USES FOR SGA HEAP SUB POOL 7
----------------------------------------------
"ksfqpn " 430 MB 23%
"gcs resources " 375 MB 20%
"gcs shadows " 207 MB 11%
"free memory " 204 MB 11%
"SQLA " 138 MB 7%
"PMR sga heap " 134 MB 7%
"db_block_hash_buckets " 101 MB 5%
"KGLH0 " 47 MB 2%
"FileOpenBlock " 24 MB 1%
"KQR L PO " 18 MB 1%-----------------------------------------
free memory 298 MB
memory alloc. 3069 MB
Sub total 3367 MB
TOTALS ---------------------------------------
Total free memory 1734 MB
Total memory alloc. 20 GB
Grand total 22 GB发现组件ksfqpn的内存占用异常，gcs resources和gcs shadows也占用偏高。
ksfqpn与IO接口相关，ksfq – kernel service functions sequential file io interface，通过查阅MOS文档发现与Bug 25058080 Excessive allocation of memory “ksfqpn” in SGA using RMAN backups in Exadata匹配。
其bug主要原因是内部算法导致ksfqpn内存重用和回收出现了异常，导致ksfqpn大小是x$ksfgp的1000倍以上。
关于作者
李翔宇，云和恩墨西区交付技术顾问，长期服务移动运营商行业客户，熟悉Oracle性能优化，故障诊断，特殊恢复。
END
推荐阅读：2021数据技术嘉年华视频回放及PPT下载
2021数据技术嘉年华50余个PPT下载、视频回放已上传墨天轮平台，可在“数据和云”公众号回复关键词“2021DTC”获得！
你知道吗？我们的视频号里已经发布了很多精彩的内容，快去看看吧！↓↓↓
点击下图查看更多 ↓
云和恩墨大讲堂 | 一个分享交流的地方
长按，识别二维码，加入万人交流社群
请备注：云和恩墨大讲堂
点个“在看”
你的喜欢会被看到❤
案例：ORA-04031 12.1.0.2 on exadata x7相关推荐
- ORA-12012 Error on auto execute of job SYS.ORA$AT_OS_OPT_SY_NN in 12.2.0 Database
报错如下: ORA-12012: error on auto execute of job "SYS"."ORA$AT_OS_OPT_SY_88" ORA-20 ...
- oracle 12.2.0.1 搭建 active dataguard
os: centos 7.4 database:12.2.0.1 + dbf 本次是以 oracle database 12.2.0.1 + dbf 的形式部署的,后面会记录 rac + asm 的形 ...
- Oracle12081,【Oracle介质】Oracle 12C Linux x86-64 最新OPatch patch 6880880 12.2.0.1.7
天萃荷净 Linux x86-64 补丁程序6880880: OPatch patch of version 12.2.0.1.7 for Oracle software releases 12.1. ...
- 手动升级11.2.0.3到12.2.0.1
手动升级11.2.0.3到12.2.0.1 参考资料: 1.如何下载并运行Oracle数据库预升级实用程序 (文档 ID 1577379.1) 2.https://docs.oracle.com/en ...
- 在Docker中使用Oracle 18c（12.2.0.2）
文章目录 一.从Docker Hub或阿里云镜像下载小麦苗上传的镜像文件 二.创建容器并启动数据库 三.尽情使用吧 3.1 数据库使用 3.2 创建数据库 3.3 EMDE的使用 3.4 使用ssh连 ...
- 【EMCC】 12.1.0.5 OEM server agent 安装配置及监控MySQL数据库
[EMCC]Oracle Enterprise Manager Cloud Control 12.1.0.5 OEM server agent 安装配置及监控MySQL数据库 原文: https:// ...
- RedHat 7.3 Oracle 12.2.0.1 RAC 安装手册(转)
1 准备工作 1.1 关于GRID的一些变化 1.1.1 简化的基于映像的Oracle Grid Infrastructure安装 从Oracle Grid Infrastructure 12 ...
- Oracle 12c(12.1.0.5) oem agent silent install(静默安装agent)
注释: 文章来自oracle support 文档 ID 1360083.1,静默安装agent采用的是把OMS服务端(即oem server端)的agent用压缩包download,远程传到agen ...
- oracle数据库gold,Oracle数据库之Oracle GoldenGate 12.2.0.1 安装、升级和删除
本文主要向大家介绍了Oracle数据库之Oracle GoldenGate 12.2.0.1 安装.升级和删除,通过具体的内容向大家展现,希望对大家学习Oracle数据库有所帮助. 1.准备工作 选择 ...
最新文章
- 机器学习01-定义、线性回归、梯度下降
- java i 原子_为什么i ++不是原子的？
- terminate和quit导致串口资源被占用
- IntelliJ IDEA 2020.2.4款 神级超级牛逼插件推荐（自用，真的超级牛逼）
- Linux性能监控工具:GTOP部署实录
- python 提升效率_@Python 程序员，如何最大化提升编码效率？
- 先天性脑部发育异常_儿童脑发育不良的3大“诱因”，若孩子出现这5种异常，要警惕...
- 横空出世！IDEA画图神器来了，比Visio快10倍！
- 【英语学习】【Daily English】U05 Places L01 How can I get to the city museum?
- executeQuery()方法
- https://gogs.io/
- VMware虚拟机中 启动Windows XP系统黑屏 的解决
- 摘：Windows系统内存计数器理解解析_备忘录_51Testing软件测试网...
- 根据前序序列创建二叉树
- python化学公式配平_配平化学方程式的C++代码实现
- 论“无常，苦、无我”
- php邮件回复功能,用PHP回复邮件（Laravel Mailgun）
- 计算机硬盘坏的原因,硬盘损坏的原因有哪些 硬盘正确使用方法介绍
- 2015实习准备之C/C++篇（未完待续）
- java 公交管理系统 代码_Java学员作品-城市公交管理系统
热门文章
- python随机生成奇数_python实现按照给定范围随机生成小数，偶数，奇数，整数，正态分布，均匀分布数据以及图片...
- 开发之前需要哪些文档_为什么在开发之前总是应该做文档
- 开源erp_大公司为何使用开源ERP
- 样条 开源_成年人在开源社区中是什么样的？
- editor编辑器为什么头部信息会不见_简单聊一聊一个前端编辑器的性能优化
- java servlet 配置_servlet与javabean配置
- # 电脑管家_如何彻底关闭电脑上的“恶心小广告”？联想电脑管家一步到位！...
- java hibernate 分页取数_java – Hibernate分页机制
- macos 编译php,TODO：macOS编译PHP7.1
- matlab实验4图形的绘制,MATLAB编程与应用实验报告（三维图形绘制）