你可能遇到过类似的数据库无法启动的问题,
postgres@digoal-> FATAL:  XX000: could not map anonymous shared memory: Cannot allocate memory
HINT:  This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory, swap space, or huge pages. To reduce the request size (currently 3322716160 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
LOCATION:  CreateAnonymousSegment, pg_shmem.c:398通过查看meminfo可以得到原因。
CommitLimit: Based on the overcommit ratio ('vm.overcommit_ratio'),this is the total amount of  memory currently available tobe allocated on the system. This limit is only adhered toif strict overcommit accounting is enabled (mode 2 in'vm.overcommit_memory').The CommitLimit is calculated with the following formula:CommitLimit = ([total RAM pages] - [total huge TLB pages]) *overcommit_ratio / 100 + [total swap pages]For example, on a system with 1G of physical RAM and 7Gof swap with a `vm.overcommit_ratio` of 30 it wouldyield a CommitLimit of 7.3G.For more details, see the memory overcommit documentationin vm/overcommit-accounting.
Committed_AS: The amount of memory presently allocated on the system.The committed memory is a sum of all of the memory whichhas been allocated by processes, even if it has not been"used" by them as of yet. A process which malloc()'s 1Gof memory, but only touches 300M of it will show up asusing 1G. This 1G is memory which has been "committed" toby the VM and can be used at any time by the allocatingapplication. With strict overcommit enabled on the system(mode 2 in 'vm.overcommit_memory'),allocations which wouldexceed the CommitLimit (detailed above) will not be permitted.This is useful if one needs to guarantee that processes willnot fail due to lack of memory once that memory has beensuccessfully allocated.依据vm.overcommit_memory设置的值,
当vm.overcommit_memory=0时,不允许普通用户overcommit, 但是允许root用户轻微的overcommit。
当vm.overcommit_memory=1时,允许overcommit.
当vm.overcommit_memory=2时,Committed_AS不能大于CommitLimit。
commit 限制 计算方法The CommitLimit is calculated with the following formula:CommitLimit = ([total RAM pages] - [total huge TLB pages]) *overcommit_ratio / 100 + [total swap pages]For example, on a system with 1G of physical RAM and 7Gof swap with a `vm.overcommit_ratio` of 30 it wouldyield a CommitLimit of 7.3G.
[root@digoal postgresql-9.4.4]# freetotal       used       free     shared    buffers     cached
Mem:       1914436     713976    1200460      72588      32384     529364
-/+ buffers/cache:     152228    1762208
Swap:      1048572     542080     506492
[root@digoal ~]# cat /proc/meminfo |grep Commit
CommitLimit:     2005788 kB
Committed_AS:     132384 kB
这个例子的2G就是以上公式计算得来。overcommit限制的初衷是malloc后,内存并不是立即使用掉,所以如果多个进程同时申请一批内存的话,不允许OVERCOMMIT可能导致某些进程申请内存失败,但实际上内存是还有的。所以Linux内核给出了几种选择,2是比较靠谱或者温柔的做法。1的话风险有点大,因为可能会导致OOM。所以当数据库无法启动时,要么你降低一下数据库申请内存的大小(例如降低shared_buffer或者max conn),要么就是修改一下overcommit的风格。[参考]
1. kernel-doc-2.6.32/Documentation/filesystems/proc.txtMemTotal: Total usable ram (i.e. physical ram minus a few reservedbits and the kernel binary code)MemFree: The sum of LowFree+HighFree
MemAvailable: An estimate of how much memory is available for starting newapplications, without swapping. Calculated from MemFree,SReclaimable, the size of the file LRU lists, and the lowwatermarks in each zone.The estimate takes into account that the system needs somepage cache to function well, and that not all reclaimableslab will be reclaimable, due to items being in use. Theimpact of those factors will vary from system to system.This line is only reported if sysctl vm.meminfo_legacy_layout = 0Buffers: Relatively temporary storage for raw disk blocksshouldn't get tremendously large (20MB or so)Cached: in-memory cache for files read from the disk (thepagecache).  Doesn't include SwapCachedSwapCached: Memory that once was swapped out, is swapped back in butstill also is in the swapfile (if memory is needed itdoesn't need to be swapped out AGAIN because it is alreadyin the swapfile. This saves I/O)Active: Memory that has been used more recently and usually notreclaimed unless absolutely necessary.Inactive: Memory which has been less recently used.  It is moreeligible to be reclaimed for other purposesHighTotal:HighFree: Highmem is all memory above ~860MB of physical memoryHighmem areas are for use by userspace programs, orfor the pagecache.  The kernel must use tricks to accessthis memory, making it slower to access than lowmem.LowTotal:LowFree: Lowmem is memory which can be used for everything thathighmem can be used for, but it is also available for thekernel's use for its own data structures.  Among manyother things, it is where everything from the Slab isallocated.  Bad things happen when you're out of lowmem.SwapTotal: total amount of swap space availableSwapFree: Memory which has been evicted from RAM, and is temporarilyon the diskDirty: Memory which is waiting to get written back to the diskWriteback: Memory which is actively being written back to the diskAnonPages: Non-file backed pages mapped into userspace page tables
AnonHugePages: Non-file backed huge pages mapped into userspace page tablesMapped: files which have been mmaped, such as librariesSlab: in-kernel data structures cache
SReclaimable: Part of Slab, that might be reclaimed, such as cachesSUnreclaim: Part of Slab, that cannot be reclaimed on memory pressurePageTables: amount of memory dedicated to the lowest level of pagetables.
NFS_Unstable: NFS pages sent to the server, but not yet committed to stablestorageBounce: Memory used for block device "bounce buffers"
WritebackTmp: Memory used by FUSE for temporary writeback buffersCommitLimit: Based on the overcommit ratio ('vm.overcommit_ratio'),this is the total amount of  memory currently available tobe allocated on the system. This limit is only adhered toif strict overcommit accounting is enabled (mode 2 in'vm.overcommit_memory').The CommitLimit is calculated with the following formula:CommitLimit = ([total RAM pages] - [total huge TLB pages]) *overcommit_ratio / 100 + [total swap pages]For example, on a system with 1G of physical RAM and 7Gof swap with a `vm.overcommit_ratio` of 30 it wouldyield a CommitLimit of 7.3G.For more details, see the memory overcommit documentationin vm/overcommit-accounting.
Committed_AS: The amount of memory presently allocated on the system.The committed memory is a sum of all of the memory whichhas been allocated by processes, even if it has not been"used" by them as of yet. A process which malloc()'s 1Gof memory, but only touches 300M of it will show up asusing 1G. This 1G is memory which has been "committed" toby the VM and can be used at any time by the allocatingapplication. With strict overcommit enabled on the system(mode 2 in 'vm.overcommit_memory'),allocations which wouldexceed the CommitLimit (detailed above) will not be permitted.This is useful if one needs to guarantee that processes willnot fail due to lack of memory once that memory has beensuccessfully allocated.
VmallocTotal: total size of vmalloc memory areaVmallocUsed: amount of vmalloc area which is used
VmallocChunk: largest contiguous block of vmalloc area which is free2. kernel-doc-2.6.32/Documentation/vm/overcommit-accounting
The Linux kernel supports the following overcommit handling modes0       -       Heuristic overcommit handling. Obvious overcommits ofaddress space are refused. Used for a typical system. Itensures a seriously wild allocation fails while allowingovercommit to reduce swap usage.  root is allowed to allocate slighly more memory in this mode. This is the default.1       -       Always overcommit. Appropriate for some scientificapplications.2       -       Don't overcommit. The total address space commitfor the system is not permitted to exceed swap + aconfigurable amount (default is 50%) of physical RAM.Depending on the amount you use, in most situationsthis means a process will not be killed while accessingpages but will receive errors on memory allocation asappropriate.The overcommit policy is set via the sysctl `vm.overcommit_memory'.The overcommit amount can be set via `vm.overcommit_ratio' (percentage)
or `vm.overcommit_kbytes' (absolute value).The current overcommit limit and amount committed are viewable in
/proc/meminfo as CommitLimit and Committed_AS respectively.Gotchas
-------The C language stack growth does an implicit mremap. If you want absolute
guarantees and run close to the edge you MUST mmap your stack for the
largest size you think you will need. For typical stack usage this does
not matter much but it's a corner case if you really really careIn mode 2 the MAP_NORESERVE flag is ignored. How It Works
------------The overcommit is based on the following rulesFor a file backed mapSHARED or READ-only     -       0 cost (the file is the map not swap)PRIVATE WRITABLE        -       size of mapping per instanceFor an anonymous or /dev/zero mapSHARED                  -       size of mappingPRIVATE READ-only       -       0 cost (but of little use)PRIVATE WRITABLE        -       size of mapping per instanceAdditional accountingPages made writable copies by mmapshmfs memory drawn from the same poolStatus
------o       We account mmap memory mappings
o       We account mprotect changes in commit
o       We account mremap changes in size
o       We account brk
o       We account munmap
o       We report the commit status in /proc
o       Account and check on fork
o       Review stack handling/building on exec
o       SHMfs accounting
o       Implement actual limit enforcementTo Do
-----
o       Account ptrace pages (this is hard)

PostgreSQL 内存OOM控制策略导致数据库无法启动的诊断一例(如何有效避免oom)相关推荐

  1. 又一例SPFILE设置错误导致数据库无法启动

    --======================================== --又一例SPFILE设置错误导致数据库无法启动 --============================== ...

  2. ORA-12547: TNS:lost contact导致数据库无法启动

    墨墨导读:一个诡异的案例:ORA-12547: TNS:lost contact导致数据库无法启动,甚至sqlplus都无法登录,让我们一一来解开这个案例的真面目. 1. 背景概述 某客户出现数据库无 ...

  3. oracle access表丢失,oracle 11.1.0.7-版本也会出现access$表丢失导致数据库无法启动

    oracle 11.1.0.7-版本也会出现access$表丢失导致数据库无法启动 发布时间:2017-06-23 20:10 来源:互联网 当前栏目:web技术类 下面我们来看看oracle 11. ...

  4. Oracle数据文件scn不一致,控制文件与数据文件头SCN不一致导致数据库无法启动故障处理...

    环境说明 OS操作系统:WINDOWS 2012 64位 数据库版 本:ORACLE 11.2.0.1 故障问题描述 客户反映数据库无法启动,报ORA-01589:要打开数据库必须使用RESETLOG ...

  5. oracle怎么启memory,修改memory内存参数,导致数据库启不来

    这几天新装完一个rac 11g的数据库,安装完成后修改memory内存参数. 重新启动时,遇到如下错误: 来自Oracle的官方解析是: Starting with Oracle Database 1 ...

  6. 缺少控制文件导致数据库无法启动

    在本人近期工作中,遇到了数据库无法正常启动的问题,以下是我对问题的整理和总结. 问题描述 1.启动数据库,报错 ORA-00205: error in identifying control file ...

  7. oracle闪回区满了,一次快速闪回区满导致数据库不能启动的解决过程

    一.事件背景描述:一个测试系统的数据库由于磁盘空间满了,清理了磁盘空间的,等待很久系统没有相应,因此通过shutdown immediate命令重新启动数据库,但是数据库一直关闭不了,所以通过shut ...

  8. oracle启动包找不到数据文件,ORA-01078,LRM-00109参数文件问题,导致数据库无法启动...

    在oracle9i和oracle10g中,数据库默认将使用spfile启动数据库,如果spfile不存在,并且找不到静态参数文件或环境变量设置错误,则就会出现ORA-01078和LRM-00109错误 ...

  9. mysql sys库 oom_MySQL 5.6因为OOM导致数据库重启

    MySQL 5.6因为OOM导致数据库重启 发布时间:2020-08-09 08:29:53 来源:ITPUB博客 阅读:89 作者:feelpurple 线上的一套MySQL 5.6的从库,因为OO ...

最新文章

  1. 2015级C++第15周程序阅读 范型程序设计
  2. Jmeter性能测试之if控制器的使用
  3. Spring MVC 返回视图时添加的模型数据------POJO
  4. olap与mysql_MySQL与OLAP:分析型SQL查询最佳实践探索
  5. stateflow中向量与矩阵
  6. BZOJ3627 [JLOI2014]路径规划
  7. Postman里如何把某个HTTP的请求和响应作为example保存
  8. Python3中的hasattr()
  9. ffmpeg抓取rtsp流并保存_详细解析RTSP框架和数据包分析(1)
  10. Android 打包keysotre文件
  11. vue表单中批量导入功能_Vue 编辑 新建表单复用的一些思考
  12. 【搜索】【usaco 4.1.4】奶牛加密术
  13. Excel 对比两列数据大小 大于等于 高亮显示
  14. OPPO设备设置第三方桌面为默认Launcher
  15. c#mvc模式进行crud_实用的微服务开发模式:CRUD与。 CQRS
  16. 渗透测试常见问题以及方法
  17. ios适配iPhone和iPad
  18. mysql gbk转utf8_MySQL字符集GBK转换到UTF8
  19. MES如何实现生产过程全追踪
  20. js日期计算,根据当前日期计算n个月后的日期

热门文章

  1. 共享可写节包含重定位_理解重定位
  2. TS高级类型内置工具类型
  3. 华为鸿蒙战略布局,继5G与鸿蒙后又一重要布局:华为发布计算战略
  4. java创建xml设置路径_java 写入xml文件 地址如何设置为局域网内的另一台服务器上...
  5. java中三种方法_Java文件I/O的三种方法
  6. php 怎么开启错误报告,总结PHP开启关闭错误报告示例代码以及PHP脚本不报错的原因...
  7. Javascript归并排序
  8. eclipse连接数据库
  9. 20190917:(leetcode习题)将有序数组转换为二叉搜索树
  10. Linux进入救援系统怎么恢复,Linux在rescue(救援)模式修复GRUB (引导菜单)