linux配置大内存技术

1、理论介绍

系统进程通过虚拟地址访问内存，CPU必须把它转换成物理内存地址才能真正访问内存。为了提高这个转换效率,CPU会缓存最近的虚拟内存地址和物理内存地址的映射关系，并保存在一个

由CPU维护的映射表中。为了尽量提高内存的访问速度，需要在映射表中保存尽量多的映射关系。在linux系统中,所有内存都是以页的形式划分的，默认情况下每页是4096B，这就意味着如果

物理内存很大，映射表的条目数会很多，系统cpu在检索的过程中，需要检索的条目数就会很。从另一个角度说，如果能够让cpu减少检索内存条目的数量，可采取的办法只有增加页的尺寸，

hugepage应时而生。hugepage，一直被cache,不会被交换出去,也就是说使用hugepage的内存不能被其他的进程使用。随着内存技术的发展，现在上T的内存配置也出现，所以内存已经不是问

题，在实际应用中，如果你的物理内存能够达到8g以上，那么都可以使用大内存技术。在oracle数据库中，sga与大内存配合使用，将会对性能产生非常大的提升。

2、oracle中大内存技术的配置建议

oracle针对大内存技术，提供了一个脚本如下：

#!/bin/bash

#

# hugepages_settings.sh

#

# Linux bash script to compute values for the

# recommended HugePages/HugeTLB configuration

#

# Note: This script does calculation for all shared memory

# segments available when the script is run, no matter it

# is an Oracle RDBMS shared memory segment or not.

#

# This script is provided by Doc ID 401749.1 from My Oracle Support

#

# Welcome text

echo "

This script is provided by Doc ID 401749.1 from My Oracle Support

() where it is intended to compute values for

the recommended HugePages/HugeTLB configuration for the current shared

memory segments. Before proceeding with the execution please make sure

that:

* Oracle Database instance(s) are up and running

* Oracle Database 11g Automatic Memory Management (AMM) is not setup

(See Doc ID 749851.1)

* The shared memory segments can be listed by command:

# ipcs -m

Press Enter to proceed..."

read

# Check for the kernel version

KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`

# Find out the HugePage size

HPG_SZ=`grep Hugepagesize /proc/meminfo | awk '{print $2}'`

# Initialize the counter

NUM_PG=0

# Cumulative number of pages required to handle the running shared memory segments

for SEG_BYTES in `ipcs -m | awk '{print $5}' | grep "[0-9][0-9]*"`

do

MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`

if [ $MIN_PG -gt 0 ]; then

NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`

fi

done

RES_BYTES=`echo "$NUM_PG * $HPG_SZ * 1024" | bc -q`

# An SGA less than 100MB does not make sense

# Bail out if that is the case

if [ $RES_BYTES -lt 100000000 ]; then

echo "***********"

echo "** ERROR **"

echo "***********"

echo "Sorry! There are not enough total of shared memory segments allocated for

HugePages configuration. HugePages can only be used for shared memory segments

that you can list by command:

# ipcs -m

of a size that can match an Oracle Database SGA. Please make sure that:

* Oracle Database instance is up and running

* Oracle Database 11g Automatic Memory Management (AMM) is not configured"

exit 1

fi

# Finish with results

case $KERN in

'2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;

echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;

'2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;

*) echo "Unrecognized kernel version $KERN. Exiting." ;;

esac

# End

[root@hdyydb1 u02]# chmod u+x hugepages_settings.sh

[root@hdyydb1 u02]# ./hugepages_settings.sh

This script is provided by Doc ID 401749.1 from My Oracle Support

() where it is intended to compute values for

the recommended HugePages/HugeTLB configuration for the current shared

memory segments. Before proceeding with the execution please make sure

that:

* Oracle Database instance(s) are up and running

* Oracle Database 11g Automatic Memory Management (AMM) is not setup

(See Doc ID 749851.1)

* The shared memory segments can be listed by command:

# ipcs -m

Press Enter to proceed...

Recommended setting: vm.nr_hugepages = 2588

可以看到，运行脚本显示，将vm.nr_hugepages设置为2588

3、hugepage在oracle中的配置应用

--检查系统是否启用hugepage，或者是否可以启用

[root@hdyydb1 u02]# cat /proc/meminfo | grep -i huge

AnonHugePages:     75776 kB

HugePages_Total:       0

HugePages_Free:        0

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

可以看到，HugePages_Total,HugePages_Free均为零，证明没有使用，但可以配置。如果上面什么都没有显示，证明不能配置。

--检查数据库的参数设置是否支持

SQL> show parameter sga_target

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

sga_target                           big integer 0

大内存技术，不支持内存自动管理，必须关闭AMM(自动内存管理)特性才能使用hugepage

调整方法如下：

ALTER SYSTEM SET sga_max_size = xxxxM SCOPE=SPFILE;

ALTER SYSTEM SET sga_target = xxxxM SCOPE=SPFILE;

ALTER SYSTEM SET PGA_AGGREGATE_TARGET = xxxxM SCOPE=SPFILE;

ALTER SYSTEM SET memory_target = 0 SCOPE=SPFILE;

ALTER SYSTEM SET memory_max_target = 0 SCOPE=SPFILE;

--根据第二步的验证结果进行配置

[root@hdyydb1 u02]# vi /etc/sysctl.conf

把vm.nr_hugepages = 2588 添加进去，并保存

[root@hdyydb1 u02]# sysctl -p

net.ipv4.ip_forward = 0

net.ipv4.conf.default.rp_filter = 1

net.ipv4.conf.default.accept_source_route = 0

kernel.sysrq = 0

kernel.core_uses_pid = 1

net.ipv4.tcp_syncookies = 1

net.bridge.bridge-nf-call-ip6tables = 0

net.bridge.bridge-nf-call-iptables = 0

net.bridge.bridge-nf-call-arptables = 0

kernel.msgmnb = 65536

kernel.msgmax = 65536

fs.aio-max-nr = 1048576

fs.file-max = 6815744

kernel.shmall = 2097152

kernel.shmmax = 7247757312

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

net.ipv4.ip_local_port_range = 9000 65500

net.core.rmem_default = 262144

net.core.rmem_max = 4194304

net.core.wmem_default = 262144

net.core.wmem_max = 1048586

vm.min_free_kbytes = 51200

vm.nr_hugepages = 2588

检查设置：

[root@hdyydb1 u02]# cat /proc/meminfo | grep -i huge

AnonHugePages:     75776 kB

HugePages_Total:     977

HugePages_Free:      977

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

--锁定内存配置

如果不配置锁定内存，在后alert_sid.log中会有类似如下的建议：

RECOMMENDATION:

Total System Global Area size is 5 GB. For optimal performance,

prior to the next instance restart:

1. Large pages are automatically locked into physical memory.

Increase the per process memlock (soft) limit to at least 5 GB to lock

100% System Global Area's large pages into physical memory

并且，hugepage并没有真正使用起来！

主要是配置

oracle soft memlock

oracle hard memlock

注意，这里设置的值均以kb为单位的！

实际物理内存 > 锁定内存 >=HugePages_Total*Hugepagesize

[root@hdyydb1 u02]# vim /etc/security/limits.conf

添加：

oracle soft memlock 6291456

oracle hard memlock 6291456

我的物理内存是8g，sga 5g左右

4、重启操作系统、数据库后生效

[oracle@hdyydb1 ~]$ cat /proc/meminfo | grep -i huge

AnonHugePages:     43008 kB

HugePages_Total:    2588

HugePages_Free:     2168

HugePages_Rsvd:     2165

HugePages_Surp:        0

Hugepagesize:       2048 kB

free比total小，证明已经生效