innodb_buffer_pool_reads、innodb_buffer_pool_read_requests分析与innodb 缓存命中率计算

澄清一下innodb buffer pool 缓存命中率（cache hit ratio）指标的计算，这个计算涉及到两个状态变量innodb_buffer_pool_reads和innodb_buffer_pool_read_requests。官方文档对这两个变量的解释如下：

Innodb_buffer_pool_reads

The number of logical reads that InnoDB could not satisfy from the buffer pool, and had to read directly from disk.

Innodb_buffer_pool_read_requests

The number of logical read requests.

从上面的解释来并不能看出来这两个变量之间有什么关系。需要从代码上去分析一下。我简单的翻了代码，不求十分准确，只要能解释其中的关系就好。

innodb_buffer_pool_reads和innodb_buffer_pool_read_requests的定义可以在下面找到，

// 下面的代码出现在ha_innodb.cc文件中的大概967-970行    (char *)&export_vars.innodb_buffer_pool_read_requests, SHOW_LONG,SHOW_SCOPE_GLOBAL},{"buffer_pool_reads", (char *)&export_vars.innodb_buffer_pool_reads,SHOW_LONG, SHOW_SCOPE_GLOBAL},

可以看到，在代码中innodb_buffer_pool_reads使用了(char *)&export_vars.innodb_buffer_pool_reads来指代；innodb_buffer_pool_read_requests使用(char *)&export_vars.innodb_buffer_pool_read_requests来指代。两个都是字符指针。

进一步的，这两个指代变量在这里赋值，

//下面这行代码出现在srv0srv.cc文件大概1524-1525行的位置export_vars.innodb_buffer_pool_read_requests =Counter::total(stat.m_n_page_gets);
//下面这行代码出现在srv0srv.cc文件大概1534行的位置
export_vars.innodb_buffer_pool_reads = srv_stats.buf_pool_reads;

可以看到export_vars.innodb_buffer_pool_read_requests使用了stat.m_n_page_gets并对其求和来计算（是因为其中分了若干个shards，累积的时候，根据shard分别累积，取值的时候，需要求和）；export_vars.innodb_buffer_pool_reads使用了srv_stats.buf_pool_reads来计算。

下面进一步看srv_stats.buf_pool_reads、stat.m_n_page_gets这两个变量实际发生变化的地方，因为代码中对这两个变量的计算在多处都有发生，这里只各取一处，能说明其中的关系就好：

//buf0buf.cc
template <typename T>
buf_block_t *Buf_fetch<T>::single_page() {buf_block_t *block;
//函数一开始，就m_buf_pool->stat.m_n_page_gets进行了累加，在inc()函数中，以m_page_id.page_no()作为shard的id，对m_buf_pool->stat.m_n_page_gets的值加1Counter::inc(m_buf_pool->stat.m_n_page_gets, m_page_id.page_no());for (;;) {//下面这行代码，调用Buf_fetch的get方法，读取blockif (static_cast<T *>(this)->get(block) == DB_NOT_FOUND) {return (nullptr);}}//此处省略非关键代码
。。。。。。。。。return (block);
}//再看看Buf_fetch的get()实现，Buf_fetch有多个子类，挑一个看看就好dberr_t Buf_fetch_normal::get(buf_block_t *&block) {/* Keep this path as simple as possible. */for (;;) {/* Lookup the page in the page hash. If it doesn't exist in thebuffer pool then try and read it in from disk. *//在这里看看block是否已经在buffer pool中block = lookup();//如果存在，则break退出if (block != nullptr) {buf_block_fix(block);/* Now safe to release page_hash S lock. */rw_lock_s_unlock(m_hash_lock);break;}/* Page not in buf_pool: needs to be read from file *///如果不存在，则进入到read_page()中进一步读取read_page();}return (DB_SUCCESS);
}// read_page()的实现
template <typename T>
void Buf_fetch<T>::read_page() {bool success{};auto sync = m_mode != Page_fetch::SCAN;if (sync) {//这里是sync的读取，意思是同步io，会等待io线程读取完成，会调用buf_read_page_low方法，并累加srv_stats.buf_pool_reads.add(count);//可以看到srv_stats.buf_pool_reads等于从磁盘读取的page的数量success = buf_read_page(m_page_id, m_page_size);} else {dberr_t err;//这里是异步io，读取一个page，如果成功的话，累加srv_stats.buf_pool_readsauto ret = buf_read_page_low(&err, false, 0, BUF_READ_ANY_PAGE, m_page_id,m_page_size, false);success = ret > 0;if (success) {srv_stats.buf_pool_reads.add(1);}ut_a(err != DB_TABLESPACE_DELETED);/* Increment number of I/O operations used for LRU policy. */buf_LRU_stat_inc_io();}if (success) {if (sync) {//如果成功，这里有一个预读操作buf_read_ahead_random(m_page_id, m_page_size, ibuf_inside(m_mtr));}m_retries = 0;} else if (m_retries < BUF_PAGE_READ_MAX_RETRIES) {++m_retries;DBUG_EXECUTE_IF("innodb_page_corruption_retries",m_retries = BUF_PAGE_READ_MAX_RETRIES;);} else {ib::fatal(ER_IB_MSG_74)<< "Unable to read page " << m_page_id << " into the buffer pool after "<< BUF_PAGE_READ_MAX_RETRIES<< " attempts. The most probable cause of this error may"" be that the table has been corrupted. Or, the table was"" compressed with with an algorithm that is not supported by ""this"" instance. If it is not a decompress failure, you can try to ""fix"" this problem by using innodb_force_recovery. Please ""see " REFMAN " for more details. Aborting...";}#if defined UNIV_DEBUG || defined UNIV_BUF_DEBUGut_ad(fsp_skip_sanity_check(m_page_id.space()) || ++buf_dbg_counter % 5771 ||buf_validate());
#endif /* UNIV_DEBUG || UNIV_BUF_DEBUG */
}bool buf_read_page(const page_id_t &page_id, const page_size_t &page_size) {ulint count;dberr_t err;count = buf_read_page_low(&err, true, 0, BUF_READ_ANY_PAGE, page_id,page_size, false);srv_stats.buf_pool_reads.add(count);if (err == DB_TABLESPACE_DELETED) {ib::error(ER_IB_MSG_141) << "trying to read page " << page_id<< " in nonexisting or being-dropped tablespace";}/* Increment number of I/O operations used for LRU policy. */buf_LRU_stat_inc_io();return (count > 0);
}//上文，如果sync读取一个page成功的话，会进一步的read_ahead预读
ulint buf_read_ahead_random(const page_id_t &page_id,const page_size_t &page_size, bool inside_ibuf) {buf_pool_t *buf_pool = buf_pool_get(page_id);ulint recent_blocks = 0;ulint ibuf_mode;ulint count;page_no_t low, high;dberr_t err;page_no_t i;const page_no_t buf_read_ahead_random_area = BUF_READ_AHEAD_AREA(buf_pool);if (!srv_random_read_ahead) {/* Disabled by user */return (0);}if (srv_startup_is_before_trx_rollback_phase) {/* No read-ahead to avoid thread deadlocks */return (0);}if (ibuf_bitmap_page(page_id, page_size) || trx_sys_hdr_page(page_id)) {/* If it is an ibuf bitmap page or trx sys hdr, we dono read-ahead, as that could break the ibuf page accessorder */return (0);}low = (page_id.page_no() / buf_read_ahead_random_area) *buf_read_ahead_random_area;high = (page_id.page_no() / buf_read_ahead_random_area + 1) *buf_read_ahead_random_area;/* Remember the tablespace version before we ask the tablespace sizebelow: if DISCARD + IMPORT changes the actual .ibd file meanwhile, wedo not try to read outside the bounds of the tablespace! */if (fil_space_t *space = fil_space_acquire(page_id.space())) {if (high > space->size) {high = space->size;}fil_space_release(space);} else {return (0);}os_rmb;if (buf_pool->n_pend_reads >buf_pool->curr_size / BUF_READ_AHEAD_PEND_LIMIT) {return (0);}/* Count how many blocks in the area have been recently accessed,that is, reside near the start of the LRU list. */for (i = low; i < high; i++) {rw_lock_t *hash_lock;const buf_page_t *bpage;bpage = buf_page_hash_get_s_locked(buf_pool, page_id_t(page_id.space(), i),&hash_lock);if (bpage != nullptr && buf_page_is_accessed(bpage) &&buf_page_peek_if_young(bpage)) {recent_blocks++;if (recent_blocks >= BUF_READ_AHEAD_RANDOM_THRESHOLD(buf_pool)) {rw_lock_s_unlock(hash_lock);goto read_ahead;}}if (bpage != nullptr) {rw_lock_s_unlock(hash_lock);}}/* Do nothing */return (0);read_ahead:/* Read all the suitable blocks within the area */if (inside_ibuf) {ibuf_mode = BUF_READ_IBUF_PAGES_ONLY;} else {ibuf_mode = BUF_READ_ANY_PAGE;}count = 0;//这里会循环读取pagefor (i = low; i < high; i++) {/* It is only sensible to do read-ahead in the non-sync aiomode: hence FALSE as the first parameter */const page_id_t cur_page_id(page_id.space(), i);if (!ibuf_bitmap_page(cur_page_id, page_size)) {count += buf_read_page_low(&err, false, IORequest::DO_NOT_WAKE, ibuf_mode,cur_page_id, page_size, false);if (err == DB_TABLESPACE_DELETED) {ib::warn(ER_IB_MSG_140) << "Random readahead trying to"" access page "<< cur_page_id<< " in nonexisting or"" being-dropped tablespace";break;}}}/* In simulated aio we wake the aio handler threads only afterqueuing all aio requests.  */os_aio_simulated_wake_handler_threads();if (count) {DBUG_PRINT("ib_buf",("random read-ahead %u pages, %u:%u", (unsigned)count,(unsigned)page_id.space(), (unsigned)page_id.page_no()));}/* Read ahead is considered one I/O operation for the purpose ofLRU policy decision. */buf_LRU_stat_inc_io();//将从磁盘读取的page的数量累加buf_pool->stat.n_ra_pages_read_rnd += count;srv_stats.buf_pool_reads.add(count);return (count);
}

从上面的代码看起来，

innodb_buffer_pool_reads：等同于物理读的page数
innodb_buffer_pool_read_requests：逻辑读请求数，但是因为一次请求一个块，所以等同于逻辑读的page数。

综上，

在禁用预读的情况下，buffer pool命中率指标计算： (1 - innodb_buffer_pool_reads/innodb_buffer_pool_read_requests) * 100

在启用预读的情况下，buffer pool命中率指标计算：
x=innodb_buffer_pool_reads中的非预读部分
y=innodb_buffer_pool_reads中的预读部分
假设x=3,y=5,innodb_buffer_pool_reads=x+y=8,innodb_buffer_pool_read_requests=100
rate1: 1-(innodb_buffer_pool_reads/innodb_buffer_pool_read_requests)=0.9992
rate2: 1-innodb_buffer_pool_reads/(innodb_buffer_pool_read_requests+innodb_buffer_pool_reads)=0.9993
rate3: innodb_buffer_pool_read_requests/(innodb_buffer_pool_read_requests+innodb_buffer_pool_reads)=0.9259

实际使用中，通常使用rate3作为命中率指标，可以看到，这个值偏小。

innodb_buffer_pool_reads、innodb_buffer_pool_read_requests分析与innodb 缓存命中率计算相关推荐

Innodb存储引擎的缓存命中率计算
数据库的慢查询是我们在生产环境中必须经常检测的,如果慢查询语句过多,说明我们应该增加buffer_pool的大小了.常常检查的指标就是查看缓存命中率是否过低. mysql> show statu ...
Redis 缓存命中率计算
1.需要先登陆到Redis服务器上 2.运行命令 info ,得到如下参数 keyspace_hits:命中的次数 keyspace_misses:没有命中的次数 3.缓存命中率 = keyspace ...
mysql qps 索引查询_【MySQL】MySQL配置调优之 QPS/TPS/索引缓存命中率、innoDB索引缓存命中率、查询缓存命中率查看...
运行中的mysql状态查看: 对正在运行的mysql进行监控,其中一个方式就是查看mysql运行状态. (1)QPS(每秒Query量) QPS = Questions(or Queries) / s ...
mysql query cache 命中率_MySQL缓存命中率概述及如何提高缓存命中率
MySQL缓存命中率概述工作原理: 查询缓存的工作原理,基本上可以概括为: 缓存SELECT操作或预处理查询(注释:5.1.17开始支持)的结果集和SQL语句: 新的SELECT语句或预处理查询语句 ...
mysql qps如何查看_mysql状态查看 QPS/TPS/缓存命中率查看
运行中的mysql状态查看对正在运行的mysql进行监控,其中一个方式就是查看mysql运行状态. (1)QPS(每秒Query量) QPS = Questions(or Queries) / up ...
如何提高缓存命中率（Redis）
缓存命中率的介绍命中:可以直接通过缓存获取到需要的数据. 不命中:无法直接通过缓存获取到想要的数据,需要再次查询数据库或者执行其它的操作.原因可能是由于缓存中根本不存在,或者缓存已经过期. 通常来讲 ...
PHP提高redis命中率,怎么提高redis缓存命中率
缓存命中率的介绍命中:可以直接通过缓存获取到需要的数据. 不命中:无法直接通过缓存获取到想要的数据,需要再次查询数据库或者执行其它的操作.原因可能是由于缓存中根本不存在,或者缓存已经过期. 通常来讲 ...
memcached 缓存命中率
缓存命中率的介绍命中:可以直接通过缓存获取到需要的数据. 不命中:无法直接通过缓存获取到想要的数据,需要再次查询数据库或者执行其它的操作.原因可能是由于缓存中根本不存在,或者缓存已经过期. 通常来讲 ...
什么是 CDN 缓存命中率以及如何计算和优化它？
新钛云服已累计为您分享694篇技术干货本文主要关注 Amazon CloudFront CDN 缓存以及如何使用它们来实现更好的缓存命中率. 在了解缓存中的命中率和未命中率之前,最好先了解缓存是什么 ...
mysql buffer 命中率_从MySQL的源码剖析Innodb buffer的命中率计算
按官方手册推荐Innodb buffer Hit Ratios的计算是: 100-((iReads / iReadRequests)*100) iReads : mysql->status-&g ...

innodb_buffer_pool_reads、innodb_buffer_pool_read_requests分析与innodb 缓存命中率计算

innodb_buffer_pool_reads、innodb_buffer_pool_read_requests分析与innodb 缓存命中率计算相关推荐

最新文章

热门文章