MySQL General、Slow日志介绍，参数及配置
2024-05-10 11:56:10
MySQL General log
MySQL General log是记录mysqld(MySQLserver进程)收到的所有请求的日志，我们可以通过这个命令查看格式：
SHOW CREATE TABLE mysql.general_log可以看出general_log的格式为：
CREATE TABLE `general_log` (`event_time` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),`user_host` mediumtext NOT NULL,`thread_id` bigint(21) unsigned NOT NULL,`server_id` int(10) unsigned NOT NULL,`command_type` varchar(64) NOT NULL,`argument` mediumblob NOT NULL
) ENGINE=CSV DEFAULT CHARSET=utf8 COMMENT='General log'这个日志包括客户端连接事件记录，客户端发送来的SQL查询记录等等。其中客户端连接还包括连接类型等等，目前MySQL支持的连接类型有TCP/IP, SSL/TLS , Socket (unix文件链接), Named Pipe (windows管道连接), or Shared Memory (共享内存)
客户端发送来的SQL查询记录也许和最后执行的顺序不同，这个只是收到请求的时间。
配置
查看参数
mysql> show variables like '%general%';+------------------+----------------------------------------------------------------+
| Variable_name | Value |
+------------------+----------------------------------------------------------------+
| general_log | OFF |
| general_log_file | D:\mysql-5.7.21-winx64\mysql-5.7.21-winx64\data\BIH-D-6885.log |
+------------------+----------------------------------------------------------------+
2 rows in set临时开启通用日志查询：set global general_log = on;
临时关闭通用日志查询：set global general_log = off;
日志格式示例：
mysqld, Version: 5.7.21 (MySQL Community Server (GPL)). started with:
TCP Port: 3306, Named Pipe: MySQL
Time Id Command Argument
2018-03-30T03:19:50.409295Z 4 Connect root@localhost on using TCP/IP
2018-03-30T03:19:50.409295Z 4 Query SET NAMES utf8mb4
2018-03-30T03:19:50.409295Z 4 Init DB test
2018-03-30T03:19:50.434300Z 4 Query SELECT @@character_set_database, @@collation_database
2018-03-30T03:19:50.448303Z 4 Query SHOW FULL TABLES WHERE Table_type != 'VIEW'
2018-03-30T03:19:53.385890Z 2 Query SHOW CHARACTER SET
2018-03-30T03:19:53.386890Z 2 Query SHOW COLLATION
2018-03-30T03:19:53.389891Z 4 Query SHOW ENGINES
2018-03-30T03:20:41.493510Z 4 Query SHOW FULL TABLES WHERE Table_type != 'VIEW'
2018-03-30T03:20:41.496510Z 4 Query SHOW FULL TABLES WHERE Table_type = 'VIEW'
2018-03-30T03:20:41.543520Z 4 Query CREATE TABLE `test` (
`id` int NOT NULL AUTO_INCREMENT ,
`name` varchar(255) NULL ,
`info` varchar(255) NULL ,
PRIMARY KEY (`id`)
)如果想永久配置，则在配置文件中加入：
general_log = ON
general_log_file=D:\mysql-5.7.21-winx64\mysql-5.7.21-winx64\data\BIH-D-6885.log
可以以文件和表格（就是上面提到的mysql.general_log表）两种方式存储：
查看当前格式
mysql> show variables like '%log_output%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| log_output | FILE |
+---------------+-------+
1 row in set设置通用日志输出为表方式：
set global log_output = 'file';设置通用日志输出为表和文件方式：
set global log_output = 'file,table';利用mysql general log日志找出查询次数最多的SQL句子
analysis-general-log.py：
#!/usr/bin/python# sort and count mysql general log
# Author: Jason
# Created: UTC 2015-02-15 17:51:53import re
import sys
import osif len(sys.argv) == 2:logPath = sys.argv[1]if not os.path.exists(logPath):print ("file " + logPath + " does not exists.")sys.exit(1)
else:print ("Usage: " + sys.argv[0] + " logPath")sys.exit(1)logFo = open(logPath)
match = 0for line in logFo:line = re.sub(r"＼n","",line)if match == 0:# match line begin with numberslineMatch = re.match(r"＼s+[0-9]+＼s+.*",line,flags=re.I)if lineMatch:lineTmp = lineMatch.group(0)match = match + 1continueelif match == 1:# match line begin with numberslineMatch = re.match(r"＼s+[0-9]+＼s+.*",line,flags=re.I)if lineMatch:# match only querylineMatchQuery = re.match(r".*Query＼s+(.*)",lineTmp,flags=re.I)if lineMatchQuery:lineTmp = lineMatchQuery.group(1)# remove extra spacelineTmp = re.sub(r"＼s+", " ",lineTmp)# replace values (value) to values (x)lineTmp = re.sub(r"values＼s*＼(.*?＼)", "values (x)",lineTmp,flags=re.I)# replace filed = 'value' to filed = 'x'lineTmp = re.sub(r"(=|>|<|>=|<=)＼s*('|＼").*?＼2","＼＼1 'x'",lineTmp)# replace filed = value to filed = xlineTmp = re.sub(r"(=|>|<|>=|<=)＼s*[0-9]+","＼＼1 x",lineTmp)# replace like 'value' to like 'x'lineTmp = re.sub(r"like＼s+('|＼").*?＼1","like 'x'",lineTmp,flags=re.I)# replace in (value) to in (x)lineTmp = re.sub(r"in＼s+＼(.*?＼)","in (x)",lineTmp,flags=re.I)# replace limit x,y to limitlineTmp = re.sub(r"limit.*","limit",lineTmp,flags=re.I)print (lineTmp)match = 1lineTmp = lineMatch.group(0)else: lineTmp += linematch = 1logFo.close()使用方法：
analysis-general-log.py general.log | sort | uniq -c | sort -nr示例输出：
1032 SELECT * FROM wp_comments WHERE ( comment_approved = 'x' OR comment_approved = 'x' ) AND comment_post_ID = x ORDER BY comment_date_gmt DESC
653 SELECT post_id, meta_key, meta_value FROM wp_postmeta WHERE post_id in (x) ORDER BY meta_id ASC
527 SELECT FOUND_ROWS()
438 SELECT t.*, tt.* FROM wp_terms AS t INNER JOIN wp_term_taxonomy AS tt ON t.term_id = tt.term_id WHERE tt.taxonomy = 'x' AND t.term_id = x limit
341 SELECT option_value FROM wp_options WHERE option_name = 'x' limit
329 SELECT t.*, tt.*, tr.object_id FROM wp_terms AS t INNER JOIN wp_term_taxonomy AS tt ON tt.term_id = t.term_id INNER JOIN wp_term_relationships AS tr ON tr.term_taxonomy_id = tt.term_taxonomy_id WHERE tt.taxonomy in (x) AND tr.object_id in (x) ORDER BY t.name ASC
311 SELECT wp_posts.* FROM wp_posts WHERE 1= x AND wp_posts.ID in (x) AND wp_posts.post_type = 'x' AND ((wp_posts.post_status = 'x')) ORDER BY wp_posts.post_date DESC
219 SELECT wp_posts.* FROM wp_posts WHERE ID in (x)
218 SELECT tr.object_id FROM wp_term_relationships AS tr INNER JOIN wp_term_taxonomy AS tt ON tr.term_taxonomy_id = tt.term_taxonomy_id WHERE tt.taxonomy in (x) AND tt.term_id in (x) ORDER BY tr.object_id ASC
217 SELECT wp_posts.* FROM wp_posts WHERE 1= x AND wp_posts.ID in (x) AND wp_posts.post_type = 'x' AND ((wp_posts.post_status = 'x')) ORDER BY wp_posts.menu_order ASC
202 SELECT SQL_CALC_FOUND_ROWS wp_posts.ID FROM wp_posts WHERE 1= x AND wp_posts.post_type = 'x' AND (wp_posts.post_status = 'x') ORDER BY wp_posts.post_date DESC limit
118 SET NAMES utf8
115 SET SESSION sql_mode= 'x'
115 SELECT @@SESSION.sql_mode
112 SELECT option_name, option_value FROM wp_options WHERE autoload = 'x'
111 SELECT user_id, meta_key, meta_value FROM wp_usermeta WHERE user_id in (x) ORDER BY umeta_id ASC
108 SELECT YEAR(min(post_date_gmt)) AS firstdate, YEAR(max(post_date_gmt)) AS lastdate FROM wp_posts WHERE post_status = 'x'
108 SELECT t.*, tt.* FROM wp_terms AS t INNER JOIN wp_term_taxonomy AS tt ON t.term_id = tt.term_id WHERE tt.taxonomy in (x) AND tt.count > x ORDER BY tt.count DESC limit
107 SELECT t.*, tt.* FROM wp_terms AS t INNER JOIN wp_term_taxonomy AS tt ON t.term_id = tt.term_id WHERE tt.taxonomy in (x) AND t.term_id in (x) ORDER BY t.name ASC
107 SELECT * FROM wp_users WHERE ID = 'x'
106 SELECT SQL_CALC_FOUND_ROWS wp_posts.ID FROM wp_posts WHERE 1= x AND wp_posts.post_type = 'x' AND (wp_posts.post_status = 'x') AND post_date > 'x' ORDER BY wp_posts.post_date DESC limit
106 SELECT SQL_CALC_FOUND_ROWS wp_posts.ID FROM wp_posts WHERE 1= x AND wp_posts.post_type = 'x' AND (wp_posts.post_status = 'x') AND post_date > 'x' ORDER BY RAND() DESC limit
105 SELECT SQL_CALC_FOUND_ROWS wp_posts.ID FROM wp_posts WHERE 1= x AND wp_posts.post_type = 'x' AND (wp_posts.post_status = 'x') AND post_date > 'x' ORDER BY wp_posts.comment_count DESC limit慢查询日志
获取锁的时间并不算在慢查询时间内，这个记录是在语句执行完后写入，所以也可能和执行顺序不一样。
Slow log 大概格式
CREATE TABLE `slow_log` (`start_time` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),`user_host` mediumtext NOT NULL,`query_time` time(6) NOT NULL,`lock_time` time(6) NOT NULL,`rows_sent` int(11) NOT NULL,`rows_examined` int(11) NOT NULL,`db` varchar(512) NOT NULL,`last_insert_id` int(11) NOT NULL,`insert_id` int(11) NOT NULL,`server_id` int(10) unsigned NOT NULL,`sql_text` mediumblob NOT NULL,`thread_id` bigint(21) unsigned NOT NULL
) ENGINE=CSV DEFAULT CHARSET=utf8 COMMENT='Slow log'配置
查看相关参数：
mysql> show variables like '%quer%';
+----------------------------------------+---------------------------------------------------------------------+
| Variable_name | Value |
+----------------------------------------+---------------------------------------------------------------------+
| binlog_rows_query_log_events | OFF |
| ft_query_expansion_limit | 20 |
| have_query_cache | YES |
| log_queries_not_using_indexes | OFF |
| log_throttle_queries_not_using_indexes | 0 |
| long_query_time | 10.000000 |
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 1048576 |
| query_cache_type | OFF |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
| slow_query_log | OFF |
| slow_query_log_file | D:\mysql-5.7.21-winx64\mysql-5.7.21-winx64\data\BIH-D-6885-slow.log |
+----------------------------------------+---------------------------------------------------------------------+
15 rows in setlog_queries_not_using_indexes:如果该值是ON，则会记录所有没有利用索引来进行查询的语句，前提是slow_query_log的值也是ON,否则，不会记录; 和log_throttle_queries_not_using_indexes配合使用，设定每分钟记录到日志的未使用索引的语句数目，超过这个数目后只记录语句数量和花费的总时间long_query_time指定了慢查询的阈值，即执行语句的时间若超过这个值则为慢查询语句
这两种日志的维护
不像binlog那样，只要flush一下，就从新文件开始写。对于Generallog还有slowlog只是关闭原有文件再打开下。为了让这些日志在新的文件中写入，可以在flush之前，将文件重命名。这样在执行flush的时候，会重新生成一个文件：
shell> cd mysql-data-directory
shell> mv mysql.log mysql.old
shell> mv mysql-slow.log mysql-slow.old
shell> mysqladmin flush-logsMySQL General、Slow日志介绍，参数及配置相关推荐
- MySQL 慢查询日志介绍
一. 慢查询介绍 MySQL的慢查询日志是MySQL提供的一种日志记录,它用来记录在MySQL中响应时间超过指定阀值的SQL语句,运行时间超过long_query_time值的SQL,会被记录到慢查询 ...
- MySQL 慢查询日志如何查看及配置
简介 MySQL 慢查询日志是排查问题 SQL 语句,以及检查当前 MySQL 性能的一个重要功能. 查看是否开启慢查询功能: 说明: slow_query_log 慢查询开启状态 slow_quer ...
- mysql数据库执行事务日志_第十章 MySQL事务及其日志介绍
一.数据库升级 #1.提出方案 1)升级的方法 2)升级的步骤 3)升级的时间 4)升级步骤可能会出现的问题 5)出现的问题怎么解决,解决时间 6)升级后出现的问题 #2.搭建新的数据库 #3.备份就 ...
- mysql general log使用介绍
目录 1.修改变量 2.修改配置文件 general log 是MySQL 日志的一种,它会记录MySQL执行的每条SQL,非常详细. 但对MySQL性能有影响,为了性能考虑,一般general lo ...
- MySQL慢查询日志的配置与使用_MySQL 慢查询日志的开启与配置
简介 MySQL 慢查询日志是排查问题 SQL 语句,以及检查当前 MySQL 性能的一个重要功能. 查看是否开启慢查询功能: mysql> show variables like 'slow_ ...
- mysql 日志 设置 set_MySQL 慢查询日志的开启与配置
简介 MySQL 慢查询日志是排查问题 SQL 语句,以及检查当前 MySQL 性能的一个重要功能. 查看是否开启慢查询功能: mysql> show variables like 'slow_ ...
- mysql general bin区别_MySQL中几种常见的日志
前言: 在 MySQL 系统中,有着诸多不同类型的日志.各种日志都有着自己的用途,通过分析日志,我们可以优化数据库性能,排除故障,甚至能够还原数据.这些不同类型的日志有助于我们更清晰的了解数据库,在日 ...
- mysql 查询日志介绍
MySQL查询日志介绍 MySQL的查询日志记录了所有MySQL数据库请求的信息.无论这些请求是否得到了正确的执行.默认文件名为hostname.log.默认情况下MySQL查询日志是关闭的.生产环境 ...
- MySQL查询日志介绍
MySQL查询日志介绍 MySQL的查询日志记录了所有MySQL数据库请求的信息.无论这些请求是否得到了正确的执行.默认文件名为hostname.log.默认情况下MySQL查询日志是关闭的.生产环境 ...
最新文章
- 修改mysql参数_mysql动态修改参数
- oracle向达梦迁移工作量,从Oracle安全移植到国产达梦数据库的DBA实践
- TestNG如何用命令行运行
- heidiSQL使用简介
- Apache孵化器主席Justin Mclean：如何成为Apache顶级开源项目
- 贪心算法——洛谷(P1106)删数问题
- Docker 安装 Tomcat
- 万稞pw80线切割编程软件_零基础如何快速学习UG数控编程？
- 虚拟机安装MySQL教程
- 卡巴斯基kis6换许可文件key教程（转）
- iOS开发全套资源，从入门到全栈IOS工程师
- Stripies POJ 1862
- 远程连接服务器突然失败
- docker部署微服务项目
- toFixed() is not a function toFixed方法数字类型才能使用
- FL Studio教程之Fruity Blood Overdrive插件
- 软著申请你还不会？我是这样申请的
- pascal语言的版本
- nmap：网络探测工具和安全/端口扫描器
- C#大华监控画面切换