SQL调优案例，MYSQL服务器性能调优

服务器环境见下表

服务器环境
序号	名称	版本	内存	CPU
1	Windows Server 2008 64bit	R2	16G	2
2	Mysql	5.7	256M	2

一、问题现象

1、服务器CPU使用率不稳定，波动比较明显，偶尔飙升至100%，内存使用率稳定在10%左右。

2、MYSQL相应异常缓慢，部分查询结果等待时间极长，切无法出数据。

二、检查思路

1、CPU飙升应该是有大的查询语句在执行，使用show full processlist检查查询语句。

2、内存只用到10%，应该是安装时使用了mysql的默认参数导致，需要提升mysql的相关参数。

3、收集用户反馈检查部分表和数据是否有异常。

三、处理方法

1、首先使用show full processlist 得到占用大量CPU时间的查询语句

  4  root    test.test.com:49357 x3  Query   23  Sending data    SELECT T.R_R_ID, T.RECORD_LOCK, T.NORM_VALUE, T.URGE_PERIOD,\n\t\t   E.ENTITY_ID, E.PARENT_ID, E.NAME ENTITY_NAME, E.NOTE ENTITY_NOTE, E.TYPE_ID ENTITY_TYPE,\n\t       N.NORM_ID, N.DEF_CODE NORM_CODE, N.TEST_COND, N.CALCULATE_MATH, N.CHINESE_MATH,\n\t       D.NAME NORM_NAME, D.DATA_TYPE, CD.NAME UNIT_NAME\n\t  FROM  TEST_NORM N\n\t INNER JOIN  TEST_NORM_DEFIN D ON D.NORM_DEFIN_ID = N.NORM_DEFIN_ID\n\t INNER JOIN  TEST_ENTITY E ON E.CODE = N.ENTITY_CODE\n\t  LEFT JOIN SYS_GENERAL_CODE CD ON CD.ENCODE_VALUE = D.UNIT_NAME AND CD.KEY_VALUE LIKE 'DWMC_%'\n\t  LEFT JOIN (SELECT DISTINCT RR.*\n\t  \t\t\t  FROM (\n\t  \t\t\t   SELECT DISTINCT A.R_R_ID,A.R_U_INFO_ID,A.ENTITY_ID,A.RECORD_LOCK,A.NORM_CODE,A.NORM_VALUE,A.URGE_PERIOD \n\t  \t\t\t   FROM  TEST_REPORT_RECORD A\n\t  \t\t\t  \t\tWHERE A.URGE_PERIOD_YEAR IN (2017)\n\t\t\t\t   \t\tAND A.URGE_PERIOD_MONTH IN (9)\n\t\t\t\t   \t\tAND A.URGE_PERIOD_DAY IN (20)\n\t\t\t\t   \t\t \n\t\t\t\t   \t\tAND A.NORM_CODE IN('SJA_FDL_D_D_D_SUM_M_MANL','SJA_GDL_D_D_D_SUM_M_CTST','SJA_SCCYDL_D_D_D_SUM_M_MANL','SJA_JJDL_D_D_D_SUM_M_MANL','SJA_ZGFH_D_D_D_MAX_M_MANL','SJA_PJFH_D_D_D_PAV_M_CTST','SJA_FHLV_D_D_D_PAV_M_CTST','SJA_FHXS_D_D_D_PAV_M_CTST','SJA_HYM_D_D_D_SUM_M_MANL','SJA_HYY_D_D_D_SUM_M_MANL','SJA_FDZJZB_D_D_D_SUM_M_MANL','SJA_FDYMH_D_D_D_SUM_M_CTST','SJA_FDRYH_D_D_D_SUM_M_CTST','SJA_FDBZMHZ_D_D_D_PAV_M_CTST','SJA_GDBZMHZ_D_D_D_PAV_M_CTST','SJA_MPRL_D_D_D_SUM_M_CTST','SJA_KCM_D_D_D_SIN_M_MANL','SJA_HLKC_D_D_D_SUM_M_CTST','SJA_JJKCM_D_D_D_SUM_M_CTST','SJA_MFHKYTS_D_D_D_SUM_M_MANL','SJA_LYXS_D_D_D_SUM_M_CTST','SJA_YXXS_D_D_D_PAV_M_CSST','SJA_KDCL_D_D_D_PAV_M_MANL','SJA_CLXS_D_D_D_PAV_M_CTST','SJA_JZYXQK_D_D_D_SIN_M_MANL','SJA_RBTBR_D_D_D_SIN_M_MANL','SJA_RBTJFZR_D_D_D_SIN_M_MANL')\n\t\t\t\t   \t\t \n\t\t\t\t   \t\n\t\t\t         \n\t\t\t         \n\t\t\t         \n\t\t\t         \n\t\t\t        UNION ALL(\n\t\t\t        \tSELECT DISTINCT A.R_R_ID,A.R_U_INFO_ID,A.ENTITY_ID,A.RECORD_LOCK,A.NORM_CODE,A.NORM_VALUE,A.URGE_PERIOD \n\t  \t\t\t   \t\tFROM  TEST_REPORT_RECORD A\n\t\t\t\t   \t\tWHERE A.NORM_CODE IN('SJA_FDL_D_D_D_SUM_M_MANL','SJA_ZGFH_D_D_D_MAX_M_MANL','SJA_KCM_D_D_D_SIN_M_MANL')\n\t\t\t\t   \t\tAND A.URGE_PERIOD_YEAR IN(2017)\n\t\t\t\t   \t\tAND A.URGE_PERIOD_MONTH IN(9)\n\t\t\t\t   \t\tAND A.URGE_PERIOD_DAY IN(20)\n\t\t\t        ) \n\t\t\t         \t\t\t        \t\t\t        \t\t\t        \n\t\t\t        ) RR\n\t\t\t\t) T ON N.DEF_CODE = T.NORM_CODE\n\t WHERE 1 = 1\n\t \n\t\tAND N.DEF_CODE IN('SJA_FDL_D_D_D_SUM_M_MANL','SJA_GDL_D_D_D_SUM_M_CTST','SJA_SCCYDL_D_D_D_SUM_M_MANL','SJA_JJDL_D_D_D_SUM_M_MANL','SJA_ZGFH_D_D_D_MAX_M_MANL','SJA_PJFH_D_D_D_PAV_M_CTST','SJA_FHLV_D_D_D_PAV_M_CTST','SJA_FHXS_D_D_D_PAV_M_CTST','SJA_HYM_D_D_D_SUM_M_MANL','SJA_HYY_D_D_D_SUM_M_MANL','SJA_FDZJZB_D_D_D_SUM_M_MANL','SJA_FDYMH_D_D_D_SUM_M_CTST','SJA_FDRYH_D_D_D_SUM_M_CTST','SJA_FDBZMHZ_D_D_D_PAV_M_CTST','SJA_GDBZMHZ_D_D_D_PAV_M_CTST','SJA_MPRL_D_D_D_SUM_M_CTST','SJA_KCM_D_D_D_SIN_M_MANL','SJA_HLKC_D_D_D_SUM_M_CTST','SJA_JJKCM_D_D_D_SUM_M_CTST','SJA_MFHKYTS_D_D_D_SUM_M_MANL','SJA_LYXS_D_D_D_SUM_M_CTST','SJA_YXXS_D_D_D_PAV_M_CSST','SJA_KDCL_D_D_D_PAV_M_MANL','SJA_CLXS_D_D_D_PAV_M_CTST','SJA_JZYXQK_D_D_D_SIN_M_MANL','SJA_RBTBR_D_D_D_SIN_M_MANL','SJA_RBTJFZR_D_D_D_SIN_M_MANL')\n\t \n\t \n\t \n    \t  AND '20170920' >= DATE_FORMAT(E.ACTIVE_START_TIME, '%Y%m%d') AND '20170920' <= DATE_FORMAT(E.ACTIVE_END_TIME, '%Y%m%d')  \n     \n\t \n       \tAND E.ENTITY_ID IN ('0202010110')\n     \n     \n     ORDER BY E.SORT, E.ENTITY_ID, FIND_IN_SET(SUBSTR(N.DEF_CODE, INSTR(N.DEF_CODE, '_') + 1), 'FDL_D_D_D_SUM_M_MANL,GDL_D_D_D_SUM_M_CTST,SCCYDL_D_D_D_SUM_M_MANL,JJDL_D_D_D_SUM_M_MANL,ZGFH_D_D_D_MAX_M_MANL,PJFH_D_D_D_PAV_M_CTST,FHLV_D_D_D_PAV_M_CTST,FHXS_D_D_D_PAV_M_CTST,HYM_D_D_D_SUM_M_MANL,HYY_D_D_D_SUM_M_MANL,FDZJZB_D_D_D_SUM_M_MANL,FDYMH_D_D_D_SUM_M_CTST,FDRYH_D_D_D_SUM_M_CTST,FDBZMHZ_D_D_D_PAV_M_CTST,GDBZMHZ_D_D_D_PAV_M_CTST,MPRL_D_D_D_SUM_M_CTST,KCM_D_D_D_SIN_M_MANL,HLKC_D_D_D_SUM_M_CTST,JJKCM_D_D_D_SUM_M_CTST,MFHKYTS_D_D_D_SUM_M_MANL,LYXS_D_D_D_SUM_M_CTST,YXXS_D_D_D_PAV_M_CSST,KDCL_D_D_D_PAV_M_MANL,CLXS_D_D_D_PAV_M_CTST,JZYXQK_D_D_D_SIN_M_MANL,RBTBR_D_D_D_SIN_M_MANL,RBTJFZR_D_D_D_SIN_M_MANL'), T.URGE_PERIOD

2、使用文本工具处理后结果如下

SELECT T.R_R_ID,T.RECORD_LOCK,T.NORM_VALUE,T.URGE_PERIOD,E.ENTITY_ID,E.PARENT_ID,E.NAME ENTITY_NAME,E.NOTE ENTITY_NOTE,E.TYPE_ID ENTITY_TYPE,N.NORM_ID,N.DEF_CODE NORM_CODE,N.TEST_COND,N.CALCULATE_MATH,N.CHINESE_MATH,D.NAME NORM_NAME,D.DATA_TYPE,CD.NAME UNIT_NAMEFROM  TEST_NORM NINNER JOIN  TEST_NORM_DEFIN D ON D.NORM_DEFIN_ID = N.NORM_DEFIN_IDINNER JOIN  TEST_ENTITY E ON E.CODE = N.ENTITY_CODELEFT JOIN SYS_GENERAL_CODE CD ON CD.ENCODE_VALUE = D.UNIT_NAMEAND CD.KEY_VALUE LIKE 'DWMC_%'LEFT JOIN (SELECT DISTINCT RR.*FROM (SELECT DISTINCT A.R_R_ID,A.R_U_INFO_ID,A.ENTITY_ID,A.RECORD_LOCK,A.NORM_CODE,A.NORM_VALUE,A.URGE_PERIODFROM  TEST_REPORT_RECORD AWHERE A.URGE_PERIOD_YEAR IN (2017)AND A.URGE_PERIOD_MONTH IN (9)AND A.URGE_PERIOD_DAY IN (20)AND A.NORM_CODE IN ('SJA_FDL_D_D_D_SUM_M_MANL','SJA_GDL_D_D_D_SUM_M_CTST','SJA_SCCYDL_D_D_D_SUM_M_MANL','SJA_JJDL_D_D_D_SUM_M_MANL','SJA_ZGFH_D_D_D_MAX_M_MANL','SJA_PJFH_D_D_D_PAV_M_CTST','SJA_FHLV_D_D_D_PAV_M_CTST','SJA_FHXS_D_D_D_PAV_M_CTST','SJA_HYM_D_D_D_SUM_M_MANL','SJA_HYY_D_D_D_SUM_M_MANL','SJA_FDZJZB_D_D_D_SUM_M_MANL','SJA_FDYMH_D_D_D_SUM_M_CTST','SJA_FDRYH_D_D_D_SUM_M_CTST','SJA_FDBZMHZ_D_D_D_PAV_M_CTST','SJA_GDBZMHZ_D_D_D_PAV_M_CTST','SJA_MPRL_D_D_D_SUM_M_CTST','SJA_KCM_D_D_D_SIN_M_MANL','SJA_HLKC_D_D_D_SUM_M_CTST','SJA_JJKCM_D_D_D_SUM_M_CTST','SJA_MFHKYTS_D_D_D_SUM_M_MANL','SJA_LYXS_D_D_D_SUM_M_CTST','SJA_YXXS_D_D_D_PAV_M_CSST','SJA_KDCL_D_D_D_PAV_M_MANL','SJA_CLXS_D_D_D_PAV_M_CTST','SJA_JZYXQK_D_D_D_SIN_M_MANL','SJA_RBTBR_D_D_D_SIN_M_MANL','SJA_RBTJFZR_D_D_D_SIN_M_MANL')UNION ALL (SELECT DISTINCT A.R_R_ID,A.R_U_INFO_ID,A.ENTITY_ID,A.RECORD_LOCK,A.NORM_CODE,A.NORM_VALUE,A.URGE_PERIODFROM  TEST_REPORT_RECORD AWHERE A.NORM_CODE IN('SJA_FDL_D_D_D_SUM_M_MANL','SJA_ZGFH_D_D_D_MAX_M_MANL','SJA_KCM_D_D_D_SIN_M_MANL')AND A.URGE_PERIOD_YEAR IN (2017)AND A.URGE_PERIOD_MONTH IN (9)AND A.URGE_PERIOD_DAY IN (20))) RR) T ON N.DEF_CODE =T.NORM_CODEWHERE 1 = 1AND N.DEF_CODE IN('SJA_FDL_D_D_D_SUM_M_MANL', 'SJA_GDL_D_D_D_SUM_M_CTST','SJA_SCCYDL_D_D_D_SUM_M_MANL', 'SJA_JJDL_D_D_D_SUM_M_MANL','SJA_ZGFH_D_D_D_MAX_M_MANL', 'SJA_PJFH_D_D_D_PAV_M_CTST','SJA_FHLV_D_D_D_PAV_M_CTST', 'SJA_FHXS_D_D_D_PAV_M_CTST','SJA_HYM_D_D_D_SUM_M_MANL', 'SJA_HYY_D_D_D_SUM_M_MANL','SJA_FDZJZB_D_D_D_SUM_M_MANL', 'SJA_FDYMH_D_D_D_SUM_M_CTST','SJA_FDRYH_D_D_D_SUM_M_CTST', 'SJA_FDBZMHZ_D_D_D_PAV_M_CTST','SJA_GDBZMHZ_D_D_D_PAV_M_CTST', 'SJA_MPRL_D_D_D_SUM_M_CTST','SJA_KCM_D_D_D_SIN_M_MANL', 'SJA_HLKC_D_D_D_SUM_M_CTST','SJA_JJKCM_D_D_D_SUM_M_CTST', 'SJA_MFHKYTS_D_D_D_SUM_M_MANL','SJA_LYXS_D_D_D_SUM_M_CTST', 'SJA_YXXS_D_D_D_PAV_M_CSST','SJA_KDCL_D_D_D_PAV_M_MANL', 'SJA_CLXS_D_D_D_PAV_M_CTST','SJA_JZYXQK_D_D_D_SIN_M_MANL', 'SJA_RBTBR_D_D_D_SIN_M_MANL','SJA_RBTJFZR_D_D_D_SIN_M_MANL')AND '20170920' >= DATE_FORMAT(E.ACTIVE_START_TIME, '%Y%m%d')AND '20170920' <= DATE_FORMAT(E.ACTIVE_END_TIME, '%Y%m%d')AND E.ENTITY_ID IN ('0202010110')ORDER BY E.SORT,E.ENTITY_ID,FIND_IN_SET(SUBSTR(N.DEF_CODE, INSTR(N.DEF_CODE, '_') + 1),'FDL_D_D_D_SUM_M_MANL,GDL_D_D_D_SUM_M_CTST,SCCYDL_D_D_D_SUM_M_MANL,JJDL_D_D_D_SUM_M_MANL,ZGFH_D_D_D_MAX_M_MANL,PJFH_D_D_D_PAV_M_CTST,FHLV_D_D_D_PAV_M_CTST,FHXS_D_D_D_PAV_M_CTST,HYM_D_D_D_SUM_M_MANL,HYY_D_D_D_SUM_M_MANL,FDZJZB_D_D_D_SUM_M_MANL,FDYMH_D_D_D_SUM_M_CTST,FDRYH_D_D_D_SUM_M_CTST,FDBZMHZ_D_D_D_PAV_M_CTST,GDBZMHZ_D_D_D_PAV_M_CTST,MPRL_D_D_D_SUM_M_CTST,KCM_D_D_D_SIN_M_MANL,HLKC_D_D_D_SUM_M_CTST,JJKCM_D_D_D_SUM_M_CTST,MFHKYTS_D_D_D_SUM_M_MANL,LYXS_D_D_D_SUM_M_CTST,YXXS_D_D_D_PAV_M_CSST,KDCL_D_D_D_PAV_M_MANL,CLXS_D_D_D_PAV_M_CTST,JZYXQK_D_D_D_SIN_M_MANL,RBTBR_D_D_D_SIN_M_MANL,RBTJFZR_D_D_D_SIN_M_MANL'),T.URGE_PERIOD

注该语句查询经过索引但是查询时间还是高达30多秒。

4、通过应用管理员反馈一条简单的SQL查询也很久无法相应，同样该查询走了索引，用时36.7s

select *from test_report_record rwhere r.NORM_CODE = 'SJC2_CJX_D_D_D_SUM_M_MANL'and r.URGE_PERIOD_YEAR = 2017and r.URGE_PERIOD_MONTH = 9and r.URGE_PERIOD_DAY = 20

test_report_record 该表记录数比较大，为了得到该表的数据进行了count操作

select count(1) from test_report_record.

但是无法count出结果，应该是test_report_record表出现问题。且经过查询发现，该表数据量较大并未做过分区表，根据应用需求该表设计为分区表比较合理。

5、新建一张test_report_record2分区表，将test_report_record表的数据导入一份到该表，使用相同语句进行查询确认是否相应速度得到提升。

CREATE TABLE `test_report_record2` (`R_R_ID` varchar(32) NOT NULL DEFAULT '',`NORM_VALUE` varchar(2000) DEFAULT NULL ,`INPUT_VALUE` varchar(2000) DEFAULT NULL,`NORM_CODE` varchar(64) NOT NULL DEFAULT '',`R_U_INFO_ID` varchar(32) DEFAULT NULL ,`PUSH_TIME` datetime DEFAULT NULL ,`STATE` int(11) DEFAULT NULL ,`SYS_USER_ID` varchar(32) DEFAULT NULL,`URGE_PERIOD_YEAR` int(11) ,`URGE_PERIOD_MONTH` int(11) DEFAULT NULL,`URGE_PERIOD_DAY` int(11) DEFAULT NULL,`URGE_PERIOD` date DEFAULT NULL,`ENTITY_ID` varchar(64) DEFAULT NULL ,`RECORD_LOCK` int(4) DEFAULT NULL ,`SEND` int(11) DEFAULT '0'  ,PRIMARY KEY (`R_R_ID`,`URGE_PERIOD_YEAR`),KEY `R_U_INFO_ID2` (`R_U_INFO_ID`) USING BTREE,KEY `NORM_CODE2` (`NORM_CODE`) USING BTREE,KEY `URGE_PERIOD2` (`URGE_PERIOD`) USING BTREE,KEY `URGE2` (`URGE_PERIOD_YEAR`,`URGE_PERIOD_MONTH`,`URGE_PERIOD_DAY`) USING BTREE,KEY `PUSH_TIME2` (`PUSH_TIME`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8  PARTITION BY RANGE(URGE_PERIOD_YEAR) (PARTITION p_2015 VALUES LESS THAN (2015),PARTITION p_2016 VALUES LESS THAN (2016),PARTITION p_2017 VALUES LESS THAN (2017),PARTITION p_2018 VALUES LESS THAN (2018),PARTITION p_2019 VALUES LESS THAN (2019),PARTITION p_2020 VALUES LESS THAN (2020),PARTITION p_2021 VALUES LESS THAN (2021),PARTITION p_2022 VALUES LESS THAN (2022),PARTITION p_2023 VALUES LESS THAN (2023),PARTITION p_2024 VALUES LESS THAN (2024),PARTITION p_2025 VALUES LESS THAN (2025),PARTITION p_catchall VALUES LESS THAN MAXVALUE);

6、停止mysql数据库停止应用。检查发现Mysql使用的是默认的参数，针对这种情况对相关参数进行了调整

myisam_sort_buffer_size=128M
key_buffer_size=2048M
read_buffer_size=64M
read_rnd_buffer_size=64M
sort_buffer_size=16M
innodb_log_buffer_size=16M
innodb_buffer_pool_size=2048M
innodb_log_file_size=128M
max_allowed_packet=32M

7、启动MYSQL，不启动应用。统计test_report_record数据，依旧无法得到结果，对该表进行大的查询时候会报1205的错误。

8、确认test_report_record该表统计信息有问题（oracle经验），决定对该表进行统计信息的收集和优化工作

mysql> analyze table test_report_record

9、分析完毕改表后，重新对test_report_record进行统计，结果如下664w行用了73s

10、使用如下方法插入并将test_report_record该表改成分区表

insert into test_report_record2 select * from test_report_record;
alter table test_report_record rename test_report_recordbak;
alter table test_report_record2 rename test_report_record;

11、验证结果现在查询仅需要0.008s就有结果，比之前快了几个数量级，目前的执行计划也走了分区

12、打开应用，检查报表，访问速度均得到较大的提升，优化完毕。

四、问题总结

随着系统的使用，应用数据会越来越多，应用在设计之初的性能问题就会慢慢表现出来，此时维护的重点就需要对系统进行优化。

1、mysql安装时初始参数一般都较小，可以根据服务器的配置进行提升，上面的参数仅供参考，实际上还可设置大点。

2、mysql的表的信息收集机制和oracle还是有比较大的不同，最近发现在oracle本来不会成为问题的在Mysql上运行会存在巨大的性能问题，再出现表无法及时返回时注意使用optimize和analyze对表进行分析和优化。

3、对于部分预计数据量很大的表可以考虑使用分区表。

4、不管是什么类型的数据库都要注意对数据进行备份，防止表数据意外丢失或者坏道时可以保证数据恢复。