@zabbix数据库历史与趋势数据占用优化（mysql存储查询）
2024-06-18 02:20:31
文章目录
- 1.数据库大小查询
- 1）数据库大小查询
- 2）数据导入不记录binlog
- 3）数据库查看语句
- 4）mysqldump（逻辑备份）
- 2.zabbix案例
- 1）磁盘占用优化
- 2）优化表的大小
- 3）服务器上查询确认表大小
- 4）数据库分区自动化脚本
1.数据库大小查询
1）数据库大小查询
进入数据库，切换到information_schema 库（这个库存放了其他的数据库的关联信息）
#切入数据库信息库
mysql> information_schema#查询所有数据的大小
mysql> select concat(round(sum(data_length/1024/1024),2),'MB') as data from tables;#查询zabbix库的大小
mysql> select concat(round(sum(data_length/1024/1024),2),'MB') as data from tables where table_schema='zabbix';#查询zabbix库hosts表大小
mysql> select concat(round(sum(data_length/1024/1024),2),'MB') as data from tables where table_schema='zabbix' and table_name='hosts';
2）数据导入不记录binlog
临时关闭binlog日志记录
#查看binlog状态
mysql> show variables like "sql_log_bin";#临时关闭binlog日志
mysql> set sql_log_bin=0;#临时开启binlog日志
mysql> set sql_log_bin=1;#刷新字段
mysql> flush privileges;
3）数据库查看语句
#查看数据库的创建语句
mysql> SHOW CREATE DATABASE zabbix #查看表的创建语句
mysql> SHOW CREATE TABLE history #查看表结构
mysql> DESC student
4）mysqldump（逻辑备份）
#排除某些表备份数据库（格式：--ignore-table=databases.tables）
shell> mysqldump -u"root" -p"password" -h"192.168.10.10" -P"3306" zabbix --single-transaction --ignore-table=zabbix.history > date.sql#排除某些表，压缩备份数据库
shell> mysqldump -u"root" -p"password" zabbix --ignore-table=zabbix.history --ignore-table=zabbix.history_uint --ignore-table=zabbix.trends_uint --ignore-table=zabbix.trends | gzip > zabbix.sql.gz#不包含数据导出数据库
shell> mysqldump -u"root" -p"zabbbix" -d databases > databases$(date +%F).sql#导出单个数据表结构（不包含数据）
shell> mysqldump -h localhost -u"root" -p"password" -d database table > table$(date +%F).sql#单独备份多张表的表结构(只备份表结构)
shell> mysqldump -u"root" -p"password" -d zabbix zabbix.history zabbix.history_uint zabbix.trends_uint zabbix.trends > table$(date +%F).sql#mysql导入数据（注意sql文件的路径）
mysql> source /root/zabbix.sql;#使用zcat导入压缩sql.gz数据
zcat ./mysql.sql.gz |mysql -u"root" -p"password" database
2.zabbix案例
1）磁盘占用优化
假设zabbix占用空间大，需要优化
zabbix占用大的表 一般为两个表：
history：历史数据存储表
trends：趋势数据存储表
#查看zabbix库容量
mysql> select concat(round(sum(data_length/1024/1024),2),'MB') as data from tables where table_schema='zabbix';
+-----------+
| data |
+-----------+
| 1760.81MB |
+-----------+
1 row in set (0.01 sec)#查询zabbix库表大小，并排序
mysql> SELECT TABLE_NAME,DATA_LENGTH,INDEX_LENGTH,(DATA_LENGTH+INDEX_LENGTH) as length,TABLE_ROWS,concat(round((DATA_LENGTH+INDEX_LENGTH)/1024/1024,3), 'MB') as total_size FROM information_schema.TABLES WHERE TABLE_SCHEMA='zabbix' order by length desc;
+----------------------------+-------------+--------------+------------+------------+------------+
| TABLE_NAME | DATA_LENGTH | INDEX_LENGTH | length | TABLE_ROWS | total_size |
+----------------------------+-------------+--------------+------------+------------+------------+
| history_uint | 1125138432 | 503332864 | 1628471296 | 9288216 | 1553.031MB |
| trends_uint | 360185856 | 0 | 360185856 | 4578474 | 343.500MB |
| history | 192610304 | 89915392 | 282525696 | 3206240 | 269.438MB |
| trends | 92454912 | 0 | 92454912 | 1207380 | 88.172MB |
| history_str | 46219264 | 25247744 | 71467008 | 822685 | 68.156MB |
| alerts | 6832128 | 2867200 | 9699328 | 14900 | 9.250MB |
| items | 4669440 | 2359296 | 7028736 | 6470 | 6.703MB |
| events | 3686400 | 3162112 | 6848512 | 26401 | 6.531MB |
| history_text | 3162112 | 16384 | 3178496 | 387 | 3.031MB |
| triggers | 2605056 | 294912 | 2899968 | 2860 | 2.766MB |
| event_recovery | 1589248 | 540672 | 2129920 | 10712 | 2.031MB |
| images | 1589248 | 16384 | 1605632 | 138 | 1.531MB |
| functions | 409600 | 475136 | 884736 | 5965 | 0.844MB |
| items_applications | 360448 | 458752 | 819200 | 5626 | 0.781MB |
| item_preproc | 475136 | 212992 | 688128 | 5571 | 0.656MB |
| item_discovery | 360448 | 294912 | 655360 | 3818 | 0.625MB |
| auditlog | 311296 | 180224 | 491520 | 2709 | 0.469MB |
| graphs_items | 229376 | 196608 | 425984 | 2523 | 0.406MB |
| graphs | 180224 | 114688 | 294912 | 863 | 0.281MB |
| trigger_depends | 114688 | 147456 | 262144 | 1362 | 0.250MB |
| item_rtdata | 196608 | 0 | 196608 | 3021 | 0.188MB |
| profiles | 114688 | 81920 | 196608 | 700 | 0.188MB |
| hosts | 98304 | 98304 | 196608 | 192 | 0.188MB |
| item_application_prototype | 81920 | 114688 | 196608 | 945 | 0.188MB |
| hostmacro | 98304 | 81920 | 180224 | 749 | 0.172MB |
| item_condition | 114688 | 49152 | 163840 | 897 | 0.156MB |
| applications | 81920 | 49152 | 131072 | 894 | 0.125MB |
| mappings | 81920 | 49152 | 131072 | 1139 | 0.125MB |
| widget_field | 16384 | 98304 | 114688 | 134 | 0.109MB |
| media_type | 98304 | 16384 | 114688 | 10 | 0.109MB |
| sysmaps_elements | 16384 | 81920 | 98304 | 0 | 0.094MB |
| auditlog_details | 81920 | 16384 | 98304 | 476 | 0.094MB |
| trigger_discovery | 65536 | 16384 | 81920 | 780 | 0.078MB |
| application_template | 49152 | 32768 | 81920 | 417 | 0.078MB |
| sysmaps | 16384 | 65536 | 81920 | 0 | 0.078MB |
| event_tag | 65536 | 16384 | 81920 | 565 | 0.078MB |
| httptest | 16384 | 65536 | 81920 | 2 | 0.078MB |
| sysmaps_links | 16384 | 49152 | 65536 | 0 | 0.063MB |
| problem | 16384 | 49152 | 65536 | 95 | 0.063MB |
| scripts | 16384 | 49152 | 65536 | 3 | 0.063MB |
| screens_items | 49152 | 16384 | 65536 | 211 | 0.063MB |
| group_prototype | 16384 | 49152 | 65536 | 14 | 0.063MB |
| acknowledges | 16384 | 49152 | 65536 | 17 | 0.063MB |
| event_suppress | 16384 | 49152 | 65536 | 0 | 0.063MB |
| escalations | 16384 | 49152 | 65536 | 9 | 0.063MB |
| host_discovery | 16384 | 32768 | 49152 | 4 | 0.047MB |
| slideshow_user | 16384 | 32768 | 49152 | 0 | 0.047MB |
| correlation | 16384 | 32768 | 49152 | 0 | 0.047MB |
| proxy_dhistory | 16384 | 32768 | 49152 | 0 | 0.047MB |
| slides | 16384 | 32768 | 49152 | 0 | 0.047MB |
| sysmaps_link_triggers | 16384 | 32768 | 49152 | 0 | 0.047MB |
| application_prototype | 16384 | 32768 | 49152 | 141 | 0.047MB |
| icon_mapping | 16384 | 32768 | 49152 | 0 | 0.047MB |
| opcommand_hst | 16384 | 32768 | 49152 | 0 | 0.047MB |
| service_alarms | 16384 | 32768 | 49152 | 0 | 0.047MB |
| sysmap_usrgrp | 16384 | 32768 | 49152 | 0 | 0.047MB |
| application_discovery | 16384 | 32768 | 49152 | 84 | 0.047MB |
| icon_map | 16384 | 32768 | 49152 | 0 | 0.047MB |
| dservices | 16384 | 32768 | 49152 | 35 | 0.047MB |
| opcommand_grp | 16384 | 32768 | 49152 | 0 | 0.047MB |
| media | 16384 | 32768 | 49152 | 3 | 0.047MB |
| sysmap_user | 16384 | 32768 | 49152 | 0 | 0.047MB |
| httptestitem | 16384 | 32768 | 49152 | 6 | 0.047MB |
| services_links | 16384 | 32768 | 49152 | 0 | 0.047MB |
| drules | 16384 | 32768 | 49152 | 0 | 0.047MB |
| optemplate | 16384 | 32768 | 49152 | 0 | 0.047MB |
| hosts_templates | 16384 | 32768 | 49152 | 191 | 0.047MB |
| actions | 16384 | 32768 | 49152 | 7 | 0.047MB |
| config | 16384 | 32768 | 49152 | 0 | 0.047MB |
| maintenances_windows | 16384 | 32768 | 49152 | 0 | 0.047MB |
| screens | 16384 | 32768 | 49152 | 53 | 0.047MB |
| opmessage_usr | 16384 | 32768 | 49152 | 6 | 0.047MB |
| hosts_groups | 16384 | 32768 | 49152 | 196 | 0.047MB |
| users_groups | 16384 | 32768 | 49152 | 4 | 0.047MB |
| maintenances_hosts | 16384 | 32768 | 49152 | 0 | 0.047MB |
| screen_usrgrp | 16384 | 32768 | 49152 | 0 | 0.047MB |
| opmessage_grp | 16384 | 32768 | 49152 | 4 | 0.047MB |
| httpstepitem | 16384 | 32768 | 49152 | 6 | 0.047MB |
| autoreg_host | 16384 | 32768 | 49152 | 0 | 0.047MB |
| maintenances_groups | 16384 | 32768 | 49152 | 0 | 0.047MB |
| screen_user | 16384 | 32768 | 49152 | 0 | 0.047MB |
| sysmap_element_trigger | 16384 | 32768 | 49152 | 0 | 0.047MB |
| dashboard_usrgrp | 16384 | 32768 | 49152 | 0 | 0.047MB |
| maintenances | 16384 | 32768 | 49152 | 0 | 0.047MB |
| rights | 16384 | 32768 | 49152 | 23 | 0.047MB |
| slideshows | 16384 | 32768 | 49152 | 0 | 0.047MB |
| dashboard_user | 16384 | 32768 | 49152 | 1 | 0.047MB |
| task | 16384 | 32768 | 49152 | 0 | 0.047MB |
| slideshow_usrgrp | 16384 | 32768 | 49152 | 0 | 0.047MB |
| opgroup | 16384 | 32768 | 49152 | 1 | 0.047MB |
| interface | 16384 | 32768 | 49152 | 28 | 0.047MB |
| tag_filter | 16384 | 32768 | 49152 | 0 | 0.047MB |
| operations | 16384 | 16384 | 32768 | 16 | 0.031MB |
| lld_macro_path | 16384 | 16384 | 32768 | 58 | 0.031MB |
| proxy_history | 16384 | 16384 | 32768 | 0 | 0.031MB |
| opconditions | 16384 | 16384 | 32768 | 0 | 0.031MB |
| corr_operation | 16384 | 16384 | 32768 | 0 | 0.031MB |
| services | 16384 | 16384 | 32768 | 0 | 0.031MB |
| globalmacro | 16384 | 16384 | 32768 | 0 | 0.031MB |
| proxy_autoreg_host | 16384 | 16384 | 32768 | 0 | 0.031MB |
| sessions | 16384 | 16384 | 32768 | 175 | 0.031MB |
| services_times | 16384 | 16384 | 32768 | 0 | 0.031MB |
| expressions | 16384 | 16384 | 32768 | 10 | 0.031MB |
| corr_condition | 16384 | 16384 | 32768 | 0 | 0.031MB |
| widget | 16384 | 16384 | 32768 | 15 | 0.031MB |
| problem_tag | 16384 | 16384 | 32768 | 10 | 0.031MB |
| opcommand | 16384 | 16384 | 32768 | 0 | 0.031MB |
| history_log | 16384 | 16384 | 32768 | 56 | 0.031MB |
| config_autoreg_tls | 16384 | 16384 | 32768 | 0 | 0.031MB |
| valuemaps | 16384 | 16384 | 32768 | 102 | 0.031MB |
| sysmap_url | 16384 | 16384 | 32768 | 0 | 0.031MB |
| httptest_field | 16384 | 16384 | 32768 | 0 | 0.031MB |
| dhosts | 16384 | 16384 | 32768 | 35 | 0.031MB |
| media_type_param | 16384 | 16384 | 32768 | 123 | 0.031MB |
| corr_condition_group | 16384 | 16384 | 32768 | 0 | 0.031MB |
| usrgrp | 16384 | 16384 | 32768 | 6 | 0.031MB |
| sysmap_shape | 16384 | 16384 | 32768 | 0 | 0.031MB |
| dchecks | 16384 | 16384 | 32768 | 1 | 0.031MB |
| group_discovery | 16384 | 16384 | 32768 | 0 | 0.031MB |
| conditions | 16384 | 16384 | 32768 | 51 | 0.031MB |
| sysmap_element_url | 16384 | 16384 | 32768 | 0 | 0.031MB |
| users | 16384 | 16384 | 32768 | 3 | 0.031MB |
| opmessage | 16384 | 16384 | 32768 | 13 | 0.031MB |
| httpstep_field | 16384 | 16384 | 32768 | 0 | 0.031MB |
| interface_discovery | 16384 | 16384 | 32768 | 0 | 0.031MB |
| host_tag | 16384 | 16384 | 32768 | 5 | 0.031MB |
| httpstep | 16384 | 16384 | 32768 | 2 | 0.031MB |
| graph_theme | 16384 | 16384 | 32768 | 4 | 0.031MB |
| regexps | 16384 | 16384 | 32768 | 5 | 0.031MB |
| trigger_tag | 16384 | 16384 | 32768 | 85 | 0.031MB |
| maintenance_tag | 16384 | 16384 | 32768 | 0 | 0.031MB |
| hstgrp | 16384 | 16384 | 32768 | 25 | 0.031MB |
| dashboard | 16384 | 16384 | 32768 | 2 | 0.031MB |
| graph_discovery | 16384 | 16384 | 32768 | 235 | 0.031MB |
| globalvars | 16384 | 0 | 16384 | 0 | 0.016MB |
| ids | 16384 | 0 | 16384 | 56 | 0.016MB |
| corr_condition_tagvalue | 16384 | 0 | 16384 | 0 | 0.016MB |
| corr_condition_tagpair | 16384 | 0 | 16384 | 0 | 0.016MB |
| timeperiods | 16384 | 0 | 16384 | 0 | 0.016MB |
| housekeeper | 16384 | 0 | 16384 | 0 | 0.016MB |
| task_remote_command_result | 16384 | 0 | 16384 | 0 | 0.016MB |
| corr_condition_tag | 16384 | 0 | 16384 | 0 | 0.016MB |
| task_remote_command | 16384 | 0 | 16384 | 0 | 0.016MB |
| task_close_problem | 16384 | 0 | 16384 | 0 | 0.016MB |
| dbversion | 16384 | 0 | 16384 | 0 | 0.016MB |
| task_check_now | 16384 | 0 | 16384 | 0 | 0.016MB |
| task_acknowledge | 16384 | 0 | 16384 | 0 | 0.016MB |
| opinventory | 16384 | 0 | 16384 | 0 | 0.016MB |
| host_inventory | 16384 | 0 | 16384 | 0 | 0.016MB |
+----------------------------+-------------+--------------+------------+------------+------------+
149 rows in set (0.01 sec)
2）优化表的大小
清理表的村存储数据：
表数据只需要保留一个月，删除某个表超时一个月的数据；
删除表数据，需要获取获取时间戳，删除表数据后，执行optimize table table_name 立刻释放磁盘空间
【时间戳在线换算】
#删除之前的1656864000存储的数据，通过时间戳换算，数据过大，可能很慢
mysql> delete from zabbix.history_uint where clock < 1656864000;
Query OK, 184017 rows affected (1 min 13.99 sec)#立即释放磁盘占用空间
mysql> optimize table zabbix.history_uint;
+---------------------+----------+----------+-------------------------------------------------------------------+
| Table | Op | Msg_type | Msg_text |
+---------------------+----------+----------+-------------------------------------------------------------------+
| zabbix.history_uint | optimize | note | Table does not support optimize, doing recreate + analyze instead |
| zabbix.history_uint | optimize | status | OK |
+---------------------+----------+----------+-------------------------------------------------------------------+
2 rows in set (3 min 7.93 sec)#继续查看表的大小（确认到表已经减少了）
mysql> select concat(round(sum(data_length/1024/1024),2),'MB') as data from tables where table_schema='zabbix' and table_name='history_uint';
+----------+
| data |
+----------+
| 542.98MB |
+----------+
1 row in set (0.00 sec)
3）服务器上查询确认表大小
筛选 查看数据库大小
#文件大小查看(查看zabbix库中表的ibd文件大小)
shell> du -s /var/lib/mysql/zabbix/* |sort -rn |head -10
847876 /var/lib/mysql/zabbix/history_uint.ibd
368644 /var/lib/mysql/zabbix/trends_uint.ibd
290820 /var/lib/mysql/zabbix/history.ibd
135172 /var/lib/mysql/zabbix/history_str.ibd
110596 /var/lib/mysql/zabbix/trends.ibd
57348 /var/lib/mysql/zabbix/history_text.ibd
18436 /var/lib/mysql/zabbix/alerts.ibd
14340 /var/lib/mysql/zabbix/events.ibd
14336 /var/lib/mysql/zabbix/items.ibd
10244 /var/lib/mysql/zabbix/event_recovery.ibd
4）数据库分区自动化脚本
【数据库分区脚本】
#数据库分区脚本
shell> wget https://dl.cactifans.com/zabbix/partitiontables_gt_zbx34.sh#执行脚本即可（脚本配置无问题）
shell> ./partitiontables_gt_zbx34.sh#查看分区后的表数据（history）
mysql> use zabbix;
mysql> select partition_name ,partition_expression,partition_description,table_rows from information_schema.partitions where table_schema=schema() and table_name='history';
脚本基础应用配置：
#优化脚本（基础配置）
shell> egrep "^[^#]" partitiontables_gt_zbx34.sh
EMAIL="zeny@foxmail.com" #邮箱
daily_history_min=30 #历史数据存储保留时间
monthly_history_min=12 #趋势数据存储时间配置
first_year=`date +"%Y"`
last_year=$first_year
cur_month=`date +"%m"|sed 's/^0*//'`
if [ $cur_month -eq 12 ]; thenlast_year=$((first_year+1))cur_month=1
fi
y=`date +"%Y"`
SQL="/tmp/partition.sql"
PATHTOCRON="/usr/local/zabbix/cron.d"
PATHTOMAILBIN="/usr/bin/mail"
DUMP_FILE=/tmp/zabbix.sql
function usage {cat <<_EOF_
$0 [-h host][-u user][-p password][-d min_days][-y startyear][-n][-s][-e email_address][-b]-h host database host-u user db user-p password user password-d min_days Minimum number of days of history to keep (default: $daily_history_min)-m min_months Minimum number of months to keep trends (default: $monthly_history_min)-y startyear First year to set up with partitions-n noninteractive Run without questions - careful, make sure you know what is going to happen. Needs my.cnf with correct permissions.-b backup Create backup of DB in $DUMP_FILE before alterations (only works with non-interactive mode, -n)-s simulate Create SQL file that would be executed for examination ($SQL)-e email Email address to receive partition update report (default: $EMAIL)
After running this script, don't forget to disable housekeeping if
you didn't have the script disable it, and add the following cronjob### Option: DisableHousekeeping# If set to 1, disables housekeeping.## Mandatory: no# Range: 0-1################### Uncomment and change the following line to 1 in ################### Then restart the zabbix serverDisableHousekeeping=1
Cron job
0 0 * * * $PATHTOCRON/housekeeping.sh
_EOF_exit
}
DBHOST=localhost #数据库连接地址
DBUSER=zabbix #数据库连接用户
DBPASS=zabbix #数据库登录密码
SIMULATE=0
NONINTERACTIVE=0
BACKUP=0
while getopts "m:nsbe:h:u:p:d:y:?h" flag; docase $flag inh) DBHOST=$OPTARG ;;u) DBUSER=$OPTARG ;;p) DBPASS=$OPTARG ;;e) EMAIL=$OPTARG ;;s) SIMULATE=1 ;;n) NONINTERACTIVE=1 ;;b) BACKUP=1 ;;d) h=$OPTARGif [ $h -gt 0 ] 2>/dev/null; thendaily_history_min=$helseecho "Invalid daily history min, exiting"exit 1fi;;m) h=$OPTARGif [ $h -gt 0 ] 2>/dev/null; thenmonthly_history_min=$helseecho "Invalid monthly history min, exiting"exit 1fi;;y) yy=$OPTARGif [ $yy -lt $y -a $yy -gt 2000 ] 2>/dev/null; thenfirst_year=$yyelseecho "Invalid year, exiting"exit 1fi;;?|h) usage ;;esac
done
shift $((OPTIND-1))
if [ $NONINTERACTIVE != 1 ]; thenecho "Ready to partition tables."
fi
if [ $SIMULATE = 0 ]; thenif [ $NONINTERACTIVE = 1 ]; thenmysql -B -h $DBHOST -e "GRANT CREATE ROUTINE ON zabbix.* TO '$DBUSER'@'localhost';"mysql -h $DBHOST -e "GRANT LOCK TABLES ON zabbix.* TO '$DBUSER'@'$DBHOST' IDENTIFIED BY '$DBPASS';"if [ $BACKUP = 1 ]; thenmysqldump --opt -h $DBHOST -u $DBUSER -p$DBPASS zabbix --result-file=$DUMP_FILErc=$?if [ $rc -ne 0 ]; thenecho "Error during mysqldump, exit code: $rc"fifielseecho -e "\nReady to update permissions of Zabbix user to create routines\n"echo -n "Enter root DB user: "read DBADMINUSERecho -n "Enter $DBADMINUSER password: "read DBADMINPASSmysql -B -h $DBHOST -u $DBADMINUSER -p$DBADMINPASS -e "GRANT CREATE ROUTINE ON zabbix.* TO '$DBUSER'@'localhost';"echo -e "\n"echo -ne "\nDo you want to backup the database (recommended) (Y/n): "read ynif [ "$yn" != "n" -a "$yn" != "N" ]; thenecho -e "\nEnter output file, press return for default of $DUMP_FILE"read df[ "$df" != "" ] && DUMP_FILE=$df## Lock tables is needed for a good mysqldump#echo "GRANT LOCK TABLES ON zabbix.* TO '${DBUSER}'@'${DBHOST}' IDENTIFIED BY '${DBPASS}';" | mysql -h${DBHOST} -u${DBADMINUSER} --password=${DBADMINPASS}mysqldump --opt -h ${DBHOST} -u ${DBUSER} -p${DBPASS} zabbix --result-file=${DUMP_FILE}rc=$?if [ $rc -ne 0 ]; thenecho "Error during mysqldump, rc: $rc"echo "Do you wish to continue (y/N): "read yn[ "yn" != "y" -a "$yn" != "Y" ] && exitelseecho "Mysqldump succeeded!, proceeding with upgrade..."fielseecho "Are you certain you have a backup (y/N): "read yn[ "$yn" != 'y' -a "$yn" != "Y" ] && exitfifi
fi
if [ $NONINTERACTIVE = 1 ]; thenyn='y'
elseecho -e "\n\nReady to proceed:"echo -e "\nStarting yearly partioning at: $first_year"echo "and ending at: $last_year"echo "With $daily_history_min days of daily history"echo -e "\n\nReady to proceed (Y/n): "read yn[ "$yn" = 'n' -o "$yn" = "N" ] && exit
fi
DAILY="history history_log history_str history_text history_uint"
DAILY_IDS="itemid id itemid id itemid"
MONTHLY="trends trends_uint"
MONTHLY_IDS=""
TABLES="$DAILY $MONTHLY"
IDS="$DAILY_IDS $MONTHLY_IDS"
if [ $NONINTERACTIVE != 1 ]; thenecho "Use zabbix; SELECT 'Altering tables';" >$SQL
elseecho "Use zabbix;" >$SQL
fi
cnt=0
for i in $TABLES; doif [ $NONINTERACTIVE != 1 ]; thenecho "Altering table: $i"echo "SELECT '$i';" >>$SQLficnt=$((cnt+1))case $i inhistory_log)#echo "ALTER TABLE $i DROP KEY history_log_2;" >>$SQL#echo "ALTER TABLE $i ADD KEY history_log_2(itemid, id);" >>$SQL#echo "ALTER TABLE $i DROP PRIMARY KEY ;" >>$SQL#id=`echo $IDS | cut -f$cnt -d" "`#echo "ALTER TABLE $i ADD KEY ${i}id ($id);" >>$SQL;;history_text)#echo "ALTER TABLE $i DROP KEY history_text_2;" >>$SQL#echo "ALTER TABLE $i ADD KEY history_text_2 (itemid, clock);" >>$SQL#echo "ALTER TABLE $i DROP PRIMARY KEY ;" >>$SQL#id=`echo $IDS | cut -f$cnt -d" "`#echo "ALTER TABLE $i ADD KEY ${i}id ($id);" >>$SQL;;esac
done
echo -en "\n" >>$SQL
for i in $MONTHLY; doif [ $NONINTERACTIVE != 1 ]; thenecho "Creating monthly partitions for table: $i"echo "SELECT '$i';" >>$SQLfiecho "ALTER TABLE $i PARTITION BY RANGE( clock ) (" >>$SQLfor y in `seq $first_year $last_year`; dolast_month=12[ $y -eq $last_year ] && last_month=$((cur_month+1))for m in `seq 1 $last_month`; do[ $m -lt 10 ] && m="0$m"ms=`date +"%Y-%m-01" -d "$m/01/$y +1 month"`pname="p${y}${m}"echo -n "PARTITION $pname VALUES LESS THAN (UNIX_TIMESTAMP(\"$ms 00:00:00\"))" >>$SQL[ $m -ne $last_month -o $y -ne $last_year ] && echo -n "," >>$SQLecho -ne "\n" >>$SQLdonedoneecho ");" >>$SQL
done
for i in $DAILY; doif [ $NONINTERACTIVE != 1 ]; thenecho "Creating daily partitions for table: $i"echo "SELECT '$i';" >>$SQLfiecho "ALTER TABLE $i PARTITION BY RANGE( clock ) (" >>$SQLfor d in `seq -$daily_history_min 2`; dods=`date +"%Y-%m-%d" -d "$d day +1 day"`pname=`date +"%Y%m%d" -d "$d day"`echo -n "PARTITION p$pname VALUES LESS THAN (UNIX_TIMESTAMP(\"$ds 00:00:00\"))" >>$SQL[ $d -ne 2 ] && echo -n "," >>$SQLecho -ne "\n" >>$SQLdoneecho ");" >>$SQL
done
if [ $NONINTERACTIVE != 1 ]; thencat >>$SQL <<_EOF_
SELECT "Installing procedures";
_EOF_
fi
cat >>$SQL <<_EOF_
/**************************************************************MySQL Auto Partitioning Procedure for Zabbix 1.8http://zabbixzone.com/zabbix/partitioning-tables/Author: Ricardo Santos (rsantos at gmail.com)Version: 20110518
**************************************************************/
DELIMITER //
DROP PROCEDURE IF EXISTS zabbix.create_zabbix_partitions; //
CREATE PROCEDURE zabbix.create_zabbix_partitions ()
BEGIN
_EOF_
for i in $DAILY; doecho " CALL zabbix.create_next_partitions(\"zabbix\",\"$i\");" >>$SQLecho " CALL zabbix.drop_old_partitions(\"zabbix\",\"$i\");" >>$SQL
done
echo -en "\n" >>$SQL
for i in $MONTHLY; doecho " CALL zabbix.create_next_monthly_partitions(\"zabbix\",\"$i\");" >>$SQLecho " CALL zabbix.drop_old_monthly_partitions(\"zabbix\",\"$i\");" >>$SQL
done
cat >>$SQL <<_EOF_
END //
DROP PROCEDURE IF EXISTS zabbix.create_next_partitions; //
CREATE PROCEDURE zabbix.create_next_partitions (SCHEMANAME varchar(64), TABLENAME varchar(64))
BEGINDECLARE NEXTCLOCK timestamp;DECLARE PARTITIONNAME varchar(16);DECLARE CLOCK int;SET @totaldays = 7;SET @i = 1;createloop: LOOPSET NEXTCLOCK = DATE_ADD(NOW(),INTERVAL @i DAY);SET PARTITIONNAME = DATE_FORMAT( NEXTCLOCK, 'p%Y%m%d' );SET CLOCK = UNIX_TIMESTAMP(DATE_FORMAT(DATE_ADD( NEXTCLOCK ,INTERVAL 1 DAY),'%Y-%m-%d 00:00:00'));CALL zabbix.create_partition( SCHEMANAME, TABLENAME, PARTITIONNAME, CLOCK );SET @i=@i+1;IF @i > @totaldays THENLEAVE createloop;END IF;END LOOP;
END //
DROP PROCEDURE IF EXISTS zabbix.drop_old_partitions; //
CREATE PROCEDURE zabbix.drop_old_partitions (SCHEMANAME varchar(64), TABLENAME varchar(64))
BEGINDECLARE OLDCLOCK timestamp;DECLARE PARTITIONNAME varchar(16);DECLARE CLOCK int;SET @mindays = $daily_history_min;SET @maxdays = @mindays+4;SET @i = @maxdays;droploop: LOOPSET OLDCLOCK = DATE_SUB(NOW(),INTERVAL @i DAY);SET PARTITIONNAME = DATE_FORMAT( OLDCLOCK, 'p%Y%m%d' );CALL zabbix.drop_partition( SCHEMANAME, TABLENAME, PARTITIONNAME );SET @i=@i-1;IF @i <= @mindays THENLEAVE droploop;END IF;END LOOP;
END //
DROP PROCEDURE IF EXISTS zabbix.create_next_monthly_partitions; //
CREATE PROCEDURE zabbix.create_next_monthly_partitions (SCHEMANAME varchar(64), TABLENAME varchar(64))
BEGINDECLARE NEXTCLOCK timestamp;DECLARE PARTITIONNAME varchar(16);DECLARE CLOCK int;SET @totalmonths = 3;SET @i = 1;createloop: LOOPSET NEXTCLOCK = DATE_ADD(NOW(),INTERVAL @i MONTH);SET PARTITIONNAME = DATE_FORMAT( NEXTCLOCK, 'p%Y%m' );SET CLOCK = UNIX_TIMESTAMP(DATE_FORMAT(DATE_ADD( NEXTCLOCK ,INTERVAL 1 MONTH),'%Y-%m-01 00:00:00'));CALL zabbix.create_partition( SCHEMANAME, TABLENAME, PARTITIONNAME, CLOCK );SET @i=@i+1;IF @i > @totalmonths THENLEAVE createloop;END IF;END LOOP;
END //
DROP PROCEDURE IF EXISTS zabbix.drop_old_monthly_partitions; //
CREATE PROCEDURE zabbix.drop_old_monthly_partitions (SCHEMANAME varchar(64), TABLENAME varchar(64))
BEGINDECLARE OLDCLOCK timestamp;DECLARE PARTITIONNAME varchar(16);DECLARE CLOCK int;SET @minmonths = $monthly_history_min;SET @maxmonths = @minmonths+24;SET @i = @maxmonths;droploop: LOOPSET OLDCLOCK = DATE_SUB(NOW(),INTERVAL @i MONTH);SET PARTITIONNAME = DATE_FORMAT( OLDCLOCK, 'p%Y%m' );CALL zabbix.drop_partition( SCHEMANAME, TABLENAME, PARTITIONNAME );SET @i=@i-1;IF @i <= @minmonths THENLEAVE droploop;END IF;END LOOP;
END //
DROP PROCEDURE IF EXISTS zabbix.create_partition; //
CREATE PROCEDURE zabbix.create_partition (SCHEMANAME varchar(64), TABLENAME varchar(64), PARTITIONNAME varchar(64), CLOCK int)
BEGINDECLARE RETROWS int;SELECT COUNT(1) INTO RETROWSFROM information_schema.partitionsWHERE table_schema = SCHEMANAME AND table_name = TABLENAME AND partition_name = PARTITIONNAME;IF RETROWS = 0 THENSELECT CONCAT( "create_partition(", SCHEMANAME, ",", TABLENAME, ",", PARTITIONNAME, ",", CLOCK, ")" ) AS msg;SET @sql = CONCAT( 'ALTER TABLE ', SCHEMANAME, '.', TABLENAME, ' ADD PARTITION (PARTITION ', PARTITIONNAME, ' VALUES LESS THAN (', CLOCK, '));' );PREPARE STMT FROM @sql;EXECUTE STMT;DEALLOCATE PREPARE STMT;END IF;
END //
DROP PROCEDURE IF EXISTS zabbix.drop_partition; //
CREATE PROCEDURE zabbix.drop_partition (SCHEMANAME varchar(64), TABLENAME varchar(64), PARTITIONNAME varchar(64))
BEGINDECLARE RETROWS int;SELECT COUNT(1) INTO RETROWSFROM information_schema.partitionsWHERE table_schema = SCHEMANAME AND table_name = TABLENAME AND partition_name = PARTITIONNAME;IF RETROWS = 1 THENSELECT CONCAT( "drop_partition(", SCHEMANAME, ",", TABLENAME, ",", PARTITIONNAME, ")" ) AS msg;SET @sql = CONCAT( 'ALTER TABLE ', SCHEMANAME, '.', TABLENAME,' DROP PARTITION ', PARTITIONNAME, ';' );PREPARE STMT FROM @sql;EXECUTE STMT;DEALLOCATE PREPARE STMT;END IF;
END //
DELIMITER ;
_EOF_
if [ $SIMULATE = 1 ]; thenexit 0
fi
if [ $NONINTERACTIVE = 1 ]; thenyn='y'
elseecho -e "\n\nReady to apply script to database, this may take a while.(Y/n): "read yn
fi
if [ "$yn" != "n" -a "$yn" != "N" ]; thenmysql --skip-column-names -h ${DBHOST} -u ${DBUSER} -p${DBPASS} <$SQL
fi
conf=/etc/zabbix/zabbix_server.conf
if [ $NONINTERACTIVE = 1 ]; thenyn='y'
elseecho -e "\nIf Zabbix Version = 2.0 \nDo you want to update the /etc/zabbix/zabbix_server.conf"echo -n "to disable housekeeping (Y/n): "read yn
fi
if [ "$yn" != "n" -a "$yn" != "N" ]; thencp $conf ${conf}.baksed -i "s/^# DisableHousekeeping=0/DisableHousekeeping=1/" $confsed -i "s/^DisableHousekeeping=0/DisableHousekeeping=1/" $conf/etc/init.d/zabbix-server stopsleep 5/etc/init.d/zabbix-server start 2>&1 > /dev/null
fi
tmpfile=/tmp/cron$$
if [ $NONINTERACTIVE = 1 ]; thenyn='y'
elseecho -ne "\nDo you want to update the crontab (Y/n): "read yn
fi
if [ "$yn" != "n" -a "$yn" != "N" ]; thenwhere=while [ "$where" = "" ]; doif [ $NONINTERACTIVE = 1 ]; thenwhere='Y'elseecho "The crontab entry can be either in /etc/cron.daily, or added"echo -e "to the crontab for root\n"echo -n "Do you want to add this to the /etc/cron.daily directory (Y/n): "read wherefi[ "$where" = "" -o "$where" = "y" ] && where="Y"if [ "$where" != "y" -a "$where" != "Y" -a "$where" != "n" -a "$where" != "N" ]; thenwhere=""echo "Response not recognized, please try again"fidoneif [ $NONINTERACTIVE != 1 ]; thenecho -en "\nEnter email of who should get the daily housekeeping reports: "read mailtofi[ "$mailto" = "" ] && mailto=$EMAILmkdir -p $PATHTOCRONcat >$PATHTOCRON/housekeeping.sh <<_EOF_
MAILTO=$mailto
tmpfile=/tmp/housekeeping\$\$
date >\$tmpfile
/usr/bin/mysql --skip-column-names -B -h localhost -u${DBUSER} -p${DBPASS} zabbix -e "CALL create_zabbix_partitions();" >>\$tmpfile 2>&1
$PATHTOMAILBIN -s "Zabbix MySql Partition Housekeeping" \$MAILTO <\$tmpfile
rm -f \$tmpfile
_EOF_chmod +x $PATHTOCRON/housekeeping.shchown -R zabbix.zabbix /usr/local/zabbixif [ "$where" = "Y" ]; thencat >/etc/cron.daily/zabbixhousekeeping <<_EOF_
$PATHTOCRON/housekeeping.sh
_EOF_chmod +x /etc/cron.daily/zabbixhousekeepingelsecrontab -l >$tmpfilecat >>$tmpfile <<_EOF_
0 0 * * * $PATHTOCRON/housekeeping.sh
_EOF_crontab $tmpfilerm $tmpfilefi
fi@zabbix数据库历史与趋势数据占用优化（mysql存储查询）相关推荐
- Tushare简单的数据使用以及mysql存储教程
Tushare简单的数据使用以及mysql存储 本人tushare ID:430903 由于某些特殊的原因,接触到了股票策略量化的那么一个项目,而这需要大量数据的支持.而我第一时间尝试的是爬虫,存在m ...
- python爬取新闻并归数据库_Python爬取数据并写入MySQL数据库操作示例
Python爬取数据并写入MySQL数据库的实例 首先我们来爬取 http://html-color-codes.info/color-names/ 的一些数据. 按 F12 或 ctrl+u 审查元 ...
- 关系型数据库 和 非关系型数据对比 以及 MySQL与Oracle对比
一.关系型数据库 关系型数据库,是指采用了关系模型来组织数据的数据库. 关系模型1970年提出的,关系模型的概念得到了充分的发展并逐渐成为主流数据库结构的主流模型. 简单来说,关系模型 ...
- 优化mysql插入数据_优化MySQL插入方法的五个妙招
以下是涉及到插入表格的查询的5种改进方法: 1)使用LOAD DATA INFILE从文本下载数据这将比使用插入语句快20倍. 2)使用带有多个VALUES列表的INSERT语句一次插入几行这将比使用 ...
- python爬取新闻存入数据库_Python爬取数据并写入MySQL数据库的实例
按 F12 或 ctrl+u 审查元素,结果如下: 结构很清晰简单,我们就是要爬 tr 标签里面的 style 和 tr 下几个并列的 td 标签,下面是爬取的代码: #!/usr/bin/env p ...
- 数据库 用Python把数据写入到MySQL数据库/如何使用Python将数据写入到阿里云的RDS-MYSQL
版权声明:本文为博主原创文章,未经博主允许不得转载. https://blog.csdn.net/weixin_42555401 需要解决的问题: 1.用Python把数据写入到MySQL数据库? 2 ...
- oracle sql 全是子查询查询速度太慢如何优化_如果面试官问你如何优化mysql分页查询，请把这篇文章甩给他！...
在我们日常开发中,分页查询是必不可少的,可以说每干后端程序猿大部分时间都是CURD,所以分页的查询也接触的不少,你们都是怎么实现的呢?前不久的一段时间,我的一个同事突然找我寻求帮助,他说他写的sql查 ...
- mysql 怎么查询慢sql语句_如何优化MySQL中查询慢的SQL语句啊？
追问 恩恩我对sql稍微了解点 不知道怎么优化慢sql 可以指导一下嘛 谢谢啦 可以加下我qq吗 1501630150来自:求助得到的回答 mysql数据库有100万+数据,查询起来很慢了,如何优化 ...
- php连接mysql并查询表数据,php连接mysql并查询数据怎么实现
php连接mysql并查询数据怎么实现2020-11-08 13:33:21 php连接mysql并查询数据的实现方法:首先利用mysqli_connect函数与数据库建立连接:然后利用mysqli_ ...
最新文章
- python 实现延迟的操作
- HDoj-1042 大数阶乘
- Android数据存储
- 收藏｜MySQL常用语法小结
- 寻找听过我讲座的大学生
- 01_sklearn--监督学习——广义线性模型，普通最小二乘法
- I - Washing clothes
- Java面向对象设计原则
- [转]那些相见恨晚的 JavaScript 技巧
- Python学习笔记：创建进程
- OpenCV（Open Source Computer Vision Library）简介
- neo4j 查 match
- 编译原理系列之四 自顶向下语法分析方法
- vb与mysql实现登录界面_VB连接SQL数据库做用户登录窗口
- AD绘制原理图最基本的操作-第一张原理图（下）-PCB系列教程1-6
- 下载pyboard的flash中的驱动程序_教你如何安装打印机驱动程序
- Java实例（1）BMI计算
- 上海市高等学校信息技术水平二三级python 模拟题 编程
- Mysql优化碎片空间
- APP测试点分析与总结