我们的mysql 备份系统遭遇严重bug

源于 开源软件 xtrabackup 的一个bug  
https://bugs.launchpad.net/percona-xtrabackup/+bug/722638

之前我们的大规模部署都没有遇到这问题。

在做计数器转mysql 后，我们部署了备份系统，屡屡备份失败，于是决定彻底的解决这个问题，

经过一系列测试后，发现在备份过程无法跨越 计数器的数据入库操作，

备份系统报错：
[code]
[01] Copying ./cnt_it/cnt_referrer_channel_2011.ibd
     to /usr/local/mysql/crontab/cnt_it/backup/innodb/full/2011-06-10_18-18-25/cnt_it/cnt_referrer_channel_2011.ibd
[01]        ...done
[01] Copying ./cnt_it/cnt_goals_abandon_201109.ibd
     to /usr/local/mysql/crontab/cnt_it/backup/innodb/full/2011-06-10_18-18-25/cnt_it/cnt_goals_abandon_201109.ibd
[01]        ...done
[01] Copying ./cnt_it/cnt_referrer_search_keyword_201107.ibd InnoDB: Error: tablespace id is 43167 in the data dictionary
InnoDB: but in file ./cnt_it/cnt_referrer_summary_work.ibd it is 43178!
110610 18:37:57  InnoDB: Assertion failure in thread 1201920320 in file /home/buildbot/slaves/percona-server-51-12/TGZ_CentOS_5_x86_64/work/xtrabackup-1.6/Percona-Server-5.5/storage/innobase/fil/fil0fil.c line 780
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.

to /usr/local/mysql/crontab/cnt_it/backup/innodb/full/2011-06-10_18-18-25/cnt_it/cnt_referrer_search_keyword_201107.ibd
[01]        ...done
[01] Copying ./cnt_it/cnt_goals_referrer_201205.ibd
     to /usr/local/mysql/crontab/cnt_it/backup/innodb/full/2011-06-10_18-18-25/cnt_it/cnt_goals_referrer_201205.ibd
[01]        ...done
./backup.sh: line 109: 24002 备份失败                xtrabackup --defaults-file=$CNF --backup --target-dir=$BACKUP/$ENGINE/full/$day --datadir=$DATADIR
+ return 1
+ critical 
+ df -h
[/code]

上面是什么问题呢？

就是说在备份过程中，数据库的表不能rebuild  操作，比如： truncate table ， drop table ，并重新建表 这样的操作。

从报错信息上看，应该是xtrabackup 已经考虑到这个问题了，只是当时没有处理，于是在相关的代码处加了一个assertion
代码这个地方出错，就退出。

这个bug 在1.5,1.5.1  ，1.6 版本都存在这个问题。 要到1.7版本才能修复。

慢慢等吧！

目前替代方案，采用备份从库解决。