jbd2 mysql_jbd2 BUG · xiewen/xiewen.github.io Wiki

现象：MySQl 挂起，不能写入数据，或者整个系统崩溃掉

1. BUG 1 - MySQL 挂起

1.1. 症状

kernel: ------------[ cut here ]------------

kernel: WARNING: at fs/jbd2/journal.c:507 __jbd2_log_start_commit+0x72/0xf0 [jbd2]()

kernel: jbd: bad log_start_commit: 2768618130 2768618130 49048062 2768618131

kernel: Modules linked in: nfs lockd fscache auth_rpcgss nfs_acl sunrpc ip_vs_wlc ip_vs nf_conntrack ipv6 libcrc32c ovmapi(U) xen_netfront pcspkr ext4 mbcache jbd2 xen_blkfront dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]

kernel: Pid: 30273, comm: mysqld Not tainted 2.6.39-200.32.1.el6uek.x86_64 #1

kernel: Call Trace:

kernel: [] warn_slowpath_common+0x7f/0xc0

kernel: [] warn_slowpath_fmt+0x46/0x50

kernel: [] __jbd2_log_start_commit+0x72/0xf0 [jbd2]

kernel: [] jbd2_log_start_commit+0x30/0x50 [jbd2]

kernel: [] ext4_sync_file+0x162/0x2d0 [ext4]

kernel: [] vfs_fsync_range+0x7f/0xa0

kernel: [] vfs_fsync+0x1c/0x20

kernel: [] do_fsync+0x3a/0x60

kernel: [] sys_fsync+0x10/0x20

kernel: [] system_call_fastpath+0x16/0x1b

kernel: ---[ end trace 75c73bd1e52b1c73 ]---

kernel: jbd2_log_wait_commit: error: j_commit_request=-1526349166, tid=49048062

# grep each /proc/fs/jbd2/*/info

/proc/fs/jbd2/dm-0-8/info:2981802046 transaction, each up to 8192 blocks

/proc/fs/jbd2/dm-1-8/info:5620 transaction, each up to 8192 blocks

1.2. BUG 描述

ext4/jbd2: don't wait (forever) for stale tid caused by wraparound (uek3, kernel.org)

1.3. rhel/ol 5 修复版本

2.6.18-371.12.1.el5 或 el/ol 5.11(2.6.18-398.el5) (1)

[fs] jbd: don't wake kjournald unnecessarily (Denys Vlasenko) [1116027

1081785]

[fs] jbd: don't wait (forever) for stale tid caused by wraparound

(Denys Vlasenko) [1116027 1081785]

[fs] ext4: fix waiting and sending of barrier in ext4_sync_file()

(Denys Vlasenko) [1116027 1081785]

[fs] jbd2: Add function jbd2_trans_will_send_data_barrier() (Denys

Vlasenko) [1116027 1081785]

[fs] jbd2: fix sending of data flush on journal commit (Denys

Vlasenko) [1116027 1081785]

[fs] ext4, jbd2: Add barriers for file systems with ext journals

(Denys Vlasenko) [1116027 1081785]

[fs] jbd: fix fsync() tid wraparound bug (Denys Vlasenko) [1116027

1081785] | 2.6.32-358.18.1.el6 或 6.5(2.6.32-431.el6) (ol)

[fs] ext4/jbd2: dont wait (forever) for stale tid caused by wraparound

(Eric Sandeen) [963557 955807]

[fs] jbd: dont wait (forever) for stale tid caused by wraparound (Eric

Sandeen) [963557 955807]

1.4. rhel/ol 6 修复版本

2.6.32-358.18.1.el6 或 6.5(2.6.32-431.el6) (ol)

[fs] ext4/jbd2: dont wait (forever) for stale tid caused by wraparound

(Eric Sandeen) [963557 955807]

[fs] jbd: dont wait (forever) for stale tid caused by wraparound (Eric

Sandeen) [963557 955807]

1.5. ol uek 修复版本

无

1.6. ol uek 3 修复版本

3.8.13-16 (1)

2. BUG 2 - 系统崩溃

2.1. 症状

# ----------- [cut here ] --------- [please bite here ] ---------

Kernel BUG at fs/jbd2/commit.c:395

invalid opcode: 0000 [1] SMP

last sysfs file: /devices/system/cpu/cpu0/topology/physical_package_id

CPU 0

Modules linked in: autofs4 i2c_dev i2c_core hidp nfs nfs_acl rfcomm l2cap bluetooth lockd sunrpc ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables x_tables ipv6 xfrm_nalgo crypto_api ext4 jbd2 crc16 dm_mirror parport_pc lp parport xennet pcspkr xenblk dm_raid45 dm_message dm_region_hash dm_log dm_mod dm_mem_cache ata_piix libata scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd

Pid: 986, comm: jbd2/dm-0-8 Not tainted 2.6.18-274.0.0.0.1.el5xen #1

RIP: e030:[] [] :jbd2:jbd2_journal_commit_transaction+0x50/0x1074

RSP: e02b:ffff8803fb97bd70 EFLAGS: 00010246

RAX: 0000000000000008 RBX: ffff8803fb968800 RCX: ffff8803fb97be80

RDX: 0000000000000008 RSI: 0000000000000000 RDI: 0000000000000000

RBP: ffff8803fb968800 R08: ffff8803fb968898 R09: 0000000000000000

R10: ffff8803e8a6a488 R11: 0000000000000000 R12: ffff8803fb968824

R13: ffff8803fb9688c0 R14: 0000000000000000 R15: ffff8803fb968890

FS: 00002aaaaaac5330(0000) GS:ffffffff8062e000(0000) knlGS:0000000000000000

CS: e033 DS: 0000 ES: 0000

Process jbd2/dm-0-8 (pid: 986, threadinfo ffff8803fb97a000, task ffff8803ffafa080)

Stack: ffff8803fc2d1000 00000f7c00000000 0000000800000000 00000001ffffffff

ffff880300000001 00000000eb6ca2da 0000000000000000 0000000000000000

0000000000000000 0000000000000000

Call Trace:

[] _spin_lock_irqsave+0x9/0x14

[] lock_timer_base+0x1b/0x3c

[] try_to_del_timer_sync+0x7f/0x88

[] :jbd2:kjournald2+0x9a/0x1ec

[] autoremove_wake_function+0x0/0x2e

[] keventd_create_kthread+0x0/0xc4

[] :jbd2:kjournald2+0x0/0x1ec

[] keventd_create_kthread+0x0/0xc4

[] kthread+0xfe/0x132

[] child_rip+0xa/0x12

[] keventd_create_kthread+0x0/0xc4

[] kthread+0x0/0x132

[] child_rip+0x0/0x12

Code: 0f 0b 68 1b a5 1c 88 c2 8b 01 eb fe 48 83 7d 50 00 74 0c 0f

RIP [] :jbd2:jbd2_journal_commit_transaction+0x50/0x1074

RSP

<0>Kernel panic - not syncing: Fatal exception

2.2. BUG 描述

Bug 735768 hitting J_ASSERT(journal->j_running_transaction != NULL) in journal_commit_transaction (el)

2.3. rhel/ol 5 修复版本

2.4. rhel/ol 6 修复版本

2.6.32-304.el6 或 el/ol 6.4(2.6.32-358.el6) (ol, el)

[fs] jbd2: fix fsync() tid wraparound bug (Dave Wysochanski) [735768]

[fs] jbd: fix fsync() tid wraparound bug (Dave Wysochanski) [735768]

[fs] jbd, jbd2: fixed typos (Dave Wysochanski) [735768]

2.5. ol uek 修复版本

2.6.39 (1, 2, 3 没找到)

2.6. ol uek 3 修复版本

3.8.13-16 (1, 2, 3)

3. 解决方案

Oracle Linux 6.x 系统升级到 6.4+，安装 uek3 内核。ol 5.x 建议重装为 6.5 操作系统，安装 uek3 内核。

RHEL 系统升级到 6.5+ 或 5.11+。

4. 系统升级方法

4.1. ol 6.x 升级到 6.5

# 配置 ol 6.5 的 yum repos

vi /etc/yum.repos.d/.....

yum clean all

yum update -x "Percona-Server-*"

# 注意升级完后时区变成系统默认的 EST，请手工修改为 CST

URL=ftp://aa.bb.cc.dd/.../linux/uek3

rpm -q xenstoreprovider && rpm -Uvh "$URL/xenstoreprovider-3.0-11.el6.x86_64.rpm"

rpm -ev kmod-ovmapi-uek

KV=3.8.13-55.1.5.el6uek

rpm -Uvh \

"$URL/kernel-uek-$KV.x86_64.rpm" \

"$URL/kernel-uek-devel-$KV.x86_64.rpm" \

"$URL/kernel-uek-firmware-$KV.noarch.rpm" \

"$URL/libdtrace-ctf-0.4.1-1.x86_64.rpm"

reboot

4.2. ol 5.x 升级到 6.5

备份数据

重装系统

升级 el5 老版本软件

4.3. ol 6.4 升级 uek3 内核

# 配置 6.5 yum

vi /etc/yum.repos.d/.....

yum -y update dracut

yum -y install elfutils-libs

URL=ftp://aa.bb.cc.dd/.../linux/uek3

rpm -q xenstoreprovider && rpm -Uvh "$URL/xenstoreprovider-3.0-11.el6.x86_64.rpm"

KV=3.8.13-55.1.5.el6uek

rpm -Uvh \

"$URL/kernel-uek-$KV.x86_64.rpm" \

"$URL/kernel-uek-devel-$KV.x86_64.rpm" \

"$URL/kernel-uek-firmware-$KV.noarch.rpm" \

"$URL/libdtrace-ctf-0.4.1-1.x86_64.rpm"

reboot

jbd2 mysql_jbd2 BUG · xiewen/xiewen.github.io Wiki · GitHub相关推荐

用username.github.io在github创建一个个人空间
对于技术人员来说,分享技术新的.学习经验都有许多平台,比如我们常用的CSDN,很多技术人员都会在上面创建自己的博客.但是以这种方式来分享,可能没有那么个性化,因此我们可以在github上创建自己的个人 ...
采取HEXO+NexT主题+github.io的方式建立自己的个人主页
采取HEXO+NexT主题+github.io的方式建立自己的个人主页主要知识点注册github帐号,建立io仓库. 本地配置HEXO博客. 克隆NexT主题到本地仓库建立本地博客与github ...
【GitHub.io/Github Pages使用教程】从头开始搭建自己的Github Pages，打造个人博客网站，展示个人简历、项目、文档或想要与世界共享的任何其他内容
巨人半边莲如果你曾征服乞力马扎罗山,留意过海拔 3,657-4,267 米处的尖顶植物,这种植物有时形似绿色大柱子(或'花序'),从中间长出花序. 那么你就可能看到许多巨人半边莲,这些植物生长在非洲 ...
修复ext4日志（jbd2）bug( Ext4 文件系统有以下 Bug)
from:http://blog.donghao.org/2013/03/20/%E4%BF%AE%E5%A4%8Dext4%E6%97%A5%E5%BF%97%EF%BC%88jbd2%EF%BC% ...
hexo搭建github.io博客
点击上方 "大前端圈"关注, 星标或置顶一起成长每天早晨08点30分, 第一时间与你相约 Introduction 回顾一下我三年搭建博客的历程,最初是选择wordpress,W ...
回归——同步更新github.io
回归已经有好长时间没写博客了,可能我比较懒,不太乐于分享,我觉得这个是一个很不好的习惯.但我坚信:Sharing changes the world! 最近搭建了自己的个人独立博客,基于Github ...
新的博客 bincoding.github.io
最近周围的人都在用静态博客,随主流也换了github page搭建hexo. bincoding.github.io 个人网站,用来上线乱七八糟的东西: bincoding.cn
ztree 标准得json数据格式_酷站推荐 - json-c.github.io/json-c - json-c API
json-c.github.io/json-c/ http://json-c.github.io/json-c/ JSON:JavaScript 对象表示法(JavaScript Object Not ...
【还是回来了】博客搬家--https://cangbean.github.io
还是弄了个自己的地址:https://cangbean.github.io 想记录下,防止万一以后迁移不好弄想练习markdown写作折腾而已以后不再博客园记录东西了,但是还是会回来看看的转载 ...
最新博客地址转移https://bravoing.github.io/
最新博客地址转移:https://bravoing.github.io/ 欢迎大家收藏

jbd2 mysql_jbd2 BUG · xiewen/xiewen.github.io Wiki · GitHub

jbd2 mysql_jbd2 BUG · xiewen/xiewen.github.io Wiki · GitHub相关推荐

最新文章

热门文章