ARM64下 Kdump Crash
2024-06-08 22:16:45
1.工具准备
1.1 kexec
kexec 是一个 Linux 内核到内核的引导加载程序，可以帮助从第一个内核的上下文引导到第二个内核。kexec会关闭第一个内核，绕过BIOS或固件阶段，并跳转到第二个内核。在没有 BIOS 阶段的情况下，重新启动变得更快。
- 下载最新的kexec-tools源码包
kexec源码下载地址
2. 解压源码包
tar xvpzf kexec-tools.tar.gz
- 进入到kexec-tools目录中，并进行配置（静态编译）
LDFLAGS=-static ./configure ARCH=arm64 --build=x86_64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu --without-xen
- 编译
make
- 生成的工具即在build目录下，可拷贝至文件系统中
1.2 crash
crash是redhat的工程师开发的，主要用来离线分析linux内核转存文件，它整合了gdb工具，功能非常强大。可以查看堆栈，dmesg日志，内核数据结构，反汇编等等。crash支持多种工具生成的转存文件格式，如kdump，LKCD，netdump和diskdump，而且还可以分析虚拟机Xen和Kvm上生成的内核转存文件。同时crash还可以调试运行时系统，直接运行crash即可，ubuntu下内核映象存放在/proc/kcore。
- 下载
crash不同版本下载地址
- 解压编译
$ tar -vxzf crash-7.3.0.tar.gz
$ cd crash-7.3.0/
$ make target=arm64
首次编译时总是报一个错误，我的PC系统是ubuntu20.04，安装一些工具后解决
sudo apt-get install libaio-dev libncurses5-dev zlib1g-dev liblzma-dev flex bison byacc
- 验证
ycy@DESKTOP-IDHR9T5:~/tool/crash-7.3.0$ ./crash --buildinfobuild_command: crashbuild_data: Wed Jun 9 11:41:35 CST 2021 by uid=1000(ycy) on DESKTOP-IDHR9T5build_target: ARM64build_version: 7.3.0
compiler version: gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
2.整体流程
2.1 使能配置项
- CONFIG_KEXEC=y
- CONFIG_SYSFS=y
- CONFIG_DEBUG_INFO=y
- CONFIG_CRASH_DUMP=y
- CONFIG_PROC_VMCORE=y
2.2 预留内存
在内核命令行中传递以下启动项，其中 @X 是可选的。
crashkernel=Y[@X]
// Y: 内核的大小
// X: 内核的起始地址（起始地址必须与2MiB (0x200000)对齐）
通过cat /proc/iomem 可以查看预留内存是否成
180000000-1efffffff : System RAM1a0000000-1a7ffffff : Crash kernel1eab38000-1eadfffff : reserved1eae11000-1efdfffff : reserved1efe30000-1efe4ffff : reserved1efe50000-1efe50fff : reserved1efe53000-1efe53fff : reserved1efe54000-1eff6ffff : reserved1eff70000-1efffffff : reserved
2.3 加载内核
kexec -p <dump-capture-kernel-Image> \--initrd=<initrd-for-dump-capture-kernel> \--append="root=<root-dev> <arch-specific-options>"//下面是当时使用的特定于arch的命令行选项For arm64:"1 maxcpus=1 reset_devices"
2.4 检查捕获内核加载状况
//查看捕获内核加载状态 0：未加载 1：已加载
root@a1000:/# cat /sys/kernel/kexec_crash_loaded
//查看捕获内核加载大小
root@a1000:/# cat /sys/kernel/kexec_crash_size
2.5 启动捕获内核
在如前所述成功加载转储-捕获内核之后，如果触发系统崩溃，系统将重新引导到转储-捕获内核。触发点位于panic()、die()、die_nmi()和sysrq处理程序(ALT-SysRq-c)中。
可通过以下指令触发panic测试：
root@a1000:~# echo c > /proc/sysrq-trigger
2.6 拷贝vmcore
将proc目录下的vmcore文件拷贝至你的存储设备中即可，vmcore就是第一个内核崩溃时的内存镜像
- 提供一个脚本：
#!/bin/sh
KDUMP_PATH="/home"
function save_dump_file()
{
# datename=$(date +%Y-%m-%d-%H:%M:%S)coredir="${KDUMP_PATH}/kdump"mkdir -p $coredirif [ ! -f /sbin/vmcore-dmesg ];thenecho “Skipping saving vmcore-dmesg.txt.File /sbin/vmcore-dmesg is not present”return;fiecho “kdump: saving vmcore-dmesg.txt to $coredir”if [ ! -f "$coredir/vmcore_1" ]; then/sbin/vmcore-dmesg /proc/vmcore > $coredir/vmcore-dmesg_1.txtcp /proc/vmcore $coredir/vmcore_1elseif [ ! -f "$coredir/vmcore_2" ]; then/sbin/vmcore-dmesg /proc/vmcore > $coredir/vmcore-dmesg_2.txtcp /proc/vmcore $coredir/vmcore_2elserm $coredir/vmcore_1 $coredir/vmcore-dmesg_1.txtmv $coredir/vmcore_2 $coredir/vmcore_1mv $coredir/vmcore-dmesg_2.txt $coredir/vmcore-dmesg_1.txt/sbin/vmcore-dmesg /proc/vmcore > $coredir/vmcore-dmesg_2.txtcp /proc/vmcore $coredir/vmcore_2fifi
}if [ -s /proc/vmcore ];thenecho “save core dump file…”save_dump_filereboot
else
# kexec setting/sbin/kexec -p --append="earlycon=uart8250,mmio32,0x20008000 console=ttyS0,115200n8 memreserve=64M@0xf8000000 rw boot_delay=32 1 maxcpus=1 reset_devices root=/dev/mmcblk0p9 boot_delay=32 rootwait init=/linuxrc" --dtb=/home/root/bsta1000-fadb.dtb /home/root/Image
fi
3.Crash使用详解
3.1 背景知识
crash是redhat的工程师开发的，主要用来离线分析linux内核转存文件，它整合了gdb工具，功能非常强大。可以查看堆栈，dmesg日志，内核数据结构，反汇编等等。crash支持多种工具生成的转存文件格式，如kdump，LKCD，netdump和diskdump，而且还可以分析虚拟机Xen和Kvm上生成的内核转存文件。同时crash还可以调试运行时系统，直接运行crash即可，ubuntu下内核映象存放在/proc/kcore
3.2 使用方法
crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form)
crash [OPTION]... [NAMELIST] (live system form)ex:
ycy@DESKTOP-IDHR9T5:~/tool/crash-7.3.0$ ./crash vmlinux ~/te/vmcorecrash 7.3.0
Copyright (C) 2002-2021 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011, 2020-2021 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=aarch64-elf-linux"...
... ...crash> crash在加载内核转存文件是会输出系统基本信息，如出问题的进程（bash - 2613），系统内存大小（7.9GB），系统架构（x86_64）等等，可以看到这个dump是sysrq触发的一个panic系统崩溃。
This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=aarch64-elf-linux"...KERNEL: vmlinuxDUMPFILE: /home/ycy/te/vmcoreCPUS: 8DATE: Sun May 16 16:50:05 CST 2021UPTIME: 00:15:09
LOAD AVERAGE: 5.06, 4.79, 3.15TASKS: 274NODENAME: a1000RELEASE: 4.19.66+1.0.0VERSION: #2 SMP Tue Jun 15 09:27:09 CST 2021MACHINE: aarch64 (unknown Mhz)MEMORY: 2.6 GBPANIC: "sysrq: SysRq : Trigger a crash"PID: 641COMMAND: "sh"TASK: ffff8001df128dc0 [THREAD_INFO: ffff8001df128dc0]CPU: 5STATE: TASK_RUNNING (SYSRQ)crash>
3.3 常用命令
Crash命令列表
| 命令 | 功能 |
|---|---|
| * | 指针快捷健 |
| alias | 命令快捷键 |
| ascii | ASCII码转换和码表 |
| bpf | eBPF - extended Berkeley Filter |
| bt | 堆栈查看 |
| btop | 地址页表转换 |
| dev | 设备数据查询 |
| dis | 反汇编 |
| eval | 计算器 |
| exit | 退出 |
| extend | 命令扩展 |
| files | 打开的文件查看 |
| foreach | 循环查看 |
| fuser | 文件使用者查看 |
| gdb | 调用gdb执行命令 |
| help | 帮助 |
| ipcs | 查看system V IPC工具 |
| irq | 查看irq数据 |
| kmem | 查看Kernel内存 |
| list | 查看链表 |
| log | 查看系统消息缓存 |
| mach | 查看平台信息 |
| mod | 加载符号表 |
| mount | Mount文件系统数据 |
| net | 网络命令 |
| p | 查看数据结构 |
| ps | 查看进程状态信息 |
| pte | 查看页表 |
| ptob | 页表地址转换 |
| ptov | 物理地址虚拟地址转换 |
| rd | 查看内存 |
| repeat | 重复执行 |
| runq | 查看run queue上的线程 |
| search | 搜索内存 |
| set | 设置线程环境和Crash内部变量 |
| sig | 查询线程消息 |
| struct | 查询结构体 |
| swap | 查看swap信息 |
| sym | 符号和虚拟地址转换 |
| sys | 查看系统信息 |
| task | 查看task_struct和thread_thread信息 |
| timer | 查看timer队列 |
| tree | 查看radix树和rb树 |
| union | 查看union结构体 |
| vm | 查看虚拟内存 |
| vtop | 虚拟地址物理地址转换 |
| waitq | 查看wait queue上的进程 |
| whatis | 符号表查询 |
| wr | 改写内存 |
| q | 退出 |
以下对常用命令进行详细介绍：
3.3.1 查看堆栈 bt
- bt:默认会dump问题线程的场景
crash> bt
PID: 641 TASK: ffff8001df128dc0 CPU: 5 COMMAND: "sh"#0 [8caa7f99223ac900] machine_kexec at ffff0000082185ecPC: 0000ffff8552f2ec LR: 0000ffff854dd6e0 SP: 0000ffffff551f40X29: 0000ffffff551f40 X28: 0000000000000000 X27: 0000000000000001X26: 00000000004cd320 X25: 000000000050b000 X24: 0000000000000002X23: 00000000020b54e0 X22: 0000000000000002 X21: 0000ffff855d2560X20: 00000000020b54e0 X19: 0000000000000001 X18: 0000000000000fffX17: 0000ffff854da1f0 X16: 00000000005030f0 X15: 0000000000000020X14: 0000000000000000 X13: 0000000000000018 X12: 0000000000000000X11: 0000000000000000 X10: 0000ffffff551f20 X9: 0000000000000001X8: 0000000000000040 X7: 0000000000000063 X6: 0000000000000001X5: 0000000000015551 X4: 0000000000000000 X3: 0000ffff855d6a78X2: 0000000000000002 X1: 00000000020b54e0 X0: 0000000000000001ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
- -t/-T:把整个stack的信息dump出来
crash> bt -t
PID: 641 TASK: ffff8001df128dc0 CPU: 5 COMMAND: "sh"START: machine_kexec at ffff0000082185f0[ffff8001e450b6a0] machine_kexec at ffff0000082185f0[ffff8001e450b6e8] __crash_kexec at ffff0000082ba048[ffff8001e450b738] crash_kexec at ffff0000082ba15c[ffff8001e450b868] __handle_sysrq at ffff00000872a080[ffff8001e450b878] sysrq_handle_crash at ffff0000087295f4[ffff8001e450b8b0] sysrq_handle_crash at ffff0000087295f4[ffff8001e450b8c8] die at ffff00000820ce00[ffff8001e450b8f8] die_kernel_fault at ffff00000821ae10[ffff8001e450b938] __do_kernel_fault at ffff00000821aea8[ffff8001e450b968] do_page_fault at ffff00000821b0d0[ffff8001e450b998] do_translation_fault at ffff00000821b3d8[ffff8001e450ba18] cont_add at ffff00000827a9e8[ffff8001e450ba28] __up_console_sem at ffff00000827abf8[ffff8001e450ba48] console_unlock at ffff00000827b3e0[ffff8001e450ba68] vprintk_emit at ffff00000827cb6c[ffff8001e450ba98] do_mem_abort at ffff000008201364[ffff8001e450baa8] el1_ia at ffff0000082030d0[ffff8001e450bac8] sysrq_handle_crash at ffff0000087295f4[ffff8001e450bad8] vprintk_emit at ffff00000827ca64[ffff8001e450bae8] vprintk_default at ffff00000827cd60[ffff8001e450bb38] _cond_resched at ffff000008d32bb0[ffff8001e450bc60] __handle_sysrq at ffff00000872a080[ffff8001e450bc70] sysrq_handle_crash at ffff0000087295f4[ffff8001e450bca8] sysrq_handle_crash at ffff0000087295f4[ffff8001e450bcb8] write_sysrq_trigger at ffff00000872a65c[ffff8001e450bcf8] proc_reg_write at ffff00000840ffa0[ffff8001e450bd18] __vfs_write at ffff000008390ea8[ffff8001e450bd38] vfs_write at ffff0000083911a4[ffff8001e450bd58] f_dupfd at ffff0000083b48d8[ffff8001e450bd78] _cond_resched at ffff000008d32bb0[ffff8001e450bd98] __sb_start_write at ffff0000083934c0[ffff8001e450bda8] vfs_write at ffff000008391270[ffff8001e450bdd8] ksys_write at ffff00000839147c[ffff8001e450be18] __arm64_sys_write at ffff000008391510[ffff8001e450be48] __arm64_sys_close at ffff00000838e59c[ffff8001e450be68] el0_svc_common at ffff0000082139f4[ffff8001e450be78] el0_svc_handler at ffff000008213b04[ffff8001e450beb8] el0_svc at ffff000008203f88
- -slf:加参数显示函数偏移，函数所在的文件和每一帧的具体内容，从而对照源码和汇编代码，查看函数入参和局部变量
crash> bt -slf
PID: 641 TASK: ffff8001df128dc0 CPU: 5 COMMAND: "sh"#0 [8caa7f99223ac900] machine_kexec+60 at ffff0000082185ec/home/ycy/kernel_tag/linux_kernel-Linux-v1.4/build/../arch/arm64/kernel/machine_kexec.c: 158PC: 0000ffff8552f2ec LR: 0000ffff854dd6e0 SP: 0000ffffff551f40X29: 0000ffffff551f40 X28: 0000000000000000 X27: 0000000000000001X26: 00000000004cd320 X25: 000000000050b000 X24: 0000000000000002X23: 00000000020b54e0 X22: 0000000000000002 X21: 0000ffff855d2560X20: 00000000020b54e0 X19: 0000000000000001 X18: 0000000000000fffX17: 0000ffff854da1f0 X16: 00000000005030f0 X15: 0000000000000020X14: 0000000000000000 X13: 0000000000000018 X12: 0000000000000000X11: 0000000000000000 X10: 0000ffffff551f20 X9: 0000000000000001X8: 0000000000000040 X7: 0000000000000063 X6: 0000000000000001X5: 0000000000015551 X4: 0000000000000000 X3: 0000ffff855d6a78X2: 0000000000000002 X1: 00000000020b54e0 X0: 0000000000000001ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
- -c/-a:查看指定cpu的堆栈或所有当前CPU
crash> bt -c 2
PID: 0 TASK: ffff8001e9f044c0 CPU: 2 COMMAND: "swapper/2"#0 [ffff8001eab77cc0] crash_save_cpu at ffff0000082ba430#1 [ffff8001eab77e80] handle_IPI at ffff000008212f08#2 [ffff8001eab77ec0] gic_handle_irq at ffff00000820189c
--- <IRQ stack> ---#3 [ffff8001e9f1bf50] el1_irq at ffff00000820332cPC: ffff000008206f18 [arch_cpu_idle+16]LR: ffff000008206f14 [arch_cpu_idle+12]SP: ffff8001e9f1bf60 PSTATE: 60c00009X29: ffff8001e9f1bf60 X28: 0000000000000000 X27: 0000000000000000X26: 0000000000000000 X25: 0000000000000000 X24: 0000000000000000X23: ffff8001e9f044c0 X22: ffff8001e9f044c0 X21: ffff0000092da748X20: 0000000000000002 X19: ffff0000092da6c8 X18: 0000000000000000X17: 0000000000000000 X16: 0000000000000000 X15: 0000000000000400X14: 0000000000000400 X13: 0000000000000001 X12: 0000000000000000X11: 000000000000bc06 X10: 00000000000009e0 X9: ffff8001e9f1beb0X8: ffff8001e9f04f00 X7: 0000000000000000 X6: 0000000000000002X5: 000000000000003f X4: ffff8001e9f1bed0 X3: ffff0000092f04c0X2: 0000000000000000 X1: 00000000002e1ab8 X0: 0000000000000008#4 [ffff8001e9f1bf60] arch_cpu_idle at ffff000008206f14#5 [ffff8001e9f1bf70] do_idle at ffff000008254678#6 [ffff8001e9f1bfb0] cpu_startup_entry at ffff000008254888#7 [ffff8001e9f1bfd0] secondary_start_kernel at ffff0000082129ccps:也可以用set命令来改变线程环境，从而查看别的cpu上的堆栈情况
3.3.2 查看堆栈 log
log命令可以用来查看系统的日志，“log-a”可以读取还没有从内核日志缓存到用户空间日志缓存的日志。也可以重定向到文件（log > logfile）。
crash> log
[ 7.141822] Mali:
[ 7.141823] Mali device driver loaded
[ 7.141929] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 7.146623] brd: module loaded
[ 7.150647] loop: module loaded
[ 7.150867] zram: Added device: zram0
[ 7.151777] null: module loaded
[ 7.151780] dummy-irq: no IRQ given. Use irq=N
[ 7.156258] i2c-core: driver [at24] registered
[ 7.156278] i2c-core: driver [eeprom] registered
.......
3.3.3 反汇编 dis -r/-f
- 显示从给定地址到例程结束的所有指令。（-f 正向，-r 反向）
crash> dis -f ffff00000820332c
d0xffff00000820332c <el1_irq+172>: blr x1
0xffff000008203330 <el1_irq+176>: mov sp, x19
0xffff000008203334 <el1_irq+180>: msr daifset, #0xf
0xffff000008203338 <el1_irq+184>: ldr x20, [sp,#288]
0xffff00000820333c <el1_irq+188>: str x20, [x28,#8]
0xffff000008203340 <el1_irq+192>: ldp x21, x22, [sp,#256]
0xffff000008203344 <el1_irq+196>: msr elr_el1, x21
0xffff000008203348 <el1_irq+200>: msr spsr_el1, x22
0xffff00000820334c <el1_irq+204>: ldp x0, x1, [sp]
0xffff000008203350 <el1_irq+208>: ldp x2, x3, [sp,#16]
0xffff000008203354 <el1_irq+212>: ldp x4, x5, [sp,#32]
0xffff000008203358 <el1_irq+216>: ldp x6, x7, [sp,#48]
0xffff00000820335c <el1_irq+220>: ldp x8, x9, [sp,#64]
0xffff000008203360 <el1_irq+224>: ldp x10, x11, [sp,#80]
0xffff000008203364 <el1_irq+228>: ldp x12, x13, [sp,#96]
0xffff000008203368 <el1_irq+232>: ldp x14, x15, [sp,#112]
0xffff00000820336c <el1_irq+236>: ldp x16, x17, [sp,#128]
0xffff000008203370 <el1_irq+240>: ldp x18, x19, [sp,#144]
0xffff000008203374 <el1_irq+244>: ldp x20, x21, [sp,#160]
0xffff000008203378 <el1_irq+248>: ldp x22, x23, [sp,#176]
0xffff00000820337c <el1_irq+252>: ldp x24, x25, [sp,#192]
0xffff000008203380 <el1_irq+256>: ldp x26, x27, [sp,#208]
0xffff000008203384 <el1_irq+260>: ldp x28, x29, [sp,#224]
0xffff000008203388 <el1_irq+264>: ldr x30, [sp,#240]
0xffff00000820338c <el1_irq+268>: add sp, sp, #0x140
0xffff000008203390 <el1_irq+272>: eret
3.3.4 查看和搜索内存 rd
- rd:查看内存内容
crash> rd ffff000008d40070 32
ffff000008d40070: 65762078756e694c 2e34206e6f697372 Linux version 4.
ffff000008d40080: 2e312b36362e3931 7963792820302e30 19.66+1.0.0 (ycy
ffff000008d40090: 504f544b53454440 355439524844492d @DESKTOP-IDHR9T5
ffff000008d400a0: 7620636367282029 39206e6f69737265 ) (gcc version 9
ffff000008d400b0: 62552820302e332e 332e392075746e75 .3.0 (Ubuntu 9.3
ffff000008d400c0: 75627537312d302e 2e30327e3175746e .0-17ubuntu1~20.
ffff000008d400d0: 2032232029293430 2065755420504d53 04)) #2 SMP Tue
ffff000008d400e0: 30203531206e754a 2039303a37323a39 Jun 15 09:27:09
ffff000008d400f0: 3132303220545343 000000000000000a CST 2021........
ffff000008d40100: 0000000000000000 0000000000000000 ................
ffff000008d40110: 0000000000000000 0000000000000000 ................
ffff000008d40120: 0000000000000000 0000000000000000 ................
ffff000008d40130: 0000000000000000 0000000000000000 ................
ffff000008d40140: 0000000000000000 0000000000000000 ................
ffff000008d40150: 0000000000000000 0000000000000000 ................
ffff000008d40160: 0000000000000000 0000000000000000 ................
- -a linux_banner:查看版本信息
crash> rd -a linux_banner
ffff000008d40070: Linux version 4.19.66+1.0.0 (ycy@DESKTOP-IDHR9T5) (gcc versi
ffff000008d400ac: on 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #2 SMP Tue Jun 15 0
ffff000008d400e8: 9:27:09 CST 2021
- -s:打印符号表
crash> rd ffff8001e450b6a0 32 -s
ffff8001e450b6a0: machine_kexec+64 0000000000000004
ffff8001e450b6b0: 0000000000000004 000000000000003f
ffff8001e450b6c0: ffff8001a0000040 0000000000000001
ffff8001e450b6d0: ffff8001a00000f8 d61f0220aa1f03e3
ffff8001e450b6e0: ffff8001e450b730 __crash_kexec+120
ffff8001e450b6f0: __stack_chk_guard crashk_res
ffff8001e450b700: ffff8001e450bb70 posix_timers_hashtable+2040
ffff8001e450b710: ffff8001e450b778 0000000000000000
ffff8001e450b720: 0000000056000000 posix_timers_hashtable+2040
ffff8001e450b730: ffff8001e450b8c0 crash_kexec+108
ffff8001e450b740: ffffffffffffffff __compound_literal.26+8
ffff8001e450b750: ffff8001e450bb70 0000000000000000
ffff8001e450b760: 0000ffffffffffff kallsyms_token_index+6456
ffff8001e450b770: ffff8001e450b860 0000000000000001
ffff8001e450b780: 0000000000000000 0000000000000000
ffff8001e450b790: 0000000000000006 0000000000000001
- -e:查看指定内存区域内容
crash> rd ffff8001e450b6a0 -e ffff8001e450b6f0
ffff8001e450b6a0: ffff0000082185f0 0000000000000004 ..!.............
ffff8001e450b6b0: 0000000000000004 000000000000003f ........?.......
ffff8001e450b6c0: ffff8001a0000040 0000000000000001 @...............
ffff8001e450b6d0: ffff8001a00000f8 d61f0220aa1f03e3 ............ ...
ffff8001e450b6e0: ffff8001e450b730 ffff0000082ba048 0.P.....H.+.....
- search:搜索指定内存
crash> search -s ffff8001e450b6a0 -e ffff8001e450b6e0 d61f0220aa1f03e3
ffff8001e450b6d8: d61f0220aa1f03e3
3.3.5 查看线程状态 ps
- ps:查看所有线程状态
crash> psPID PPID CPU TASK ST %MEM VSZ RSS COMM
> 0 0 0 ffff0000092e5240 RU 0.0 0 0 [swapper/0]
> 0 0 1 ffff8001e9f06040 RU 0.0 0 0 [swapper/1]
> 0 0 2 ffff8001e9f044c0 RU 0.0 0 0 [swapper/2]
> 0 0 3 ffff8001e9f02940 RU 0.0 0 0 [swapper/3]
> 0 0 4 ffff8001e9f00000 RU 0.0 0 0 [swapper/4]0 0 5 ffff8001e9f03700 RU 0.0 0 0 [swapper/5]
> 0 0 6 ffff8001e9f01b80 RU 0.0 0 0 [swapper/6]
> 0 0 7 ffff8001e9f3b700 RU 0.0 0 0 [swapper/7]1 0 5 ffff8001e9ec6040 IN 0.1 157336 6688 systemd2 0 3 ffff8001e9ec44c0 IN 0.0 0 0 [kthreadd]3 2 0 ffff8001e9ec2940 ID 0.0 0 0 [rcu_gp]4 2 0 ffff8001e9ec0000 ID 0.0 0 0 [rcu_par_gp]7 2 3 ffff8001e9ec5280 ID 0.0 0 0 [kworker/u16:0]
... ...
- -p:查看父线程树
crash> ps -p 167
PID: 0 TASK: ffff0000092e5240 CPU: 0 COMMAND: "swapper/0"PID: 2 TASK: ffff8001e9ec44c0 CPU: 3 COMMAND: "kthreadd"PID: 167 TASK: ffff8001e6a5ee00 CPU: 5 COMMAND: "bst_lw0"
- -c:查看子线程
crash> ps -c 2
PID: 2 TASK: ffff8001e9ec44c0 CPU: 3 COMMAND: "kthreadd"PID: 3 TASK: ffff8001e9ec2940 CPU: 0 COMMAND: "rcu_gp"PID: 4 TASK: ffff8001e9ec0000 CPU: 0 COMMAND: "rcu_par_gp"PID: 7 TASK: ffff8001e9ec5280 CPU: 3 COMMAND: "kworker/u16:0"PID: 8 TASK: ffff8001e9ec0dc0 CPU: 0 COMMAND: "mm_percpu_wq"PID: 9 TASK: ffff8001e9ec6e00 CPU: 0 COMMAND: "ksoftirqd/0"PID: 10 TASK: ffff8001e9f05280 CPU: 1 COMMAND: "rcu_sched"PID: 11 TASK: ffff8001e9f00dc0 CPU: 0 COMMAND: "rcu_bh"PID: 12 TASK: ffff8001e9f06e00 CPU: 0 COMMAND: "migration/0"
... ...
- -t:查看线程运行时间
crash> ps -t 167
PID: 167 TASK: ffff8001e6a5ee00 CPU: 5 COMMAND: "bst_lw0"RUN TIME: 00:15:01START TIME: 8177699104UTIME: 0STIME: 0
- -A:查看活动线程
crash> ps -APID PPID CPU TASK ST %MEM VSZ RSS COMM
> 0 0 0 ffff0000092e5240 RU 0.0 0 0 [swapper/0]
> 0 0 1 ffff8001e9f06040 RU 0.0 0 0 [swapper/1]
> 0 0 2 ffff8001e9f044c0 RU 0.0 0 0 [swapper/2]
> 0 0 3 ffff8001e9f02940 RU 0.0 0 0 [swapper/3]
> 0 0 4 ffff8001e9f00000 RU 0.0 0 0 [swapper/4]
> 0 0 6 ffff8001e9f01b80 RU 0.0 0 0 [swapper/6]
> 0 0 7 ffff8001e9f3b700 RU 0.0 0 0 [swapper/7]
> 641 478 5 ffff8001df128dc0 RU 0.0 7352 3764 sh
- -k:查看内核线程
crash> ps -kPID PPID CPU TASK ST %MEM VSZ RSS COMM
> 0 0 0 ffff0000092e5240 RU 0.0 0 0 [swapper/0]
> 0 0 1 ffff8001e9f06040 RU 0.0 0 0 [swapper/1]
> 0 0 2 ffff8001e9f044c0 RU 0.0 0 0 [swapper/2]
> 0 0 3 ffff8001e9f02940 RU 0.0 0 0 [swapper/3]
> 0 0 4 ffff8001e9f00000 RU 0.0 0 0 [swapper/4]0 0 5 ffff8001e9f03700 RU 0.0 0 0 [swapper/5]
> 0 0 6 ffff8001e9f01b80 RU 0.0 0 0 [swapper/6]
> 0 0 7 ffff8001e9f3b700 RU 0.0 0 0 [swapper/7]2 0 3 ffff8001e9ec44c0 IN 0.0 0 0 [kthreadd]3 2 0 ffff8001e9ec2940 ID 0.0 0 0 [rcu_gp]4 2 0 ffff8001e9ec0000 ID 0.0 0 0 [rcu_par_gp]
... ...
- -u:查看用户态线程
crash> ps -uPID PPID CPU TASK ST %MEM VSZ RSS COMM1 0 5 ffff8001e9ec6040 IN 0.1 157336 6688 systemd259 1 1 ffff8001e86be040 IN 0.1 25836 10456 systemd-journal277 1 1 ffff8001e4468dc0 IN 0.0 13860 3152 systemd-udevd372 1 6 ffff8001e97a9b80 IN 0.1 7868 4276 systemd-network378 1 5 ffff8001e584c4c0 IN 0.1 7516 4132 systemd-resolve379 1 1 ffff8001e446e040 IN 0.0 81452 3936 systemd-timesyn381 1 4 ffff8001e8103700 IN 0.1 7492 4076 systemd-logind382 1 4 ffff8001e8105280 IN 0.0 3128 1620 klogd407 1 7 ffff8001e8106040 IN 0.0 3128 1596 syslogd408 1 7 ffff8001e81044c0 IN 0.1 602040 9028 SafetyService409 1 1 ffff8001e8102940 IN 0.0 4648 3184 dbus-daemon
... ...
- -l:查看最后运行时间戳
crash> ps -l
[909022209204] [IN] PID: 535 TASK: ffff8001df37c4c0 CPU: 2 COMMAND: "SecurityService"
[909021738959] [IN] PID: 507 TASK: ffff8001e584a940 CPU: 3 COMMAND: "CameraService"
[909021687516] [IN] PID: 510 TASK: ffff8001e5848dc0 CPU: 7 COMMAND: "SecurityService"
[909017588390] [IN] PID: 514 TASK: ffff8001df18e040 CPU: 6 COMMAND: "PersistenceServ"
[909016957627] [IN] PID: 561 TASK: ffff8001dd951b80 CPU: 0 COMMAND: "UpgradeService"
[909014829938] [IN] PID: 470 TASK: ffff8001e2aa2940 CPU: 2 COMMAND: "DiagnoseService"
[909012664304] [IN] PID: 490 TASK: ffff8001e0778dc0 CPU: 0 COMMAND: "SensorService"
... ...
- -r:查看线程资源限制
crash> ps -r 167
PID: 167 TASK: ffff8001e6a5ee00 CPU: 5 COMMAND: "bst_lw0"RLIMIT CURRENT MAXIMUMCPU (unlimited) (unlimited)FSIZE (unlimited) (unlimited)DATA (unlimited) (unlimited)STACK 8388608 (unlimited)CORE 0 (unlimited)RSS (unlimited) (unlimited)NPROC 6630 6630NOFILE 1024 4096MEMLOCK 65536 65536AS (unlimited) (unlimited)LOCKS (unlimited) (unlimited)SIGPENDING 6630 6630MSGQUEUE 819200 819200NICE 0 0RTPRIO 0 0RTTIME (unlimited) (unlimited)
3.3.6 Context切换 set
- 切换panic线程
crash> set -pPID: 641
COMMAND: "sh"TASK: ffff8001df128dc0 [THREAD_INFO: ffff8001df128dc0]CPU: 5STATE: TASK_RUNNING (SYSRQ)
crash> set ffff8001df128dc0PID: 641
COMMAND: "sh"TASK: ffff8001df128dc0 [THREAD_INFO: ffff8001df128dc0]CPU: 5STATE: TASK_RUNNING (SYSRQ)
3.3.7 加载module符号表 mod
- mod:查看当前加载的module
crash> modMODULE NAME BASE SIZE OBJECT FILE
ffff000000d42040 panic_module ffff000000d40000 16384 (not loaded) [CONFIG_KALLSYMS]
- -S:加载所有module符号表
ARM64下 Kdump Crash相关推荐
- linux6下kdump的配置
linux6下kdump的配置 背景知识: kexec是一个快速启动机制,允许通过已经运行的内核的上下文启动一个Linux内核,不需要经过BIOS.BIOS可能会消耗很多时间,特别是带有众多数量的外设 ...
- linux6.5kdump,GitHub - figozhang/linux-5.0-kdump: Kdump+crash lab
Linux 5.0 for Kdump+crash lab On ARM64 ============================================= 笨叔已经录制好视频课程节目,全 ...
- Linux配置kdump大小,linux6下kdump的配置
linux6下kdump的配置 背景知识: kexec是一个快速启动机制,允许通过已经运行的内核的上下文启动一个Linux内核,不需要经过BIOS.BIOS可能会消耗很多时间,特别是带有众多数量的外设 ...
- 第26月第22天 iOS瘦身之armv7 armv7s arm64选用 iOS crash
1.iOS瘦身之armv7 armv7s arm64选用 机器对指令集的支持是向下兼容的,因此armv7的指令集是可以运行在iphone5S以上的,只是效率没那么高而已~ 但是由于苹果要求必须支持ar ...
- Linux配置kdump大小,RedHat linux 下kdump的配置
转载地址:http://www.lvluo.net/blog/2011/05/redhat-linux-%E4%B8%8Bkdump%E9%85%8D%E7%BD%AE.html 如果服务器经常死机( ...
- eclipse在ubuntu13.04下崩溃crash
错误信息: # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0 ...
- Fix chrome 下flash crash的问题
2019独角兽企业重金招聘Python工程师标准>>> 本来好好的,结果不知道为什么,在MAC下使用chrome不断出现flash插件的错误,网上搜了一下,看这里,要把chrome自 ...
- linux firefox 段错误,中标麒麟v10在arm64下360浏览器(chromium)访问崩溃(段错误)
浏览器下载地址: https://browser.360.cn/se/linux/index.html ``` [root@localhost browser360]# dmesg [ 0.00000 ...
- Electron 开发环境下总是 crash
全局安装一个 electron devtool 关掉 崩溃时选择重新打开 转载于:https://www.cnblogs.com/savokiss/p/9107916.html
最新文章
- 检测jQuery.js是否已加载的判断代码
- RabbitMQ学习笔记-安装
- win10安装mysql无服务_win10装sql没有SQL Server 服务
- hash：奶牛看地图（洛谷P3405 [USACO16DEC]Cities and States S）
- 【转】phpize学习
- android activity查询,android中activity.findViewById()方法查找的是什么？
- 美股数据获取 python_python3+tesseract获取美股PEG图像上的数据
- kendo实现grid的inline编辑
- iOS获取camera的yuv数据
- 数据和判定(二)------运算符
- java提高篇-----字符串
- 扒一扒国产 Linux 操作系统架构是怎么设计的 ？
- INVEST模型生境质量评价之威胁源数据处理
- Python functools模块之cmp_to_key
- 【精华】多目标跟踪MOT
- 【STC15系列】STC15软串口输出打印示例程序
- 一篇文章教会你利用Python网络爬虫抓取王者荣耀图片
- 工作杂谈：由IT行业技术变革 谈当今技术趋势
- ArrayList类的详解
- Kafka之Controller(Broker的领导者)
热门文章
- requireJs笔记
- 计算机毕业设计Java乒乓球俱乐部管理(源码+系统+mysql数据库+lw文档）
- 【TWS API使用教程3】---如何使用TWS API从盈透证券中设置contract及获取contract的信息？
- 24V转5V降压芯片PW2205，PCB布局图册
- Multisim基础 有极性的电容 添加元件的位置
- wav文件提取音频数据_python
- Linux SSH命令大全
- AutoCAD.Net开发问题之：层表事件的响应（续）
- HTML5期末大作业：校园网站设计——云南艺术学院文化学院(3页) HTML+CSS+JavaScript web前端开发技术 web课程设计 网页规划与设计
- python快速实现NPV净现值计算