prometheus 异常退出报错：opening storage failed

prometheus 异常退出日志如下：

解决方法：

1. 进入到prometheus数据目录内，移走报错文件夹，再次启动后日志如下：

ts=2022-06-30T03:52:56.187Z caller=manager.go:956 level=info component="rule manager" msg="Rule manager stopped"
ts=2022-06-30T03:52:56.187Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..."
ts=2022-06-30T03:52:56.187Z caller=main.go:1068 level=info msg="Notifier manager stopped"
ts=2022-06-30T03:52:56.187Z caller=main.go:1077 level=error err="opening storage failed: reloadBlocks: 15 errors: corrupted block 01G2XWKD8X0VPKWNJGN5YE64E7: read symbols: invalid checksum; corrupted block 01G3E6EMA9NMC5P3GEASJD7A4H: invalid magic number 0; corrupted block 01G1TFMH0WD2YXH0GJ9D86QKFY: invalid magic number 0; corrupted block 01G22VDVS27547SZ1BRMG8R4JY: invalid magic number 0; corrupted block 01G3VG5NRWAVJ8WGN0PZ2T5TAH: read symbols: invalid checksum; corrupted block 01G548SQ8V8YFVCR5XP3XS6GGW: read symbols: invalid checksum; corrupted block 01G313KA0WPYMK53APAPGGQ3RD: read symbols: invalid checksum; corrupted block 01G361H0RV5HA7ZMJA37GJJ605: invalid magic number 0; corrupted block 01G36P4714CSMRBZJMV5XV2W63: read symbols: invalid checksum; corrupted block 01G3X07RGVREACT0YTJP3HMVH2: read symbols: invalid checksum; corrupted block 01G578XWRW3E0VSY0X35FMMDPD: read symbols: invalid checksum; corrupted block 01G32CSNGWD0SAFSDF2HSXT6J5: read symbols: invalid checksum; corrupted block 01G37AQC8WVV2AWQNZWNF05WYA: read symbols: invalid checksum; corrupted block 01G386698WS46Q9566K0X9VX25: invalid magic number 0; corrupted block 01G4A9YT0WJDKKWEM33B017D0T: read symbols: invalid checksum"

2. 将日志中的所有错误块移走

$ mv 01G313KA0WPYMK53APAPGGQ3RD 01G361H0RV5HA7ZMJA37GJJ605 01G36P4714CSMRBZJMV5XV2W63 01G3X07RGVREACT0YTJP3HMVH2 01G578XWRW3E0VSY0X35FMMDPD 01G32CSNGWD0SAFSDF2HSXT6J5 01G37AQC8WVV2AWQNZWNF05WYA 01G386698WS46Q9566K0X9VX25 01G4A9YT0WJDKKWEM33B017D0T bak/

3. 再次启动成功。

ts=2022-06-30T03:56:12.892Z caller=db.go:1157 level=error component=tsdb msg="Failed to read meta.json for a block during reloadBlocks. Skipping" dir=/data/01G313KA0WPYMK53APAPGGQ3RD err="open /data/01G313KA0WPYMK53APAPGGQ3RD/meta.json: no such file or directory"
ts=2022-06-30T03:56:13.355Z caller=db.go:1157 level=error component=tsdb msg="Failed to read meta.json for a block during reloadBlocks. Skipping" dir=/data/01G32CSNGWD0SAFSDF2HSXT6J5 err="open /data/01G32CSNGWD0SAFSDF2HSXT6J5/meta.json: no such file or directory"
ts=2022-06-30T03:56:15.150Z caller=db.go:1157 level=error component=tsdb msg="Failed to read meta.json for a block during reloadBlocks. Skipping" dir=/data/01G361H0RV5HA7ZMJA37GJJ605 err="open /data/01G361H0RV5HA7ZMJA37GJJ605/meta.json: no such file or directory"
ts=2022-06-30T03:56:15.381Z caller=db.go:1157 level=error component=tsdb msg="Failed to read meta.json for a block during reloadBlocks. Skipping" dir=/data/01G36P4714CSMRBZJMV5XV2W63 err="open /data/01G36P4714CSMRBZJMV5XV2W63/meta.json: no such file or directory"
ts=2022-06-30T03:56:15.641Z caller=db.go:1157 level=error component=tsdb msg="Failed to read meta.json for a block during reloadBlocks. Skipping" dir=/data/01G37AQC8WVV2AWQNZWNF05WYA err="open /data/01G37AQC8WVV2AWQNZWNF05WYA/meta.json: no such file or directory"
ts=2022-06-30T03:56:16.006Z caller=db.go:1157 level=error component=tsdb msg="Failed to read meta.json for a block during reloadBlocks. Skipping" dir=/data/01G386698WS46Q9566K0X9VX25 err="open /data/01G386698WS46Q9566K0X9VX25/meta.json: no such file or directory"
ts=2022-06-30T03:56:26.474Z caller=db.go:1157 level=error component=tsdb msg="Failed to read meta.json for a block during reloadBlocks. Skipping" dir=/data/01G3X07RGVREACT0YTJP3HMVH2 err="open /data/01G3X07RGVREACT0YTJP3HMVH2/meta.json: no such file or directory"
ts=2022-06-30T03:56:33.312Z caller=db.go:1157 level=error component=tsdb msg="Failed to read meta.json for a block during reloadBlocks. Skipping" dir=/data/01G4A9YT0WJDKKWEM33B017D0T err="open /data/01G4A9YT0WJDKKWEM33B017D0T/meta.json: no such file or directory"
ts=2022-06-30T03:56:46.171Z caller=db.go:1157 level=error component=tsdb msg="Failed to read meta.json for a block during reloadBlocks. Skipping" dir=/data/01G578XWRW3E0VSY0X35FMMDPD err="open /data/01G578XWRW3E0VSY0X35FMMDPD/meta.json: no such file or directory"
ts=2022-06-30T03:57:08.413Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any"
ts=2022-06-30T03:57:35.681Z caller=head.go:520 level=error component=tsdb msg="Loading on-disk chunks failed" err="iterate on on-disk chunks: corruption in head chunk file /data/chunks_head/001701: checksum mismatch expected:00000000, actual:cfead048"
ts=2022-06-30T03:57:35.684Z caller=head.go:689 level=info component=tsdb msg="Deleting mmapped chunk files"
ts=2022-06-30T03:57:35.764Z caller=head.go:699 level=info component=tsdb msg="Deletion of mmap chunk files successful, reattempting m-mapping the on-disk chunks"
ts=2022-06-30T03:57:35.869Z caller=head.go:536 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=27.456523956s
ts=2022-06-30T03:57:35.869Z caller=head.go:542 level=info component=tsdb msg="Replaying WAL, this may take a while"
ts=2022-06-30T03:58:34.193Z caller=head.go:578 level=info component=tsdb msg="WAL checkpoint loaded"
ts=2022-06-30T03:58:48.103Z caller=db.go:750 level=warn component=tsdb msg="Encountered WAL read error, attempting repair" err="read records: corruption in segment /data/wal/00003413 at 77221215: snappy: corrupt input"
ts=2022-06-30T03:58:48.103Z caller=wal.go:364 level=warn component=tsdb msg="Starting corruption repair" segment=3413 offset=77221215
ts=2022-06-30T03:58:48.455Z caller=wal.go:372 level=warn component=tsdb msg="Deleting all segments newer than corrupted segment" segment=3413
ts=2022-06-30T03:59:54.766Z caller=wal.go:394 level=warn component=tsdb msg="Rewrite corrupted segment" segment=3413
ts=2022-06-30T04:00:44.492Z caller=main.go:958 level=info fs_type=65735546
ts=2022-06-30T04:00:44.492Z caller=main.go:961 level=info msg="TSDB started"

prometheus 异常退出报错：opening storage failed相关推荐

DM7数据库DMAP服务异常，报错“ dmap init failed, code[-7157]: 管道文件已存在”
问题描述:centos7下达梦数据库DM7启动dmap服务报错" dmap init failed, code[-7157]: 管道文件已存在" 1)查看DmAPService服务 ...
神奇的python(五)之python强制退出报错(异常)解决方法
简介:python程序在执行时,有时候由于某些模块冲突或者是定时器冲突,会出现很多奇怪的错误,为此我们需要让python程序在退出时免受这些问题的干扰. (1) 常用的信号类型: SIGINT 终止进 ...
centos6.5报错:checking filesystems failed问题处理
centos系统重启报错:checking filesystems failed checking filesystems /dev/mapper/vg_0-root: 搜了下可能是文件系统损坏根据 ...
quartz集群报错but has failed to stop it. This is very likely to create a memory leak.
quartz集群报错but has failed to stop it. This is very likely to create a memory leak. 在一台配置1核2G内存的阿里云服务器 ...
解决只读模式U盘保护格式化或者dd写报错：ERROR: failed to open ‘/dev/sdb‘ in read-write mode: Read-only file system.
解决只读模式U盘保护格式化或者dd写报错:ERROR: failed to open '/dev/sdb' in read-write mode: Read-only file system. 或者: ...
gitlab报错 fatal: index-pack failed error: RPC failed; result=18, HTTP code = 200解决方案
gitlab报错 "fatal: index-pack failed error: RPC failed; result=18, HTTP code = 200",如下图这个问题 ...
SAP WM 采购订单收货后LT06报错-Allowed storage unit types not defined for storage type Z03-
SAP WM 采购订单收货后LT06报错-Allowed storage unit types not defined for storage type Z03- 1,如下的storage type ...
cordova编译报错：Execution failed for task ':processDebugResources'
cordova编译报错:Execution failed for task ':processDebugResources' 引发这个错误的最扩祸首就是一个中文命名的文件,不知道什么时候加入的,我写了 ...
git配置报错fatal: Authentication failed for ‘‘问题解决
git配置报错fatal: Authentication failed for ''问题解决参考文章: (1)git配置报错fatal: Authentication failed for ''问题 ...
mysql主从skip1677_解决字符集不同引起的主从同步异常1677报错问题
之前的一篇文章中,我们遇到了主从同步的一个问题,错误代码:1236,详细请看今天,突然发现测试环境的主从同步关系断开了,报错代码:1677 一.问题错误信息 Last_SQL_Errno: 1677 ...

prometheus 异常退出报错：opening storage failed

prometheus 异常退出报错：opening storage failed相关推荐

最新文章

热门文章

prometheus 异常退出 报错：opening storage failed

prometheus 异常退出 报错：opening storage failed相关推荐

最新文章

热门文章

prometheus 异常退出报错：opening storage failed

prometheus 异常退出报错：opening storage failed相关推荐