某集团的ebs系统因磁盘空间不足把redo和undo存放到raid 0之上，而且该库无任何备份。最终悲剧发生了,raid 0异常导致redo undo全部丢失,数据库无法正常启动(我接手之时数据库已经resetlogs过,但是未成功)

Sun Jul 27 11:31:27 2014

SMON: enabling cache recovery

SMON: enabling tx recovery

Sun Jul 27 11:31:27 2014

Database Characterset is ZHS16GBK

Sun Jul 27 11:31:27 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/bdump/prod_smon_454754.trc:

ORA-00604: error occurred at recursive SQL level 1

ORA-00376: file 42 cannot be read at this time

ORA-01110: data file 42: '/prod/oracle/PROD/logdata/undo/undo1.dbf'

Sun Jul 27 11:31:27 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/bdump/prod_smon_454754.trc:

ORA-00604: error occurred at recursive SQL level 1

ORA-00376: file 42 cannot be read at this time

ORA-01110: data file 42: '/prod/oracle/PROD/logdata/undo/undo1.dbf'

Sun Jul 27 11:31:27 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/bdump/prod_smon_454754.trc:

ORA-00604: error occurred at recursive SQL level 1

ORA-00376: file 42 cannot be read at this time

ORA-01110: data file 42: '/prod/oracle/PROD/logdata/undo/undo1.dbf'

Sun Jul 27 11:31:27 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_663670.trc:

ORA-00604: error occurred at recursive SQL level 1

ORA-00376: file 41 cannot be read at this time

ORA-01110: data file 41: '/prod/oracle/PROD/logdata/undo/undo2.dbf'

Error 604 happened during db open, shutting down database

USER: terminating instance due to error 604

Instance terminated by USER, pid = 663670

ORA-1092 signalled during: ALTER DATABASE OPEN...

查询相关文件状态发现,undo表空间文件丢失,被offline处理

因为以前alert日志被清理,通过这里大概猜测是offline丢失的undo文件,然后resetlogs了数据库,现在处理方式为

使用_corrupted_rollback_segments屏蔽回滚段,然后尝试启动数据库

Tue Jul 29 11:40:39 2014

SMON: enabling cache recovery

SMON: enabling tx recovery

Tue Jul 29 11:40:39 2014

Database Characterset is ZHS16GBK

Tue Jul 29 11:40:39 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/bdump/prod_smon_569378.trc:

ORA-00604: error occurred at recursive SQL level 1

ORA-01555: snapshot too old: rollback segment number with name "" too small

Tue Jul 29 11:40:39 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/bdump/prod_smon_569378.trc:

ORA-00604: error occurred at recursive SQL level 1

ORA-01555: snapshot too old: rollback segment number with name "" too small

Tue Jul 29 11:40:39 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/bdump/prod_smon_569378.trc:

ORA-00604: error occurred at recursive SQL level 1

ORA-01555: snapshot too old: rollback segment number with name "" too small

Tue Jul 29 11:40:39 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_585786.trc:

ORA-00604: error occurred at recursive SQL level 1

ORA-01555: snapshot too old: rollback segment number with name "" too small

Error 604 happened during db open, shutting down database

USER: terminating instance due to error 604

Instance terminated by USER, pid = 585786

ORA-1092 signalled during: alter database open...

该错误是由于数据库启动需要找到对应的回滚段,但是由于undo异常导致该回滚段无法找到，因此出现该错误，解决方法是通过修改数据scn，让其不找回滚段,从而屏蔽该错误.数据库启动后,删除undo重新创建新undo

Tue Jul 29 15:59:22 2014

drop tablespace undo2 including contents and datafiles

Tue Jul 29 15:59:23 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_782490.trc:

ORA-01122: database file 41 failed verification check

ORA-01110: data file 41: '/prod/oracle/PROD/logdata/undo/undo2.dbf'

ORA-01565: error in identifying file '/prod/oracle/PROD/logdata/undo/undo2.dbf'

ORA-27037: unable to obtain file status

IBM AIX RISC System/6000 Error: 2: No such file or directory

Additional information: 3

Tue Jul 29 15:59:23 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_782490.trc:

ORA-01259: unable to delete datafile /prod/oracle/PROD/logdata/undo/undo2.dbf

Tue Jul 29 15:59:23 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_782490.trc:

ORA-01122: database file 42 failed verification check

ORA-01110: data file 42: '/prod/oracle/PROD/logdata/undo/undo1.dbf'

ORA-01565: error in identifying file '/prod/oracle/PROD/logdata/undo/undo1.dbf'

ORA-27037: unable to obtain file status

IBM AIX RISC System/6000 Error: 2: No such file or directory

Additional information: 3

ORA-01259: unable to delete datafile /prod/oracle/PROD/logdata/undo/undo2.dbf

Tue Jul 29 15:59:23 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_782490.trc:

ORA-01259: unable to delete datafile /prod/oracle/PROD/logdata/undo/undo1.dbf

Tue Jul 29 15:59:23 2014

Completed: drop tablespace undo2 including contents and datafiles

Tue Jul 29 15:59:56 2014

create undo tablespace undotbs1 datafile '/prod/oracle/PROD/logdata/undo_new01.dbf' size 100M autoextend on next 128M maxsize 30G

Tue Jul 29 15:59:57 2014

Completed: create undo tablespace undotbs1 datafile '/prod/oracle/PROD/logdata/undo_new01.dbf' size 100M autoextend on next 128M maxsize 30G

Tue Jul 29 16:00:03 2014

alter tablespace undotbs1 add datafile '/prod/oracle/PROD/logdata/undo_new02.dbf' size 100M autoextend on next 128M maxsize 30G

Completed: alter tablespace undotbs1 add datafile '/prod/oracle/PROD/logdata/undo_new02.dbf' size 100M autoextend on next 128M maxsize 30G

业务运行过程中,数据库报大量ORA-600 4097,ORA-600 kdsgrp1,ORA-600 kcfrbd_3错误

Tue Jul 29 16:07:03 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_950484.trc:

ORA-00600: internal error code, arguments: [4097], [], [], [], [], [], [], []

Tue Jul 29 16:07:06 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_950484.trc:

ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []

Tue Jul 29 16:10:06 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_917702.trc:

ORA-00600: internal error code, arguments: [4097], [], [], [], [], [], [], []

Tue Jul 29 16:10:07 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_917702.trc:

ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], []

Tue Jul 29 16:12:45 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/bdump/prod_m000_880692.trc:

ORA-00600: internal error code, arguments: [4097], [], [], [], [], [], [], []

Tue Jul 29 16:21:23 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_1040638.trc:

ORA-00600: 内部错误代码, 参数: [kcfrbd_3], [41], [231381], [1], [12800], [12800], [], []

Tue Jul 29 16:21:37 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_1040638.trc:

ORA-00600: 内部错误代码, 参数: [kcfrbd_3], [41], [231381], [1], [12800], [12800], [], []

Tue Jul 29 16:21:56 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_1040638.trc:

ORA-00600: 内部错误代码, 参数: [kcfrbd_3], [41], [231381], [1], [12800], [12800], [], []

Tue Jul 29 16:22:18 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_1040638.trc:

ORA-00600: 内部错误代码, 参数: [kcfrbd_3], [41], [231381], [1], [12800], [12800], [], []

Tue Jul 29 16:22:28 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_1105950.trc:

ORA-00600: 内部错误代码, 参数: [4097], [], [], [], [], [], [], []

Tue Jul 29 16:22:33 2014

Errors in file /prod/oracle/PROD/db/tech_st/10.2.0/admin/PROD_erpserver/udump/prod_ora_1159232.trc:

ORA-00600: 内部错误代码, 参数: [kcfrbd_3], [42], [61235], [1], [12800], [12800], [], []

出现该错误有几个原因和解决方法：

ORA-600 kdsgrp1 是因为相关坏块引起(tab,index,memory,cr block等),结合日志分析对象异常原因,根据具体情况确定对象然后选择合适处理方案(具体参考NOTE:1332252.1)

ORA-600 4097 由于数据库异常关闭然后open,创建回滚段,可能触发bug导致该问题(虽然说在当前版本修复,但是实际处理我确实按照NOTE:1030620.6解决)

ORA-600 kcfrbd_3 有事务的block被访问之后,根据回滚槽信息定位到相关回滚段,而正好新建的回滚段信息又和以前的名字编号一致,从而反馈出来是数据文件大小不够,从而出现该错误(具体参考NOTE:601798.1)

最终该数据库虽然恢复了,抢救了大量数据,但是对于ebs系统来说,丢失redo和undo数据的损失还是巨大的.再次温馨提示:数据库的redo,undo也很重要,数据库的备份更加重要

mysql undo丢失_某集团ebs数据库redo undo丢失导致悲剧相关推荐

  1. 阿里云 mysql 表锁死_解决阿里云数据库MySQL实例空间满自动锁问题

    MySQL实例可能会由于SQL语句.外部攻击等原因导致实例空间满,为避免数据丢失,RDS会对实例进行自动锁定,磁盘锁定之后,将无法进行写入操作. 背景信息 当实例由于实例空间满自动锁定时,控制台可以在 ...

  2. ef mysql 读写分离_基于 EntityFramework 的数据库主从读写分离服务插件

    基于 EntityFramework 的数据库主从读写分离服务插件 1. 版本信息和源码 1.1版本信息 v1.01 beta(2015-04-07),基于 EF 6.1 开发,支持 EF 6.1 之 ...

  3. mysql节假日表_节假日常见的数据库磁盘空间处理小结

    数据库的报警可以拆分为很多类别,但是有一点是无论如何都跑不掉的,而且花样百出,那就是磁盘空间报警. 在我的认知中,磁盘空间报警可以从上向下,从下向上的看待,如果从下向上看待,磁盘空间类报警的处理方法相 ...

  4. php mysql主从延迟_如何解决主从数据库同步延迟问题？php连接 mysql 数据库如何添加一个公共的配置文件50...

    在上一篇文章中,小编为您详细介绍了关于<图上属标注的什么样元器件?火车购票明明显示无座为什么样乘车后却发现有很多空座>相关知识.本篇中小编将再为您讲解标题如何解决主从数据库同步延迟问题?p ...

  5. 测试mysql连接服务器_实现服务器与数据库的连接

    如何实现服务器与数据库的连接 第一步:先建立数据库,通过apache将数据库据发布到服务器上 1.配置数据库 安装Apache2.2 2.在计算机管理 --->服务--->启动myApac ...

  6. 编写mysql的工具_自己编写的数据库工具类

    /** * 数据库工具类 * 1.连接数据库 * 2.执行增删改查功能功能并报错 * 3.没有参数传递是则使用默认信息连接 */ class mysql { private $link = null; ...

  7. mysql 嵌入式 稳定性_常用的嵌入式数据库的比较

    2.1 Berkeley DB 技术特点: 1. Berkeley DB是一个开放源代码的内嵌式数据库管理系统,能够为应用程序提供高性能的数据管理服务.应用它程序员只需要调用一些简单的API就可以完成 ...

  8. php pdo 支持mysql表类型_全新的PDO数据库操作类php版（仅适用Mysql）

    复制代码 代码如下: /** * 作者:胡睿 * 日期:2012/07/21 * 电邮:hooray0905@foxmail.com */ class HRDB{ protected $pdo; pr ...

  9. goods购物表MySQL的代码_第一节：数据库及表

    在Mysql中建立一个数据库cart,在库中建立两个表,分别是:goods(商品表),order(订单表). 本教程用命令创建数据库,用命令创建数据库习惯了,比用PhpMyAdmin快,因为用phpM ...

最新文章

  1. mybatis CRUD操作
  2. [转]ASP.NET1.0升级ASP.NET2.0问题总结
  3. 终于有人把 Python 讲清楚了！
  4. mysql-mybatis 8.0版本配置====解决could not create connection to database server.
  5. 综述|深度学习在SLAM定位与建图中的应用（近250篇参考文献）_3Ｄ视觉工坊的博客-CSDN博客
  6. 行业软件和鸿蒙,华为鸿蒙负责人王成录：育人才，打造国产软件“根”能力
  7. const 与 static readonly 的区别
  8. matlab 在图片上绘制热力图_python可视化之seaborn——热力图
  9. delphi tdxdbgrid 导出 excel_9个免费的Excel图表工具，1键即可做出高大上图表，月薪3万没问题...
  10. auto_ptr解析
  11. 命令行开发、编译、打包Android应用程序
  12. Matlab聚类分析（Kmeans）
  13. Dell电脑重装系统
  14. linux中批量删除用户,Linux shell批量删除用户
  15. USB驱动之U盘驱动
  16. Ubuntu16.04《解压7z.001文件》
  17. Golang实现验证码图片OCR
  18. 身份证扫描到一个PDF或图片
  19. linux进程冻结,Linux进程冻结技术
  20. 开源IDS系列--【2015】获取snort vrt 规则（talo）

热门文章

  1. 微信公众平台和开放平台的区别
  2. 坚持创新驱动发展，上海控安入选《2021中国网络安全产业势能榜》
  3. yolov3-tiny　训练。以及yolov3 画图。
  4. 怎样给Tab Control控件加上背景颜色或图片
  5. C语言02基础深入理解（二）
  6. shell脚本 简介 及 第一个脚本程序
  7. Linux系统知识全景个人笔记（一）——Linux结构与组成
  8. 海康威视设备在Web端显示实时（回放）视频
  9. 屏蔽ip段访问 html,php屏蔽ip、ip段、省份地区的访客
  10. 深度学习入门笔记（一）：机器学习基础