mysql ogg kafka,OGG到kafka替换目标端OGG
1.--源端
--停源端所有进程
stop *
stop mgr
2.--目标端进程
info REP_K4 showch---看RBA是否变化,查询checkpoint 表
send REP_K4 status--查看 有没有大事物
--确认上述信息OK,则停目标端所有进程
stop *
stop mgr
3.----新目标端
备份之前安装的软件
cd /data/ogg
mkdir bak
ls
bak ggs12
cd ggs12
mv * ./../bak
cd /data/ogg
rm -rf ggs12
---目标库
[root@OGG-CLIENT ogg]# mkdir bak
[root@OGG-CLIENT ogg]# cd ggs12/
[root@OGG-CLIENT ggs12]# mv * ./../bak/
[root@OGG-CLIENT ggs12]# ls
4.---旧目标端,将全部信息传递到新的目标段
scp -r ggs12/ app@1.3.10.11:/data/ogg
5.启动所有进程
drwxr-x--- 2 app app 206 Jul 5 17:52 dirchk
drwxr-x--- 2 app app 6 Jul 2 20:23 dircrd--空
drwxr-x--- 2 app app 4096 Aug 20 22:15 dirdat
drwxr-x--- 2 app app 4096 Aug 20 20:52 dirdef
drwxr-x--- 2 app app 108 Jul 23 19:35 dirdmp
drwxr-x--- 4 app app 38 Aug 29 2017 diretc
drwxr-x--- 2 app app 24 Aug 29 2017 dirout
drwxr-x--- 2 app app 121 Aug 20 20:53 dirpcs
drwxr-x--- 2 app app 326 Jul 23 19:49 dirprm
drwxr-x--- 2 app app 8192 Aug 20 20:53 dirrpt
drwxr-x--- 2 app app 6 Jul 2 20:23 dirsql--空
drwxr-x--- 2 app app 6 Aug 20 20:53 dirtmp--空
drwxr-x--- 2 app app 6 Jul 2 20:23 dirwlt--空
drwxr-x--- 5 app app 46 Aug 29 2017 dirwww
---报错
2019-08-20 23:31:23 ERROR OGG-01091 Unable to open file "/data/ggs12/dirdat/t1002322" (error 2, No such file or directory).
cd /data/ogg/ggs12/dirdat
[app@OGG-CLIENT dirdat]$ ll t1002322
cd /data/ogg/ggs12
./ggsci
stop mgr
[app@OGG-CLIENT data]$ ps -ef|grep mgr
app 96817 1 0 23:30 ? 00:00:00 ./mgr PARAMFILE /data/ogg/ggs12/dirprm/mgr.prm REPORTFILE /data/ogg/ggs12/dirrpt/MGR.rpt PROCESSID MGR
app 97030 96606 0 23:35 pts/0 00:00:00 grep --color=auto mgr
[app@OGG-CLIENT data]$ kill -9 96817
[app@OGG-CLIENT data]$ cd ggs12/
[app@OGG-CLIENT ggs12]$ ./ggsci
[app@OGG-CLIENT ogg]$ mv ggs12/ ./..
---再次报错
[app@OGG-CLIENT ggs12]$ ./ggsci
Oracle GoldenGate for Big Data
Version 12.3.1.1.1
Oracle GoldenGate Command Interpreter
Version 12.3.0.1.0 OGGCORE_OGGADP.12.3.0.1.0GA_PLATFORMS_170828.1608
Linux, x64, 64bit (optimized), Generic on Aug 28 2017 17:13:45
Operating system character set identified as UTF-8.
Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved.
Source Context :
SourceModule : [ggparam.repo]
SourceID : [/scratch/aime/adestore/views/aime_adc4150324/oggcore/OpenSys/src/gglib/ggparam/paramrepo.cpp]
SourceMethod : [open_specfile]
SourceLine : [1221]
ThreadBacktrace : [15] elements
: [/data/ggs12/libgglog.so(CMessageContext::AddThreadContext()+0x1e) [0x7fc673a2e54e]]
: [/data/ggs12/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x6ac) [0x7fc673a1ed4c]]
: [/data/ggs12/libgglog.so(_MSG_String_Int32_String(CSourceContext*, int, char const*, int, char const*, CMessageFactory::MessageDisposition)+0x4a) [0x7fc673a1226a]]
: [/data/ggs12/libggparam.so(ggs::ggparam::repo::open_specfile(char const*)+0x2f0) [0x7fc671b833b0]]
: [/data/ggs12/libggparam.so(ggs::ggparam::repo::open()+0x51) [0x7fc671b849e1]]
: [/data/ggs12/libggparam.so(ggs::ggparam::dict::dict()+0x104) [0x7fc671b8ccf4]]
: [/data/ggs12/libggparam.so(ggs::ggparam::dict::instance()+0x43) [0x7fc671b8cea3]]
: [./ggsci(readGlobalConfigFile()+0x59) [0x54b929]]
: [./ggsci() [0x62cecb]]
: [./ggsci(ggs::gglib::MultiThreading::MainThread::ExecMain()+0x5e) [0x5550de]]
: [./ggsci(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*)+0x173) [0x5594a3]]
: [./ggsci(ggs::gglib::MultiThreading::MainThread::Run(int, char**)+0x140) [0x559d10]]
: [./ggsci(main+0x3b) [0x64c1cb]]
: [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fc66d119445]]
: [./ggsci(__gxx_personality_v0+0x299) [0x4b6529]]
2019-08-20 23:38:14 ERROR OGG-10111 Error opening parameter definition file: /data/ogg/ggs12/ggparam.dat.
errno: 2 - No such file or directory.
2019-08-20 23:38:14 WARNING OGG-01525 Failed to open trace output file, '/data/ogg/ggs12/dirdmp/gglog-ggsci.dmp', error 2 (No such file or directory).
2019-08-20 23:38:14 ERROR OGG-01668 PROCESS ABENDING.
[app@OGG-CLIENT ggs12]$ cd
[app@OGG-CLIENT ~]$ vi .bash_profile
[app@OGG-CLIENT ~]$ export OGG_HOME=
[app@OGG-CLIENT ~]$ cd /data/ggs12/
[app@OGG-CLIENT ggs12]$ ./ggsci
Oracle GoldenGate for Big Data
Version 12.3.1.1.1
Oracle GoldenGate Command Interpreter
Version 12.3.0.1.0 OGGCORE_OGGADP.12.3.0.1.0GA_PLATFORMS_170828.1608
Linux, x64, 64bit (optimized), Generic on Aug 28 2017 17:13:45
Operating system character set identified as UTF-8.
Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved.
GGSCI (OGG-CLIENT) 1> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER STOPPED
REPLICAT ABENDED REP_K1 00:00:00 01:07:47
REPLICAT ABENDED REP_K2 00:00:00 01:07:47
REPLICAT ABENDED REP_K3 00:00:00 01:07:46
REPLICAT ABENDED REP_K4 00:00:00 01:07:45
REPLICAT ABENDED REP_K5 00:00:00 01:07:45
GGSCI (OGG-CLIENT) 2> start mgr
Manager started.
GGSCI (OGG-CLIENT) 3> info mgr
Manager is running (IP port OGG-CLIENT.8909, Process ID 97094).
GGSCI (OGG-CLIENT) 4> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT ABENDED REP_K1 00:00:00 01:07:56
REPLICAT ABENDED REP_K2 00:00:00 01:07:56
REPLICAT ABENDED REP_K3 00:00:00 01:07:55
REPLICAT ABENDED REP_K4 00:00:00 01:07:54
REPLICAT ABENDED REP_K5 00:00:00 01:07:53
GGSCI (OGG-CLIENT) 5> start REP_K1
Sending START request to MANAGER ...
REPLICAT REP_K1 starting
GGSCI (OGG-CLIENT) 6> info REP_K1
REPLICAT REP_K1 Last Started 2019-08-20 23:41 Status RUNNING
Checkpoint Lag 00:00:00 (updated 01:08:03 ago)
Process ID 97103
Log Read Checkpoint File /data/ggs12/dirdat/t1002322
First Record RBA 49652169
GGSCI (OGG-CLIENT) 7> !
info REP_K1
REPLICAT REP_K1 Last Started 2019-08-20 23:41 Status RUNNING
Checkpoint Lag 01:11:06 (updated 00:00:00 ago)
Process ID 97103
Log Read Checkpoint File /data/ggs12/dirdat/t1002324
2019-08-20 22:30:41.200193 RBA 8347811
[app@OGG-CLIENT ~]$ vi .bash_profile
[app@OGG-CLIENT ~]$ cat .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
#export OGG_HOME=/data/ogg/ggs12
export OGG_HOME=/data/ggs12
JAVA_HOME="/home/app/jdk1.8.0_101"
CLASSPATH="$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib"
NODE_HOME="/home/app/node-v8.11.2-linux-x64"
PATH=$PATH:$HOME/.local/bin:$HOME/bin:$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$NODE_HOME/bin
export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64:$JAVA_HOME/jre/lib/amd64/libjsig.so:$JAVA_HOME/jre/lib/amd64/server:$JAVA_HOME/jre/lib/amd64/server/libjvm.so:$LD_LIBRARY_PATH
stty erase ^h
export PATH
[app@OGG-CLIENT ~]$ source .bash_profile
[app@OGG-CLIENT ~]$ cd /data/ggs12/
ogg for kafka 平滑过渡方案落地 注意:原目标端ogg目录是啥,到目标端 ogg目录还得一致。
1.--源端
--停源端所有进程stop *stop mgr
2.--目标端进程info REP_K4 showch---看RBA是否变化,查询checkpoint 表send REP_K4 status--查看 有没有大事物
--确认上述信息OK,则停目标端所有进程stop *stop mgr
3.----新目标端备份之前安装的软件cd /data/oggmkdir bak
ls bak ggs12
cd ggs12mv * ./../bak
cd /data/oggrm -rf ggs12
---目标库[root@OGG-CLIENT ogg]# mkdir bak[root@OGG-CLIENT ogg]# cd ggs12/[root@OGG-CLIENT ggs12]# mv * ./../bak/[root@OGG-CLIENT ggs12]# ls
4.---旧目标端,将全部信息传递到新的目标段scp -r ggs12/ app@10.3.120.151:/data/ogg
5.启动所有进程
drwxr-x--- 2 app app 206 Jul 5 17:52 dirchkdrwxr-x--- 2 app app 6 Jul 2 20:23 dircrd--空drwxr-x--- 2 app app 4096 Aug 20 22:15 dirdatdrwxr-x--- 2 app app 4096 Aug 20 20:52 dirdefdrwxr-x--- 2 app app 108 Jul 23 19:35 dirdmpdrwxr-x--- 4 app app 38 Aug 29 2017 diretcdrwxr-x--- 2 app app 24 Aug 29 2017 diroutdrwxr-x--- 2 app app 121 Aug 20 20:53 dirpcsdrwxr-x--- 2 app app 326 Jul 23 19:49 dirprmdrwxr-x--- 2 app app 8192 Aug 20 20:53 dirrptdrwxr-x--- 2 app app 6 Jul 2 20:23 dirsql--空drwxr-x--- 2 app app 6 Aug 20 20:53 dirtmp--空drwxr-x--- 2 app app 6 Jul 2 20:23 dirwlt--空drwxr-x--- 5 app app 46 Aug 29 2017 dirwww
---报错2019-08-20 23:31:23 ERROR OGG-01091 Unable to open file "/data/ggs12/dirdat/t1002322" (error 2, No such file or directory).
cd /data/ogg/ggs12/dirdat[app@OGG-CLIENT dirdat]$ ll t1002322
cd /data/ogg/ggs12./ggscistop mgr
[app@OGG-CLIENT data]$ ps -ef|grep mgrapp 96817 1 0 23:30 ? 00:00:00 ./mgr PARAMFILE /data/ogg/ggs12/dirprm/mgr.prm REPORTFILE /data/ogg/ggs12/dirrpt/MGR.rpt PROCESSID MGRapp 97030 96606 0 23:35 pts/0 00:00:00 grep --color=auto mgr[app@OGG-CLIENT data]$ kill -9 96817[app@OGG-CLIENT data]$ cd ggs12/[app@OGG-CLIENT ggs12]$ ./ggsci
[app@OGG-CLIENT ogg]$ mv ggs12/ ./..
---再次报错
[app@OGG-CLIENT ggs12]$ ./ggsci Oracle GoldenGate for Big DataVersion 12.3.1.1.1
Oracle GoldenGate Command InterpreterVersion 12.3.0.1.0 OGGCORE_OGGADP.12.3.0.1.0GA_PLATFORMS_170828.1608Linux, x64, 64bit (optimized), Generic on Aug 28 2017 17:13:45Operating system character set identified as UTF-8.
Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved.
Source Context : SourceModule : [ggparam.repo] SourceID : [/scratch/aime/adestore/views/aime_adc4150324/oggcore/OpenSys/src/gglib/ggparam/paramrepo.cpp] SourceMethod : [open_specfile] SourceLine : [1221] ThreadBacktrace : [15] elements : [/data/ggs12/libgglog.so(CMessageContext::AddThreadContext()+0x1e) [0x7fc673a2e54e]] : [/data/ggs12/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x6ac) [0x7fc673a1ed4c]] : [/data/ggs12/libgglog.so(_MSG_String_Int32_String(CSourceContext*, int, char const*, int, char const*, CMessageFactory::MessageDisposition)+0x4a) [0x7fc673a1226a]] : [/data/ggs12/libggparam.so(ggs::ggparam::repo::open_specfile(char const*)+0x2f0) [0x7fc671b833b0]] : [/data/ggs12/libggparam.so(ggs::ggparam::repo::open()+0x51) [0x7fc671b849e1]] : [/data/ggs12/libggparam.so(ggs::ggparam::dict::dict()+0x104) [0x7fc671b8ccf4]] : [/data/ggs12/libggparam.so(ggs::ggparam::dict::instance()+0x43) [0x7fc671b8cea3]] : [./ggsci(readGlobalConfigFile()+0x59) [0x54b929]] : [./ggsci() [0x62cecb]] : [./ggsci(ggs::gglib::MultiThreading::MainThread::ExecMain()+0x5e) [0x5550de]] : [./ggsci(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*)+0x173) [0x5594a3]] : [./ggsci(ggs::gglib::MultiThreading::MainThread::Run(int, char**)+0x140) [0x559d10]] : [./ggsci(main+0x3b) [0x64c1cb]] : [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fc66d119445]] : [./ggsci(__gxx_personality_v0+0x299) [0x4b6529]]
2019-08-20 23:38:14 ERROR OGG-10111 Error opening parameter definition file: /data/ogg/ggs12/ggparam.dat.errno: 2 - No such file or directory.
2019-08-20 23:38:14 WARNING OGG-01525 Failed to open trace output file, '/data/ogg/ggs12/dirdmp/gglog-ggsci.dmp', error 2 (No such file or directory).
2019-08-20 23:38:14 ERROR OGG-01668 PROCESS ABENDING.[app@OGG-CLIENT ggs12]$ cd [app@OGG-CLIENT ~]$ vi .bash_profile [app@OGG-CLIENT ~]$ export OGG_HOME=[app@OGG-CLIENT ~]$ cd /data/ggs12/[app@OGG-CLIENT ggs12]$ ./ggsci Oracle GoldenGate for Big DataVersion 12.3.1.1.1
Oracle GoldenGate Command InterpreterVersion 12.3.0.1.0 OGGCORE_OGGADP.12.3.0.1.0GA_PLATFORMS_170828.1608Linux, x64, 64bit (optimized), Generic on Aug 28 2017 17:13:45Operating system character set identified as UTF-8.
Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved.
GGSCI (OGG-CLIENT) 1> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER STOPPED REPLICAT ABENDED REP_K1 00:00:00 01:07:47 REPLICAT ABENDED REP_K2 00:00:00 01:07:47 REPLICAT ABENDED REP_K3 00:00:00 01:07:46 REPLICAT ABENDED REP_K4 00:00:00 01:07:45 REPLICAT ABENDED REP_K5 00:00:00 01:07:45
GGSCI (OGG-CLIENT) 2> start mgrManager started.
GGSCI (OGG-CLIENT) 3> info mgr
Manager is running (IP port OGG-CLIENT.8909, Process ID 97094).
GGSCI (OGG-CLIENT) 4> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING REPLICAT ABENDED REP_K1 00:00:00 01:07:56 REPLICAT ABENDED REP_K2 00:00:00 01:07:56 REPLICAT ABENDED REP_K3 00:00:00 01:07:55 REPLICAT ABENDED REP_K4 00:00:00 01:07:54 REPLICAT ABENDED REP_K5 00:00:00 01:07:53
GGSCI (OGG-CLIENT) 5> start REP_K1
Sending START request to MANAGER ...REPLICAT REP_K1 starting
GGSCI (OGG-CLIENT) 6> info REP_K1
REPLICAT REP_K1 Last Started 2019-08-20 23:41 Status RUNNINGCheckpoint Lag 00:00:00 (updated 01:08:03 ago)Process ID 97103Log Read Checkpoint File /data/ggs12/dirdat/t1002322 First Record RBA 49652169
GGSCI (OGG-CLIENT) 7> !info REP_K1
REPLICAT REP_K1 Last Started 2019-08-20 23:41 Status RUNNINGCheckpoint Lag 01:11:06 (updated 00:00:00 ago)Process ID 97103Log Read Checkpoint File /data/ggs12/dirdat/t1002324 2019-08-20 22:30:41.200193 RBA 8347811
[app@OGG-CLIENT ~]$ vi .bash_profile [app@OGG-CLIENT ~]$ cat .bash_profile # .bash_profile
# Get the aliases and functionsif [ -f ~/.bashrc ]; then. ~/.bashrcfi
# User specific environment and startup programs
#export OGG_HOME=/data/ogg/ggs12export OGG_HOME=/data/ggs12JAVA_HOME="/home/app/jdk1.8.0_101"CLASSPATH="$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib"NODE_HOME="/home/app/node-v8.11.2-linux-x64"PATH=$PATH:$HOME/.local/bin:$HOME/bin:$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$NODE_HOME/binexport LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64:$JAVA_HOME/jre/lib/amd64/libjsig.so:$JAVA_HOME/jre/lib/amd64/server:$JAVA_HOME/jre/lib/amd64/server/libjvm.so:$LD_LIBRARY_PATHstty erase ^h
export PATH
[app@OGG-CLIENT ~]$ source .bash_profile[app@OGG-CLIENT ~]$ cd /data/ggs12/
ogg for kafka 平滑过渡方案落地 注意:原目标端ogg目录是啥,到目标端 ogg目录还得一致。
mysql ogg kafka,OGG到kafka替换目标端OGG相关推荐
- 使用ogg实现oracle到kafka的增量数据实时同步
Oracle Golden Gate软件是一种基于日志的结构化数据复制备份软件,它通过解析源数据库在线日志或归档日志获得数据的增量变化,再将这些变化应用到目标数据库,从而实现源数据库与目标数据库同步. ...
- ogg oracle 测试kafka_利用ogg实现oracle到kafka的增量数据实时同步
前言 ogg即Oracle GoldenGate是Oracle的同步工具,本文讲如何配置ogg以实现Oracle数据库增量数据实时同步到kafka中,其中同步消息格式为json. 下面是我的源端和目标 ...
- 利用OGG实现Oracle到Kafka到Greenplum的增量数据同步
墨墨导读:本文来自墨天轮用户 肖杰 的投稿,介绍用OGG实现Oracle到Kafka到Greenplum的增量数据同步的全过程. 墨天轮主页:https://www.modb.pro/u/6722 背 ...
- oracle oci.dll无法加载_基于OGG 实现Oracle到Kafka增量数据实时同步
背景 在大数据时代,存在大量基于数据的业务.数据需要在不同的系统之间流动.整合.通常,核心业务系统的数据存在OLTP数据库系统中,其它业务系统需要获取OLTP系统中的数据.传统的数仓通过批量数据同步的 ...
- oracle指定源位置怎么弄,ORACLE Goldengate测试解决源端和目标端表结构字段位置不同的2种实现方法...
测试环境: 源端操作系统:rhel5.6 目标端操作系统:windows2008 R2 源和目标使用的数据版本均为:11.2.0.4.0 一.先模拟问题现象 源和目标端搭建好OGG软件和oracle数 ...
- 目标端的trail文件损坏或误删除,如何重新进行抽取
本实验主要用于:目标端OGG的trail文件损坏或人为误删除,在不丢失任何数据的情况下, 如何进行操作 1 重新启动目标端并关闭,查看检查点的信息:---此处是模拟trail文件被删除,复制进程读取的 ...
- ORACLE 源端ogg同步到mysql目标端
source源端 oracle 11.2.0.4 ogg12c ip:10.200.3.193 系统centos 7.1 1. 创建source表和一些初始化数据 su - oracle sqlplu ...
- 面试官问:为什么MySQL的索引不采用Kafka的索引机制
点击上方"方志朋",选择"设为星标" 回复"666"获取新整理的面试资料 第一眼看到这个问题,也是很迷惑的,谁没事会问这种问题.然而,事实上 ...
- mysql实时监听canal+kafka
1.首先安装并启动mysql mysql5.7版本安装部署详细步骤_怪只怪满眼尽是人间烟火-CSDN博客最新版mysql下载地址:MySQL :: Download MySQL Community S ...
最新文章
- 实例讲解《Microsoft AJAX Library》(2):DomEvent类
- leetcode刷题 66 67
- python文件输出中文_【python】中文的输出,打印,文件编码问题解决方法
- 网速慢?NO可能是路由器的原因?
- python内函数名加括号和不加括号的区别,python中 函数名加括号与不加括号
- iOS 点转成字符串,再字符串转换成点
- springboot项目中mybatis实现数据的基本查询
- javascript arguments(转)
- 高中信息技术——进制与编码刷题点整理
- Web项目常用技术整理
- js实现发送验证码倒计时按钮
- 黑苹果 惠普笔记本电池补丁_小米笔记本Pro 黑苹果 改装内置WIFI和蓝牙
- Pos应用与扫盲一路通
- PHP傻瓜式一键建站系统源码+无需授权
- CodeForces 312B Archer
- 【科目一】你必须知道的驾考交通标志大全
- 互联网公司的中台到底是什么
- 顺丰终端服务器断网,顺丰手持终端第六代,实现收派件智能化操作
- insmod过程详解
- 11月初.wang域名总量TOP15:13家中国域名商上榜
热门文章
- BP神经网络的基本思想,一文搞定bp神经网络
- Pthreads并行编程之spin lock与mutex性能对比分析
- Python是个什么鬼?为什么建筑设计都要学它?
- 问题 A: 鸡兔同笼
- 区块链具有诸多技术优势 在国外基础教育中的应用案例
- linux云自动化运维,Liunx运维一线大神亲授 全新Linux云计算运维基础与Linux Shell自动化运维实战课程...
- 艾永亮:区域性企业如何进入全国市场?
- Python 封闭科赫曲线绘制
- 《Python 深度学习》刷书笔记 Chapter 4 关于电影评论模型的进一步探讨
- linux中如何解压.tgz