debezium-connect-oracle使用

测试环境：

a.数据库：Oracle19c 19.3 已开归档日志

b.Kafka：2.8，内置zookeeper

c.连接器：debezium-connector-oracle-1.6.2.Final-plugin.tar

d.windows10

1.创建用户赋权

SQL> show con_name;CON_NAME
------------------------------
CDB$ROOT
SQL> CREATE TABLESPACE logminer_tbs DATAFILE 'C:\oracle\app\oradata\ORCL\logminer_tbs.dbf'2      SIZE 25M REUSE AUTOEXTEND ON MAXSIZE UNLIMITED;--表空间已创建。SQL> alter session set container=orclpdb;--会话已更改。SQL> show con_name;CON_NAME
------------------------------
ORCLPDB
SQL> CREATE TABLESPACE logminer_tbs DATAFILE 'C:\oracle\app\oradata\ORCL\orclpdb\logminer_tbs.dbf'2      SIZE 25M REUSE AUTOEXTEND ON MAXSIZE UNLIMITED;--表空间已创建。SQL> alter session set container =CDB$ROOT;--会话已更改。
SQL> show con_name;CON_NAME
------------------------------
CDB$ROOT
SQL> CREATE USER c##dbzuser IDENTIFIED BY dbz2      DEFAULT TABLESPACE logminer_tbs3      QUOTA UNLIMITED ON logminer_tbs4      CONTAINER=ALL;--用户已创建。GRANT CREATE SESSION TO c##dbzuser CONTAINER=ALL;GRANT SET CONTAINER TO c##dbzuser CONTAINER=ALL;GRANT SELECT ON V_$DATABASE to c##dbzuser CONTAINER=ALL;GRANT FLASHBACK ANY TABLE TO c##dbzuser CONTAINER=ALL;GRANT SELECT ANY TABLE TO c##dbzuser CONTAINER=ALL;GRANT SELECT_CATALOG_ROLE TO c##dbzuser CONTAINER=ALL;GRANT EXECUTE_CATALOG_ROLE TO c##dbzuser CONTAINER=ALL;GRANT SELECT ANY TRANSACTION TO c##dbzuser CONTAINER=ALL;GRANT LOGMINING TO c##dbzuser CONTAINER=ALL;GRANT CREATE TABLE TO c##dbzuser CONTAINER=ALL;GRANT LOCK ANY TABLE TO c##dbzuser CONTAINER=ALL;GRANT ALTER ANY TABLE TO c##dbzuser CONTAINER=ALL;GRANT CREATE SEQUENCE TO c##dbzuser CONTAINER=ALL;GRANT EXECUTE ON DBMS_LOGMNR TO c##dbzuser CONTAINER=ALL;GRANT EXECUTE ON DBMS_LOGMNR_D TO c##dbzuser CONTAINER=ALL;GRANT SELECT ON V_$LOG TO c##dbzuser CONTAINER=ALL;GRANT SELECT ON V_$LOG_HISTORY TO c##dbzuser CONTAINER=ALL;GRANT SELECT ON V_$LOGMNR_LOGS TO c##dbzuser CONTAINER=ALL;GRANT SELECT ON V_$LOGMNR_CONTENTS TO c##dbzuser CONTAINER=ALL;GRANT SELECT ON V_$LOGMNR_PARAMETERS TO c##dbzuser CONTAINER=ALL;GRANT SELECT ON V_$LOGFILE TO c##dbzuser CONTAINER=ALL;GRANT SELECT ON V_$ARCHIVED_LOG TO c##dbzuser CONTAINER=ALL;GRANT SELECT ON V_$ARCHIVE_DEST_STATUS TO c##dbzuser CONTAINER=ALL;

2.创建测试表，对该表开启归档日志、附加日志

C:\Users\CHEN_he>sqlplus / as sysdbaSQL*Plus: Release 19.0.0.0.0 - Production on 星期五 1月 21 13:17:20 2022
Version 19.3.0.0.0Copyright (c) 1982, 2019, Oracle.  All rights reserved.连接到:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0SQL> conn c##dbzuser/dbz;
已连接。
SQL> CREATE TABLE STU ( "s_id" INT PRIMARY KEY, "s_name" VARCHAR ( 255 ) );表已创建。SQL> ALTER TABLE C##DBZUSER.STU ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;表已更改。

3.修改kafka配置文件

vi config/connect-dis...
#添加连接器插件路径，即上边解压的路径，windows下，路径的 \ 要转义为 \\ ,否则启动连接器回报错invalid plugins.path
plugin.path=debezium-connect-oracle

4.复制所需jar包（Oracle 必须为ojdbc8）

#解压连接器插件，复制jar包到kafka/libs目录下
cp debezium-connect-oracle/*.jar  $kafka_home/libs/

5.启动

./bin/zookeeper-server-start.sh config/zookeeper.con...
./bin/kafka-server-start.sh con.....
./bin/connect-dis.....  con/connect-dis...

6.查看连接器注册情况

curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/

查看现有连接器

curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d

7.注册连接器

命令行容易出错（建议用postman）

{"name": "test_dbz",  // 连接器的名称"config": {"connector.class" : "io.debezium.connector.oracle.OracleConnector",  "database.hostname" : "192.168.211.3",  "database.port" : "1521",  "database.user" : "c##dbzuser",  "database.password" : "dbz",   "database.dbname" : "ORCLCDB",  //数据库的名称"database.server.name" : "server2",  //服务名"tasks.max" : "1",    //连接器创建的最大任务数"database.pdb.name" : "ORCLPDB",  //连接器从中捕获更改的 Oracle 可插拔数据库的名称，不是必须的，但是配置上后，pdb需打开"table.include.list": "C##DBZUSER.STU","database.history.kafka.bootstrap.servers" : "localhost:9092", //kafka服务地址"database.history.kafka.topic": "schema-changes.stu" , //连接器写入和恢复 DDL 语句的数据库历史主题的名称。本主题仅供内部使用，消费者不得使用。"decimal.handling.mode": "string", //number类型乱码解决"log.mining.strategy": "online_catalog",//读取redolog,redolog写满才会生成归档日志，导致topic接收数据慢"snapshot.mode": "schema_only"//取消快照}
}

[root@localhost ~]# curl -H "Accept:application/json" localhost:8083/
{"version":"2.8.0","commit":"ebb1d6e21cc92130","kafka_cluster_id":"5m7EcwSpTQ2qQMSGCfZY2w"}
[root@localhost ~]# curl -H "Accept:application/json" localhost:8083/connectors/
[]
[root@localhost ~]# curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{"name": "stu2","config": {"connector.class": "io.debezium.connector.oracle.OracleConnector","tasks.max": "1","database.server.name": "server2","database.hostname": "192.168.211.3","database.port": "1521","database.user": "c##dbzuser","database.password": "dbz","database.dbname": "ORCL","table.include.list": "C##DBZUSER.STU","database.history.kafka.bootstrap.servers": "localhost:9092","database.history.kafka.topic": "schema-changes.stu"}}'
HTTP/1.1 500 Internal Server Error
Date: Fri, 21 Jan 2022 05:39:16 GMT
Content-Type: application/json
Content-Length: 48
Server: Jetty(9.4.39.v20210325)

{"name": "stu2","config": {"connector.class": "io.debezium.connector.oracle.OracleConnector","tasks.max": "1","database.server.name": "server2","database.hostname": "192.168.211.3","database.port": "1521","database.user": "c##dbzuser","database.password": "dbz","database.dbname": "ORCL","table.include.list": "C##DBZUSER.STU","database.history.kafka.bootstrap.servers": "localhost:9092","database.history.kafka.topic": "schema-changes.stu"}}

8.查看连接器状态

有报错会在下方body中输出

9.查看topic

C:\Kafka\kafka_2.12-2.8.0>bin\windows\kafka-topics.bat --bootstrap-server localhost:9092 --list
__consumer_offsets
connect-configs
connect-offsets
connect-status
schema-changes.test_dbz
server2
server2.C__DBZUSER.STU  ---创建主题时，## 会被自动转换成 __

10.查看消费者消息

注意！！！

测试发现，连接器对于这几个用户的变更也会被捕获到

删除连接器

curl -s http://192.168.211.3:8083/connectors/TEST_MORE -X DELETE

{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"GA=="},"NAME":"kobe"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301452,"snapshot":"true","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301452,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"Fw=="},"NAME":"jodan"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301453,"snapshot":"true","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301453,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"FA=="},"NAME":"ccc"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301453,"snapshot":"true","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301453,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"FQ=="},"NAME":"ccc"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301453,"snapshot":"true","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301453,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"AQ=="},"NAME":"CH"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301453,"snapshot":"true","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301453,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"Ag=="},"NAME":"CHh"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301453,"snapshot":"true","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301454,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"Cg=="},"NAME":"qqq"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301454,"snapshot":"true","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301454,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"Cg=="},"NAME":"qqq"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301454,"snapshot":"true","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301454,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"Cg=="},"NAME":"qqq"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301454,"snapshot":"true","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301454,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"Cg=="},"NAME":"qqq"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642932301454,"snapshot":"last","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":null,"scn":"3278144","commit_scn":null,"lcr_position":null},"op":"r","ts_ms":1642932301454,"transaction":null}}
{"schema":{"type":"struct","fields":[{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"before"},{"type":"struct","fields":[{"type":"struct","fields":[{"type":"int32","optional":false,"field":"scale"},{"type":"bytes","optional":false,"field":"value"}],"optional":true,"name":"io.debezium.data.VariableScaleDecimal","version":1,"doc":"Variable scaled decimal","field":"ID"},{"type":"string","optional":true,"field":"NAME"}],"optional":true,"name":"server2.PDBUSER.TESTPDB.Value","field":"after"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"version"},{"type":"string","optional":false,"field":"connector"},{"type":"string","optional":false,"field":"name"},{"type":"int64","optional":false,"field":"ts_ms"},{"type":"string","optional":true,"name":"io.debezium.data.Enum","version":1,"parameters":{"allowed":"true,last,false"},"default":"false","field":"snapshot"},{"type":"string","optional":false,"field":"db"},{"type":"string","optional":true,"field":"sequence"},{"type":"string","optional":false,"field":"schema"},{"type":"string","optional":false,"field":"table"},{"type":"string","optional":true,"field":"txId"},{"type":"string","optional":true,"field":"scn"},{"type":"string","optional":true,"field":"commit_scn"},{"type":"string","optional":true,"field":"lcr_position"}],"optional":false,"name":"io.debezium.connector.oracle.Source","field":"source"},{"type":"string","optional":false,"field":"op"},{"type":"int64","optional":true,"field":"ts_ms"},{"type":"struct","fields":[{"type":"string","optional":false,"field":"id"},{"type":"int64","optional":false,"field":"total_order"},{"type":"int64","optional":false,"field":"data_collection_order"}],"optional":true,"field":"transaction"}],"optional":false,"name":"server2.PDBUSER.TESTPDB.Envelope"},"payload":{"before":null,"after":{"ID":{"scale":0,"value":"Hg=="},"NAME":"CCC"},"source":{"version":"1.6.2.Final","connector":"oracle","name":"server2","ts_ms":1642961344000,"snapshot":"false","db":"ORCLPDB","sequence":null,"schema":"PDBUSER","table":"TESTPDB","txId":"09001800a1020000","scn":"3630790","commit_scn":"3635039","lcr_position":null},"op":"c","ts_ms":1642933132429,"transaction":null}}

报错：

未开启表的附加日志

{"name": "TEST_MORE","connector": {"state": "RUNNING","worker_id": "192.168.211.3:8083"},"tasks": [{"id": 0,"state": "FAILED","worker_id": "192.168.211.3:8083","trace": "org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.\r\n\tat io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)\r\n\tat io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:211)\r\n\tat io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:63)\r\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:159)\r\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:122)\r\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\r\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\r\n\tat java.lang.Thread.run(Thread.java:748)\r\nCaused by: io.debezium.DebeziumException:
*************************************************
Supplemental logging not configured for table ORCL.C##DBZUSER.TEST2.  Use command: ALTER TABLE C##DBZUSER.TEST2 ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS\r\n\tat
**************************************************
io.debezium.connector.oracle.logminer.LogMinerHelper.checkSupplementalLogging(LogMinerHelper.java:408)\r\n\tat io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:130)\r\n\t... 8 more\r\n"}],"type": "source"
}

未开启pdb会报可插拔数据库未打开错误

alter pluggable database orclpdb open;

debezium-connect-oracle使用相关推荐

Debezium 抽取oracle数据
1.环境介绍操作系统:centos 7.9 jdk版本:11.0.12 kafka版本:2.8.0 Debezium版本:1.6(debezium-connector-oracle-1.6.1.Fi ...
How to connect oracle databse
1. 下載客戶端Oracle Developer Tools for Visual Studio_32bit 安裝后通過配置tnsnames.ora指定連接 C:\app\user name\prod ...
sqlplus connect oracle
为什么80%的码农都做不了架构师?>>> lsnrctl status find the sid: lsnrctl service sqlplus apps/apps@&quo ...
基于debezium实时数据同步（Oracle为主）
基于debezium实时数据同步全部需要下载的内容链接 1.下载zookeeper-3.4.10 2.下载kafka_2.13-2.8.0 3.下载Kafka Connector:建议使用1.6以上 ...
基于Debezium 1.6和Oracle 11g 的 Debezium-Oracle实战
一.应用场景 1.事件触发数据库的触发器(trigger)能够一定程度上实现我们需要的事件触发,但通常需要配合定时任务才能实现. 2.数据同步典型的解决方法有 • 双写 • 变更数据抓取(chan ...
debezium集成Oralce攻略(上)
本期我们做一个debezium集成oralce的步骤,比较全,因为本人在这段时间内也踩了不少的坑,如果你有幸看到了这篇文章,就不需要绕弯路了,直接完成debezium监控oralce的全步骤,全部按照 ...
flink实时消费kafka中oracle的DML数据写入mysql
1.需要环境 zookeeper,小编安装环境为zookeeper-3.4.10 kakfa,小编安装环境为kafka_2.13-2.8.0 kafka-connect-oracle,此为kafka- ...
【硬刚大数据】大数据同步工具之FlinkCDC/Canal/Debezium对比
欢迎关注博客主页:微信搜:import_bigdata,大数据领域硬核原创作者_王知无(import_bigdata)_CSDN博客欢迎点赞.收藏.留言 ,欢迎留言交流! 本文由[王知无]原创,首发 ...
oracle proc 定义宿主,oracle proc 编程基础及最小化案例
oracle proc 编程是在我的概念中是非常陌生的,学习了一天下来发现这东西真的好古老,但是依然健壮.稳定,其中间是编辑一个 .pc 的文件使用 proc 进行预处理,预处理后生成一个真正的.c文 ...
python连接oracle批量写入_oracle大数据量python导入实践-1w/s
在项目中需要将一个80w+的csv数据(200+m)导入到oracle库,一开始使用的是Navicat for Oracle的导入工具.跑了五六分钟之后绝望了,因为才跑了2%,按这样的速度跑半天都跑不 ...

debezium-connect-oracle使用

测试环境：

1.创建用户赋权

2.创建测试表，对该表开启归档日志、附加日志

3.修改kafka配置文件

4.复制所需jar包（Oracle 必须为ojdbc8）

5.启动

6.查看连接器注册情况

7.注册连接器

8.查看连接器状态

9.查看topic

10.查看消费者消息

注意！！！

删除连接器

报错：

未开启pdb会报可插拔数据库未打开错误

debezium-connect-oracle使用相关推荐

最新文章

热门文章