kettle-java代码执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn‘t be found!

1.在java项目中执行hive相关的ktr:

 KettleEnvironment.init();TransMeta transMeta = new TransMeta("C:\\Users\\lixiang\\Desktop\\hivetorizhi.ktr");// 设置执行模式TransExecutionConfiguration config = new TransExecutionConfiguration();config.setExecutingClustered(true);config.setExecutingLocally(false);config.setExecutingRemotely(false);config.setClusterPosting(true);config.setClusterPreparing(true);config.setClusterStarting(true);TransSplitter transSplitter = Trans.executeClustered(transMeta, config);System.out.println(transSplitter.getCarteObjectMap());System.out.println(transSplitter.getMaster());System.out.println(transSplitter.getSlaves()[0]);System.out.println(transSplitter.getSlaves()[1].getStepNames()[0]);
}

2.报错：

Exception in thread "main" org.pentaho.di.core.exception.KettleXMLException:
错误从XML文件读取转换错误从XML文件读取转换Unable to load database connection info from XML nodeUnable to create new database interfacedatabase type with plugin id [HIVE2] couldn't be found!at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:3402)at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2758)at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2710)at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2687)at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2667)at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2632)at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2595)at com.kettle.KettleExamle.getRemoteServer(KettleExamle.java:55)at com.kettle.KettleExamle.main(KettleExamle.java:140)
Caused by: org.pentaho.di.core.exception.KettleXMLException:
错误从XML文件读取转换Unable to load database connection info from XML nodeUnable to create new database interfacedatabase type with plugin id [HIVE2] couldn't be found!at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:3384)... 8 more
Caused by: org.pentaho.di.core.exception.KettleXMLException:
Unable to load database connection info from XML nodeUnable to create new database interfacedatabase type with plugin id [HIVE2] couldn't be found!at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:1013)at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:2976)... 8 more
Caused by: org.pentaho.di.core.exception.KettleXMLException:
Unable to create new database interfacedatabase type with plugin id [HIVE2] couldn't be found!at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:977)... 9 more
Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
database type with plugin id [HIVE2] couldn't be found!at org.pentaho.di.core.database.DatabaseMeta.findDatabaseInterface(DatabaseMeta.java:584)at org.pentaho.di.core.database.DatabaseMeta.getDatabaseInterface(DatabaseMeta.java:558)at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:974)... 9 more

3.报错分析：kettle程序在读取database type的过程中读不到hive2这个类型。

查看kettle-core-7.1.0.0-12.jar中的kettle-database-types.xml文件，此文件定义了database type ,其中没hive2.。

<database-types><database-type id="AS/400">                                                        <description>AS/400</description>                                                <classname>org.pentaho.di.core.database.AS400DatabaseMeta</classname>            </database-type>                                                                   <database-type id="CACHE"><description>Intersystems Cache</description><classname>org.pentaho.di.core.database.CacheDatabaseMeta</classname></database-type>                                                       <database-type id="DB2"><description>IBM DB2</description><classname>org.pentaho.di.core.database.DB2DatabaseMeta</classname></database-type>                                                     <database-type id="DBASE"><description>dBase III, IV or 5</description><classname>org.pentaho.di.core.database.DbaseDatabaseMeta</classname></database-type>                                                       <database-type id="DERBY"><description>Apache Derby</description><classname>org.pentaho.di.core.database.DerbyDatabaseMeta</classname></database-type>                                                       <database-type id="EXASOL4"><description>Exasol 4</description><classname>org.pentaho.di.core.database.Exasol4DatabaseMeta</classname></database-type>                                                         <database-type id="EXTENDB"><description>ExtenDB</description><classname>org.pentaho.di.core.database.ExtenDBDatabaseMeta</classname></database-type>                                                         <database-type id="FIREBIRD"><description>Firebird SQL</description><classname>org.pentaho.di.core.database.FirebirdDatabaseMeta</classname></database-type>                                                          <database-type id="GENERIC"><description>Generic database</description><classname>org.pentaho.di.core.database.GenericDatabaseMeta</classname></database-type>                                                         <database-type id="GREENPLUM"><description>Greenplum</description><classname>org.pentaho.di.core.database.GreenplumDatabaseMeta</classname></database-type>                                                           <database-type id="INFINIDB"><description>Calpont InfiniDB</description><classname>org.pentaho.di.core.database.InfiniDbDatabaseMeta</classname></database-type><database-type id="SQLBASE"><description>Gupta SQL Base</description><classname>org.pentaho.di.core.database.GuptaDatabaseMeta</classname></database-type>                                                        <database-type id="H2"><description>H2</description><classname>org.pentaho.di.core.database.H2DatabaseMeta</classname></database-type>                                                    <database-type id="HYPERSONIC"><description>Hypersonic</description><classname>org.pentaho.di.core.database.HypersonicDatabaseMeta</classname></database-type>                                                            <database-type id="INFOBRIGHT"><description>Infobright</description><classname>org.pentaho.di.core.database.InfobrightDatabaseMeta</classname></database-type>                                                            <database-type id="INFORMIX"><description>Informix</description><classname>org.pentaho.di.core.database.InformixDatabaseMeta</classname></database-type>                                                          <database-type id="INGRES"><description>Ingres</description><classname>org.pentaho.di.core.database.IngresDatabaseMeta</classname></database-type>                                                        <database-type id="VECTORWISE"><description>Ingres VectorWise</description><classname>org.pentaho.di.core.database.VectorWiseDatabaseMeta</classname></database-type>                                                        <database-type id="INTERBASE"><description>Borland Interbase</description><classname>org.pentaho.di.core.database.InterbaseDatabaseMeta</classname></database-type>                                                           <database-type id="KINGBASEES"><description>KingbaseES</description><classname>org.pentaho.di.core.database.KingbaseESDatabaseMeta</classname></database-type>                                                            <database-type id="LucidDB"><description>LucidDB</description><classname>org.pentaho.di.core.database.LucidDBDatabaseMeta</classname></database-type>                                                         <database-type id="MARIADB"><description>MariaDB</description><classname>org.pentaho.di.core.database.MariaDBDatabaseMeta</classname></database-type><database-type id="MONETDB"><description>MonetDB</description><classname>org.pentaho.di.core.database.MonetDBDatabaseMeta</classname></database-type>                                                         <database-type id="MSACCESS"><description>MS Access</description><classname>org.pentaho.di.core.database.MSAccessDatabaseMeta</classname></database-type>                                                          <database-type id="MSSQLNATIVE"><description>MS SQL Server (Native)</description><classname>org.pentaho.di.core.database.MSSQLServerNativeDatabaseMeta</classname></database-type>                                                             <database-type id="MSSQL"><description>MS SQL Server</description><classname>org.pentaho.di.core.database.MSSQLServerDatabaseMeta</classname></database-type>                                                             <database-type id="MYSQL"><description>MySQL</description><classname>org.pentaho.di.core.database.MySQLDatabaseMeta</classname></database-type>                                                       <database-type id="MONDRIAN"><description>Native Mondrian</description><classname>org.pentaho.di.core.database.MondrianNativeDatabaseMeta</classname></database-type>                                                       <database-type id="NEOVIEW"><description>Neoview</description><classname>org.pentaho.di.core.database.NeoviewDatabaseMeta</classname></database-type>                                                         <database-type id="NETEZZA"><description>Netezza</description><classname>org.pentaho.di.core.database.NetezzaDatabaseMeta</classname></database-type>                                                         <database-type id="ORACLE"><description>Oracle</description><classname>org.pentaho.di.core.database.OracleDatabaseMeta</classname></database-type>                                                        <database-type id="ORACLERDB"><description>Oracle RDB</description><classname>org.pentaho.di.core.database.OracleRDBDatabaseMeta</classname></database-type>                                                           <database-type id="POSTGRESQL"><description>PostgreSQL</description><classname>org.pentaho.di.core.database.PostgreSQLDatabaseMeta</classname></database-type><database-type id="REDSHIFT"><description>Redshift</description><classname>org.pentaho.di.core.database.RedshiftDatabaseMeta</classname></database-type><database-type id="REMEDY-AR-SYSTEM"><description>Remedy Action Request System</description><classname>org.pentaho.di.core.database.RemedyActionRequestSystemDatabaseMeta</classname>                                                                                           </database-type>                                                                          <database-type id="SAPDB"><description>MaxDB (SAP DB)</description><classname>org.pentaho.di.core.database.SAPDBDatabaseMeta</classname></database-type>                                                       <database-type id="SQLITE"><description>SQLite</description><classname>org.pentaho.di.core.database.SQLiteDatabaseMeta</classname></database-type><database-type id="SYBASE"><description>Sybase</description><classname>org.pentaho.di.core.database.SybaseDatabaseMeta</classname></database-type><database-type id="SYBASEIQ"><description>SybaseIQ</description><classname>org.pentaho.di.core.database.SybaseIQDatabaseMeta</classname></database-type><database-type id="TERADATA"><description>Teradata</description><classname>org.pentaho.di.core.database.TeradataDatabaseMeta</classname></database-type><database-type id="UNIVERSE"><description>UniVerse database</description><classname>org.pentaho.di.core.database.UniVerseDatabaseMeta</classname></database-type><database-type id="VERTICA"><description>Vertica</description><classname>org.pentaho.di.core.database.VerticaDatabaseMeta</classname></database-type><database-type id="VERTICA5"><description>Vertica 5+</description><classname>org.pentaho.di.core.database.Vertica5DatabaseMeta</classname></database-type></database-types>

4.报错解决：

hive的api的连接方式是通过jdbc连接的。因此kettle连接hive的思路与连接postgresql的思路相同。因此，仿照postgresql的连接方法自定义一个database-type为hive2。第一步，在kettle-core-7.1.0.0-12.jar的kettle-database-types.xml文件中添加下面内容

   <database-type id="HIVE2"><description>HIVE2</description><classname>org.pentaho.di.core.database.Hive2SQLDatabaseMeta</classname></database-type>

注意：其中会有对应的实现类org.pentaho.di.core.database.Hive2SQLDatabaseMeta，而这个实现类是不存在的，需要自己手动创建，如何创建呢？请看下一步。

第二步：仿照postgresql的实现类org.pentaho.di.core.database.PostgreSQLDatabaseMeta来写即可，只需将连接的url以及jdbc driver名称更换为hive相关。具体操作：PostgreSQLDatabaseMeta复制一份重命名为Hive2SQLDatabaseMeta，修改其中的其中的getDriverClass 方法的返回值为hive的jdbc Driver和getDriverClass方法的返回值为hive的url。

将org.pentaho.di.core.database.Hive2SQLDatabaseMeta类编译为.class文件（kettle的源码github上面有，自行下载，下载完成以后就可以进行第2步的复制类修改类然后反编译成class文件）放进kettle-core-7.1.0.0-12.jar的org\pentaho\di\core\database路径下

6.将修改后kettle-core-7.1.0.0-12.jar放入执行项目中，执行ktr：

执行成功：结果：

{/hivetorizhi (master)=1cc5f5bb-b0c4-4938-96f7-85363f88564d, /hivetorizhi (local:slave1-8081)=09ea3633-f5bd-4e43-9352-62b958255469, /hivetorizhi (local:slave2-8082)=35e00dc2-f570-4f26-8540-156fb84d9524}
/hivetorizhi (master)
/hivetorizhi (local:slave2-8082)
写日志

over

附加：

适配hive后的kettle-core-7.1.0.0-12.jar 下载链接：https://download.csdn.net/download/m0_37618809/11081777

kettle-java代码执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn‘t be found!相关推荐

java代码解压rar文件时报错，badRarArchive
针对java代码解压rar文件时报错浅谈 1.针对所报的异常: de.innosystec.unrar.Archive setFile 警告: exception in archive constru ...
HIVE的安装配置、mysql的安装、hive创建表、创建分区、修改表等内容、hive beeline使用、HIVE的四种数据导入方式、使用Java代码执行hive的sql命令
1.上传tar包这里我上传的是apache-hive-1.2.1-bin.tar.gz 2.解压 mkdir -p /home/tuzq/software/hive/ tar -zxvf apach ...
java栈属于哪个代,Java 代码执行原理
原标题:Java 代码执行原理西安鸥鹏it教育将在本篇文章中,将重点研究java源代码的执行原理,即从程序员编写JAVA源代码,到最终形成产品,在整个过程中,都经历了什么?每一步又是怎么执行的?执行 ...
java build path entries 为空_TOOLFK工具-在线JAVA代码执行工具
本文要推荐的[TOOLFK]在线JAVA代码执行工具 ,提供JAVA代码在线执行功能. 網站名稱:ToolFk 網站鏈結:https://www.toolfk.com/ 工具链接:https://ww ...
java解析shell命令_Android中执行java命令的方法及java代码执行并解析shell命令
这篇文章给大家介绍Android中执行java命令的方法及java代码执行并解析shell命令,需要的朋友一起学习 android中执行java命令的方法大家都晓得吗,下面一段内容给大家带来了具体解析 ...
java 调用wget_通过命令行执行WGET下载会更快，而通过Java代码执行时会更慢
我正在使用WGET通过java代码下载文件,这需要大约10分钟才能下载20 MB文件.但是通过命令行执行wget下载,同样的文件以10MbPs的速度在7秒内下载.有人知道为什么吗?我该如何改进我的Ja ...
hive跑mr时报错，java.lang.IllegalArgumentException: The value of property yarn.resourcemanager.zk-address
E1:hive跑mr时报错,java.lang.IllegalArgumentException: The value of property yarn.resourcemanager.zk-addr ...
Java代码执行顺序
Java代码执行顺序 1. Java初步认知 2. Javayun.java例子 3. 反编译Javayun.class文件 4. 分析Javayun_dxdump文件 5. 再来一个网上的例子Jav ...
linux怎么卸载webpack,安装webpack后，执行webpack -v命令时报错：SyntaxError: Block-sc
安装webpack后,执行webpack -v命令时报错如下: [root@FreeServer ~]# webpack -v /usr/local/node-v4.4.7-linux-x64/lib ...
java数据库防火墙,数据库centos7防火墙导致java程序访问mongodb3.0.1时报错的问题分析...
环境描述: 数据库:mongodb3.0.1 数据库系统:centos7,(虚拟机,最小安装) 数据库驱动:mongo-Java-driver-3.0.0.jar 问题描述:shell环境下用mong ...

kettle-java代码执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn‘t be found!

kettle-java代码执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn‘t be found!相关推荐

最新文章

热门文章