Cannot find class ‘org.apache.hudi.hadoop.HoodieParquetInputFormat‘

使用hudi报错, 原因需要把hudi的jar包放到hive的auxlib目录下.

/usr/local/hive/auxlib

Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Cannot find class 'org.apache.hudi.hadoop.HoodieParquetInputFormat'at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:231)at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:217)at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:254)at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:480)... 50 more
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Cannot find class 'org.apache.hudi.hadoop.HoodieParquetInputFormat'at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:400)at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:187)at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:271)at org.apache.hive.service.cli.operation.Operation.run(Operation.java:337)at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:439)at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:416)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:422)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)at com.sun.proxy.$Proxy22.executeStatementAsync(Unknown Source)at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:282)at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:501)at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)

Cannot find class ‘org.apache.hudi.hadoop.HoodieParquetInputFormat‘相关推荐

hive查询hudi表使用count报java.lang.ClassNotFoundException: org.apache.hudi.hadoop.HoodieParquetInputFormat
问题描述: hive 查询hudi 报错, 已将hudi 对应包hudi-hadoop-mr-bundle-0.9.0.jar放入lib目录下并重启hive select count(1) from ...
bigint hive java类型_详解Apache Hudi如何配置各种类型分区
1. 引入 Apache Hudi支持多种分区方式数据集,如多级分区.单分区.时间日期分区.无分区数据集等,用户可根据实际需求选择合适的分区方式,下面来详细了解Hudi如何配置何种类型分区. 2. 分 ...
Apache+Hudi入门指南: Spark+Hudi+Hive+Presto
一.整合 hive集成hudi方法:将hudi jar复制到hive lib下 cp ./packaging/hudi-hadoop-mr-bundle/target/hudi-hadoop-mr-b ...
Apache Hudi - 2 - 基础功能特性实践
前言本文对Hudi官网提到的部分特性(功能)做了测试,具体的测试数据均由以下代码直接生成: from faker import Fakerdef fake_data(faker: Faker, ...
使用Apache Hudi + Amazon S3 + Amazon EMR + AWS DMS构建数据湖
1. 引入数据湖使组织能够在更短的时间内利用多个源的数据,而不同角色用户可以以不同的方式协作和分析数据,从而实现更好.更快的决策.Amazon Simple Storage Service(amaz ...
Apache Hudi 0.7.0 和 0.8.0 新功能已在 Amazon EMR 中可用
文末限时福利倒计时3天,不要错过! 前言 Apache Hudi 是一个开源事务性数据湖框架,通过提供记录级插入.更新和删除功能,极大地简化了增量数据处理和数据管道开发.如果您要在 Amazon Si ...
大数据Hadoop之——新一代流式数据湖平台 Apache Hudi
文章目录一.概述二.Hudi 架构三.Hudi的表格式 1)Copy on Write(写时复制) 2)Merge On Read(读时合并) 3)COW vs MOR 四.元数据表(Metad ...
技术干货｜基于Apache Hudi 的CDC数据入湖
简介:阿里云技术专家李少锋(风泽)在Apache Hudi 与 Apache Pulsar 联合 Meetup 杭州站上的演讲整理稿件,本议题将介绍典型 CDC 入湖场景,以及如何使用 Pulsar/ ...
技术干货｜基于Apache Hudi 的CDC数据入湖「内附干货PPT下载渠道」
简介: 阿里云技术专家李少锋(风泽)在Apache Hudi 与 Apache Pulsar 联合 Meetup 杭州站上的演讲整理稿件,本议题将介绍典型 CDC 入湖场景,以及如何使用 Pulsar ...

Cannot find class ‘org.apache.hudi.hadoop.HoodieParquetInputFormat‘

Cannot find class ‘org.apache.hudi.hadoop.HoodieParquetInputFormat‘相关推荐

最新文章

热门文章