Spark 创建 hive表报错 ROW FORMAT DELIMITED is only compatible with 'textfile', not 'parquet'

场景：

在spark分析数据时，创建hive表失败。

提示异常：

19/10/09 10:59:18 INFO execution.SparkSqlParser: Parsing command: CREATE EXTERNAL TABLE IF NOT EXISTS
dm_xxx.user_area(biFollowersCount String,city String,created_at String,description String,experience String,followers_count String,friends_count String,name String,profileImageUrl String,province String,statuses_count String,uid String,url String,verified String,verified_reason String,verified_type String,verified_type_ext String)
partitioned by (pt_province string,pt_city string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\001'
STORED AS PARQUET TBLPROPERTIES('parquet.compression'='SNAPPY')
Exception in thread "main" org.apache.spark.sql.catalyst.parser.ParseException:
Operation not allowed: ROW FORMAT DELIMITED is only compatible with 'textfile', not 'parquet'(line 1, pos 0)== SQL ==
CREATE EXTERNAL TABLE IF NOT EXISTS
^^^
dm_xxx.user_area(biFollowersCount String,city String,created_at String,description String,experience String,followers_count String,friends_count String,name String,profileImageUrl String,province String,statuses_count String,uid String,url String,verified String,verified_reason String,verified_type String,verified_type_ext String)
partitioned by (pt_province string,pt_city string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\001'
STORED AS PARQUET TBLPROPERTIES('parquet.compression'='SNAPPY')at org.apache.spark.sql.catalyst.parser.ParserUtils$.operationNotAllowed(ParserUtils.scala:39)at org.apache.spark.sql.execution.SparkSqlAstBuilder.org$apache$spark$sql$execution$SparkSqlAstBuilder$$validateRowFormatFileFormat(SparkSqlParser.scala:1320)at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitCreateHiveTable$1.apply(SparkSqlParser.scala:1089)at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitCreateHiveTable$1.apply(SparkSqlParser.scala:1064)at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:99)at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitCreateHiveTable(SparkSqlParser.scala:1064)at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitCreateHiveTable(SparkSqlParser.scala:55)at org.apache.spark.sql.catalyst.parser.SqlBaseParser$CreateHiveTableContext.accept(SqlBaseParser.java:1121)at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:42)at org.apache.spark.sql.catalyst.parser.AstBuilder$$anonfun$visitSingleStatement$1.apply(AstBuilder.scala:71)at org.apache.spark.sql.catalyst.parser.AstBuilder$$anonfun$visitSingleStatement$1.apply(AstBuilder.scala:71)at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:99)at org.apache.spark.sql.catalyst.parser.AstBuilder.visitSingleStatement(AstBuilder.scala:70)at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:69)at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:68)at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:97)at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:48)at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:68)at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:632)at com.izhonghong.data.filter.weibo.WeiBoAccountArea$.main(WeiBoAccountArea.scala:93)at com.izhonghong.data.filter.weibo.WeiBoAccountArea.main(WeiBoAccountArea.scala)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

原因：

spark 不为 parquet文件提供DELIMITERS。

参考：

https://issues.apache.org/jira/browse/SPARK-15279

Spark 创建 hive表报错 ROW FORMAT DELIMITED is only compatible with 'textfile', not 'parquet'相关推荐

Spark创建hive表报错 ROW FORMAT DELIMITED is only compatible with ‘textfile‘, not ‘orc‘
创建hive表失败: 报错异常信息: Operation not allowed: ROW FORMAT DELIMITED is only compatible with 'textfile', n ...
row format delimited fields terminated by “,“含义是以‘，‘结尾的行格式分隔字段
row format delimited fields terminated by ","含义是以','结尾的行格式分隔字段欢迎使用Markdown编辑器 hive中 creat ...
为什么要加上ROW FORMAT DELIMITED NULL DEFINED AS ‘‘；？
create table if not exists db_app.app_pageview_rate(daystr string,step string,numb int,rate double,r ...
SAP RETAIL WA01创建分配表报错- No allocation rule allowed for allocation strategies or variants-
SAP RETAIL WA01创建分配表报错- No allocation rule allowed for allocation strategies or variants- 1,执行事务代码WA ...
SAP RETAIL WA01 创建分配表报错 - Plant 0000000039 Confirmation date not maintained.-
SAP RETAIL WA01 创建分配表报错 - Plant 0000000039 Confirmation date not maintained.- 在SAP零售系统里,试图执行事务代码WA01 ...
mysql 1005 - can't create table_关于创建数据表报错一例(ERROR 1005 Can’t create table (errno: 121))...
问题描述曾遇到创建数据表报错问题,报错如下: ERROR 1005 (HY000) at line 18: Can't create table 'db1.t2' (errno: 121) 通过日志 ...
row format delimited fields terminated by ','
row format delimited fields terminated by ',' 以','结尾的行格式分隔字段
sparksql查询hive表报错：codec com.hadoop.compression.lzo.LzoCodec not found 问题探究
1.问题描述在某次测试时,我想用复用已有的表结构,所以拷贝了原来表的部分建表语句重新建了一个表,通过命令行往这个表添加数据正常,但是通过sparksql从该表查询数据时报错: Exception i ...
MySql创建临时表报错 Statement violates GTID consistency
MySql 在创建临时表时候报错 :Statement violates GTID consistency 一.问题重现 drop table if exists temp_01; create ta ...

Spark 创建 hive表报错 ROW FORMAT DELIMITED is only compatible with 'textfile', not 'parquet'

场景：

提示异常：

原因：

参考：

Spark 创建 hive表报错 ROW FORMAT DELIMITED is only compatible with 'textfile', not 'parquet'相关推荐

最新文章

热门文章

Spark 创建 hive表报错 ROW FORMAT DELIMITED is only compatible with 'textfile', not 'parquet'

场景：

提示异常 ：

原因：

参考：

Spark 创建 hive表报错 ROW FORMAT DELIMITED is only compatible with 'textfile', not 'parquet'相关推荐

最新文章

热门文章

提示异常：