Can‘t get JDBC type for struct＜date:int,day:int,hours:int,minutes:int,month:int,seconds:int,time:big

使用Spark将DataFrame类型数据保存到Oracle的过程中报错:

Exception in thread "main" java.lang.IllegalArgumentException: Can't get JDBC type for struct <data:int, day:int, hours:int, minutes:int, month:int, seconds:int, time:bigint, timeZoneOffset:int, year:int>

Oracle表数据结构是这样的:

pd_id-->varchar
ywxtmc-->varchar
sqljb-->varchar
rzlx-->varchar
cjbbh-->varchar
rzsj-->date
rksj-->date

我一开始创建的pojo类是这样的:

rzsj和rksj都是string类型，导致保存失败，报错。

然后将string类型改成了date类型就报了标题的错误:

java.lang.IllegalArgumentException: Can't get JDBC type for struct<date:int,day:int,hours:int,minutes:int,month:int,seconds:int,time:bigint,timezoneOffset:int,year:int>

解决方法：
思路：时间字段以timestamp格式进行保存

1、首先依旧采用string类型

2、获取到dataFrame之后，使用sparkSQL内置函数转换成timestamp格式

RDD<String> stringRDD = sparkContext.textFile("C:\\Users\\yanni\\Documents\\WeChat Files\\yanning1994-4\\FileStorage\\File\\2020-07\\catalina.out", 5);
JavaRDD<String> stringJavaRDD = stringRDD.toJavaRDD();JavaRDD<MGOutput> rowRDD = stringJavaRDD .map(line -> {String parts[] = line.split("&##&");MGOutput stu = new MGOutput();stu.setPd_id(parts[0]);stu.setSqljb(parts[1]);stu.setRzsj(parts[2]);stu.setYwxtmc("业务系统");stu.setRzlx("采集类型");stu.setCjbbh("版本号");SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");String format = df.format(new Date());stu.setRksj(format);return stu;
});//完整数据的DataFrame
Dataset<Row> dataFrame = sparkSession.createDataFrame(rowRDD, MGOutput.class);
dataFrame.createTempView("rowDataset");
Dataset<Row> result = sparkSession.sql("select pd_id ,to_timestamp(rzsj) as rzsj,to_timestamp(rzsj) as rzsj from rowDataset");//保存数据
result.write().mode("overwrite").format("jdbc").option("url","jdbc:oracle:thin:@//192.168.1.xxx:11521/xxx").option("user","xxx").option("password","xxx").option("dbtable","xxx").option("batchsize","10").save();

3、得到新的dataFrame即可成功保存。

Can‘t get JDBC type for struct＜date:int,day:int,hours:int,minutes:int,month:int,seconds:int,time:big相关推荐

C: error: incompatible types when initializing type ‘int’ using type ‘atomic_t’ {aka ‘struct ＜anonym
typedef.c:17:10: error: incompatible types when initializing type 'int' using type 'atomic_t' {aka ' ...
毕设问题小记——No Dialect mapping for JDBC type:-1错误
为了实现在用户误操作后能够对数据进行修正,打算添加一个数据库管理功能,主要实现对数据表的查询和执行sql语句.费了半天功夫完成以后查询有的表数据正常显示,但是个别表出现如下错误: No Dialect ...
error: incompatible types when assigning to type ‘const struct wiphy_wowlan_support‘
环境说明系统:Centos7 (3.10.0) 驱动版本:00014996-RTL8192EU_linux_v5.6.3.1_34030_COEX20171113-0047.20190613.zip ...
Go 接口实现原理【高阶篇】： type _interface struct
Go 接口实现原理[高阶篇]: type _interface struct The Internal Definition Of Interface Types https://www.tapirg ...
JDBC和Ibatis中的Date,Time,Timestamp处理
JDBC和Ibatis中的Date,Time,Timestamp处理 November 25th, 2010西坪 Leave a commentGo to comments 在此前,遇到过使用Ibat ...
解决pandas（Python）的报错：unsupported operand type(s) for -: ‘datetime.date’ and ‘Timestamp’
在使用Pandas包的时候,遇到时间加减出现的报错 'datetime.date'是datetime的一种时间格式: 'Timestamp'(注意是大写的字母)是Pandas的一种时间格式. 这两个虽 ...
AnalysisException: cannot resolve ‘`Magnitude Type`‘ given input columns: [Date, Day, Depth, Latitud
Ubuntu下Jupyter报错:AnalysisException: cannot resolve '`Magnitude Type`' given input columns: [Date, Da ...
maven ＜type＞pom＜/type＞＜scope＞import＜/scope＞
我们知道Maven的继承和Java的继承一样,只能单继承,无法实现多继承,你是否想过我们如果要继承多个父模块的时候应该怎么做呢?或许你会想只往一个父模块中添加jar包依赖,只继承一个父模块就可以了,但 ...
oracle 方言报错,ORACLE11g：No Dialect mapping for JDBC type: -9解决方案详解
解决方案由于hibernate4.2.0和Oracle10g数据库结合使用自定义方言时,需要和老版本做区别,原来的Oracle10gDialect中没有注册NVARCHAR和NCLOB数据类型, 此 ...

Can‘t get JDBC type for struct＜date:int,day:int,hours:int,minutes:int,month:int,seconds:int,time:big

Can‘t get JDBC type for struct＜date:int,day:int,hours:int,minutes:int,month:int,seconds:int,time:big相关推荐

最新文章

热门文章