在日志分析大作业中, 有要求把hive数据导入mysql的,

hive数据文件如下:

[root@cdh-1 sqoop]# hadoop fs -ls -R /user/hive/warehouse/sogouquery/

Found 1 items

drwxr-xr-x - root supergroup 0 2016-07-17 09:27 /user/hive/warehouse/sogouquery/year=2016

[root@cdh-1 sqoop]# hadoop fs -ls -R /user/hive/warehouse/sogouquery/

drwxr-xr-x - root supergroup 0 2016-07-17 09:27 /user/hive/warehouse/sogouquery/year=2016

drwxr-xr-x - root supergroup 0 2016-07-17 09:28 /user/hive/warehouse/sogouquery/year=2016/month=7

drwxr-xr-x - root supergroup 0 2016-07-17 09:27 /user/hive/warehouse/sogouquery/year=2016/month=7/day=1

-rwxr-xr-x 1 root supergroup 93273 2016-07-17 09:20 /user/hive/warehouse/sogouquery/year=2016/month=7/day=1/kafkadate0.1468772436053.Done

drwxr-xr-x - root supergroup 0 2016-07-17 09:28 /user/hive/warehouse/sogouquery/year=2016/month=7/day=2

-rwxr-xr-x 1 root supergroup 93273 2016-07-17 09:20 /user/hive/warehouse/sogouquery/year=2016/month=7/day=2/kafkadate1.1468772440741.Done

drwxr-xr-x - root supergroup 0 2016-07-17 09:28 /user/hive/warehouse/sogouquery/year=2016/month=7/day=3

-rwxr-xr-x 1 root supergroup 93273 2016-07-17 09:20 /user/hive/warehouse/sogouquery/year=2016/month=7/day=3/kafkadate0.1468772445986.Done

Done文件信息如下:

00:00:56        7098052750676105        [9aiyy] 5       2       www.9aiyy.com/html/pic/yz/25460/

00:00:56        2512392400865138        [拳皇2003人物出招表]    1       23      zhidao.baidu.com/question/42150698.html?fr=qrl3

00:00:56        4691862369525524        [朝鲜能不能打败韩国]    2       1       club.cqvip.com/topic.asp?k=13,10,0,194069

00:00:56        07276358834409019       [夜晚文化晚会主持稿]    30      37      www.99admin.com/article/2006-12-04/24727.html

......

现在有一个疑问， 就是在mysql创建对应表时, mysql表字段中要不要包含hive分区表year, month,day字段, 即用

CREATE TABLE `sogouquery`

(`time` varchar(10),

`userid` varchar(30),

`query` varchar(2000),

`pagerank` int,

`clickrank` int,

`site` varchar(2000),

`year` int,

`month` int,

`day` int

)

还是

CREATE TABLE `sogouquery`

(`time` varchar(8),

`userid` varchar(30),

`query` varchar(2000),

`pagerank` int,

`clickrank` int,

`site` varchar(2000)

),

用sqoop命令 bin/sqoop export --connect "jdbc:mysql://cdh-2:3306/test" --username root --password 123456 --table sogouquery -m 1 --export-dir '/user/hive/warehouse/sogouquery/year=2016/month=7/day=3/kafkadate1.1468746179357.Done' --input-fields-terminated-by '\t'  测试过

如果mysql表不包含year,month,day三个字段可以把从数据hive导入mysql，但是这样就无法实现按年月日查询，因为根本就没这三个日期字段.

如果包含year,month,day三个字段从hive导入mysql时报" Can't export data, please check failed map task logs",  不知道大家是怎样解决的?