hive explode函数入门及其与lateral view的结合

简单用法

举个例子

select collect_list(name) from t_order;

结果

+------------------------------------------------------------------------------------------------------+
|c0                                                                                                    |
+------------------------------------------------------------------------------------------------------+
|["saml","saml","tony","saml","tony","tony","saml","mart","saml","mart","mart","mart","neil","neil",""]|
+------------------------------------------------------------------------------------------------------+

select explode(collect_list(name)) from t_order;

+----+
|col |
+----+
|saml|
|saml|
|tony|
|saml|
|tony|
|tony|
|saml|
|mart|
|saml|
|mart|
|mart|
|mart|
|neil|
|neil|
|    |
+----+

select explode(collect_set(name)) from t_order;

+----+
|col |
+----+
|saml|
|tony|
|mart|
|neil|
|    |
+----+

与lateral view结合等的复杂用法

设某表有一个字段extinfo数据类似如下,类型为json,需要提取其中target_ids并展开

{"target_type":"4","target_keys":"20402","target_order":"31","target_ids":"[\"4320402595801\",\"4320402133801\",\"4320402919201\",\"4320402238501\"]"}

分布
首先提取对应json字段的值

select device_num,get_json_object(extinfo,'$.target_ids')

+----------+-----------------------------------------------------------------+
|device_num|c1                                                               |
+----------+-----------------------------------------------------------------+
|586344    |["4320402595801","4320402133801","4320402919201","4320402238501"]|
+----------+-----------------------------------------------------------------+

利用正则替换去掉多余的符号

select device_num,regexp_replace(get_json_object(extinfo, '$.target_ids'),'[\\[\\"\\]]','')

结果

+----------+-------------------------------------------------------+
|device_num|c1                                                     |
+----------+-------------------------------------------------------+
|586344    |4320402595801,4320402133801,4320402919201,4320402238501|
+----------+-------------------------------------------------------+

利用,进行拆分成数组

select device_num,split(regexp_replace(get_json_object(extinfo, '$.target_ids'),'[\\[\\"\\]]',''),",")

结果

+----------+-----------------------------------------------------------------+
|device_num|c1                                                               |
+----------+-----------------------------------------------------------------+
|586344    |["4320402595801","4320402133801","4320402919201","4320402238501"]|
+----------+-----------------------------------------------------------------+

数组就可以用explode结合lateral view炸开了.
注意explode的位置在from xx的后面,此外,表别名subview,炸开的列别名target_id

select device_num,target_id
from xx
lateral view explode(split(regexp_replace(get_json_object(extinfo, '$.target_ids'),'[\\[\\"\\]]',''),",")) subview as target_id

结果如下

+----------+-------------+
|device_num|target_id    |
+----------+-------------+
|586344    |4320402595801|
|586344    |4320402133801|
|586344    |4320402919201|
|586344    |4320402238501|
+----------+-------------+

总结

list,set可以使用explode炸开
如果炸开后还要和原来的表连接,需要搭配使用lateral view
explode结合lateral view需要给表和列起别名
split函数生成的是数组
使用explode需要先构建list或者set等等

hive explode函数入门及其与lateral view的结合相关推荐

mysql explode函数_hive中，lateral view 与 explode函数
hive中常规处理json数据,array类型json用get_json_object(#,"$.#")这个方法足够了,map类型复合型json就需要通过数据处理才能解析. exp ...
Hive自定义函数入门实例
概念 UDF函数一共分为三种类型: UDF-一进一出,给定一个参数,输出一个处理后的数据 UDAF-多进一出,属于聚合函数,类似于count.sum等函数 UDTF-一进多出,属于一个参数,返 ...
Hive date_format函数入门
date_format 的语法: date_format(date/timestamp/string ts, string fmt) 举几个例子 select date_format('2015-04 ...
Hive排名函数入门
row_number 第一种函数:row_number从1开始,按照顺序,生成分组内记录的序列,row_number()的值不会存在重复,当排序的值相同时,按照表中记录的顺序进行排列效果如下: 98 ...
Hive from_unixtime函数入门
目的转换时间戳为指定格式的时间案例两个参数,一个是时间戳,单位是秒! 不是ms,另一个是需要输出的格式 select from_unixtime(1567896035000,'HH'); 04 ...
hive时间函数入门
1)当前系统时间函数:current_date().current_timestamp().unix_timestamp() -- 函数1:current_date(); 当前系统日期格式:&quo ...
Hive之explode()函数和posexplode()函数和lateral view函数
目录 1.explode()函数 2.posexplode()函数 3.lateral view函数 1.explode()函数英文释义: explode() takes in an array ( ...
Hive Lateral View + explode 详解
hive中的函数分为3类,UDF函数.UDAF函数.UDTF函数 UDF:一进一出 UDAF:聚集函数,多进一出,类似于:count/max/min UDTF:一进多出,如explore().pose ...
Hive 连续多次 lateral view explode 踩坑
table1表结构: c1 c2arr c3arr c4arr select c1, c2arr_item, c3arr_item, c4arr_item from table1 lateral vi ...

hive explode函数入门及其与lateral view的结合

简单用法

与lateral view结合等的复杂用法

总结

hive explode函数入门及其与lateral view的结合相关推荐

最新文章

热门文章