用户订单指标业务开发
2024-05-10 02:25:44
文章目录
- 3.1 需求分析
- 3.2 创建dw层表
- 3.3 订单宽表ETL处理
- 1、加载用户维度数据
- 2、导入订单数据
- 3、时间近30天、90天、180天、订单上午、下午时间拉宽
- 4、与地址表合并加入收货地址信息
- 3.4 指标开发
- 1 指标开发 一
- 2 指标开发二
- 3 指标开发三
- 4 指标开发四
- 5 指标开发五
- 6 指标开发六
- 7 创建ads层表、加载数据
- 3.5 退货表指标统计
- ads层开发
- 常见错误
3.1 需求分析
电商平台往往需要根据用户的购买数据来分析用户的行为，此处。我们基于用户的订单情况进行一些统计分析，用于将来的用户行为分析。根据用户的消费行为习惯，对运营部门提供用户分析数据指标。表是订单表！！
- 以下为本需求需要统计的基于用户的订单指标：
- 以下为基于用户的退货指标：
订单状态 - 1)等待支付
如果您选择“网上支付”和“银行转账”这两种付款方式，在暂未收到您款项时，订单会显示“等待支付”，建议您在订单保留期限内及时付款。 - 2)等待预售商品到货
若您订购的商品为预售商品，商品到货前订单会显示“等待预售商品到货”。 - 3)正在配货
此状态说明您的订单正在库房配货。 - 4)等待移仓
若您订单中的商品当地库房缺货，我们将从北京仓库调货至当地仓库，此时订单显示“等待移仓”。 - 5)正在移仓
若您订单中的商品正在从北京仓库发往当地仓库，订单会显示“正在移仓”。 - 6)已配货
此状态说明您的订单已完成配货，正在等待发货。 - 7)已发货
若您的订单已从库房发出，正在配送途中，订单会显示“已发货”。 - 8)已送达
1）若您已收到商品并在“我的订单”中进行了“收货反馈”，订单会显示“已送达”。
2）若您未进行“收货反馈”操作，系统在发货后的20天默认变为“已送达”。
3） 订单状态为“已发货”，但订单中的物流配送信息显示“配送成功”时，系统会默认将您的订单状态显示为“已送达”。
4）国内平邮订单，如果您未进行“确认收货”操作，系统会在发货25天后默认您收到商品，订单状态显示为“已送达”。
5）海外订单，如果您未进行“确认收货”操作，系统会在发货60天后默认您收到商品，订单状态显示为“已送达”。 - 9)交易成功
若您的订单状态为“已送达”，且此状态后的15天内未发生退货，系统将默认变为“交易成功”。 - 10)交易未成功（拒收）——reject
若订单未送达、送达后未签收或签收后办理了退货，订单状态都会默认显示“交易未成功”。 - 11)取消（退货）——return
若您订单中的商品缺货，或您的订单超出了订单保留期限，或您将订单进行了取消操作，订单都将显示“取消”状态。
3.2 创建dw层表
1、创建itcast_dw.dim_user表
drop table if exists `itcast_dw`.`dim_user`;create table `itcast_dw`.`dim_user`(userId bigint,loginName string,userSex bigint
)
partitioned by (dt string)
STORED AS PARQUET;
2、创建订单临时表tmp_order_wide
在dw层创建 订单临时表tmp_order_wide
-- 订单临时订单表
drop table if exists `itcast_dw`.`tmp_order_wide`;create table `itcast_dw`.`tmp_order_wide`(orderId bigint,orderStatus bigint,payType bigint,userId bigint,userAddressId bigint,payTime string,payMoney double,createtime string
)
partitioned by(dt string)
STORED AS PARQUET;
3、创建订单时间标志宽表tmp_order_datetag_wide
drop table `itcast_dw`.`tmp_order_datetag_wide`;create table `itcast_dw`.`tmp_order_datetag_wide`(orderId bigint,orderStatus bigint,payType bigint,userId bigint,userAddressId bigint,payTime string,payMoney double,createtime string,flag30 bigint,flag60 bigint,flag90 bigint,flag180 bigint,flagTimeBucket string
)
partitioned by(dt string)
STORED AS PARQUET;
4、创建订单时间标志、地址标志宽表 fact_order_wide
-- 地址拉宽
drop table `itcast_dw`.`fact_order_wide`;create table `itcast_dw`.`fact_order_wide`(orderId bigint,orderStatus bigint,payType bigint,userId bigint,userAddressId bigint,payTime string,payMoney double,createtime string,flag30 bigint,flag60 bigint,flag90 bigint,flag180 bigint,flagTimeBucket string,othername string
)
partitioned by(dt string)
STORED AS PARQUET;
3.3 订单宽表ETL处理
1、加载用户维度数据
insert overwrite table `itcast_dw`.`dim_user` partition(dt='20190909')
select userId, loginName,userSex
from`itcast_ods`.`itcast_users` ;--验证
select * from itcast_dw.dim_user limit 10;
2、导入订单数据
insert overwrite table `itcast_dw`.`tmp_order_wide` partition (dt='20190909')
selectorderid,orderstatus,paytype,userid,useraddressid,paytime,totalmoney,createtime
from `itcast_ods`.`itcast_orders` where dt='20190909' ;-- 测试
select * from `itcast_dw`.`tmp_order_wide` limit 10;
3、时间近30天、90天、180天、订单上午、下午时间拉宽
insert overwrite table `itcast_dw`.`tmp_order_datetag_wide` partition(dt='20190909')
selectorderId,orderStatus,payType,userId,userAddressId,payTime,paymoney,createtime,case when datediff(current_timestamp, createtime) <= 30then 1else 0end as flag_30,case when datediff(current_timestamp, createtime) <= 60then 1else 0end as flag_60,case when datediff(current_timestamp, createtime) <= 90then 1else 0end as flag_90, case when datediff(current_timestamp, createtime) <= 180then 1else 0end as flag_180,case when hour(createtime) >= 0 and hour(createtime) < 6then '凌晨'when hour(createtime) >= 6 and hour(createtime) < 12then '上午'when hour(createtime) >= 12 and hour(createtime) < 14then '中午'when hour(createtime) >= 14 and hour(createtime) < 18then '下午'else '晚上'end as flag_time_bucket
from `itcast_dw`.`tmp_order_wide`
where dt='20190909';-- 测试语句
select * from `itcast_dw`.`tmp_order_datetag_wide` limit 5;
4、与地址表合并加入收货地址信息
--创建dw层dim_user_address表
drop table if exists `itcast_dw`.`dim_user_address`;
create table `itcast_dw`.`dim_user_address`(addressId bigint,userId bigint,userName string,otherName string,userPhone string,areaIdPath string,areaId bigint,userAddress string,isDefault bigint,dataFlag bigint,createTime string
)
partitioned by (dt string)
STORED AS PARQUET;
--从ods层itcast_user_address导出数据到dim_user_address表
insert overwrite table `itcast_dw`.`dim_user_address` partition(dt="20190909")
select
addressId,
userId,
userName,
otherName,
userPhone,
areaIdPath,
areaId,
userAddress,
isDefault,
dataFlag,
createTime
from `itcast_ods`.`itcast_user_address` where dt="20190909";
--地址表合并加入收货地址信息
insert overwrite table `itcast_dw`.`fact_order_wide` partition(dt='20190909')
selectt1.orderId,t1.orderStatus,t1.payType,t1.userId,t1.userAddressId,t1.payTime,t1.paymoney,t1.createtime,t1.flag30,t1.flag60,t1.flag90,t1.flag180,t1.flagTimeBucket,t2.othername
from(select * from `itcast_dw`.`tmp_order_datetag_wide` where dt='20190909') t1left join(select * from `itcast_dw`.`dim_user_address` where dt='20190909') t2on t1.userAddressId = t2.addressId;
-- 测试
select * from `itcast_dw`.`fact_order_wide` limit 10;
3.4 指标开发
1 指标开发 一
参考代码：
selectt1.userid,t1.loginname,MIN(t2.payTime) as first_paytime, --首次下单时间MAX(t2.payTime) as lastest_paytime, --尾单时间DATEDIFF(CURRENT_TIMESTAMP, MIN(t2.payTime)) as first_day_during_days,--首单距今DATEDIFF(CURRENT_TIMESTAMP, MAX(t2.payTime)) as lastest_day_durning_days, --尾单距今MIN(t2.paymoney) as min_paymoney,MAX(t2.paymoney) as max_paymoney
from(select * from `itcast_dw`.`fact_order_wide` where dt='20190909') as t2left join(select * from `itcast_dw`.`dim_user` where dt='20190909') as t1on t1.userId = t2.userId
group by t1.userid,t1.loginname
limit 5;
2 指标开发二
参考代码：
selectt2.userid,t2.loginname,MIN(t1.payTime) as first_paytime,MAX(t1.payTime) as lastest_paytime,DATEDIFF(CURRENT_TIMESTAMP, MIN(t1.payTime)) as first_day_during_days,DATEDIFF(CURRENT_TIMESTAMP, MAX(t1.payTime)) as lastest_day_durning_days,MIN(t1.paymoney) as min_paymoney,MAX(t1.paymoney) as max_paymoney,
sum(
case when t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end
) as total_count_without_back,--累计消费次数不含退拒，
sum(case when t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back, --累计消费金额不含退拒--累计近30天消费次数不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end) as total_count_without_back_30,--累计近30天消费金额不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back_30,--累计近30天消费次数含退拒sum(case when t1.flag30 =1 then 1else 0end) as total_count_without_30,--累计近30天消费金额含退拒sum(case when t1.flag30 =1 then t1.paymoneyelse 0end) as total_money_with_back_30from
(select * from itcast_dw.fact_order_wide where dt="20190909") t1
left join
(select * from itcast_dw.dim_user where dt="20190909") t2 on
t1.userid=t2.userid
group by t2.userid,t2.loginname limit 5;
3 指标开发三
参考代码：
-- 指标开发三
-- 1. 客单价（含退拒）
-- 2. 客单价（不含退拒）
-- 3. 近60天客单价（含退拒）-----分析用户消费水平
-- 4. 近60天客单价（不含退拒）
selectt2.userid,t2.loginname,MIN(t1.payTime) as first_paytime,MAX(t1.payTime) as lastest_paytime,DATEDIFF(CURRENT_TIMESTAMP, MIN(t1.payTime)) as first_day_during_days,DATEDIFF(CURRENT_TIMESTAMP, MAX(t1.payTime)) as lastest_day_durning_days,MIN(t1.paymoney) as min_paymoney,MAX(t1.paymoney) as max_paymoney,
sum(
case when t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end
) as total_count_without_back,--累计消费次数不含退拒，
sum(case when t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back, --累计消费金额不含退拒--累计近30天消费次数不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end) as total_count_without_back_30,--累计近30天消费金额不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back_30,--累计近30天消费次数含退拒sum(case when t1.flag30 =1 then 1else 0end) as total_count_without_30,--累计近30天消费金额含退拒sum(case when t1.flag30 =1 then t1.paymoneyelse 0end) as total_money_with_back_30,--客单价含退拒SUM(t1.paymoney) / SUM(1) AS atv,--客单价不含退拒SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then t1.paymoney else 0 end) / SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then 1 else 0end) AS atv_withoutback,--近60天客单价含退拒SUM(case when t1.flag60 = 1 then t1.paymoney else 0 end) / SUM(case when t1.flag60 = 1then 1else 0end) AS atv_60,--近60天不含退拒SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 and t1.flag60 = 1then t1.paymoneyelse 0end) / SUM(case when (t1.orderStatus != 10 or t1.orderStatus != 11) and t1.flag60 = 1then 1else 0end) AS atv_60_withoutbackfrom
(select * from itcast_dw.fact_order_wide where dt="20190909") t1
left join
(select * from itcast_dw.dim_user where dt="20190909") t2 on
t1.userid=t2.userid
group by t2.userid,t2.loginname limit 5;
4 指标开发四
1、加载订单地址分析表
参考代码：
-- 创建订单地址分析表
drop table if exists `itcast_ads`.`tmp_order_address`;
create table `itcast_ads`.`tmp_order_address`(userId bigint, -- 用户IdotherName string, -- 地址类型（家里、学校...)totalCount bigint, -- 下单数rn bigint -- 下单排名
)
partitioned by (dt string)
STORED AS PARQUET;
--从tmp_order_datetag_wide统计最常用地址
insert overwrite table `itcast_ads`.`tmp_order_address` partition(dt='20190909')
select
t3.userid,
t3.othername,
t3.ordercount,
row_number() over( partition by t3.userid order by ordercount desc ) rn --partiton by userid:按照用户分组，order by ordercount :按照订单数量排序 降序 ，rn:组内的排序
from
(selectt1.userId as userid,t1.othername as othername,count(t1.orderid) as ordercount -->每个用户每个订单的数量
from(select * from `itcast_dw`.`fact_order_wide` where dt='20190909') t1group by t1.userid,t1.otherName order by t1.userid ) t3 ;
-- 测试
select * from `itcast_ads`.`tmp_order_address` order by userId, rn limit 10;
2、统计常用收货地址指标
参考代码：
selectt2.userid,t2.loginname,MIN(t1.payTime) as first_paytime,MAX(t1.payTime) as lastest_paytime,DATEDIFF(CURRENT_TIMESTAMP, MIN(t1.payTime)) as first_day_during_days,DATEDIFF(CURRENT_TIMESTAMP, MAX(t1.payTime)) as lastest_day_durning_days,MIN(t1.paymoney) as min_paymoney,MAX(t1.paymoney) as max_paymoney,
sum(
case when t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end
) as total_count_without_back,--累计消费次数不含退拒，
sum(case when t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back, --累计消费金额不含退拒--累计近30天消费次数不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end) as total_count_without_back_30,--累计近30天消费金额不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back_30,--累计近30天消费次数含退拒sum(case when t1.flag30 =1 then 1else 0end) as total_count_without_30,--累计近30天消费金额含退拒sum(case when t1.flag30 =1 then t1.paymoneyelse 0end) as total_money_with_back_30,SUM(t1.paymoney) / SUM(1) AS atv,SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then t1.paymoney else 0 end) / SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then 1 else 0end) AS atv_withoutback,SUM(case when t1.flag60 = 1 then t1.paymoney else 0 end) / SUM(case when t1.flag60 = 1then 1else 0end) AS atv_60,SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 and t1.flag60 = 1then t1.paymoneyelse 0end) / SUM(case when (t1.orderStatus != 10 or t1.orderStatus != 11) and t1.flag60 = 1then 1else 0end) AS atv_60_withoutback,--最常用地址
max(case when t3.rn =1
then t3.othername
else ''
end) as most_usual_address from
(select * from itcast_dw.fact_order_wide where dt="20190909") t1
left join
(select * from itcast_dw.dim_user where dt="20190909") t2 on
t1.userid=t2.useridleft join(select * from `itcast_ads`.`tmp_order_address` where dt='20190909') as t3on t1.userId = t3.userId
group by t2.userid,t2.loginname limit 5;
5 指标开发五
更新mysql表中的模拟数据：
SET FOREIGN_KEY_CHECKS=0;
-- ----------------------------
-- Table structure for `itcast_payments`
-- ----------------------------
DROP TABLE IF EXISTS `itcast_payments`;
CREATE TABLE `itcast_payments` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`payCode` varchar(20) DEFAULT NULL,
`payName` varchar(255) DEFAULT NULL,
`payDesc` text,
`payOrder` int(11) DEFAULT '0',
`payConfig` text,
`enabled` tinyint(4) DEFAULT '0',
`isOnline` tinyint(4) DEFAULT '0',
`payFor` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `payCode` (`payCode`,`enabled`,`isOnline`)
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8;
-- ----------------------------
-- Records of itcast_payments
-- ----------------------------
INSERT INTO `itcast_payments` VALUES ('0', 'unkown', '未知方式', '未知', '0', null, '0', '0', null);
INSERT INTO `itcast_payments` VALUES ('1', 'alipays', '支付宝(及时到帐)', '支付宝(及时到帐)', '4', '', '0', '1', '1,2,4');
INSERT INTO `itcast_payments` VALUES ('2', 'weixinpays', '微信支付', '微信支付', '0', '', '0', '1', '1,2,3');
INSERT INTO `itcast_payments` VALUES ('3', 'wallets', '余额支付', '余额支付', '5', '', '1', '1', '1,2,3,4');
INSERT INTO `itcast_payments` VALUES ('4', 'cod', '货到付款', '开通城市', '1', '', '1', '0', '1,2,3,4');
--更新ods层数据
创建dw层dim_payments表
drop table if exists `itcast_dw`.`dim_payments`;
create table `itcast_dw`.`dim_payments`(id bigint,payCode string,payName string,payDesc string,payOrder bigint,payConfig string,enabled bigint,isOnline bigint,payFor string
)
partitioned by (dt string)
STORED AS PARQUET;
--从ods层itcast_payments导出数据到dw层dim_payments表中
insert overwrite table `itcast_dw`.`dim_payments` partition(dt="20190909")
select
id ,
payCode ,
payName ,
payDesc ,
payOrder ,
payConfig,
enabled ,
isOnline ,
payFor
from `itcast_ods`.`itcast_payments` where dt="20190909";
1、加载支付方式排名
-- 创建支付方式分析表
drop table if exists `itcast_ads`.`tmp_order_paytype`;
create table `itcast_ads`.`tmp_order_paytype`(userid bigint, -- 用户idpayType bigint, -- 支付类型idpayCode string, -- 支付码totalCount bigint, -- 订单总数rn bigint -- 等级
)
partitioned by (dt string)
STORED AS PARQUET;
-- 加载支付方式分析
insert overwrite table `itcast_ads`.`tmp_order_paytype` partition(dt='20190909')
selectt3.*,row_number() over(partition by userId order by totalCount desc) rn
from(selectt1.userId,t1.payType,t2.payCode,sum(1) as totalCount --sum(1)等同于count效果from`itcast_dw`.`dim_payments` t2right join`itcast_dw`.`tmp_order_datetag_wide` t1on t2.id = t1.payTypegroup by t1.userId, t1.payType, t2.payCode) t3;-- 测试
select * from `itcast_ads`.`tmp_order_paytype` limit 5;
2、统计常用支付方式指标
参考代码：
selectt2.userid,t2.loginname,MIN(t1.payTime) as first_paytime,MAX(t1.payTime) as lastest_paytime,DATEDIFF(CURRENT_TIMESTAMP, MIN(t1.payTime)) as first_day_during_days,DATEDIFF(CURRENT_TIMESTAMP, MAX(t1.payTime)) as lastest_day_durning_days,MIN(t1.paymoney) as min_paymoney,MAX(t1.paymoney) as max_paymoney,
sum(
case when t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end
) as total_count_without_back,--累计消费次数不含退拒，
sum(case when t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back, --累计消费金额不含退拒--累计近30天消费次数不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end) as total_count_without_back_30,--累计近30天消费金额不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back_30,--累计近30天消费次数含退拒sum(case when t1.flag30 =1 then 1else 0end) as total_count_without_30,--累计近30天消费金额含退拒sum(case when t1.flag30 =1 then t1.paymoneyelse 0end) as total_money_with_back_30,SUM(t1.paymoney) / SUM(1) AS atv,SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then t1.paymoney else 0 end) / SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then 1 else 0end) AS atv_withoutback,SUM(case when t1.flag60 = 1 then t1.paymoney else 0 end) / SUM(case when t1.flag60 = 1then 1else 0end) AS atv_60,SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 and t1.flag60 = 1then t1.paymoneyelse 0end) / SUM(case when (t1.orderStatus != 10 or t1.orderStatus != 11) and t1.flag60 = 1then 1else 0end) AS atv_60_withoutback,--最常用地址
max(case when t3.rn =1
then t3.othername
else ''
end) as most_usual_address,
--常用的支付方式MAX(case when t4.rn = 1then t4.payCodeelse ''end) as most_usual_paytype
from
(select * from itcast_dw.fact_order_wide where dt="20190909") t1
left join
(select * from itcast_dw.dim_user where dt="20190909") t2 on
t1.userid=t2.useridleft join(select * from `itcast_ads`.`tmp_order_address` where dt='20190909') as t3on t1.userId = t3.userIdleft join (select * from `itcast_ads`.`tmp_order_paytype` where dt='20190909') as t4on t1.userId = t4.userId
group by t2.userid,t2.loginname limit 5;
6 指标开发六
参考代码
selectt2.userid,t2.loginname,MIN(t1.payTime) as first_paytime,MAX(t1.payTime) as lastest_paytime,DATEDIFF(CURRENT_TIMESTAMP, MIN(t1.payTime)) as first_day_during_days,DATEDIFF(CURRENT_TIMESTAMP, MAX(t1.payTime)) as lastest_day_durning_days,MIN(t1.paymoney) as min_paymoney,MAX(t1.paymoney) as max_paymoney,
sum(
case when t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end
) as total_count_without_back,--累计消费次数不含退拒，
sum(case when t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back, --累计消费金额不含退拒--累计近30天消费次数不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end) as total_count_without_back_30,--累计近30天消费金额不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back_30,--累计近30天消费次数含退拒sum(case when t1.flag30 =1 then 1else 0end) as total_count_without_30,--累计近30天消费金额含退拒sum(case when t1.flag30 =1 then t1.paymoneyelse 0end) as total_money_with_back_30,SUM(t1.paymoney) / SUM(1) AS atv,SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then t1.paymoney else 0 end) / SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then 1 else 0end) AS atv_withoutback,SUM(case when t1.flag60 = 1 then t1.paymoney else 0 end) / SUM(case when t1.flag60 = 1then 1else 0end) AS atv_60,SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 and t1.flag60 = 1then t1.paymoneyelse 0end) / SUM(case when (t1.orderStatus != 10 or t1.orderStatus != 11) and t1.flag60 = 1then 1else 0end) AS atv_60_withoutback,--最常用地址max(case when t3.rn =1then t3.othernameelse ''end) as most_usual_address,--最常用支付方式max(case when t4.rn = 1then t4.payCodeelse ''end) as most_usual_paytype,SUM(case when t1.otherName = '学校'then 1else 0end) as school_order_count, -- 学校下单总数SUM(case when t1.otherName = '单位'then 1else 0end) as company_order_count, -- 单位下单总数SUM(case when t1.otherName = '家里'then 1else 0end) as home_order_count, -- 家里下单总数SUM(case when t1.flagTimeBucket = '上午'then 1else 0end) as am_order_count, -- 上午下单总数SUM(case when t1.flagTimeBucket = '下午'then 1else 0end) as pm_order_count, -- 下午下单总数SUM(case when t1.flagTimeBucket = '晚上'then 1else 0end) as night_order_count-- 晚上下单总数
from
(select * from itcast_dw.fact_order_wide where dt="20190909") t1
left join
(select * from itcast_dw.dim_user where dt="20190909") t2 on
t1.userid=t2.useridleft join(select * from `itcast_ads`.`tmp_order_address` where dt='20190909') as t3on t1.userId = t3.userIdleft join (select * from `itcast_ads`.`tmp_order_paytype` where dt='20190909') as t4on t1.userId = t4.userId
group by t2.userid,t2.loginname limit 5;
7 创建ads层表、加载数据
drop table if exists `itcast_ads`.`tmp_user_order_measure`;create table `itcast_ads`.`tmp_user_order_measure`(userid string, -- 用户idusername string, -- 用户名称first_paytime string, -- 第一次消费时间lastest_paytime string, -- 最近一次消费时间first_day_during_days bigint, -- 首单距今时间lastest_day_durning_days bigint, -- 尾单距今时间min_paymoney double, -- 最小消费金额max_paymoney double, -- 最大消费金额total_count_without_back bigint, -- 累计消费次数（不含退拒）total_money_without_back double, -- 累计消费金额（不含退拒）total_count_without_back_30 bigint, -- 近30天累计消费次数（不含退拒）total_money_without_back_30 double, -- 近30天累计消费金额（不含退拒）total_count_30 bigint, -- 近30天累计消费次数（含退拒）total_money_30 double, -- 近30天累计消费金额（含退拒）atv double, -- 客单价（含退拒）atv_withoutback double, -- 客单价（不含退拒）atv_60 double, -- 近60天客单价（含退拒）atv_60_withoutback double, -- 近60天客单价（不含退拒）most_usual_address string, -- 常用收货地址most_usual_paytype string, -- 常用支付方式school_order_count bigint, -- 学校下单总数company_order_count bigint, -- 单位下单总数home_order_count bigint, -- 家里下单总数am_order_count bigint, -- 上午下单总数pm_order_count bigint, -- 下午下单总数night_order_count bigint -- 晚上下单总数
)
partitioned by (dt string)
STORED AS PARQUET;
insert overwrite table `itcast_ads`.`tmp_user_order_measure` partition (dt='20190909')
selectt2.userid,t2.loginname,MIN(t1.payTime) as first_paytime,MAX(t1.payTime) as lastest_paytime,DATEDIFF(CURRENT_TIMESTAMP, MIN(t1.payTime)) as first_day_during_days,DATEDIFF(CURRENT_TIMESTAMP, MAX(t1.payTime)) as lastest_day_durning_days,MIN(t1.paymoney) as min_paymoney,MAX(t1.paymoney) as max_paymoney,
sum(
case when t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end
) as total_count_without_back,--累计消费次数不含退拒，
sum(case when t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back, --累计消费金额不含退拒--累计近30天消费次数不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then 1else 0end) as total_count_without_back_30,--累计近30天消费金额不含退拒sum(case when t1.flag30 =1 and t1.orderstatus !=10 and t1.orderstatus !=11then t1.paymoneyelse 0end) as total_money_without_back_30,--累计近30天消费次数含退拒sum(case when t1.flag30 =1 then 1else 0end) as total_count_without_30,--累计近30天消费金额含退拒sum(case when t1.flag30 =1 then t1.paymoneyelse 0end) as total_money_with_back_30,SUM(t1.paymoney) / SUM(1) AS atv,SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then t1.paymoney else 0 end) / SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 then 1 else 0end) AS atv_withoutback,SUM(case when t1.flag60 = 1 then t1.paymoney else 0 end) / SUM(case when t1.flag60 = 1then 1else 0end) AS atv_60,SUM(case when t1.orderStatus != 10 or t1.orderStatus != 11 and t1.flag60 = 1then t1.paymoneyelse 0end) / SUM(case when (t1.orderStatus != 10 or t1.orderStatus != 11) and t1.flag60 = 1then 1else 0end) AS atv_60_withoutback,--最常用地址max(case when t3.rn =1then t3.othernameelse ''end) as most_usual_address,max(case when t4.rn = 1then t4.payCodeelse ''end) as most_usual_paytype,SUM(case when t1.otherName = '学校'then 1else 0end) as school_order_count, -- 学校下单总数SUM(case when t1.otherName = '单位'then 1else 0end) as company_order_count, -- 单位下单总数SUM(case when t1.otherName = '家里'then 1else 0end) as home_order_count, -- 家里下单总数SUM(case when t1.flagTimeBucket = '上午'then 1else 0end) as am_order_count, -- 上午下单总数SUM(case when t1.flagTimeBucket = '下午'then 1else 0end) as pm_order_count, -- 下午下单总数SUM(case when t1.flagTimeBucket = '晚上'then 1else 0end) as night_order_count-- 晚上下单总数
from
(select * from itcast_dw.fact_order_wide where dt="20190909") t1
left join
(select * from itcast_dw.dim_user where dt="20190909") t2 on
t1.userid=t2.useridleft join(select * from `itcast_ads`.`tmp_order_address` where dt='20190909') as t3on t1.userId = t3.userIdleft join (select * from `itcast_ads`.`tmp_order_paytype` where dt='20190909') as t4on t1.userId = t4.userId
group by t2.userid,t2.loginname ;
-- 测试
select * from `itcast_ads`.`tmp_user_order_measure` limit 10;
3.5 退货表指标统计
参考代码：
--dw层创建fact_order_refunds表
drop table if exists `itcast_dw`.`fact_order_refunds`;
create table `itcast_dw`.`fact_order_refunds`(id bigint,orderId bigint,goodsId bigint,refundTo bigint,refundReson bigint,refundOtherReson string,backMoney double,refundTradeNo string,refundRemark string,refundTime string,shopRejectReason string,refundStatus bigint,createTime string
)
partitioned by (dt string)
STORED AS PARQUET;
--从ods抽取数据到dw层中的fact_order_refunds
INSERT OVERWRITE TABLE `itcast_dw`.`fact_order_refunds` PARTITION (dt='20190909')
SELECT id
,orderId
,goodsId
,refundTo
,refundReson
,refundOtherReson
,backMoney
,refundTradeNo
,refundRemark
,refundTime
,shopRejectReason
,refundStatus
,createTime
FROM `itcast_ods`.`itcast_order_refunds` WHERE dt='20190909' ;
-- 退货表指标统计
drop table if exists `itcast_ads`.`tmp_user_refund_measure`;
create table `itcast_ads`.`tmp_user_refund_measure`(userId string, -- 用户idreturnGoodsCount bigint, -- 退货商品数量returnGoodsMoney double, -- 退货商品金额rejectGoodsCount bigint, -- 拒收商品数量rejectGoodsMoney double, -- 拒收商品金额latestRefundTime string -- 最后一次退拒时间
)
partitioned by (dt string)
STORED AS PARQUET;
-- 订单明细拉宽
drop table if exists `itcast_dw`.`tmp_order_goods_wide`;
create table `itcast_dw`.`tmp_order_goods_wide`(orderId bigint,orderStatus bigint,payType bigint,userId bigint,userAddressId bigint,payTime string,payPrice double,goodsId bigint,goodsNum bigint,createtime string
)
partitioned by(dt string)
STORED AS PARQUET;
-- 订单明细拉宽表导入数据
insert overwrite table `itcast_dw`.`tmp_order_goods_wide` partition (dt='20190909')
selectt1.orderId,t1.orderStatus,t1.payType,t1.userId,t1.userAddressId,t1.payTime,t2.payPrice,t2.goodsId,t2.goodsNum,t2.createtime
from(select * from `itcast_ods`.`itcast_orders` where dt='20190909') t1join(select * from `itcast_ods`.`itcast_order_goods` where dt='20190909') t2on t1.orderId = t2.orderId;-- 测试
select * from `itcast_dw`.`tmp_order_goods_wide` limit 10;
insert overwrite table `itcast_ads`.`tmp_user_refund_measure` partition(dt='20190909')
selectt1.userId,sum(case when t1.orderStatus = 11 then t1.goodsnum else 0 end) as returnGoodsCount, -- 退货商品数量sum(case when t1.orderStatus = 11 then t2.backMoney else 0 end) as returnGoodsMoney, -- 退货金额sum(case when t1.orderStatus = 10 then t1.goodsnum else 0 end) as rejectGoodsCount, -- 拒收商品数量sum(case when t1.orderStatus = 10 then t2.backMoney else 0 end) as rejectGoodsMoney, -- 拒收商品金额MAX(t2.createtime) as latestRefundTime -- 最近一次退拒时间
from(select * from `itcast_dw`.`tmp_order_goods_wide` where orderStatus = 10 or orderStatus = 11) t1 ---订单状态是退和拒的订单(以订单明细为基础的表)left join(select * from `itcast_dw`.`fact_order_refunds` where dt='20190909') t2 on t1.orderId = t2.orderId and t1.goodsid= t2.goodsid --此时才能保证订单明细中数据与退货表数据是一对一join
where t2.id is not null
group by t1.userId ;
ads层开发
-- 用户订单分析指标统计
drop table if exists `itcast_ads`.`ads_user_order_measure`;
create table `itcast_ads`.`ads_user_order_measure`(userid string, -- 用户idusername string, -- 用户名称first_paytime string, -- 第一次消费时间lastest_paytime string, -- 最近一次消费时间first_day_during_days bigint, -- 首单距今时间lastest_day_durning_days bigint, -- 尾单距今时间min_paymoney double, -- 最小消费金额max_paymoney double, -- 最大消费金额total_count_without_back bigint, -- 累计消费次数（不含退拒）total_money_without_back double, -- 累计消费金额（不含退拒）total_count_without_back_30 bigint, -- 近30天累计消费次数（不含退拒）total_money_without_back_30 double, -- 近30天累计消费金额（不含退拒）total_count_30 bigint, -- 近30天累计消费次数（含退拒）total_money_30 double, -- 近30天累计消费金额（含退拒）atv double, -- 客单价（含退拒）atv_withoutback double, -- 客单价（不含退拒）atv_60 double, -- 近60天客单价（含退拒）atv_60_withoutback double, -- 近60天客单价（不含退拒）most_usual_address string, -- 常用收货地址most_usual_paytype string, -- 常用支付方式school_order_count bigint, -- 学校下单总数company_order_count bigint, -- 单位下单总数home_order_count bigint, -- 家里下单总数am_order_count bigint, -- 上午下单总数pm_order_count bigint, -- 下午下单总数night_order_count bigint, -- 晚上下单总数returnGoodsCount bigint, -- 退货商品数量returnGoodsMoney double, -- 退货商品金额rejectGoodsCount bigint, -- 拒收商品数量rejectGoodsMoney double, -- 拒收商品金额latestRefundTime string -- 最后一次退拒时间
)
partitioned by (dt string)
STORED AS PARQUET;
insert overwrite table `itcast_ads`.`ads_user_order_measure` partition(dt='20190909')
selectt1.userid,t1.username,t1.first_paytime, t1.lastest_paytime, t1.first_day_during_days, t1.lastest_day_durning_days, t1.min_paymoney, t1.max_paymoney, t1.total_count_without_back, t1.total_money_without_back, t1.total_count_without_back_30, t1.total_money_without_back_30, t1.total_count_30, t1.total_money_30, t1.atv, t1.atv_withoutback, t1.atv_60, t1.atv_60_withoutback, t1.most_usual_address, t1.most_usual_paytype, t1.school_order_count, t1.company_order_count, t1.home_order_count, t1.am_order_count, t1.pm_order_count, t1.night_order_count,t2.returnGoodsCount, t2.returnGoodsMoney, t2.rejectGoodsCount, t2.rejectGoodsMoney,t2.latestRefundTime
from `itcast_ads`.`tmp_user_order_measure` t1left join `itcast_ads`.`tmp_user_refund_measure` t2on t1.userId = t2.userId;--验证数据select * from itcast_ads.ads_user_order_measure where dt="20190909" limit 5;
常见错误
缺少union all
查询数据表字段没有对应上
用户订单指标业务开发相关推荐
- Day651.NoSQL与RDBMS合理搭配问题 -Java业务开发常见错误
NoSQL与RDBMS合理搭配问题 Hello,阿昌来也!今天学习分享记录的关于Nosql数据库和Mysql数据库的一系列对比,架构安排,发挥出每种数据库的优势案例. 近几年,各种非关系型数据库,也就 ...
- redis 用户订单缓存_Redis实战(12)-基于Key失效和定时任务实现订单支付超时自动失效...
"商城平台用户下单"这一业务场景相信很多小伙伴并不陌生,在正常的情况下,用户在提交完订单/下完单之后,应该是前往"收银台"选择支付方式进行支付,之后只需要提供相 ...
- 「订单」业务的设计与实现
一.背景简介 订单业务一直是系统研发中的核心模块.订单的产生过程与系统中的许多模块高度关联,例如账户体系.支付中心.运营管理等等.即使仅单独考虑订单本身,它也足够复杂了. 在业务发展过程中,订单量必然 ...
- 大数据【企业级360°全方位用户画像】业务数据调研及ETL
写在前面: 博主是一名大数据的初学者,昵称来源于<爱丽丝梦游仙境>中的Alice和自己的昵称.作为一名互联网小白,写博客一方面是为了记录自己的学习历程,一方面是希望能够帮助到很多和自己一样 ...
- 浅谈电商核心「订单」业务如何设计
三分恶 2023-05-30 10:20 发表于北京 以下文章来源于知了一笑 ,作者知了一笑 知了一笑. 积累.总结.用心记录. 订单,业务的核心模块: 一.背景简介 订单业务一直都是系统研发中的核心 ...
- Spring 声明式事务在业务开发中容易碰到的坑总结
Spring 声明式事务,在业务开发使用上可能遇到的三类坑,包括: 第一,因为配置不正确,导致方法上的事务没生效.我们务必确认调用 @Transactional 注解标记的方法是 public 的,并 ...
- 灵魂拷问：后端业务开发要会用 K8s 到什么程度？
来源 | 阿里巴巴云原生公众号 很多人看着 K8s 成为最热门的开源技术,都纷纷开始学习 K8s,但也有很多人在抱怨 K8s 太复杂了.用 CNCF 新晋 TOC 张磊的话来说:这里的根本问题在于,K ...
- 游戏用户体验指标_电子游戏如何超越游戏化的用户体验
游戏用户体验指标 游戏UX (GAMES UX) During a time when the time spent on video games has reached record breakin ...
- 百度王一男： DevOps 的前提是拆掉业务-开发-测试-运维中间的三面墙
这是一个创建于 375 天前的主题,其中的信息可能已经有所发展或是发生改变. 由数人云.优维科技.中生代社区联合发起的 系列 Meetup < DevOps&SRE 超越传统运维之道&g ...
最新文章
- MySql中管理百万级要注意些什么东西(转载)
- 【C语言】单链表的所有操作的实现（包括PopBack、PushBack、PopFront、PushFront、Insert）...
- 【Python刷题】_5
- 3.JAVA中的多态
- 强网杯2021 CipherMan (内存取证分析)
- when to book didi?
- 数学大家闵嗣鹤：生死哥德巴赫猜想
- mysql 插入数据时 自动设置创建时间和更新时间
- 房价python爬取_python爬取并解析 重庆2015-2019房价走势
- 微信可以远程控制电脑吗_上车| 手机远程控制手机，还可以控制电脑
- [转] Agile Software Development 敏捷软件开发
- 用 DocFetcher 全文搜索
- 北京二手房上周成交环比增六成 个别业主涨价出售
- Torrent File Editor——好软推荐
- python等比例压缩图片_python使用pil进行图像处理(等比例压缩、裁剪)实例代码
- 虚拟机桌面切换命令行
- 小猫排队 （思维 模拟
- php去掉二维数组中某key的值
- Coursera | Andrew Ng (02-week-1-1.12)—梯度的数值逼近
- 图像处理之几何变换(python实现)