1, 概念介绍: parcel,csd

名称	含义
parcel	普通tarball + metadata ( xx_env.sh ，parcel.json, alternatives.json )
csd (Custom Service Descriptors )	普通jar包 ( descriptor/service.sdl, images/xx.png, scripts/control.sh )

parcel 详细说明： https://github.com/cloudera/cm_ext/wiki/The-parcel.json-file
csd 详细说明：https://github.com/cloudera/cm_ext/wiki/CSD-Overview， https://github.com/cloudera/cm_ext/wiki/Service-Descriptor-Language-Reference

2, 自定义parcel: 把jar包封装为parcel

a, 生成parcel

获取parcel制作/验证的工具：https://github.com/cloudera/cm_ext/
百度盘-提取码：abcd

[root@localhost Downloads]# tree DEMO-2.0/
DEMO-2.0/
├── lib
│   └── demo-2.0.jar
└── meta├── demo_env.sh└── parcel.json
2 directories, 3 files[root@localhost Downloads]# java -jar DEMO-2.0/lib/demo-2.0.jar.   ____          _            __ _ _/\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \\\/  ___)| |_)| | | | | || (_| |  ) ) ) )'  |____| .__|_| |_|_| |_\__, | / / / /=========|_|==============|___/=/_/_/_/:: Spring Boot ::                (v2.5.0)
2021-06-02 09:33:04.650  INFO 9524 --- [           main] com.example.demo3.Demo3Application
^C[root@localhost Downloads]#
[root@localhost Downloads]# cd DEMO-2.0/[root@localhost DEMO-2.0]# cat meta/demo_env.sh
#!/bin/bash
DEMO_DIRNAME=${PARCEL_DIRNAME:-"DEMO-2.0"}
export DEMO_HOME=$PARCELS_ROOT/$DEMO_DIRNAME/lib[root@localhost DEMO-2.0]# cat meta/parcel.json
{"schema_version":1,"name":"DEMO","version":"2.0","setActiveSymlink":true,"conflicts":"","provides":["demo"],"scripts":{"defines":"demo_env.sh"},"packages":[{"name":"demo","version":"2.0"}],"components":[{"name":"demo","version":"2.0","pkg_version":"2.0","pkg_release":"2.0"}],"users":{ },"groups":[  ]
}[root@localhost DEMO-2.0]# cd ..
[root@localhost Downloads]# mkdir parcels
[root@localhost Downloads]# tar -zcvf DEMO-2.0-el6.parcel DEMO-2.0/
DEMO-2.0/
DEMO-2.0/lib/
DEMO-2.0/lib/demo-2.0.jar
DEMO-2.0/meta/
DEMO-2.0/meta/parcel.json
DEMO-2.0/meta/demo_env.sh
[root@localhost Downloads]# mv DEMO-2.0-el6.parcel parcels/
[root@localhost Downloads]# cd parcels/
[root@localhost parcels]# ls
DEMO-2.0-el6.parcel    make_manifest.py  validator.jar
[root@localhost parcels]# java -jar validator.jar -f DEMO-2.0-el6.parcel
Validating: DEMO-2.0-el6.parcel
Validating: DEMO-2.0/meta/parcel.json
Validation succeeded.[root@localhost parcels]# python make_manifest.py
Scanning directory: .
Found parcel DEMO-2.0-el6.parcel
[root@localhost parcels]# ls
DEMO-2.0-el6.parcel   make_manifest.py  manifest.json  validator.jar
[root@localhost parcels]# cat manifest.json
{"parcels": [{"conflicts": "","hash": "faac5c55c6e918f4f77bac572ba69ae577386750","parcelName": "DEMO-2.0-el6.parcel","components": [{"pkg_version": "2.0","version": "2.0","name": "demo","pkg_release": "2.0"}]}],"lastUpdated": 1622598123811
}[root@localhost parcels]# cat manifest.json | grep hash | awk -F"\"" '{print$4}' > DEMO-2.0-el6.parcel.sha
[root@localhost parcels]# cat DEMO-2.0-el6.parcel.sha
faac5c55c6e918f4f77bac572ba69ae577386750
[root@localhost parcels]# sha1sum DEMO-2.0-el6.parcel
faac5c55c6e918f4f77bac572ba69ae577386750  DEMO-2.0-el6.parcel

b, 生成csd jar包

csd 配置模板(kafka, ECHO, spark , sqoop ) https://github.com/cloudera/cm_csds
csd jar包配置样例（ECHO service )：https://github.com/cloudera/cm_ext/wiki/CSD-Primer
png图片大小: 16*16 最好，否则页面显示有问题

service.sdl

{"name":"DEMO","label":"Demo","description":"spring boot demo application","version":"1.0","runAs":{"user":"root","group":"root"},"parcel":{"requiredTags":["demo"]},"compatibility":{"generation":1},"icon":"images/binocular.png","parameters":[{"name":"service_usage","label":"Service usage for ","description":"what is the tomcat used for, ie:测试,运行服务1...","configName":"service.usage","type":"string","default":"default tomcat service","configurableInWizard":true},{"name":"start_port","label":"start  port: listener port","description":"服务访问/监听端口","required":"true","type":"long","default":8810,"min":8000,"configurableInWizard":true},{"name":"stop_port","label":"shutdown port","description":"服务关闭端口","required":"true","type":"long","default":8005,"min":8000,"configurableInWizard":true}],"roles":[{"name":"DEMO_WEBSERVER","label":"Web Server","pluralLabel":"Web Servers","startRunner":{"program":"scripts/control.sh","args":["start"],"environmentVariables":{"stop_port":"${stop_port}","start_port":"${start_port}","service_usage":"${service_usage}"}},"logging":{"dir":"/var/log/demo","filename":"demo-${start_port}.log","configName":"log.dir","loggingType":"other"}}]
}

control.sh

#!/bin/bash
# Time marker for both stderr and stdout
date; date 1>&2cmd=$1
timestamp=$(date)
export DEMO_HOME=$PARCELS_ROOT/$DEMO_DIRNAME/lib
echo "service_usage:"$service_usage
echo "stop_port:"$stop_port
echo "start_port:"$start_portif [ "start" = "$cmd" ]; thenecho "$timestamp Starting Server" exec java -Dserver.port=$start_port -jar $DEMO_HOME/*.jarelif [ "stop" = "$cmd" ]; thenecho "$timestamp Stopping Server"
elseecho "$timestamp Don't understand [$cmd]"
fi

c, 部署服务到cloudera-manager

发布parcel和csd包, 并重启cloudera-manager 服务（7180所在的主机)

[root@localhost ~]# ls /opt/cloudera/parcel-repo/DEMO-2.0-el6.parcel* -l
-rw-rw-r-- 1 root  root  15511313 Jun  2 08:57 /opt/cloudera/parcel-repo/DEMO-2.0-el6.parcel
-rw-rw-r-- 1 root  root        41 Jun  2 08:57 /opt/cloudera/parcel-repo/DEMO-2.0-el6.parcel.sha[root@localhost ~]# ls /opt/cloudera/csd/DEMO-2.0.jar
/opt/cloudera/csd/DEMO-2.0.jar

自定义CDH parcel相关推荐

CDH Parcel包制作流程详解
CDH Parcel包制作流程详解背景介绍 CDH可以很方便的添加一些大数据相关服务,但这仅限于cloudera公司提供.若想将第三方服务(如公司自己开发的组件)添加到CDH集群(托管在CDH上), ...
CDH Hadoop 基于CM方式半在线安装步骤详解
CDH Hadoop基于Cloudera Manager半在线方式安装详解 1 CM简介 Cloudera Manager是一个端到端的管理CDH集群的工具.它通过管理CDH集群(组件安装.服务管理. ...
离线部署 CDH 6.2 及使用 CDH 部署 Hadoop3 大数据平台集群服务
Cloudera Manager Cloudera Manager 分为两个部分:CDH和CM. CDH是Cloudera Distribution Hadoop的简称,顾名思义,就是cloudera ...
安装 Cloudera Manager、CDH、KUDU 随手记
转自:https://my.oschina.net/balajinima/blog/862969 一.关于CDH和Cloudera Manager CDH (Cloudera's Distributi ...
CDH简介及CDH部署、原理和使用介绍( 版本6.3.1 )
CDH简介及CDH部署.原理和使用介绍( 版本6.3.1 ) 第一章:CDH简介 CDH概念 CDH是Cloudera的100%开源平台发行版,包括Apache Hadoop,专为满足企业需求而构 ...
【CDH】超详细-搭建本地大数据研发环境（16G内存+CDH）
工欲善其事必先利其器,在经过大量的理论学习以后,需要有一个本地的研发环境来进行练手.已经工作的可以不依赖于公司的环境,在家也可以随意的练习.而自学大数据的同学,也可以进行本地练习,大数据是一门偏实践的 ...
CDH 5 升级操作
现在大数据一直在使用CDH来作为基础平台.常会面临升级版本,操作中也经常出现各种各样的问题. 这里就整理一个操作步骤指引. 1.先升级CM,大部分情况下都应该是离线升级的场景.所以要先从官方网站上下载 ...
超详细！搭建本地大数据研发环境（16G内存+CDH）
工欲善其事必先利其器,在经过大量的理论学习以后,需要有一个本地的研发环境来进行练手.已经工作的可以不依赖于公司的环境,在家也可以随意的练习.而自学大数据的同学,也可以进行本地练习,大数据是一门偏实践的 ...
Cloudera CDH 常见的版本
7.1.4 7.1.3 7.1.2 7.1.1 7.0.3 6.3.2 5.16.2 2020/10/14 2020/09/24 2020/09/24 2020/6/4 2019/11/24 20 ...
CM/CDH 问题记录
本文档为使用 CDH 6.3.1 时,记录安装过程/使用中出现的一些问题,供后续备查. 从2021年1月31日开始,所有Cloudera软件都需要有效的订阅,并且只能通过付费进行访问.之前的在线安装, ...

自定义CDH parcel

文章目录

1, 概念介绍: parcel,csd

2, 自定义parcel: 把jar包封装为parcel

a, 生成parcel

b, 生成csd jar包

c, 部署服务到cloudera-manager

自定义CDH parcel相关推荐

最新文章

热门文章