删除druid中的数据可参考 https://blog.csdn.net/qq_34864753/article/details/102861322

1. 下载imply ，解压

https://imply.io/get-startedtar -xzf imply-3.0.12.tar.gz

impy quickstart链接地址（供参考）

https://docs.imply.io/on-prem/quickstart

2. 修改 _common 目录下的 common.runtime.properties 文件

mysql 库的创建可以参考下面，需要先创建库，这边使用了imply_druid用户（注意数据库需要是utf8，否则会报错）

Exception creating table
org.skife.jdbi.v2.exceptions.CallbackFailedException: org.apache.druid.java.util.common.ISE: Druid requires its MySQL database to be created with an UTF8 charset, found `latin1`. The recommended charset is `utf8mb4`.

create database imply_druid;CREATE USER 'imply_druid'@'%' IDENTIFIED BY 'imply_druid';grant all privileges on imply_druid.* to 'imply_druid'@'%' identified by 'imply_druid';

3. 将hadoop的配置文件 core-site.xml，hdfs-site.xml，mapred-site.xml，yarn-site.xml拷贝到 _common目录下

4. 启动集群 master节点/data节点/query节点

###master节点  /usr/local/bigdata/imply-3.0.12/bin/supervise -c /usr/local/bigdata/imply-3.0.12/conf/supervise/master-no-zk.conf -daemon###data节点
/usr/local/bigdata/imply-3.0.12/bin/supervise -c /usr/local/bigdata/imply-3.0.12/conf/supervise/data.conf -daemon###query节点/usr/local/bigdata/imply-3.0.12/bin/supervise -c /usr/local/bigdata/imply-3.0.12/conf/supervise/query.conf -daemon###重启服务
/usr/local/bigdata/imply-3.0.12/bin/service -restart  （broker,router,imply-ui,middleManager,historical,coordinator,overlord)
eg：/usr/local/bigdata/imply-2.8.6/bin/service -restart router 一般一个节点上有两个服务，要一个一个restart###下掉服务/usr/local/bigdata/imply-3.0.12/bin/service --down

5. 端口及界面

主要有 8081（master节点），8090（master节点），8088（query节点），9095（query节点）四个端口

#8081端口界面

#imply.io界面，对使用方很友好

#8088端口

6. 使用案例：数据从hdfs到druid (表的列太多，删除一部分)

数据，需先要上传到hdfs

{"cc_transform_rate_history": "0.05", "cc_sex": "1", "user_id": "1000888550", "cc_call_timevstimes_latest_month": "249", "cc_entry_days": "605", "cc_transform_rate_latest_month": "0.19", "cc_transform_numbers": "174", "cc_grade": "0", "row_key": "0558880001:1", "cc_work_city_code": "310100", "cc_call_timevstimes_history": "270", "cc_connecting_numbers_valid": "3281"}
{"cc_sex": "3", "user_id": "1018039550", "cc_origin_place_code": "420000", "cc_call_timevstimes_latest_month": "41", "cc_entry_days": "138", "cc_grade": "6", "row_key": "0559308101:1", "cc_work_city_code": "420100", "cc_call_timevstimes_history": "82", "cc_connecting_numbers_valid": "2043"}
{"cc_sex": "0", "user_id": "1028090650", "cc_origin_place_code": "410000", "cc_call_timevstimes_latest_month": "230", "cc_entry_days": "32", "cc_grade": "6", "row_key": "0560908201:1", "cc_work_city_code": "310100", "cc_call_timevstimes_history": "230", "cc_connecting_numbers_valid": "348"}
{"cc_sex": "0", "user_id": "1015562650", "cc_origin_place_code": "420000", "cc_call_timevstimes_latest_month": "34", "cc_entry_days": "178", "cc_grade": "6", "row_key": "0562655101:1", "cc_work_city_code": "420100", "cc_call_timevstimes_history": "61", "cc_connecting_numbers_valid": "1890"}
{"cc_transform_rate_history": "-1.00", "cc_sex": "3", "user_id": "1003846650", "cc_entry_days": "500", "cc_transform_rate_latest_month": "-1.00", "cc_grade": "6", "row_key": "0566483001:1", "cc_call_timevstimes_history": "-1"}
{"cc_sex": "0", "user_id": "1029658650", "cc_origin_place_code": "360000", "cc_call_timevstimes_latest_month": "175", "cc_entry_days": "9", "cc_grade": "6", "row_key": "0568569201:1", "cc_work_city_code": "310100", "cc_call_timevstimes_history": "175", "cc_connecting_numbers_valid": "49"}
{"cc_transform_rate_history": "-1.00", "cc_sex": "3", "user_id": "1006958650", "cc_entry_days": "388", "cc_transform_rate_latest_month": "-1.00", "cc_grade": "6", "row_key": "0568596001:1", "cc_call_timevstimes_history": "-1"}
{"cc_sex": "3", "user_id": "1016289650", "cc_entry_days": "168", "cc_grade": "6", "row_key": "0569826101:1", "cc_call_timevstimes_history": "36", "cc_connecting_numbers_valid": "1407"}
{"cc_transform_rate_history": "0.75", "cc_sex": "1", "user_id": "40750", "cc_call_timevstimes_latest_month": "629", "cc_entry_days": "1464", "cc_transform_rate_latest_month": "0.00", "cc_transform_numbers": "141", "cc_grade": "5", "row_key": "05704:1", "cc_work_city_code": "310100", "cc_call_timevstimes_history": "337", "cc_connecting_numbers_valid": "189"}
{"cc_transform_rate_history": "-1.00", "cc_sex": "0", "user_id": "1008884750", "cc_origin_place_code": "410000", "cc_entry_days": "270", "cc_transform_rate_latest_month": "-1.00", "cc_grade": "6", "row_key": "0574888001:1", "cc_work_city_code": "310100", "cc_call_timevstimes_history": "-1"}

{"type": "index_hadoop","id": "index_hadoop_user_profile_cc_2019-08-16T07:28:53.194Z","spec": {"dataSchema": {"dataSource": "user_profile_cc","parser": {"parseSpec": {"timestampSpec": {"column": "timestamp","missingValue": "2019-08-15","format": "auto"},"dimensionsSpec": {"dimensions": ["row_key","user_id","cc_work_city_code","cc_call_timevstimes_latest_month","cc_entry_days","cc_origin_place_code","cc_transform_numbers","cc_transform_rate_latest_month","cc_call_timevstimes_history","cc_connecting_numbers_valid","cc_grade","cc_sex","cc_transform_rate_history"]},"format": "json"},"type": "hadoopyString"},"metricsSpec": [],"granularitySpec": {"type": "uniform","segmentGranularity": "DAY","queryGranularity": {"type": "none"},"rollup": false,"intervals": ["2019-08-15T00:00:00.000Z/2019-08-16T00:00:00.000Z"]},"transformSpec": {"filter": null,"transforms": []}},"ioConfig": {"type": "hadoop","inputSpec": {"paths": "hdfs://nameservice1/tmp/zjf_druid/user_profile_cc","type": "static"},"metadataUpdateSpec": null,"segmentOutputPath": null},"tuningConfig": {"type": "hadoop","workingPath": null,"version": "2019-08-16T07:28:53.194Z","partitionsSpec": {"type": "hashed","targetPartitionSize": 2000000,"maxPartitionSize": 3000000,"assumeGrouped": false,"numShards": -1,"partitionDimensions": []},"shardSpecs": {},"indexSpec": {"bitmap": {"type": "concise"},"dimensionCompression": "lz4","metricCompression": "lz4","longEncoding": "longs"},"maxRowsInMemory": 1000000,"maxBytesInMemory": 0,"leaveIntermediate": false,"cleanupOnFailure": true,"overwriteFiles": false,"ignoreInvalidRows": false,"jobProperties": {"mapreduce.job.queuename":"root.zm_yarn_pool.production","mapreduce.job.user.classpath.first": "true","mapreduce.job.classloader": "true","mapreduce.job.classloader.system.classes": "-javax.validation.,java.,javax.,org.apache.commons.logging.,org.apache.log4j.,org.apache.hadoop.,org.xerial.snappy."},"combineText": false,"useCombiner": false,"buildV9Directly": true,"numBackgroundPersistThreads": 0,"forceExtendableShardSpecs": true,"useExplicitVersion": false,"allowedHadoopPrefix": [],"logParseExceptions": true,"maxParseExceptions": 1000},"uniqueId": "83b32f38334d48ab99598d9c725c885b"},"hadoopDependencyCoordinates": null,"classpathPrefix": null,"context": {},"groupId": "index_hadoop_user_profile_cc_2019-08-16T07:28:53.194Z","dataSource": "user_profile_cc","resource": {"availabilityGroup": "index_hadoop_user_profile_cc_2019-08-16T07:28:53.194Z","requiredCapacity": 1}
}

7.任务提交可以用 imply.io那个界面，也可以用 8081那个界面的Tasks / Submit task 来提交

8.注意事项

1. history目录的runtime.properties

2. middleManager目录下的runtime.properties

Imply方式安装0.15.0版本Druid和实例（hdfs2druid）分享相关推荐

mysql8.0.15mis百度网盘_mysql-installer-community-8.0.15.0数据库的安装
数据库mysql-installer-community-8.0.15.0的安装教程,如果想下载的常看下面的百度网盘连接. 链接:https://pan.baidu.com/s/13-m_dG85LN ...
linux 天文软件,新闻|开源新闻速递：天文软件 Stellarium 0.15.0 发布
今日关注 Stellarium 0.15.0 发布.这是一款全世界最棒的免费.开源.跨平台的天文软件应用,用户可以通过该软件来观看实时的星星.行星还有星云.最新版本进行了非常多的功能完善,修复了若干b ...
WIN7实现安装node14.15.0版本
WIN7系统相对较老了,但还是有人用,其下只能安装nodejs12.X的版本,高版本的node无法安装在WIN7下了,下面提供一个解决方案: 1.进入node.js官网:https://nodejs. ...
安装CDH5.15.0过程详细记录￥坑与解决办法（20180724）
参考了网上很多教程,踩过很多坑,总结一篇,仅供参考: node1 , node2, node3 :其中node1 为主节点,2 3为从节点. 机子内存太小不建议安装测试学习,否则过程很难受 ...说实 ...
Jetson Agx Orin使用最正确的方式安装torch1.11.0,torchvision0.12.0,torchaudio0.11.0保姆级教程
BSP版本使用35.1,Jetpack版本是5.0.2 We recommend Anaconda as Python package management system. Please refer ...
未能加载文件或程序集“log4net, Version=2.0.15.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a”或它的某一个依赖项
问题环境: (1)WInform (2)启动项目调用类库的方法,类库里log记录日志问题原因: (1)启动项目的Log4net的版本是12.0.11,而类库里的是12.0.15 解决办法: (1)统 ...
elasticsearch (一): Centos7 RPM 方式安装最新单机版elasticsearch版本7.10.2 并配置中文(ik)、拼音(pinyin)插件
目录 Centos7 RPM 安装 ES 操作系统介绍: 部署方式选择RPM包管理工具修改远程访问和更改IP ,重启es常见问题安装插件: ik pinyin 中文分词插件地址: 安装中文分词 ...
【Druid IO】Druid的imply方式安装
1.下载 https://imply.io/get-started 2. 解压 lcc@lcc druid$ ll total 655376 drwxr-xr-x@ 7 lcc staff 224 9 ...
30系显卡安装tensorflow-gpu1.15.0
安培架构下的30系显卡仅支持CUDA11以上的版本,目前最新的Tensorflow和PyTorch虽然都可以直接使用,然而谷歌不再维护的tensorflow1.x却无法安装在CUDA11环境下.好在N ...

Imply方式安装0.15.0版本Druid和实例（hdfs2druid）分享

8.注意事项

Imply方式安装0.15.0版本Druid和实例（hdfs2druid）分享相关推荐

最新文章

热门文章