CHD-5.3.6集群上hive安装

解压过后：

[hadoop@master CDH5.3.6]$ ls -rlt
total 8
drwxr-xr-x. 17 hadoop hadoop 4096 Jun 2 16:07 hadoop-2.5.0-cdh5.3.6
drwxr-xr-x. 11 hadoop hadoop 4096 Jun 2 16:28 hive-0.13.1-cdh5.3.6

1.配置hive-env.sh

export JAVA_HOME=/usr/local/jdk1.8
export HADOOP_HOME=/home/hadoop/CDH5.3.6/hadoop-2.5.0-cdh5.3.6
export HIVE_HOME=/home/hadoop/CDH5.3.6/hive-0.13.1-cdh5.3.6
export HIVE_CONF_DIR=/home/hadoop/CDH5.3.6/hive-0.13.1-cdh5.3.6/conf

2.配置hive-log4j.properties

hive.log.dir=/home/hadoop/CDH5.3.6/hive-0.13.1-cdh5.3.6/log

3.配置hive-site.xml

这个寻找Apache-hadoop下的就可以，直接考过来就可以，在conf 目录下

4.配置环境变量

vi .bash_profile

export HADOOP_HOME=/home/hadoop/CDH5.3.6/hadoop-2.5.0-cdh5.3.6
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HIVE_HOME=/home/hadoop/CDH5.3.6/hive-0.13.1-cdh5.3.6
export HADOOP_INSTALL=$HADOOP_HOME
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin:$MAVEN_HOME/bin:$HIVE_HOME/bin

5.拷贝MySQL包

cp /home/hadoop/hive/lib/mysql-connector-java-5.1.46.jar ./

6.hive命令报错：

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
解决方法：

格式化MySQL：
schematool -dbType mysql -initSchema

7.进入hive

hive (default)> > CREATE TABLE dept(>   deptno int, >   dname string, >   loc string)> ROW FORMAT DELIMITED FIELDS TERMINATED BY ','  STORED AS textfile;
OK
Time taken: 0.57 seconds

8.准备数据：

vi detp.txt
10,ACCOUNTING,NEW YORK
20,RESEARCH,DALLAS
30,SALES,CHICAGO
40,OPERATIONS,BOSTON

9.装数据：

load data local inpath '/home/hadoop/tmp/detp.txt' overwrite into table dept;

10.查询：

hive (default)> select count(1) from dept;
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:set mapreduce.job.reduces=<number>
Starting Job = job_1559517371869_0001, Tracking URL = http://master:8088/proxy/application_1559517371869_0001/
Kill Command = /home/hadoop/CDH5.3.6/hadoop-2.5.0-cdh5.3.6/bin/hadoop job  -kill job_1559517371869_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2019-06-02 17:13:27,541 Stage-1 map = 0%,  reduce = 0%
2019-06-02 17:13:33,988 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2019-06-02 17:13:40,268 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1.7 sec
MapReduce Total cumulative CPU time: 1 seconds 700 msec
Ended Job = job_1559517371869_0001
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1  Reduce: 1   Cumulative CPU: 1.7 sec   HDFS Read: 292 HDFS Write: 101 SUCCESS
Total MapReduce CPU Time Spent: 1 seconds 700 msec
OK
_c0
4
Time taken: 22.14 seconds, Fetched: 1 row(s)

页面验证：

http://192.168.1.30:8088/cluster

http://192.168.1.30:50070/dfshealth.html#tab-overview

转载于:https://www.cnblogs.com/hello-wei/p/10964658.html

CHD-5.3.6集群上hive安装相关推荐

hadoop集群搭建+hive安装
1.系统环境 VMware-workstation:VMware-workstation-full-16.2.3 ubuntu:ubuntu-21.10 hadoop:hadoop2.7.2 mysq ...
jar包在Hadoop集群上测试(MapReduce)
本片使用MapReduce--统计输出给定的文本文档每一个单词出现的总次数的案例进行,jar包在集群上测试 1.添加打包插件依赖 <build><plugins><plu ...
anaconda3环境整体打包放在Spark集群上运行
一.将虚拟Python环境打包创建好环境后,进入到环境所在的文件夹,例如环境是/home/hadoop/anaconda3/envs, cd到envs下,使用打包命令将当前目录下的文件打成zip包: ...
Spark在集群上执行代码案例（中文切词）
Spark在集群上执行代码案例 java的切词使用案例(Demo) Spark中文切词代码需求:利用jieba进行中文分词,并打包上传到集群进行执行 java的切词使用案例(Demo) @Test ...
如何在集群上运行Shark
如何在集群上运行Shark 本文介绍在计算机集群上如何启动和运行Shark.如果对Amazon EC2上运行Shark感兴趣,请点击这里查看如何使用EC2脚本快速启动预先配置好的集群. 依赖: 注意: ...
Pyspark让pandas特征工程代码在集群上飞起来——天猫复购率baseline
Pyspark让pandas特征工程代码在集群上飞起来--天猫复购率baseline 环境与数据准备导包创建spark应用第一种特征工程方式:pyspark.pandas.dataframe 第 ...
Hadoop集群上使用JNI，调用资源文件
hadoop是基于java的数据计算平台,引入第三方库,例如C语言实现的开发包将会大大增强数据分析的效率和能力. 通常在是用一些工具的时候都要用到一些配置文件.资源文件等.接下来,借一个例子来说明ha ...
在Kubernetes集群上部署高可用Harbor镜像仓库
这里主要介绍使用kubectl部署Harbor镜像仓库到Kubernetes集群中. 使用Helm部署,参考: https://my.oschina.net/u/2306127/blog/181969 ...
在生产集群上运行topology
2019独角兽企业重金招聘Python工程师标准>>> 在生产集群上运行topology 博客分类: 分布式计算在生产集群上运行topology跟本地模式差不多.下面是步骤: 1) ...

CHD-5.3.6集群上hive安装

CHD-5.3.6集群上hive安装相关推荐

最新文章

热门文章