安装CDH出现file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel...does not exist

使用Cloudera Manager采取parcel方式安装CDH的时候，部分agent节点出现异常：

Src file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel does not exist

查看对应agent的日志文件查看异常信息：

[root@cdh-01 parcel-cache]# tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log

发现日志里有大量的如下输出：

20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader   INFO     Fetching torrent: http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.torrent
[20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader   INFO     Starting download of: http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel
[20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader   INFO     Failed adding torrent: file:///opt/cloudera/parcel-cache/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.torrent Already present torrent: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel
[20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader   INFO     Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=2087665645 totalSize=2087665645 upload=322898691 state=seeding seed=['http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel'] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=1000000]
[20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader   INFO     Completed download of http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel code=200 state=downloaded
[20/May/2019 21:32:17 +0000] 8949 Thread-13 parcel_cache WARNING  No checksum in header, skipping verification
[20/May/2019 21:32:17 +0000] 8949 Thread-13 parcel_cache INFO     Unpacking /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel into /opt/cloudera/parcels
[20/May/2019 21:32:17 +0000] 8949 Thread-13 downloader   ERROR    Failed op: Src file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel does not exist

也就是提示Src file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel不存在，但是parcel应该从server节点下载下来了才对，为什么提示不存在？想不通，于是停止了该agent服务：

[root@cdh-01 parcel-cache]# systemctl stop cloudera-scm-agent

然后清理该agent节点上的parcel缓存(目录默认在 /opt/cloudera/)：

然后重启该agent服务，跟踪完整的日志输出，以确认出现问题的地方(不用重启server节点，agent节点重启后会主动向server节点通信，然后开始安装的流程)：

[root@cdh-01 parcel-cache]# systemctl start cloudera-scm-agent
[root@cdh-01 parcel-cache]# tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log

启动成功之后首先是Fetching torrent，然后开始下载：

[20/May/2019 19:51:18 +0000] 8949 Thread-13 downloader   INFO     Fetching torrent: http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.torrent
[20/May/2019 19:51:19 +0000] 8949 Thread-13 downloader   INFO     Starting download of: http://cdh-03.echatsoft.com:7180/cmf/parcel/download/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel
......[20/May/2019 19:51:31 +0000] 8949 Thread-13 downloader   INFO     Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=94896207 totalSize=2087665645 upload=16947965 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=45455]
[20/May/2019 19:51:35 +0000] 8949 Thread-13 downloader   INFO     Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=95420495 totalSize=2087665645 upload=17018436 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=45706]
[20/May/2019 19:51:37 +0000] 8949 Thread-13 downloader   INFO     Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=101712030 totalSize=2087665645 upload=17018436 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=48720]
[20/May/2019 19:51:39 +0000] 8949 Thread-13 downloader   INFO     Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=115343360 totalSize=2087665645 upload=18874368 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=55249]
[20/May/2019 19:51:41 +0000] 8949 Thread-13 downloader   INFO     Current state: CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel [totalDownloaded=116916224 totalSize=2087665645 upload=23068672 state=downloading seed=[] location=/opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel progress=56003]
......

下载成功之后，开始解压：

[20/May/2019 20:07:23 +0000] 8949 Thread-13 parcel_cache INFO     Unpacking /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel into /opt/cloudera/parcels

这些流程都是正常进行的，下载也是成功了的，但是发现在解压一段时间之后就抛出了异常：

[20/May/2019 20:41:43 +0000] 8949 Thread-13 throttling_logger ERROR    Error while unpacking /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel
Traceback (most recent call last):File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/parcel_cache.py", line 194, in cbunpack_parcel(op.path, unpack_dir, self.path)File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/parcel_cache.py", line 220, in unpack_parcelraise Exception("Untar failed with return code: %d, with tar output:\n stdout: [%s],\n stderr: [%s]" % (return_code, output[0], output[1]))
Exception: Untar failed with return code: 2, with tar output:stdout: [],stderr: [tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-release/kudu-tserver: Wrote only 2048 of 10240 bytes
tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-release/kudu-master: Cannot open: No space left on device
tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug: Cannot mkdir: No space left on device
tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug: Cannot mkdir: No space left on device
tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug/kudu-tserver: Cannot open: No such file or directory
tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug: Cannot mkdir: No space left on device
tar: CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu/sbin-debug/kudu-master: Cannot open: No such file or directory
......

提示磁盘空间不足o(╯□╰)o。通过df命令查看，果然磁盘空间不够了，最后通过清理磁盘解决。

注：由于parcel文件较大（将近两个G），agent从server下载下来后还需要解压，然后再激活，所以我们要保证充足的磁盘空间，特别是用虚拟机搭建集群的同学。

安装CDH出现file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel...does not exist相关推荐

CDH安装时报错：/opt/cloudera/parcels/.flood/.../CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel does not exist
原因:磁盘空间不足. 解决:优先查看节点的磁盘空间,根目录大小不要小于30G. 根目录空间扩容到30G 以上,(每台都要).
CDH 6.3.2部署flink 1.14.3出现/opt/cloudera/parcels/FLINK/lib/flink/bin/flink-yarn.sh:17: rotateLogFilesW
/opt/cloudera/parcels/FLINK/lib/flink/bin/flink-yarn.sh:17: rotateLogFilesW CDH 6.3.2部署flink 1.14. ...
Azkaban停留在 Logging initialized using configuration in jar:file:/application/cloudera/parcels/XXXXXX
现象 :Azkaban停留在 Logging initialized using configuration in jar:file:/application/cloudera/parcels/CDH ...
CDH集群安装配置（五）- Cloudera Manager Server
在线安装 sudo yum install cloudera-manager-daemons cloudera-manager-server 离线安装资源下载地址 https://archive.c ...
Cloudera Manager 和 CDH 4 终极安装
转载请注明出处:http://www.cnblogs.com/thinkCoding/p/3567408.html 系统环境操作系统:CentOS 6.5 Cloudera Manager 版本:4 ...
CentOS 7下Cloudera Manager及CDH 6.0.1安装教程
概念介绍 CDH 概览 CDH是Apache Hadoop和相关项目的最完整.最受测试和最流行的发行版.CDH提供Hadoop的核心元素-可伸缩存储和分布式计算-以及基于web的用户界面和重要的企业功 ...
Centos7 安装包方式（离线）安装cloudera manager 和 CDH
Centos7 环境下安装CDH 首先要安装cloudera manager 然后通过cm的图形界面来安装CDH 和一些相关组件需要安装mysql来存储cloudera manager的一些数据也 ...
Centos6.9 下的 CM（Cloudera Manager）CDH 大数据环境部署
(系统采用CentOS-6.9-x86_64-bin-DVD1.iso镜像,百度下载方式) 环境采用的是一台物理机ESXI 虚拟镜像系统下三台虚拟机 n1.n2.n3 (n1 是master&am ...
cdh的集成phoenix安装_环境篇：Kylin3.0.1集成CDH6.2.0
环境篇:Kylin3.0.1集成CDH6.2.0 Kylin是什么? Apache Kylin™是一个开源的.分布式的分析型数据仓库,提供Hadoop/Spark 之上的 SQL 查询接口及多维分析( ...

安装CDH出现file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel...does not exist

安装CDH出现file /opt/cloudera/parcels/.flood/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel...does not exist相关推荐

最新文章

热门文章