远程过程中出现的一些错误

Cannot delete /tmp/hadoop-yarn/staging/hadoop/.staging/job_1477796535608_0001. Name node is in safe mode.

上述问题解决：Linux集群中的namenode没有关闭safemode

2017-12-05 18:32:27,979 INFO  [main] mapred.ClientServiceDelegate (ClientServiceDelegate.java:getProxy(276)) - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history serverRetrying connect to server: 192.168.146.130/192.168.146.130:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)Exception in thread "main" java.io.IOException: java.net.ConnectException: Call From MSI/118.202.43.35 to 192.168.146.130:10020 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

上述问题解决：开启historyserver服务 mr-jobhistory-daemon.sh start historyserver

Exception in thread "main" java.io.IOException: Job status not available

上述问题解决：在mapred-site.xml中添加如下配置：

<property>   <name>yarn.app.mapreduce.am.staging-dir</name>   <value>/tmp/hadoop-yarn/staging</value></property> <property>   <name>mapreduce.jobhistory.intermediate-done-dir</name>   <value>${yarn.app.mapreduce.am.staging-dir}/history/done_intermediate</value> </property> <property>   <name>mapreduce.jobhistory.done-dir</name>   <value>${yarn.app.mapreduce.am.staging-dir}/history/done</value> </property>

**注意**：在本地PC可以访问虚拟机集群的
hdfs  监控：xxx.xxx.xxx.xxx:50070
mr job监控：xxx.xxx.xxx.xxx:19888

WebUI无权访问hdfs文件夹/tmpPermission denied: user=dr.who, access=READ_EXECUTE, inode="/tmp":hadoop:supergroup:drwx------

上述问题解决：hadoop dfs -chmod -R 755 /tmp
注意：其显示是弃用的方法，不过仍然有效

hadoop put 机制

特别注意！关于hdfs的底层原理(上传一个文件的整个历程)
一定要看这个文章、文章的备用连接

.wp-r{font-size: 20px!important;font-weight: 700!important;}