java连接hadoop报错

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

原因查找

找到报错位置hadoop包中的xxx/Shell.java

public static final String getQualifiedBinPath(String executable)
throws IOException {// construct hadoop bin path to the specified executableString fullExeName = HADOOP_HOME_DIR + File.separator + "bin" + File.separator + executable;File exeFile = new File(fullExeName);if (!exeFile.exists()) {throw new IOException("Could not locate executable " + fullExeName+ " in the Hadoop binaries.");}return exeFile.getCanonicalPath();
}

发现需要HADOOP_HOME_DIR变量

private static String HADOOP_HOME_DIR = checkHadoopHome();
private static String checkHadoopHome() {// first check the Dflag hadoop.home.dir with JVM scopeString home = System.getProperty("hadoop.home.dir");// fall back to the system/user-global env variableif (home == null) {home = System.getenv("HADOOP_HOME");}
……return home;
}

再次查找发现通过System.getProperty("hadoop.home.dir")或者环境变量HADOOP_HOME获取

故两种处理方式

1.直接设置属性System.setProperty("hadoop.home.dir", "D:\\Program\\hadoop-common-2.6.0-bin-master");

2.设置环境变量HADOOP_HOME

下载winutils地址https://github.com/srccodes/hadoop-common-2.2.0-bin下载解压

或者新版的https://github.com/amihalik/hadoop-common-2.6.0-bin

……感谢两位的编译

配置环境变量

增加用户变量HADOOP_HOME，值是下载的zip包解压的目录，然后在系统变量path里增加%HADOOP_HOME%\bin 即可。　　

连接hadoop java.io.IOException:Could not locate executable null\bin\winutils.exe in the Hadoop binarie相关推荐

【Spark】Spark报错：Could not locate executable null\bin\winutils.exe in the Hadoop binaries
IDEA集成Scala和Spark之后,运行demo可能会报错: java.io.IOException: Could not locate executable null\bin\winutils. ...
[Spark]Could not locate executable null\bin\winutils.exe in the Hadoop binaries
这里集群环境不同,但遇到的问题及解决办法其实一样,时间有限,就不详细修改了.我主要是在Windows本地运行Spark程序出现该错误. 本文转自http://www.cnblogs.com/hyl82 ...
Could not locate executable null\bin\winutils.exe in the Hadoop binaries全网最强windows10安装hadoop教程
简述详细安装配置步骤 1.安装jdk 2.下载安装hadoop 2.1 下载 2.2 解压安装 2.3 配置环境变量 2.4 配置cmd与xml文件设置完毕启动hadoop 简述报错内容:Cou ...
【Spark】Could not locate executable null\bin\winutils.exe in the Hadoop binaries
项目场景: 在ideal中运行spark项目进行数据存储文件时报错. 问题描述: 报错:Could not locate executable null\bin\winutils.exe in the ...
安装spark，Could not locate executable null\bin\winutils.exe in the Hadoop binaries报错
安装spark,Could not locate executable null\bin\winutils.exe in the Hadoop binaries报错缺少winutils.exe程序 ...
解决Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
在Windows环境下使用IDEA.eclipse运行hadoop.spark程序时产生如下异常: 1.java.io.IOException: Could not locate executable ...
IDEA启动hadoop报Could not locate executable null\bin\winutils.exe in the Hadoop binaries.错误的解决办法
打开了一个hadoop项目,项目代码确定无误可运行,但是本地运行报错如下仔细看一下报错路径,显示的是null\bin\winutils.exe 而我的winutils已经下载下来并且复制到bin目录 ...
源码追踪，解决Could not locate executable null\bin\winutils.exe in the Hadoop binaries.问题
在windows系统本地运行spark的wordcount程序,会出现一个异常,但不影响现有程序运行. >>提君博客原创 http://www.cnblogs.com/tijun/ & ...
bug-hadoop:Could not locate executable null\bin\winutils.exe in the Hadoop binaries
原因:缺少winutils.exe程序 Hadoop都是运行在Linux系统下的,在windows下eclipse中运行mapreduce程序,要首先安装Windows下运行的支持插件下载: htt ...

连接hadoop java.io.IOException:Could not locate executable null\bin\winutils.exe in the Hadoop binarie

原因查找

连接hadoop java.io.IOException:Could not locate executable null\bin\winutils.exe in the Hadoop binarie相关推荐

最新文章

热门文章