spark本地项目报错：Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

idea本地运行spark程序时出现winutils.exe报错

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:278)at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:300)at org.apache.hadoop.util.Shell.<clinit>(Shell.java:293)at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:447)at org.apache.spark.sql.execution.datasources.json.JsonFileFormat.createBaseRdd(JsonFileFormat.scala:131)at org.apache.spark.sql.execution.datasources.json.JsonFileFormat.inferSchema(JsonFileFormat.scala:64)at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$7.apply(DataSource.scala:184)at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$7.apply(DataSource.scala:184)at scala.Option.orElse(Option.scala:289)at org.apache.spark.sql.execution.datasources.DataSource.org$apache$spark$sql$execution$datasources$DataSource$$getOrInferFileFormatSchema(DataSource.scala:183)at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:387)at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:152)at org.apache.spark.sql.DataFrameReader.json(DataFrameReader.scala:298)at org.apache.spark.sql.DataFrameReader.json(DataFrameReader.scala:251)

原因：缺少winutils.exe程序。Hadoop都是运行在Linux系统下的，如果再Windows下运行需要安装一个插件

解决方案

下载 https://github.com/srccodes/hadoop-common-2.2.0-bin 到本地并解压

我这里将下载下来的压缩文件解压到了本地的hadoop目录(将文件解压重命名为hadoop也可)

配置环境变量

添加HADOOP_HOME

配置CLASSPATH

将;E:\hadoop-2.7.1\bin\winutils.exe添加到最后

修改Path
将%HADOOP_HOME%\bin 添加到Path中

最后重启电脑，再次执行报错即可消除。

spark本地项目报错：Could not locate executable null\bin\winutils.exe in the Hadoop binaries.相关推荐

【Spark】Spark报错：Could not locate executable null\bin\winutils.exe in the Hadoop binaries
IDEA集成Scala和Spark之后,运行demo可能会报错: java.io.IOException: Could not locate executable null\bin\winutils. ...
安装spark，Could not locate executable null\bin\winutils.exe in the Hadoop binaries报错
安装spark,Could not locate executable null\bin\winutils.exe in the Hadoop binaries报错缺少winutils.exe程序 ...
【Spark】Could not locate executable null\bin\winutils.exe in the Hadoop binaries
项目场景: 在ideal中运行spark项目进行数据存储文件时报错. 问题描述: 报错:Could not locate executable null\bin\winutils.exe in the ...
[Spark]Could not locate executable null\bin\winutils.exe in the Hadoop binaries
这里集群环境不同,但遇到的问题及解决办法其实一样,时间有限,就不详细修改了.我主要是在Windows本地运行Spark程序出现该错误. 本文转自http://www.cnblogs.com/hyl82 ...
Could not locate executable null\bin\winutils.exe in the Hadoop binaries全网最强windows10安装hadoop教程
简述详细安装配置步骤 1.安装jdk 2.下载安装hadoop 2.1 下载 2.2 解压安装 2.3 配置环境变量 2.4 配置cmd与xml文件设置完毕启动hadoop 简述报错内容:Cou ...
解决Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
在Windows环境下使用IDEA.eclipse运行hadoop.spark程序时产生如下异常: 1.java.io.IOException: Could not locate executable ...
IDEA启动hadoop报Could not locate executable null\bin\winutils.exe in the Hadoop binaries.错误的解决办法
打开了一个hadoop项目,项目代码确定无误可运行,但是本地运行报错如下仔细看一下报错路径,显示的是null\bin\winutils.exe 而我的winutils已经下载下来并且复制到bin目录 ...
源码追踪，解决Could not locate executable null\bin\winutils.exe in the Hadoop binaries.问题
在windows系统本地运行spark的wordcount程序,会出现一个异常,但不影响现有程序运行. >>提君博客原创 http://www.cnblogs.com/tijun/ & ...
【收藏】Hadoop解决Could not locate executable null\bin\winutils.exe in the Hadoop binaries.问题
https://www.cnblogs.com/tijun/p/7567664.html
bug-hadoop:Could not locate executable null\bin\winutils.exe in the Hadoop binaries
原因:缺少winutils.exe程序 Hadoop都是运行在Linux系统下的,在windows下eclipse中运行mapreduce程序,要首先安装Windows下运行的支持插件下载: htt ...

spark本地项目报错：Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

idea本地运行spark程序时出现winutils.exe报错

解决方案

spark本地项目报错：Could not locate executable null\bin\winutils.exe in the Hadoop binaries.相关推荐

最新文章

热门文章