java.net.UnknownHostException: dbmtimehadoop

完整报错如下:

$ spark-submit --class WordCountLocal --master yarn --deploy-mode cluster --driver-memory 1g --executor-memory 1g --executor-cores 1 /home/appleyuchi/IdeaProjects/scala-learn/target/scala-learn-1.0-SNAPSHOT.jar
Exception in thread "main" java.lang.IllegalArgumentException: java.net.UnknownHostException: dbmtimehadoop
   at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378)
   at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
   at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
   at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
   at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
   at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
   at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
   at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
   at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:528)
   at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:525)
   at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
   at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:525)
   at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:869)
   at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:169)
   at org.apache.spark.deploy.yarn.Client.run(Client.scala:1152)
   at org.apache.spark.deploy.yarn.YarnClusterApplication.start(Client.scala:1520)
   at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
   at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
   at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
   at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
   at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.UnknownHostException: dbmtimehadoop
   ... 24 more

摸索的思路:

①搜到[1]中发现这么个信息:

--conf spark.eventLog.dir=hdfs://dbmtimehadoop/tmp/spark2 \

②猜测是spark.eventLog.dir没设置的关系

最终解决方案:

hdfs dfs -mkdir /logs
编辑conf/spark-defaults.conf内容如下:

spark.master                     spark://Desktop:9000
spark.eventLog.enabled           true
spark.eventLog.dir               hdfs://Desktop:9000/logs

Reference:

[1]spark任务提交到yarn上命令总结

java.net.UnknownHostException: dbmtimehadoop相关推荐

解决Linux中java.net.UnknownHostException: oracledb.sys.iflashbuy.com问题
Linux环境报java.net.UnknownHostException: oracledb.sys.iflashbuy.com,原因为Linux下无法解析oracledb.sys.iflashbu ...
Caused by: java.net.UnknownHostException: localhost.localdomain: localhost.localdomain的问题解决
Caused by: java.net.UnknownHostException: localhost.localdomain: localhost.localdomain的问题解决参考文章: (1 ...
java.net.UnknownHostException: unknown host:xxxx异常解决办法
java.net.UnknownHostException: unknown host:xxxx异常解决办法参考文章: (1)java.net.UnknownHostException: unkno ...
struts2 的错误信息： java.net.UnknownHostException: struts.apache.org
http://4925054.blog.51cto.com/4915054/1148538 错误: aused by: struts.apache.org - [unknown location] ...
hadoop启动碰到java.net.UnknownHostException
STARTUP_MSG: host = java.net.UnknownHostException: appleyuchi: appleyuchi: 未知的名稱或服務解决方案: 1.ubuntu的h ...
Linux调用外部服务出现java.net.UnknownHostException
windows下调用某外部服务的接口正常,发布到测试环境中的linux下,短信服务接口.验证码接口齐刷刷的出现了UnknownHostException, java.net.UnknownHostEx ...
DNS解释问题:java.net.UnknownHostException
javax.naming.CommunicationException [Root exception is java.rmi.UnknownHostException: Unknown host: ...
java.net.UnknownHostException 未知的名称或服务
在服务器上部署jar包或者war包时遇见类似:"**java.net.UnknownHostException 未知的名称或服务"**这个的错误,百思不得其解,总以为是项目代码里面 ...
一脸懵逼加从入门到绝望学习hadoop之Caused by: java.net.UnknownHostException: master报错
windows下开发hadoop应用程序,hadoop部署在linux环境中, 在运行调试时可能会出现无法找到主机,类似异常信息如下: java.net.UnknownHostException: u ...

java.net.UnknownHostException: dbmtimehadoop

java.net.UnknownHostException: dbmtimehadoop相关推荐

最新文章

热门文章