check your cluster UI to ensure that workers are registered and have sufficient resources

完整报错如下:

WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

------------------------------------------------------------------------------------------------------------------------------------------------------------

①访问http://master:8080/

点击下面的Running Applications

②选择下面RUNNING中的stderr

得到下面的报错:

java.io.InvalidClassException: scala.collection.mutable.WrappedArray$ofRef; local class incompatible: stream classdesc serialVersionUID = 3456489343829468865, local class serialVersionUID = 1028182004549731694at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616)at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:76)at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:109)at org.apache.spark.rpc.netty.NettyRpcEnv.$anonfun$deserialize$2(NettyRpcEnv.scala:292)at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)at org.apache.spark.rpc.netty.NettyRpcEnv.deserialize(NettyRpcEnv.scala:345)at org.apache.spark.rpc.netty.NettyRpcEnv.$anonfun$deserialize$1(NettyRpcEnv.scala:291)at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)at org.apache.spark.rpc.netty.NettyRpcEnv.deserialize(NettyRpcEnv.scala:291)at org.apache.spark.rpc.netty.NettyRpcEnv.$anonfun$askAbortable$7(NettyRpcEnv.scala:246)at org.apache.spark.rpc.netty.NettyRpcEnv.$anonfun$askAbortable$7$adapted(NettyRpcEnv.scala:246)at org.apache.spark.rpc.netty.RpcOutboxMessage.onSuccess(Outbox.scala:90)at org.apache.spark.network.client.TransportResponseHandler.handle(TransportResponseHandler.java:194)at org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:142)at org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:53)at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:102)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)at java.lang.Thread.run(Thread.java:748)

最终办法:

jps检查两个节点是否都有启动worker

修改File->Project Structure->Libraries中的Scala SDK为自己的$SCALA_HOME。

mvn clean

mvn package

check your cluster UI to ensure that workers are registered and have sufficient resources相关推荐

Scheduler: Initial job has not accepted any resources； check your cluster UI to ensure that workers
今天试着用Spark on YARN集群提交一个任务,读取的文件路径是本地的,结果报了这个错误. Scheduler: Initial job has not accepted any resourc ...
WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure
1:遇到这个问题是在启动bin/spark-shell以后,然后呢,执行spark实现wordcount的例子的时候出现错误了,如: scala> sc.textFile("hdfs: ...
redis集群添加节点报错Either the node already knows other nodes (check with CLUSTER NODES) or contains some k
[README] redis集群添加节点报错 [ERR] Node 192.168.163.202:6380 is not empty. Either the node already knows o ...
pyspark汇总小结
20220402 Spark报Total size of serialized results of 12189 tasks is bigger than spark.driver.maxResult ...
spark 序列化错误集群提交时_【问题解决】本地提交任务到Spark集群报错：Initial job has not accepted any resources...
本地提交任务到Spark集群报错:Initial job has not accepted any resources 错误信息如下: 18/04/17 18:18:14 INFO TaskSched ...
Spark源码阅读02-Spark核心原理之作业执行原理
概述 Spark的作业调度主要是指基于RDD的一系列操作构成的一个作业,在Executor中执行的过程.其中,在Spark作业调度中最主要的是DAGScheduler和TaskScheduler两个调 ...
【数据平台】Eclipse+Scala远程开发调试关于hostname的问题
1.代码: import org.apache.spark.SparkConf import org.apache.spark.SparkContextobject wc {def main(args ...
深入理解Spark 2.1 Core （三）：任务调度器的原理与源码分析
提交Task 调用栈如下: TaskSchedulerImpl.submitTasks CoarseGrainedSchedulerBackend.reviveOffers CoarseGrained ...
Spark修炼之道（高级篇）——Spark源码阅读：第六节 Task提交
Task提交在上一节中的 Stage提交中我们提到,最终stage被封装成TaskSet,使用taskScheduler.submitTasks提交,具体代码如下: taskScheduler.su ...

check your cluster UI to ensure that workers are registered and have sufficient resources

check your cluster UI to ensure that workers are registered and have sufficient resources相关推荐

最新文章

热门文章