问题详情

　 电脑8G，目前搭建3节点的spark集群，采用YARN模式。

master分配2G，slave1分配1G,slave2分配1G。（在安装虚拟机时）

export SPARK_WORKER_MERMORY=1g  （在spark-env.sh）

export JAVA_HOME=/usr/local/jdk/jdk1.8.0_60         （必须写）
export SCALA_HOME=/usr/local/scala/scala-2.10.5    （必须写）
export HADOOP_HOME=/usr/local/hadoop/hadoop-2.6.0    （必须写）
export HADOOP_CONF_DIR=/usr/local/hadoop/hadoop-2.6.0/etc/hadoop   （必须写）
export SPARK_MASTER_IP=192.168.80.10　　
export SPARK_WORKER_MERMORY=1G     （官网上说，至少1g）

　其实这个问题解决办法很简单，就是3个节点的是，你的内存最好还是大点。如master分配4G，slave1分配2G,slave2分配2G！！！！（尽可能的大）

当然，很多博友们，跟我的情况是一样的，在学校阶段，电脑只有8g已经是最大电脑内存限制了。

一般是由于有多个用户同时向集群提交任务或一个用户向集群同时提交了多个任务导致Yarn资源的分配错误。解决这个问题，只需要更改Hadoop的配置文件：/etc/hadoop/conf/capacity-scheduler.xml，把选项：yarn.scheduler.capacity.maximum-am-resource-percent从0.1改成0.5。顾名思义，这个选项是增加Yarn可调度的资源量，当然也可以视具体情况增加更多。也可见，默认情况下，Yarn没有将很多资源分配给任务的能力。

[spark@master logs]$  $SPARK_HOME/bin/spark-submit  \
> --class org.apache.spark.examples.JavaSparkPi \
> --master yarn-cluster \
> --num-executors 1 \
> --driver-memory 512m \
> --executor-memory 512m \
> --executor-cores 1 \
>  /usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar注意：
driver-memory不指定也可以，默认使用512M
executor-memory不指定的化， 默认是1G

17/04/09 17:03:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/04/09 17:03:55 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.80.10:8032
17/04/09 17:03:56 INFO yarn.Client: Requesting a new application from cluster with 2 NodeManagers
17/04/09 17:03:56 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (8192 MB per container)
17/04/09 17:03:56 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/04/09 17:03:56 INFO yarn.Client: Setting up container launch context for our AM
17/04/09 17:03:56 INFO yarn.Client: Setting up the launch environment for our AM container
17/04/09 17:03:56 INFO yarn.Client: Preparing resources for our AM container
17/04/09 17:03:59 INFO yarn.Client: Uploading resource file:/usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar -> hdfs://master:9000/user/spark/.sparkStaging/application_1491728358337_0001/spark-assembly-1.6.1-hadoop2.6.0.jar
17/04/09 17:04:19 INFO yarn.Client: Uploading resource file:/usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar -> hdfs://master:9000/user/spark/.sparkStaging/application_1491728358337_0001/spark-examples-1.6.1-hadoop2.6.0.jar
17/04/09 17:04:49 INFO yarn.Client: Uploading resource file:/tmp/spark-d152ed1b-09ca-47c8-8457-58f7e52ff419/__spark_conf__6499474209714260387.zip -> hdfs://master:9000/user/spark/.sparkStaging/application_1491728358337_0001/__spark_conf__6499474209714260387.zip
17/04/09 17:04:50 INFO spark.SecurityManager: Changing view acls to: spark
17/04/09 17:04:50 INFO spark.SecurityManager: Changing modify acls to: spark
17/04/09 17:04:50 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spark); users with modify permissions: Set(spark)
17/04/09 17:04:50 INFO yarn.Client: Submitting application 1 to ResourceManager
17/04/09 17:04:51 INFO impl.YarnClientImpl: Submitted application application_1491728358337_0001
17/04/09 17:04:52 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:52 INFO yarn.Client: client token: N/Adiagnostics: N/AApplicationMaster host: N/AApplicationMaster RPC port: -1queue: defaultstart time: 1491728691236final status: UNDEFINEDtracking URL: http://master:8088/proxy/application_1491728358337_0001/user: spark
17/04/09 17:04:53 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:54 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:55 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:56 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:57 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:58 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:59 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:00 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:01 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:02 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:03 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:04 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:05 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:06 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:07 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
^C17/04/09 17:05:08 INFO util.ShutdownHookManager: Shutdown hook called
17/04/09 17:05:08 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-d152ed1b-09ca-47c8-8457-58f7e52ff419.........
然后最后失败

问题分析

　　　　其实，这一情况，是我刚开始启动了太多的Application导致累积，使得内存超出了导致！

解决方法

　　　　按照办法1，办法2，办法3...的顺序来

 办法1:（一般能解决）

　　先，

[spark@master ~]$ cd $SPARK_HOME
[spark@master spark-1.6.1-bin-hadoop2.6]$ sbin/stop-all.sh

[spark@master spark-1.6.1-bin-hadoop2.6]$ cd $HADOOP_HOME
[spark@master hadoop-2.6.0]$ sbin/stop-all.sh

再，

[spark@master spark-1.6.1-bin-hadoop2.6]$ cd $HADOOP_HOME
[spark@master hadoop-2.6.0]$ sbin/start-all.sh

[spark@master ~]$ cd $SPARK_HOME
[spark@master spark-1.6.1-bin-hadoop2.6]$ sbin/start-all.sh

在spark-env.sh，至少得

export SPARK_WORKER_MERMORY=1G

　　改成

[spark@master spark-1.6.1-bin-hadoop2.6]$ $SPARK_HOME/bin/spark-submit \
--class org.apache.spark.examples.JavaSparkPi \
--master yarn-cluster \
--num-executors 1 \
--driver-memory 1g \
--executor-memory 1g \
--executor-cores 1 \
$SPARK_HOME/lib/spark-examples-1.6.1-hadoop2.6.0.jar[spark@master spark-1.6.1-bin-hadoop2.6]$ $SPARK_HOME/bin/spark-submit --class org.apache.spark.examples.JavaSparkPi --master yarn-cluster --num-executors 1 --driver-memory 1g --executor-memory 1g --executor-cores 1 $SPARK_HOME/lib/spark-examples-1.6.1-hadoop2.6.0.jar17/04/09 20:01:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/04/09 20:01:27 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.80.10:8032
17/04/09 20:01:31 INFO yarn.Client: Requesting a new application from cluster with 2 NodeManagers
17/04/09 20:01:32 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (8192 MB per container)
17/04/09 20:01:32 INFO yarn.Client: Will allocate AM container, with 1408 MB memory including 384 MB overhead
17/04/09 20:01:32 INFO yarn.Client: Setting up container launch context for our AM
17/04/09 20:01:32 INFO yarn.Client: Setting up the launch environment for our AM container
17/04/09 20:01:32 INFO yarn.Client: Preparing resources for our AM container
17/04/09 20:01:37 INFO yarn.Client: Uploading resource file:/usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar -> hdfs://master:9000/user/spark/.sparkStaging/application_1491735225663_0006/spark-assembly-1.6.1-hadoop2.6.0.jar
17/04/09 20:01:57 INFO yarn.Client: Uploading resource file:/usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar -> hdfs://master:9000/user/spark/.sparkStaging/application_1491735225663_0006/spark-examples-1.6.1-hadoop2.6.0.jar
17/04/09 20:02:18 INFO yarn.Client: Uploading resource file:/tmp/spark-8d8662b1-6410-4b72-9751-57bb0e2785d0/__spark_conf__5191259430263541003.zip -> hdfs://master:9000/user/spark/.sparkStaging/application_1491735225663_0006/__spark_conf__5191259430263541003.zip
17/04/09 20:02:18 INFO spark.SecurityManager: Changing view acls to: spark
17/04/09 20:02:19 INFO spark.SecurityManager: Changing modify acls to: spark
17/04/09 20:02:19 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spark); users with modify permissions: Set(spark)
17/04/09 20:02:22 INFO yarn.Client: Submitting application 6 to ResourceManager
17/04/09 20:02:23 INFO impl.YarnClientImpl: Submitted application application_1491735225663_0006
17/04/09 20:02:25 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:25 INFO yarn.Client: client token: N/Adiagnostics: N/AApplicationMaster host: N/AApplicationMaster RPC port: -1queue: defaultstart time: 1491739342315final status: UNDEFINEDtracking URL: http://master:8088/proxy/application_1491735225663_0006/user: spark
17/04/09 20:02:26 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:32 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:35 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:36 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:37 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:38 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:39 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:42 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:43 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:44 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:45 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:46 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:47 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:48 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:49 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:50 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:51 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:53 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:54 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:55 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:56 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:57 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:58 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:59 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:00 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:01 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:02 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:04 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)17/04/09 20:03:05 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:06 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:07 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:08 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:09 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:10 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:11 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:12 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:13 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:14 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:15 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:16 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:17 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:18 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:19 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:20 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:21 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:23 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:24 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:25 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:26 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:27 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:28 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:29 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:30 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:31 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:32 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:33 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:34 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:36 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:38 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:39 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:40 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:41 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:42 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:43 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:44 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:45 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:46 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:47 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:48 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:50 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:51 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:52 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:53 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:54 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:55 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:56 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:57 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:58 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:59 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:00 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:01 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:02 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:03 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:04 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:05 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:06 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:07 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:08 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:09 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:10 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:11 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:12 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:13 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:15 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:16 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:17 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:18 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:19 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:20 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:21 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:22 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:23 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:24 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:25 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:26 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:27 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:28 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:29 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:30 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:31 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:32 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:33 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:34 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:35 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:36 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:37 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:38 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:39 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:40 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:41 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:43 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:44 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:45 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:46 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:47 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:48 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:49 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:50 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:51 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:52 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:53 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)   只要最后出现这个了，就说明成功了！
17/04/09 20:04:53 INFO yarn.Client: client token: N/Adiagnostics: N/AApplicationMaster host: 192.168.80.11ApplicationMaster RPC port: 0queue: defaultstart time: 1491739342315final status: UNDEFINEDtracking URL: http://master:8088/proxy/application_1491735225663_0006/user: spark
17/04/09 20:04:54 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:55 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:56 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:57 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:58 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:59 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:00 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:01 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:02 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:04 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:05 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:06 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:07 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:08 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:09 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:10 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:11 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:12 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:13 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:14 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:15 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:16 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:17 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:19 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:20 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:21 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:22 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:23 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:25 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:26 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:27 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:28 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:29 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:30 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:31 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:32 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:33 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:34 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:35 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:36 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:37 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:38 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:39 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:40 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:41 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:42 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:43 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:44 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:45 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:46 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:47 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
....
17/04/09 20:06:57 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:06:58 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:06:59 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:00 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:01 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:02 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:03 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:04 INFO yarn.Client: Application report for application_1491735225663_0006 (state: FINISHED)
17/04/09 20:07:04 INFO yarn.Client: client token: N/Adiagnostics: N/AApplicationMaster host: 192.168.80.11ApplicationMaster RPC port: 0queue: defaultstart time: 1491739342315final status: SUCCEEDEDtracking URL: http://master:8088/proxy/application_1491735225663_0006/Auser: spark
17/04/09 20:07:04 INFO yarn.Client: Deleting staging directory .sparkStaging/application_1491735225663_0006
17/04/09 20:07:07 INFO util.ShutdownHookManager: Shutdown hook called
17/04/09 20:07:07 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-8d8662b1-6410-4b72-9751-57bb0e2785d0
[spark@master spark-1.6.1-bin-hadoop2.6]$

　　如果之后，再次去执行别的jar包，出现又是无法运行了，则

spark on yarn模式里需要有时手工释放linux内存

然后执行

[spark@master spark-1.6.1-bin-hadoop2.6]$ su root
Password:
[root@master spark-1.6.1-bin-hadoop2.6]# echo 1 > /proc/sys/vm/drop_caches
[root@master spark-1.6.1-bin-hadoop2.6]# echo 2 > /proc/sys/vm/drop_caches
[root@master spark-1.6.1-bin-hadoop2.6]# echo 3 > /proc/sys/vm/drop_caches

方法2： 首先想到是集群中内存资源不足，可以检查下每台机器是否有足够剩余内存( free -g)；也可能是其他已经提交的Spark应用占了大部分资源；

[spark@master spark-1.6.1-bin-hadoop2.6]$ free -gtotal       used       free     shared    buffers     cached
Mem:             1          1          0          0          0          0
-/+ buffers/cache:          0          0
Swap:            1          0          1
[spark@master spark-1.6.1-bin-hadoop2.6]$

　　那么，就要去杀死之前那些还正在运行（多余没用）的Application。

　　怎么做？

　　进入你的$HADOOP_HOME/bin下

执行命令

[spark@master bin]$ yarn application -list

然后可以看到应用的信息

如需杀死当前某个作业，使用kill application-id的命令如下

yarn application -kill application_1437456051228_1725

方法3： 如果1>正常，我们可以看看YARN集群是否启动成功。注意“坑”可能就在这里： 即使Slave上的nodemanager进程存在，要注意检查resource manager日志，看看各个node manager是否启动成功，有些人的问题就出现在这里：进程在，但是日志显示node manager状态为UNHEALTHY，所以YARN集群能识别到的总内存资源为0。检查了UNHEALTHY的原因，是因为/tmp下一个目录被识别为bad, 因为是临时目录，把每个node manager的对应目录删掉，然后重启YARN集群，最终问题解决。

ResourceManager日志存放位置是${HADOOP_HOME}/logs/yarn-*-resourcemanager-*.log

NodeManager日志存放位置是各个NodeManager节点上${HADOOP_HOME}/logs/yarn-*-nodemanager-*.log

　　先，

[spark@master ~]$ cd $SPARK_HOME
[spark@master spark-1.6.1-bin-hadoop2.6]$ sbin/stop-all.sh

[spark@master spark-1.6.1-bin-hadoop2.6]$ cd $HADOOP_HOME
[spark@master hadoop-2.6.0]$ sbin/stop-all.sh

　　再，

[spark@master spark-1.6.1-bin-hadoop2.6]$ cd $HADOOP_HOME
[spark@master hadoop-2.6.0]$ sbin/start-all.sh

[spark@master ~]$ cd $SPARK_HOME
[spark@master spark-1.6.1-bin-hadoop2.6]$ sbin/start-all.sh

方法4：如果还是有问题的话，只能去日志文件中查看出错的异常，查看更多一些的报错信息。

　　　　　　但是，方法1和方法2是足够可以解决该问题的!

方法5：

　　　　除非你的内存足够，可以扩大内存。我的只是自己电脑最大才8G

方法6：

　　在yarn-site.xml下

<property><name>yarn.nodemanager.pmem-check-enabled</name><value>false</value>
</property>
<property><name>yarn.nodemanager.vmem-check-enabled</name><value>false</value>
</property>

spark yarn模式提交任务不成功（application state: ACCEPTED）相关推荐

  1. 关于spark yarn模式下的常用属性

    前言 整理了spark官网提供的一些常用的spark属性. Spark属性 属性名 默认值 描述 spark.yarn.am.memory 512m 在Client模式下用于YARN Applicat ...

  2. flink yarn模式提交及查看日志

    一.Yarn session(一般测试环境) yarn session会初始启动指定的tm数量.  job提交后再指定的session 内运行. 其它job运行,如果资源不够,就会一直等待直到占用的j ...

  3. Java代码使用Spark on Yarn 方式提交任务到带Kerberos认证的Hadoop集群

    2019独角兽企业重金招聘Python工程师标准>>> 项目中遇到Spark Yarn方式提交到Hadoop集群,访问集群HDFS时发现使用的当前用户,没有访问权限,经过排查后发现H ...

  4. spark执行优化——依赖上传到HDFS二(-conf spark.yarn.dist.jars或者--jars 的使用)

    1.说明 之前整理过一篇类似文章,但是这个spark.yarn.jar配置的目录最好只是放spark jars目录下的jar包,如果放入其他的jar包,很大概率会有冲突,而且如果项目比较多,jar包引 ...

  5. Spark通过YARN提交任务不成功（包含YARN cluster和YARN client)

    无论用YARN cluster和YARN client来跑,均会出现如下问题. [spark@master spark-1.6.1-bin-hadoop2.6]$ jps 2049 NameNode ...

  6. Spark Standalone -- 独立集群模式、Spark 提交任务的两种模式、spark在yarn上运行的环境搭建、自己写的spark代码如何提交到yarn上并运行...

    目录 Spark Standalone -- 独立集群模式 Standalone 架构图 Standalone 的搭建 1.上传.解压.重命名 2.配置环境变量 3.修改配置文件 conf 4.同步到 ...

  7. Spark _05Standalone模式两种提交任务方式

    Standalone模式两种提交任务方式 Standalone-client提交任务方式 提交命令 ./spark-submit --master spark://node1:7077 --class ...

  8. 详解Spark运行模式（local+standalone+yarn）

    一.简述 Spark 有多种运行模式: 1.可以运行在一台机器上,称为 Local(本地)运行模式. 2.可以使用 Spark 自带的资源调度系统,称为 Standalone 模式. 3.可以使用 Y ...

  9. Spark内核（上）——附：两种Yarn模式源码解析

    文章目录 一.Spark内核概述 1.1 Spark核心组件回顾 1.1.1 Driver 1.1.2 Executor 1.2 Spark通用运行流程概述 二.Spark通信架构概述 2.1 Spa ...

最新文章

  1. rtklib 天线相位中心_基于传输线匹配理论的OAM透射阵天线设计
  2. pandas loc iloc用法
  3. 初探Vue之环境搭建
  4. leetcode_linearList03
  5. 2018.03.03、android-照虎画猫搭建简易Rest服务器
  6. 功能式Python中的探索性数据分析
  7. Linux 查看电脑基本配置
  8. mysql数据意外删了怎么办_MySQL数据库意外崩溃导致表数据文件损坏无法启动怎么办...
  9. c语言一级考试题目第四季度,模拟试题8套
  10. es6 调用 Iterator 接口的场合
  11. 【图像处理】形态学及其它集合运算（Morphological and Other Set Operations）
  12. MyBatis的一级缓存、二级缓存演示以及讲解，序列化异常的处理
  13. Unity 2D Skeletal Animation
  14. 图数据集之cora数据集介绍- 用pyton处理 - 可用于GCN任务
  15. python中的snip用法_腾讯mac截图软件Snip使用教程
  16. pycharm上传GitHub时报错：push failed: fatal: unable to access 'https://github.com/***port 443: Timed out
  17. Avalonia的Snoop
  18. Unity 性能优化之合批
  19. java接入短信_java接入创蓝253短信验证码的实例讲解
  20. 温州兰博基尼车主遭绑架案1名嫌犯获死刑-兰博基尼-女车主-绑架

热门文章

  1. 微软云计算产品详解 主打Windows
  2. 【HCIA-cloud】【1】云计算的定义、什么是云计算、云计算的架构与技术说明、华为云计算产品、华为内存DDR配置工具说明
  3. 三维重建 医学图像的可视化 PyQt Python VTK 四视图（横断面，冠状面，矢状面，3D）
  4. 华为RS2 传输介质
  5. 微信小程序黑马优购遇到的问题和心得
  6. java笔试题---用*打印漏斗形，信雅达笔试题
  7. 机器人教育的中心地段
  8. 以太坊POA共识算法解析
  9. 数据分析模型-(RFM,5W2H,AARRR,PEST,4P)
  10. redis实现计时器