YARN-2.7.3-源码分析系列2：启动脚本原理的分析

2019独角兽企业重金招聘Python工程师标准>>>

上一节，我们把二进制和源码都下载好了，这一把，我们来启动脚本的研究，看看究竟是如何把RM和NM启动起来的！

-------------启动脚本是 ./sbin/start-yarn.sh 那就让我们来分析一下

不重要的不说，我们主要看yarn这个文件里核心的


# figure out which class to run
if [ "$COMMAND" = "classpath" ] ; thenif [ "$#" -gt 0 ]; thenCLASS=org.apache.hadoop.util.Classpathelseif $cygwin; thenCLASSPATH=$(cygpath -p -w "$CLASSPATH" 2>/dev/null)fiecho $CLASSPATHexit 0fi
elif [ "$COMMAND" = "rmadmin" ] ; thenCLASS='org.apache.hadoop.yarn.client.cli.RMAdminCLI'YARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"
elif [ "$COMMAND" = "scmadmin" ] ; thenCLASS='org.apache.hadoop.yarn.client.SCMAdmin'YARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"
elif [ "$COMMAND" = "application" ] || [ "$COMMAND" = "applicationattempt" ] || [ "$COMMAND" = "container" ]; thenCLASS=org.apache.hadoop.yarn.client.cli.ApplicationCLIYARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"set -- $COMMAND $@
elif [ "$COMMAND" = "node" ] ; thenCLASS=org.apache.hadoop.yarn.client.cli.NodeCLIYARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"
elif [ "$COMMAND" = "queue" ] ; thenCLASS=org.apache.hadoop.yarn.client.cli.QueueCLIYARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"
elif [ "$COMMAND" = "resourcemanager" ] ; thenCLASSPATH=${CLASSPATH}:$YARN_CONF_DIR/rm-config/log4j.propertiesCLASS='org.apache.hadoop.yarn.server.resourcemanager.ResourceManager'YARN_OPTS="$YARN_OPTS $YARN_RESOURCEMANAGER_OPTS"if [ "$YARN_RESOURCEMANAGER_HEAPSIZE" != "" ]; thenJAVA_HEAP_MAX="-Xmx""$YARN_RESOURCEMANAGER_HEAPSIZE""m"fi
elif [ "$COMMAND" = "historyserver" ] ; thenecho "DEPRECATED: Use of this command to start the timeline server is deprecated." 1>&2echo "Instead use the timelineserver command for it." 1>&2CLASSPATH=${CLASSPATH}:$YARN_CONF_DIR/ahs-config/log4j.propertiesCLASS='org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer'YARN_OPTS="$YARN_OPTS $YARN_HISTORYSERVER_OPTS"if [ "$YARN_HISTORYSERVER_HEAPSIZE" != "" ]; thenJAVA_HEAP_MAX="-Xmx""$YARN_HISTORYSERVER_HEAPSIZE""m"fi
elif [ "$COMMAND" = "timelineserver" ] ; thenCLASSPATH=${CLASSPATH}:$YARN_CONF_DIR/timelineserver-config/log4j.propertiesCLASS='org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer'YARN_OPTS="$YARN_OPTS $YARN_TIMELINESERVER_OPTS"if [ "$YARN_TIMELINESERVER_HEAPSIZE" != "" ]; thenJAVA_HEAP_MAX="-Xmx""$YARN_TIMELINESERVER_HEAPSIZE""m"fi
elif [ "$COMMAND" = "sharedcachemanager" ] ; thenCLASSPATH=${CLASSPATH}:$YARN_CONF_DIR/scm-config/log4j.propertiesCLASS='org.apache.hadoop.yarn.server.sharedcachemanager.SharedCacheManager'YARN_OPTS="$YARN_OPTS $YARN_SHAREDCACHEMANAGER_OPTS"if [ "$YARN_SHAREDCACHEMANAGER_HEAPSIZE" != "" ]; thenJAVA_HEAP_MAX="-Xmx""$YARN_SHAREDCACHEMANAGER_HEAPSIZE""m"fi
elif [ "$COMMAND" = "nodemanager" ] ; thenCLASSPATH=${CLASSPATH}:$YARN_CONF_DIR/nm-config/log4j.propertiesCLASS='org.apache.hadoop.yarn.server.nodemanager.NodeManager'YARN_OPTS="$YARN_OPTS -server $YARN_NODEMANAGER_OPTS"if [ "$YARN_NODEMANAGER_HEAPSIZE" != "" ]; thenJAVA_HEAP_MAX="-Xmx""$YARN_NODEMANAGER_HEAPSIZE""m"fi
elif [ "$COMMAND" = "proxyserver" ] ; thenCLASS='org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer'YARN_OPTS="$YARN_OPTS $YARN_PROXYSERVER_OPTS"if [ "$YARN_PROXYSERVER_HEAPSIZE" != "" ]; thenJAVA_HEAP_MAX="-Xmx""$YARN_PROXYSERVER_HEAPSIZE""m"fi
elif [ "$COMMAND" = "version" ] ; thenCLASS=org.apache.hadoop.util.VersionInfoYARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"
elif [ "$COMMAND" = "jar" ] ; thenCLASS=org.apache.hadoop.util.RunJarYARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"
elif [ "$COMMAND" = "logs" ] ; thenCLASS=org.apache.hadoop.yarn.client.cli.LogsCLIYARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"
elif [ "$COMMAND" = "daemonlog" ] ; thenCLASS=org.apache.hadoop.log.LogLevelYARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"
elif [ "$COMMAND" = "cluster" ] ; thenCLASS=org.apache.hadoop.yarn.client.cli.ClusterCLIYARN_OPTS="$YARN_OPTS $YARN_CLIENT_OPTS"
elseCLASS=$COMMAND
fi

这个就是主类: org.apache.hadoop.yarn.server.resourcemanager.ResourceManager

对应的NodeManager你懂的

启动的时候，如果报错：

machine2: Error: JAVA_HOME is not set and could not be found.
machine1: Error: JAVA_HOME is not set and could not be found.

直接修改/root/hadoop-2.7.3/etc/hadoop/hadoop-env.sh里面，添加

#export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=/root/jdk1.8.0_181

就可以正常启动了

下面是resourceManager的启动命令

/root/jdk1.8.0_181/bin/java -Dproc_resourcemanager -Xmx90m -Dhadoop.log.dir=/root/hadoop-2.7.3/logs -Dyarn.log.dir=/root/hadoop-2.7.3/logs -Dhadoop.log.file=yarn-root-resourcemanager-machine0.log -Dyarn.log.file=yarn-root-resourcemanager-machine0.log -Dyarn.home.dir= -Dyarn.id.str=root -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA -Djava.library.path=/root/hadoop-2.7.3/lib/native -Dyarn.policy.file=hadoop-policy.xml -Dhadoop.log.dir=/root/hadoop-2.7.3/logs -Dyarn.log.dir=/root/hadoop-2.7.3/logs -Dhadoop.log.file=yarn-root-resourcemanager-machine0.log -Dyarn.log.file=yarn-root-resourcemanager-machine0.log -Dyarn.home.dir=/root/hadoop-2.7.3 -Dhadoop.home.dir=/root/hadoop-2.7.3 -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA -Djava.library.path=/root/hadoop-2.7.3/lib/native -classpath /root/hadoop-2.7.3/etc/hadoop:/root/hadoop-2.7.3/etc/hadoop:/root/hadoop-2.7.3/etc/hadoop:/root/hadoop-2.7.3/share/hadoop/common/lib/*:/root/hadoop-2.7.3/share/hadoop/common/*:/root/hadoop-2.7.3/share/hadoop/hdfs:/root/hadoop-2.7.3/share/hadoop/hdfs/lib/*:/root/hadoop-2.7.3/share/hadoop/hdfs/*:/root/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/hadoop-2.7.3/share/hadoop/yarn/*:/root/hadoop-2.7.3/share/hadoop/mapreduce/lib/*:/root/hadoop-2.7.3/share/hadoop/mapreduce/*:/contrib/capacity-scheduler/*.jar:/contrib/capacity-scheduler/*.jar:/contrib/capacity-scheduler/*.jar:/root/hadoop-2.7.3/share/hadoop/yarn/*:/root/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/hadoop-2.7.3/etc/hadoop/rm-config/log4j.properties org.apache.hadoop.yarn.server.resourcemanager.ResourceManager

下面是nodemanager的启动命令

/root/jdk1.8.0_181/bin/java -Dproc_nodemanager -Xmx1000m -Dhadoop.log.dir=/root/hadoop-2.7.3/logs -Dyarn.log.dir=/root/hadoop-2.7.3/logs -Dhadoop.log.file=yarn-root-nodemanager-machine1.log -Dyarn.log.file=yarn-root-nodemanager-machine1.log -Dyarn.home.dir= -Dyarn.id.str=root -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA -Djava.library.path=/root/hadoop-2.7.3/lib/native -Dyarn.policy.file=hadoop-policy.xml -server -Dhadoop.log.dir=/root/hadoop-2.7.3/logs -Dyarn.log.dir=/root/hadoop-2.7.3/logs -Dhadoop.log.file=yarn-root-nodemanager-machine1.log -Dyarn.log.file=yarn-root-nodemanager-machine1.log -Dyarn.home.dir=/root/hadoop-2.7.3 -Dhadoop.home.dir=/root/hadoop-2.7.3 -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA -Djava.library.path=/root/hadoop-2.7.3/lib/native -classpath /root/hadoop-2.7.3/etc/hadoop:/root/hadoop-2.7.3/etc/hadoop:/root/hadoop-2.7.3/etc/hadoop:/root/hadoop-2.7.3/share/hadoop/common/lib/*:/root/hadoop-2.7.3/share/hadoop/common/*:/root/hadoop-2.7.3/share/hadoop/hdfs:/root/hadoop-2.7.3/share/hadoop/hdfs/lib/*:/root/hadoop-2.7.3/share/hadoop/hdfs/*:/root/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/hadoop-2.7.3/share/hadoop/yarn/*:/root/hadoop-2.7.3/share/hadoop/mapreduce/lib/*:/root/hadoop-2.7.3/share/hadoop/mapreduce/*:/contrib/capacity-scheduler/*.jar:/contrib/capacity-scheduler/*.jar:/root/hadoop-2.7.3/share/hadoop/yarn/*:/root/hadoop-2.7.3/share/hadoop/yarn/lib/*:/root/hadoop-2.7.3/etc/hadoop/nm-config/log4j.properties org.apache.hadoop.yarn.server.nodemanager.NodeManager

然后根据这个命令修改成debug命令就行了

转载于:https://my.oschina.net/qiangzigege/blog/2054888

YARN-2.7.3-源码分析系列2：启动脚本原理的分析相关推荐

TiKV 源码解析系列文章（二）raft-rs proposal 示例情景分析
作者:屈鹏本文为 TiKV 源码解析系列的第二篇,按照计划首先将为大家介绍 TiKV 依赖的周边库 raft-rs .raft-rs 是 Raft 算法的 Rust 语言实现.Raft 是分布式领域 ...
一起谈.NET技术，.NET Framework源码研究系列之---万法归宗Object
经过前面三篇关于.NET Framework源码研究系列的随笔,相信大家都发现其实.NET Framework的实现其实并不复杂,也许跟我们自己做的项目开发差不多.本人也是这样的看法.不过,经过仔细深 ...
【Dubbo源码阅读系列】之远程服务调用（上）
今天打算来讲一讲 Dubbo 服务远程调用.笔者在开始看 Dubbo 远程服务相关源码的时候,看的有点迷糊.后来慢慢明白 Dubbo 远程服务的调用的本质就是动态代理模式的一种实现.本地消费者无须知道 ...
TiDB 源码阅读系列文章（六）Select 语句概览
在先前的 TiDB 源码阅读系列文章(四) 中,我们介绍了 Insert 语句,想必大家已经了解了 TiDB 是如何写入数据,本篇文章介绍一下 Select 语句是如何执行.相比 Insert,Sel ...
TiDB 源码阅读系列文章（十九）tikv-client（下）
上篇文章中,我们介绍了数据读写过程中 tikv-client 需要解决的几个具体问题,本文将继续介绍 tikv-client 里的两个主要的模块--负责处理分布式计算的 copIterator 和执 ...
TiDB 源码阅读系列文章（十五）Sort Merge Join
2019独角兽企业重金招聘Python工程师标准>>> 什么是 Sort Merge Join 在开始阅读源码之前, 我们来看看什么是 Sort Merge Join (SMJ),定 ...
SpringMVC源码阅读系列汇总
1.前言 1.1 导入 SpringMVC是基于Servlet和Spring框架设计的Web框架,做JavaWeb的同学应该都知道本文基于Spring4.3.7源码分析,(不要被图片欺骗了,手动滑稽 ...
Tomcat源码解析系列二：Tomcat总体架构
Tomcat即是一个HTTP服务器,也是一个servlet容器,主要目的就是包装servlet,并对请求响应相应的servlet,纯servlet的web应用似乎很好理解Tomcat是如何装载serv ...
prometheus变量_TiKV 源码解析系列文章（四）Prometheus（下）
本文为 TiKV 源码解析系列的第四篇,接上篇继续为大家介绍 rust-prometheus.上篇主要介绍了基础知识以及最基本的几个指标的内部工作机制,本篇会进一步介绍更多高级功能的实现原理. 与上篇 ...

YARN-2.7.3-源码分析系列2：启动脚本原理的分析

YARN-2.7.3-源码分析系列2：启动脚本原理的分析相关推荐

最新文章

热门文章