一、安装JDK

yum install java-1.8*

二、关闭防火墙

systemctl status firewalld
systemctl stop firewalld
systemctl disable firewalld

三、配置ip地址和主机名映射

vi /etc/hosts
加入：
192.168.1.1  hadoop001

四、配置免密登录

1、生成公私钥

ssh-keygen -t rsa

2、写入公钥到授权文件

cd /root/.ssh/
cat id_rsa.pub >> authorized_keys
chmod 600 authorized_keys

五、建立安装目录

mkdir -p /appserver/hadoop

六、下载安装包

cd /root
wget https://archive.apache.org/dist/hadoop/common/hadoop-3.3.4/hadoop-3.3.4.tar.gz

七、解压安装包

tar -zxvf hadoop-3.3.4.tar.gz -C /appserver/hadoop/

解压到目录为：/appserver/hadoop/hadoop-3.3.4

Hadoop目录结构
bin  etc  include  lib  libexec  LICENSE-binary  licenses-binary  LICENSE.txt  logs  NOTICE-binary  NOTICE.txt  README.txt  sbin  share

1、bin目录
会用到hdfs、mapred、yarn命令

2、etc目录
提供大量的配置文件
core-site.xml，核心配置
hdfs-site.xml，和hdfs相关的
mapred-site.xml，和计算相关的
yarn-site.xml，和yarn相关的
workers，后期会用到

3、sbin目录
启动脚本
start-dfs.sh，启动HDFS
start-yarn.sh，启动yarn
mr-jobhistory-daemon.sh，启动历史服务器

4、share目录
学习资料和官方案例

八、hadoop的配置

0、配置HADOOP_HOME和JAVA_HOME

vi /etc/profile添加：
export HADOOP_HOME=/appserver/hadoop/hadoop-3.3.4
export PATH=$PATH:$HADOOP_HOME/bin
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.362.b08-1.el7_9.x86_64
export PATH=$PATH:$JAVA_HOME/binsource /etc/profile

1、修改hadoop-env.sh
配置JAVA_HOME，yum安装openjdk的路径在/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.362.b08-1.el7_9.x86_64

cd /appserver/hadoop/hadoop-3.3.4/etc/hadoop/
vi hadoop-env.sh
加入：
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.362.b08-1.el7_9.x86_64

2、测试hadoop环境

bin/hadoop versionHadoop 3.3.4
Source code repository https://github.com/apache/hadoop.git -r a585a73c3e02ac62350c136643a5e7f6095a3dbb
Compiled by stevel on 2022-07-29T12:32Z
Compiled with protoc 3.7.1
From source with checksum fb9dd8918a7b8a5b430d61af858f6ec
This command was run using /appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-common-3.3.4.jar

3、修改core-site.xml
（1）指定namenode的hdfs协议文件系统的通信地址
（2）指定hadoop存储临时文件的目录

vi core-site.xml<configuration><property><!--指定 namenode 的 hdfs 协议文件系统的通信地址--><name>fs.defaultFS</name><value>hdfs://hadoop001:8020</value></property><property><!--指定 hadoop 存储临时文件的目录--><name>hadoop.tmp.dir</name><value>/appserver/hadoop/tmp</value></property>
</configuration>

4、修改hdfs-site.xml
（1）设置namenode节点数据保存目录
（2）设置datanode节点数据保存目录
（3）指定副本数量，在分布式文件系统中，数据通常会被冗余的存储多份，以保证可靠性和安全性，但是这里用的是伪分布式模式，节点只有一个，也有就只有一个副本

vi hdfs-site.xml<configuration><property><!--设置namenode节点数据保存目录--><name>dfs.namenode.name.dir</name><value>/appserver/hadoop/namenode/data</value></property><property><!--datanode节点数据保存目录--><name>dfs.datanode.data.dir</name><value>/appserver/hadoop/datanode/data</value></property><property><!--指定副本数量--><name>dfs.replication</name><value>1</value></property>
</configuration>

5、修改mapred-site.xml
（1）指定mapreduce作业运行在yarn上
（2）配置历史服务器地址

vi mapred-site.xml<configuration><property><!--指定 mapreduce 作业运行在 yarn 上--><name>mapreduce.framework.name</name><value>yarn</value></property><!-- 配置历史服务器的内部服务端地址 --><property><name>mapreduce.jobhistory.address</name><value>hadoop001:10020</value></property><!-- 配置历史服务器暴露的web地址 --><property><name>mapreduce.jobhistory.webapp.address</name><value>hadoop001:19888</value></property>
</configuration>

6、修改yarn-site.xml
（1）配置NodeManager上运行的附属服务。需要配置成mapreduce_shuffle后才可以在Yarn上运行MapReduce程序
（2）配置resourcemanager在哪个节点上
（3）配置yarn的classpath
（4）配置日志聚集功能

vi yarn-site.xml<configuration><property><!--配置 NodeManager 上运行的附属服务。需要配置成 mapreduce_shuffle 后才可以在 Yarn 上运行 MapReduce 程序。--><name>yarn.nodemanager.aux-services</name><value>mapreduce_shuffle</value></property><property><!--resourcemanager 的主机名--><name>yarn.resourcemanager.hostname</name><value>hadoop001</value></property><!-- 配置classpath --><property><name>yarn.application.classpath</name><value>/appserver/hadoop/hadoop-3.3.4/etc/hadoop:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/*:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/*</value></property><!-- 配置日志聚集功能，yarn-default.xml中该项默认为false --><property><name>yarn.log-aggregation-enable</name><value>true</value></property><!-- 设置日志聚集服务器地址，设置聚集到历史日志服务器上，方便查看 --><property><name>yarn.log.server.url</name><value>http://hadoop001:19888/jobhistory/logs</value></property><!-- 设置日志保留时间：7天 --><property><name>yarn.log-aggregation.retain-seconds</name><value>604800</value></property>
</configuration>

7、slaves
（1）在/appserver/hadoop/hadoop-3.3.4/etc/hadoop/workers文件，配置节点的主机名或IP地址
（2）单机版本，指定本机即可
（3）真集群环境下，所有从属节点上的DataNode服务和NodeManager服务都会被启动

vi workershadoop001

九、首次启动格式化HDFS

cd /appserver/hadoop/hadoop-3.3.4/bin
./hdfs namenode -formatWARNING: /appserver/hadoop/hadoop-3.3.4/logs does not exist. Creating.
2023-01-30 00:24:08,201 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = myhost/192.168.1.1
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 3.3.4
STARTUP_MSG:   classpath = /appserver/hadoop/hadoop-3.3.4/etc/hadoop:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/hadoop-auth-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jersey-core-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jsp-api-2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jersey-json-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/checker-qual-2.5.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-util-ajax-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/re2j-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-util-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jsch-0.1.55.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-security-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/paranamer-2.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-util-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-webapp-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/avro-1.7.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-util-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/hadoop-shaded-guava-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-databind-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-net-3.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-server-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-core-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/netty-3.10.6.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-logging-1.1.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-compress-1.21.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/metrics-core-3.2.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-codec-1.15.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-text-1.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/guava-27.0-jre.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/reload4j-1.2.22.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-client-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/failureaccess-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/zookeeper-jute-3.5.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-common-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/httpclient-4.5.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-collections-3.2.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/snappy-java-1.1.8.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/gson-2.8.9.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-config-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/woodstox-core-5.3.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-http-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-io-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jsr305-3.0.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/curator-recipes-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/dnsjava-2.1.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/zookeeper-3.5.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/asm-5.0.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jersey-server-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/curator-client-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/stax2-api-4.2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/curator-framework-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/token-provider-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/accessors-smart-2.4.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerb-core-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-io-2.8.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jersey-servlet-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jettison-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-servlet-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-math3-3.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/json-smart-2.4.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-xml-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/httpcore-4.4.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/nimbus-jose-jwt-9.8.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/commons-cli-1.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/hadoop-annotations-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/jetty-server-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-nfs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-registry-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-common-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-common-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/common/hadoop-kms-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/hadoop-auth-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/okio-2.8.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/re2j-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-redis-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.77.Final-osx-x86_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-security-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/paranamer-2.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.77.Final-osx-aarch_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-webapp-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/avro-1.7.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-util-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-databind-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-net-3.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-3.10.6.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-compress-1.21.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kotlin-stdlib-1.4.10.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-buffer-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-text-1.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/zookeeper-jute-3.5.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-http2-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-udt-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/okhttp-4.9.3.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-xml-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kotlin-stdlib-common-1.4.10.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.77.Final-linux-aarch_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-handler-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/snappy-java-1.1.8.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-dns-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/gson-2.8.9.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.77.Final-linux-x86_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/woodstox-core-5.3.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-http-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-io-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-all-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-socks-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-common-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/curator-recipes-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/zookeeper-3.5.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.77.Final-osx-aarch_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/asm-5.0.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/curator-client-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/curator-framework-4.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/accessors-smart-2.4.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-io-2.8.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-resolver-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jettison-1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-codec-http-4.1.77.Final.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-servlet-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.77.Final-osx-x86_64.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/json-smart-2.4.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-xml-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.8.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/hadoop-annotations-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/lib/jetty-server-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-nfs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-client-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-client-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.4-tests.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/asm-tree-9.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jetty-plus-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/fst-2.50.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax.inject-1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/guice-4.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/snakeyaml-1.26.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/bcprov-jdk15on-1.60.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/aopalliance-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jetty-annotations-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/asm-analysis-9.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jna-5.2.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/asm-commons-9.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-servlet-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jline-3.9.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-api-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/objenesis-2.6.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/json-io-2.5.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-server-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-common-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jetty-client-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/bcpkix-jdk15on-1.60.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jersey-client-1.19.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/websocket-client-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/java-util-1.9.0.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jetty-jndi-9.4.43.v20210629.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-api-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-client-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-registry-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-common-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-tests-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-services-core-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-router-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-server-common-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.3.4.jar:/appserver/hadoop/hadoop-3.3.4/share/hadoop/yarn/hadoop-yarn-services-api-3.3.4.jar
STARTUP_MSG:   build = https://github.com/apache/hadoop.git -r a585a73c3e02ac62350c136643a5e7f6095a3dbb; compiled by 'stevel' on 2022-07-29T12:32Z
STARTUP_MSG:   java = 1.8.0_362
************************************************************/
2023-01-30 00:24:08,217 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2023-01-30 00:24:08,364 INFO namenode.NameNode: createNameNode [-format]
2023-01-30 00:24:09,371 INFO common.Util: Assuming 'file' scheme for path /appserver/hadoop/namenode/data in configuration.
2023-01-30 00:24:09,373 INFO common.Util: Assuming 'file' scheme for path /appserver/hadoop/namenode/data in configuration.
2023-01-30 00:24:09,383 INFO namenode.NameNode: Formatting using clusterid: CID-4adad145-e4cb-4b11-bb30-526830848a22
2023-01-30 00:24:09,441 INFO namenode.FSEditLog: Edit logging is async:true
2023-01-30 00:24:09,489 INFO namenode.FSNamesystem: KeyProvider: null
2023-01-30 00:24:09,492 INFO namenode.FSNamesystem: fsLock is fair: true
2023-01-30 00:24:09,494 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2023-01-30 00:24:09,502 INFO namenode.FSNamesystem: fsOwner                = root (auth:SIMPLE)
2023-01-30 00:24:09,502 INFO namenode.FSNamesystem: supergroup             = supergroup
2023-01-30 00:24:09,526 INFO namenode.FSNamesystem: isPermissionEnabled    = true
2023-01-30 00:24:09,526 INFO namenode.FSNamesystem: isStoragePolicyEnabled = true
2023-01-30 00:24:09,527 INFO namenode.FSNamesystem: HA Enabled: false
2023-01-30 00:24:09,674 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2023-01-30 00:24:09,701 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2023-01-30 00:24:09,701 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
2023-01-30 00:24:09,711 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2023-01-30 00:24:09,711 INFO blockmanagement.BlockManager: The block deletion will start around 2023 Jan 30 00:24:09
2023-01-30 00:24:09,715 INFO util.GSet: Computing capacity for map BlocksMap
2023-01-30 00:24:09,715 INFO util.GSet: VM type       = 64-bit
2023-01-30 00:24:09,718 INFO util.GSet: 2.0% max memory 3.4 GB = 70.6 MB
2023-01-30 00:24:09,718 INFO util.GSet: capacity      = 2^23 = 8388608 entries
2023-01-30 00:24:09,899 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled
2023-01-30 00:24:09,899 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
2023-01-30 00:24:09,917 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.999
2023-01-30 00:24:09,918 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
2023-01-30 00:24:09,918 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2023-01-30 00:24:09,920 INFO blockmanagement.BlockManager: defaultReplication         = 1
2023-01-30 00:24:09,920 INFO blockmanagement.BlockManager: maxReplication             = 512
2023-01-30 00:24:09,920 INFO blockmanagement.BlockManager: minReplication             = 1
2023-01-30 00:24:09,920 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
2023-01-30 00:24:09,921 INFO blockmanagement.BlockManager: redundancyRecheckInterval  = 3000ms
2023-01-30 00:24:09,921 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
2023-01-30 00:24:09,921 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
2023-01-30 00:24:10,010 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911
2023-01-30 00:24:10,011 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215
2023-01-30 00:24:10,011 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215
2023-01-30 00:24:10,011 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215
2023-01-30 00:24:10,059 INFO util.GSet: Computing capacity for map INodeMap
2023-01-30 00:24:10,060 INFO util.GSet: VM type       = 64-bit
2023-01-30 00:24:10,060 INFO util.GSet: 1.0% max memory 3.4 GB = 35.3 MB
2023-01-30 00:24:10,060 INFO util.GSet: capacity      = 2^22 = 4194304 entries
2023-01-30 00:24:10,110 INFO namenode.FSDirectory: ACLs enabled? true
2023-01-30 00:24:10,110 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
2023-01-30 00:24:10,110 INFO namenode.FSDirectory: XAttrs enabled? true
2023-01-30 00:24:10,111 INFO namenode.NameNode: Caching file names occurring more than 10 times
2023-01-30 00:24:10,125 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536
2023-01-30 00:24:10,129 INFO snapshot.SnapshotManager: SkipList is disabled
2023-01-30 00:24:10,138 INFO util.GSet: Computing capacity for map cachedBlocks
2023-01-30 00:24:10,138 INFO util.GSet: VM type       = 64-bit
2023-01-30 00:24:10,139 INFO util.GSet: 0.25% max memory 3.4 GB = 8.8 MB
2023-01-30 00:24:10,139 INFO util.GSet: capacity      = 2^20 = 1048576 entries
2023-01-30 00:24:10,156 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2023-01-30 00:24:10,157 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2023-01-30 00:24:10,157 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2023-01-30 00:24:10,165 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
2023-01-30 00:24:10,165 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2023-01-30 00:24:10,168 INFO util.GSet: Computing capacity for map NameNodeRetryCache
2023-01-30 00:24:10,168 INFO util.GSet: VM type       = 64-bit
2023-01-30 00:24:10,169 INFO util.GSet: 0.029999999329447746% max memory 3.4 GB = 1.1 MB
2023-01-30 00:24:10,169 INFO util.GSet: capacity      = 2^17 = 131072 entries
2023-01-30 00:24:10,248 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1636510195-173.249.200.110-1675056250223
2023-01-30 00:24:10,284 INFO common.Storage: Storage directory /appserver/hadoop/namenode/data has been successfully formatted.
2023-01-30 00:24:10,347 INFO namenode.FSImageFormatProtobuf: Saving image file /appserver/hadoop/namenode/data/current/fsimage.ckpt_0000000000000000000 using no compression
2023-01-30 00:24:10,567 INFO namenode.FSImageFormatProtobuf: Image file /appserver/hadoop/namenode/data/current/fsimage.ckpt_0000000000000000000 of size 399 bytes saved in 0 seconds .
2023-01-30 00:24:10,844 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
2023-01-30 00:24:10,916 INFO namenode.FSNamesystem: Stopping services started for active state
2023-01-30 00:24:10,917 INFO namenode.FSNamesystem: Stopping services started for standby state
2023-01-30 00:24:10,928 INFO namenode.FSImage: FSImageSaver clean checkpoint: txid=0 when meet shutdown.
2023-01-30 00:24:10,929 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at myhost/192.168.1.1
************************************************************/

十、添加root启动

可以加在profile内，也可以加在hadoop-env.sh内

vi /etc/profile添加：
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=rootsource /etc/profile

十一、启动服务

进入到sbin目录下

1、启动dfs服务

./start-dfs.shStarting namenodes on [hadoop001]
Last login: Mon Jan 30 01:26:56 EST 2023 on pts/1
hadoop001: Warning: Permanently added 'hadoop001,192.168.1.1' (ECDSA) to the list of known hosts.
Starting datanodes
Last login: Mon Jan 30 01:31:55 EST 2023 on pts/1
localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Starting secondary namenodes [myhost]
Last login: Mon Jan 30 01:31:57 EST 2023 on pts/1
myhost: Warning: Permanently added 'myhost' (ECDSA) to the list of known hosts.

2、启动yarn服务

./start-yarn.shStarting resourcemanager
Last login: Mon Jan 30 01:32:03 EST 2023 on pts/1
Starting nodemanagers
Last login: Mon Jan 30 01:35:43 EST 2023 on pts/1

3、手动启动历史服务器
进入到bin目录下

./mapred --daemon start historyserver

十二、查看启动的服务

jps15570 SecondaryNameNode
20229 NodeManager
20502 Jps
15179 NameNode
19901 ResourceManager
20445 JobHistoryServer
15358 DataNode

十三、管理页面

1、hadoop NameNode管理界面
http://192.168.1.1:9870/

2、yarn ResourceManager管理界面
http://192.168.1.1:8088/

3、hadoop SecondaryNameNode管理界面
http://192.168.1.1:9868

4、yarn历史服务器管理界面
http://192.168.1.1:19888

参考资料：
https://blog.csdn.net/weixin_44433834/article/details/122408455
https://blog.csdn.net/cai750415222/article/details/116530976
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html

Hadoop基础之《（6）—Hadoop单机伪集群安装》相关推荐

  1. Zookeeper单机伪集群

    Zookeeper单机伪集群 1.配置 zookeeper下载地址:http://apache.mirrors.lucidnetworks.net/zookeeper/ 可以选择需要的版本,我下载的是 ...

  2. Redis学习笔记之Redis单机,伪集群,Sentinel主从复制的安装和配置

    0x00 Redis简介 Redis是一款开源的.高性能的键-值存储(key-value store).它常被称作是一款数据结构服务器(data structure server). Redis的键值 ...

  3. centos7 kafka2.3.1单机伪集群部署

    接上篇文章centos7 zookeeper单点部署,准备好相应的包 cp config/server.properties config/server0.properties vi config/s ...

  4. centos7 zookeeper3.5.6单机伪集群部署

    接上篇文章centos7 zookeeper单点部署准备好zookeeper包,进行集群部署 单机伪集群部署 zookeeper1 zookeeper2 zookeeper3 三个目录分别部署一个服务 ...

  5. ELK 系列九、elasticsearch扩容（从单机至单机伪集群）

    目录 一.简介 二.升级操作 2.1 es配置 2.2 logstash设置 2.3 守护进程配置 2.4 查看结果 一.简介 单机伪集群是什么概念呢,就是同一台服务器有多个es节点 先介绍一下,原先 ...

  6. 多台es服务器搭建elasticsearch集群，es单机伪集群搭建，在一台服务器上以三个不同的端口模拟三台主机完成es伪集群搭建，windows和Linux操作方法一致

    目录 (一)es单机伪集群搭建 一.准备三台elasticsearch服务器(复制三个es目录) 二.修改每台服务器的配置 1.修改es配置文件 2.配置注意 (二)多台服务器es集群搭建: 1 安装 ...

  7. 03_zookeeper伪集群安装

    一句话说明白:在1台机器上模拟多台机器,对外提供服务 在理解zookeeper集群安装方法的基础上,本文描述如何将1个机器模拟为3个节点的zookeeper集群,建议先参考阅读本文的前一期 zooke ...

  8. redis伪集群安装linux,redis伪集群搭建（亲测无坑）

    一.单机版安装部署,伪集群只需要操作前1-8步即可,再往下浏览找到:二.单机版-伪集群  安装部署继续搭建,如需设置密码参照第10步 1.安装基本工具 yum install -y gcc-c++ v ...

  9. Clickhouse入门学习、单机、集群安装部署

    参考:https://blog.csdn.net/qq_37933018/article/details/108019566 Clickhouse官网:https://clickhouse.tech/ ...

最新文章

  1. Intent以及IntentFilter详解
  2. JMP跳转指令的指令长度、直接转移与间接转移、段内跳转与段间跳转
  3. win2003 服务器超出最大连接+强制重启服务器（方法）
  4. ZOJ 3702 Gibonacci number（数学推导题）
  5. 什么是RS232/RS485转换器？
  6. Air Data System
  7. 2021高校暑假时间汇总！最长70天
  8. Python生成器和类
  9. python基础教程多少页_看完这篇文章，你的Python基础就差不多了（附200页《Python400集》）...
  10. 计算机科学素养大赛,第六届全国大学生计算机应用能力与信息素养大赛圆满结束...
  11. 15.UNIX 环境高级编程--进程间的通信
  12. 解决Sqlite Developer过期的办法
  13. 【eoeAndroid社区索引】android开发混淆
  14. 《显微镜下的大明》马伯庸2019年1月新书mobi、epub、PDF资源共享
  15. .jks或者.keystore文件查看MD5、SHA-1和SHA-256
  16. 微信小程序---配置微信商户平台进行微信支付操作指南以及踩坑记录
  17. MySQL数据库执行Update卡死问题解决
  18. CPU的后缀到底是什么意思？
  19. SpringBoot个人博客项目——相册模块开发（一）
  20. python 操作 word 文件

热门文章

  1. 中国没有真正意义上的海滩比基尼(图)
  2. 学会这27种编程语言，你还怕找不到女朋友？！那是要几个有几个！
  3. 为什么男人总是欺负你？
  4. w ndows10摄像头设置,windows10系统电脑摄像头怎么打开
  5. pb调用计算机默认游览器,PB打开ole控件IE浏览器版本问题_指定Webbrowser控件所用IE内核版本(转)...
  6. word分栏 公式居中对齐 自动编号
  7. wps 符号操作（仅符号）
  8. keil startup_stm32f10x_hd.s编译报错
  9. 【图像增强】Learning Enriched Features for Real Image Restoration and Enhancement 阅读笔记
  10. 一个排列是某些对象的有序组合，例如，3124 就是数字 1，2，3，4 的一种可能排列。 如果所有的排列按照数值或者字母序排序，我们称其为一个字典序。 0, 1, 2 的字典排列有：012 021 1