【原创 HadoopSpark 动手实践 1】Hadoop2.7.3 安装部署实践
目录:
第一部分:操作系统准备工作:
1. 安装部署CentOS7.3 1611
2. CentOS7软件安装(net-tools, wget, vim等)
3. 更新CentOS7的Yum源,更新软件速度更快
4. CentOS 用户配置,Sudo授权
第二部分:Java环境准备
1. JDK1.8 安装与配置
第三部分:Hadoop配置,启动与验证
1. 解压Hadoop2.7.3更新全局变量
2. 更新Hadoop配置文件
3. 启动Hadoop
4. 验证Hadoop
=============================================================================================
第一部分:操作系统准备工作:
1. 安装部署CentOS7.3 1611
2. CentOS7软件安装(net-tools, wget, vim等)
3. 更新CentOS7的Yum源,更新软件速度更快
4. CentOS 用户配置,Sudo授权
1. 安装部署CentOS7.3 1611
2. CentOS7软件安装(net-tools, wget, vim等)
sudo yum install -y net-tools
sudo yum install -y wget
sudo yum install -y vim
3. 更新CentOS7的Yum源(更新为阿里云的CentOS7的源),更新软件速度更快
http://mirrors.aliyun.com/help/centos
1、备份
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
2、下载新的CentOS-Base.repo 到/etc/yum.repos.d/
CentOS 5
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-5.repo
CentOS 6
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
CentOS 7
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
3、之后运行yum makecache生成缓存
4、sudo yum -y update 对系统进行升级
sudo vim /etc/hosts # 更新hosts文件,便于用spark02代表本机IP
第二部分:Java环境准备
1. JDK1.8 安装与配置
通过FileZilla 上传实验所需要用到的文件(JDK,Hadoop,Spark)
对JDK和Hadoop进行解压
tar -zxvf jdk-8u121-linux-x64.tar.gz
tar -zxvf hadoop-2.7.3.tar.gz
在 .bash_profile文件内增加环境便利,便于Java和Hadoop更容易操作
#Add JAVA_HOME and HADOOP_HOME
export JAVA_HOME=/home/spark/jdk1.8.0_121
export PATH=$PATH:$JAVA_HOME/bin
export HADOOP_HOME=/home/spark/hadoop-2.7.3
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
source .bash_profile #配置生效
第三部分:Hadoop配置,启动与验证
1. 解压Hadoop2.7.3更新全局变量
2. 更新Hadoop配置文件
3. 启动Hadoop
4. 验证Hadoop
参考:Hadoop2.7.3 官方文档进行伪分布式的配置
http://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/SingleCluster.html
Pseudo-Distributed Operation
Hadoop can also be run on a single-node in a pseudo-distributed mode where each Hadoop daemon runs in a separate Java process.
Configuration
Use the following:
etc/hadoop/core-site.xml:
<configuration><property><name>fs.defaultFS</name><value>hdfs://localhost:9000</value></property> </configuration>
etc/hadoop/hdfs-site.xml:
<configuration><property><name>dfs.replication</name><value>1</value></property> </configuration>
Setup passphraseless ssh
Now check that you can ssh to the localhost without a passphrase:
$ ssh localhost
If you cannot ssh to localhost without a passphrase, execute the following commands:
$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys$ chmod 0600 ~/.ssh/authorized_keys
Execution
The following instructions are to run a MapReduce job locally. If you want to execute a job on YARN, see YARN on Single Node.
Format the filesystem:
$ bin/hdfs namenode -format
Start NameNode daemon and DataNode daemon:
$ sbin/start-dfs.sh
The hadoop daemon log output is written to the $HADOOP_LOG_DIR directory (defaults to $HADOOP_HOME/logs).
Browse the web interface for the NameNode; by default it is available at:
- NameNode - http://localhost:50070/
Make the HDFS directories required to execute MapReduce jobs:
$ bin/hdfs dfs -mkdir /user$ bin/hdfs dfs -mkdir /user/<username>
Copy the input files into the distributed filesystem:
$ bin/hdfs dfs -put etc/hadoop input
Run some of the examples provided:
$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep input output 'dfs[a-z.]+'
Examine the output files: Copy the output files from the distributed filesystem to the local filesystem and examine them:
$ bin/hdfs dfs -get output output$ cat output/*
or
View the output files on the distributed filesystem:
$ bin/hdfs dfs -cat output/*
When you’re done, stop the daemons with:
$ sbin/stop-dfs.sh
YARN on a Single Node
You can run a MapReduce job on YARN in a pseudo-distributed mode by setting a few parameters and running ResourceManager daemon and NodeManager daemon in addition.
The following instructions assume that 1. ~ 4. steps of the above instructions are already executed.
Configure parameters as follows:etc/hadoop/mapred-site.xml:
<configuration><property><name>mapreduce.framework.name</name><value>yarn</value></property> </configuration>
etc/hadoop/yarn-site.xml:
<configuration><property><name>yarn.nodemanager.aux-services</name><value>mapreduce_shuffle</value></property> </configuration>
Start ResourceManager daemon and NodeManager daemon:
$ sbin/start-yarn.sh
Browse the web interface for the ResourceManager; by default it is available at:
- ResourceManager - http://localhost:8088/
Run a MapReduce job.
When you’re done, stop the daemons with:
$ sbin/stop-yarn.sh
配置免密码,否则运行的时候会报错。
Setup passphraseless ssh
Now check that you can ssh to the localhost without a passphrase:
$ ssh localhost
If you cannot ssh to localhost without a passphrase, execute the following commands:
$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys$ chmod 0600 ~/.ssh/authorized_keys
Hadoop配置文件具体的配置信息如下:
1. vim etc/hadoop/hadoop-env.sh
#export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=/home/spark/jdk1.8.0_1212. vim etc/hadoop/core-site.xml
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://spark01:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/spark/hadoopdata</value>
</property>
</configuration>3. vim etc/hadoop/hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>4. vim etc/hadoop/mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>5.vim etc/hadoop/yarn-site.xml<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
</configuration>1. 对HDFS进行格式化hdfs namenode -format2.启动HDFS
start-dfs.sh
3. 启动YARN
start-yarn.sh
CentOS 7.2关闭防火墙
CentOS 7.0默认使用的是firewall作为防火墙,这里改为iptables防火墙步骤。
firewall-cmd --state #查看默认防火墙状态(关闭后显示notrunning,开启后显示running)
?123[root@localhost ~]
#firewall-cmd --state
not running
检查防火墙的状态:
从centos7开始使用systemctl来管理服务和程序,包括了service和chkconfig。
?123[root@localhost ~]
#systemctl list-unit-files|grep firewalld.service --防火墙处于关闭状态
firewalld.service disabled
或者
?1234567[root@localhost ~]
#systemctl status firewalld.service
firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (
/usr/lib/systemd/system/firewalld
.service; disabled; vendor preset: enabled)
Active: inactive (dead)
关闭防火墙:
systemctl stop firewalld.service #停止firewall
systemctl disable firewalld.service #禁止firewall开机启动?123[root@localhost ~]
#systemctl stop firewalld.service
[root@localhost ~]
#systemctl disable firewalld.service
?123456789101112131415启动一个服务:systemctl start firewalld.service
关闭一个服务:systemctl stop firewalld.service
重启一个服务:systemctl restart firewalld.service
显示一个服务的状态:systemctl status firewalld.service
在开机时启用一个服务:systemctl enable firewalld.service
在开机时禁用一个服务:systemctl disable firewalld.service
查看服务是否开机启动:systemctl is-enabled firewalld.service;echo $?
查看已启动的服务列表:systemctl list-unit-files|grep enabled
Centos 7 firewall 命令:
查看已经开放的端口:
?1firewall-cmd --list-ports
开启端口
?1firewall-cmd --zone=public --add-port=80
/tcp
--permanent
命令含义:
–zone #作用域
–add-port=80/tcp #添加端口,格式为:端口/通讯协议
–permanent #永久生效,没有此参数重启后失效
重启防火墙
?1234firewall-cmd --reload
#重启firewall
systemctl stop firewalld.service
#停止firewall
systemctl disable firewalld.service
#禁止firewall开机启动
firewall-cmd --state
#查看默认防火墙状态(关闭后显示notrunning,开启后显示running)
1. 启动防火墙(Firewalld):
sudo systemctl start firewalld.service
2. 查看防火墙(Firewalld)运行状态:
sudo systemctl status firewalld.service
3. 配置防火墙(Firewalld)访问规则,打开8088(YARN)和50070(HDFS)两个端口:
sudo firewall-cmd --zone=public --add-port=8088
/tcp
--permanent
sudo firewall-cmd --zone=public --add-port=50070
/tcp
--permanent
4. 载入防火墙(Firewalld)规则:
sudo firewall-cmd --reload
5. 重新启动防火墙(Firewalld):
sudo systemctl restart firewalld.service
6. 验证防火墙(Firewalld)生效:
http://spark02:8080
http://spark02:50070
http://spark02:8088
http://spark02:50070
使用HDFS创建目录,拷贝文件和查看文件
hdfs dfs -mkdir hdfs://user/jonson/inputhdfs dfs -cp etc/hadoop hdfs://user/jonson/inputhdfs dfs -ls hdfs://user/jonson/inputhdfs dfs -mkdir hdfs://user/jonson/outputhdfs dfs -rmdir hdfs://user/jonson/outputhdfs dfs -ls hdfs://user/jonson尝试使用MapReduce计算框架[spark@Spark02 hadoop-2.7.3]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep hdfs:///user/jonson/input/hadoop hdfs:///user/jonson/output 'dfs[a-z.]+'[spark@Spark02 hadoop-2.7.3]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep hdfs:///user/jonson/input/hadoop hdfs:///user/jonson/output 'dfs[a-z.]+'
17/05/07 23:36:17 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/05/07 23:36:18 INFO input.FileInputFormat: Total input paths to process : 30
17/05/07 23:36:18 INFO mapreduce.JobSubmitter: number of splits:30
17/05/07 23:36:19 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1494169715431_0003
17/05/07 23:36:19 INFO impl.YarnClientImpl: Submitted application application_1494169715431_0003
17/05/07 23:36:19 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1494169715431_0003/
17/05/07 23:36:19 INFO mapreduce.Job: Running job: job_1494169715431_0003
17/05/07 23:36:28 INFO mapreduce.Job: Job job_1494169715431_0003 running in uber mode : false
17/05/07 23:36:28 INFO mapreduce.Job: map 0% reduce 0%
17/05/07 23:36:58 INFO mapreduce.Job: map 20% reduce 0%
17/05/07 23:37:25 INFO mapreduce.Job: map 37% reduce 0%
17/05/07 23:37:26 INFO mapreduce.Job: map 40% reduce 0%
17/05/07 23:37:50 INFO mapreduce.Job: map 47% reduce 0%
17/05/07 23:37:51 INFO mapreduce.Job: map 57% reduce 0%
17/05/07 23:37:54 INFO mapreduce.Job: map 57% reduce 19%
17/05/07 23:38:04 INFO mapreduce.Job: map 60% reduce 19%
17/05/07 23:38:06 INFO mapreduce.Job: map 60% reduce 20%
17/05/07 23:38:12 INFO mapreduce.Job: map 73% reduce 20%
17/05/07 23:38:15 INFO mapreduce.Job: map 73% reduce 24%
17/05/07 23:38:18 INFO mapreduce.Job: map 77% reduce 24%
17/05/07 23:38:21 INFO mapreduce.Job: map 77% reduce 26%
17/05/07 23:38:33 INFO mapreduce.Job: map 83% reduce 26%
17/05/07 23:38:34 INFO mapreduce.Job: map 90% reduce 26%
17/05/07 23:38:35 INFO mapreduce.Job: map 93% reduce 26%
17/05/07 23:38:36 INFO mapreduce.Job: map 93% reduce 31%
17/05/07 23:38:43 INFO mapreduce.Job: map 100% reduce 31%
17/05/07 23:38:44 INFO mapreduce.Job: map 100% reduce 100%
17/05/07 23:38:45 INFO mapreduce.Job: Job job_1494169715431_0003 completed successfully
17/05/07 23:38:45 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=345
FILE: Number of bytes written=3690573
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=81841
HDFS: Number of bytes written=437
HDFS: Number of read operations=93
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=30
Launched reduce tasks=1
Data-local map tasks=30
Total time spent by all maps in occupied slots (ms)=653035
Total time spent by all reduces in occupied slots (ms)=77840
Total time spent by all map tasks (ms)=653035
Total time spent by all reduce tasks (ms)=77840
Total vcore-milliseconds taken by all map tasks=653035
Total vcore-milliseconds taken by all reduce tasks=77840
Total megabyte-milliseconds taken by all map tasks=668707840
Total megabyte-milliseconds taken by all reduce tasks=79708160
Map-Reduce Framework
Map input records=2103
Map output records=24
Map output bytes=590
Map output materialized bytes=519
Input split bytes=3804
Combine input records=24
Combine output records=13
Reduce input groups=11
Reduce shuffle bytes=519
Reduce input records=13
Reduce output records=11
Spilled Records=26
Shuffled Maps =30
Failed Shuffles=0
Merged Map outputs=30
GC time elapsed (ms)=8250
CPU time spent (ms)=13990
Physical memory (bytes) snapshot=6025490432
Virtual memory (bytes) snapshot=64352063488
Total committed heap usage (bytes)=4090552320
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=78037
File Output Format Counters
Bytes Written=437
17/05/07 23:38:45 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/05/07 23:38:46 INFO input.FileInputFormat: Total input paths to process : 1
17/05/07 23:38:46 INFO mapreduce.JobSubmitter: number of splits:1
17/05/07 23:38:46 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1494169715431_0004
17/05/07 23:38:46 INFO impl.YarnClientImpl: Submitted application application_1494169715431_0004
17/05/07 23:38:46 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1494169715431_0004/
17/05/07 23:38:46 INFO mapreduce.Job: Running job: job_1494169715431_0004
17/05/07 23:39:00 INFO mapreduce.Job: Job job_1494169715431_0004 running in uber mode : false
17/05/07 23:39:00 INFO mapreduce.Job: map 0% reduce 0%
17/05/07 23:39:06 INFO mapreduce.Job: map 100% reduce 0%
17/05/07 23:39:13 INFO mapreduce.Job: map 100% reduce 100%
17/05/07 23:39:14 INFO mapreduce.Job: Job job_1494169715431_0004 completed successfully
17/05/07 23:39:14 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=291
FILE: Number of bytes written=237535
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=566
HDFS: Number of bytes written=197
HDFS: Number of read operations=7
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=1
Launched reduce tasks=1
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=3838
Total time spent by all reduces in occupied slots (ms)=3849
Total time spent by all map tasks (ms)=3838
Total time spent by all reduce tasks (ms)=3849
Total vcore-milliseconds taken by all map tasks=3838
Total vcore-milliseconds taken by all reduce tasks=3849
Total megabyte-milliseconds taken by all map tasks=3930112
Total megabyte-milliseconds taken by all reduce tasks=3941376
Map-Reduce Framework
Map input records=11
Map output records=11
Map output bytes=263
Map output materialized bytes=291
Input split bytes=129
Combine input records=0
Combine output records=0
Reduce input groups=5
Reduce shuffle bytes=291
Reduce input records=11
Reduce output records=11
Spilled Records=22
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=143
CPU time spent (ms)=980
Physical memory (bytes) snapshot=306675712
Virtual memory (bytes) snapshot=4157272064
Total committed heap usage (bytes)=165810176
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=437
File Output Format Counters
Bytes Written=197
利用命令行来查看运行结果:
[spark@Spark02 hadoop-2.7.3]$ hadoop fs -cat hdfs:///user/jonson/output/*
6 dfs.audit.logger
4 dfs.class
3 dfs.server.namenode.
2 dfs.period
2 dfs.audit.log.maxfilesize
2 dfs.audit.log.maxbackupindex
1 dfsmetrics.log
1 dfsadmin
1 dfs.servers
1 dfs.replication
1 dfs.file
[spark@Spark02 hadoop-2.7.3]$ hadoop fs -cat hdfs:///user/jonson/output/part-r-00000
6 dfs.audit.logger
4 dfs.class
3 dfs.server.namenode.
2 dfs.period
2 dfs.audit.log.maxfilesize
2 dfs.audit.log.maxbackupindex
1 dfsmetrics.log
1 dfsadmin
1 dfs.servers
1 dfs.replication
1 dfs.file
====================================
免密码登录原理和方法
背景:搭建Hadoop环境需要设置无密码登陆,所谓无密码登陆其实是指通过证书认证的方式登陆,使用一种被称为"公私钥"认证的方式来进行ssh登录。
在linux系统中,ssh是远程登录的默认工具,因为该工具的协议使用了RSA/DSA的加密算法.该工具做linux系统的远程管理是非常安全的。telnet,因为其不安全性,在linux系统中被搁置使用了。
" 公私钥"认证方式简单的解释:首先在客户端上创建一对公私钥 (公钥文件:~/.ssh/id_rsa.pub; 私钥文件:~/.ssh/id_rsa)。然后把公钥放到服务器上(~/.ssh/authorized_keys), 自己保留好私钥.在使用ssh登录时,ssh程序会发送私钥去和服务器上的公钥做匹配.如果匹配成功就可以登录了。
工具/原料
- linux系统
方法/步骤
确认系统已经安装了SSH。
rpm –qa | grep openssh
rpm –qa | grep rsync
-->出现如下图的信息表示已安装
假设没有安装ssh和rsync,可以通过下面命令进行安装。
yum install ssh -->安装SSH协议
yum install rsync -->rsync是一个远程数据同步工具,可通过LAN/WAN快速同步多台主机间的文件
service sshd restart -->启动服务
生成秘钥对
ssh-keygen –t rsa –P '' -->直接回车生成的密钥对:id_rsa和id_rsa.pub,默认存储在"/home/hadoop/.ssh"目录下。
把id_rsa.pub追加到授权的key里面去。
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
修改授权key的权限
chmod 600 ~/.ssh/authorized_keys
修改SSH配置文件
su - -->登陆root用户修改配置文件
vim /etc/ssh/sshd_config -->去掉下图中三行的注释
测试连接
service sshd restart -->重启ssh服务,
exit -->退出root用户,回到普通用户
ssh localhost -->连接普通用户测试
这只是配置好了单机环境上的SSH服务,要远程连接其它的服务器,接着看下面。
现在秘钥对已经生成好了,客户端SSH服务也已经配置好了,现在就把我们的钥匙(公钥)送给服务器。
scp ~/.ssh/id_rsa.pub 远程用户名@远程服务器IP:~/ -->将公钥复制到远程服务器的~/目录下
如: scp ~/.ssh/id_rsa.pub hadoop@192.168.1.134:~/
可以看到我们复制的时候需要我们输入服务器的密码,等我们把SSH配置好之后这些步骤就可以不用输入密码了。
- 8
上一步把公钥发送到192.168.1.134服务器上去了,我们去134机器上把公钥追加到授权key中去。(注意:如果是第一次运行SSH,那么.ssh目录需要手动创建,或者使用命令ssh-keygen -t rsa生成秘钥,它会自动在用户目录下生成.ssh目录。特别注意的是.ssh目录的权限问题,记得运行下chmod 700 .ssh命令)
在134机器上使用命令:
cat ~/id_rsa.pub >> ~/.ssh/authorized_keys -->追加公钥到授权key中
rm ~/id_rsa.pub -->保险起见,删除公钥
同样在134机器上重复第四步和第五步,
service sshd restart -->重启ssh服务
- 9
回到客户机来,输入:
ssh 192.168.1.134 -->应该就能直接连接服务器咯。
【原创 HadoopSpark 动手实践 1】Hadoop2.7.3 安装部署实践相关推荐
- centos 安装mysql5.7_Zabbix 4.2.5 安装部署实践详解
[导读]云计算背景下,无论是大数据.物联网还是边缘计算,规模化后大量的设备需要保证正常运行,在人员一定的情况下,就需要提高运行维护效率.同时随着智能化被应用在人们生活的方方面面,关联性也越来越紧密,即 ...
- 时空大数据实践之GeoWave安装部署实践
2019独角兽企业重金招聘Python工程师标准>>> 时空大数据实践之GeoWave安装部署实践 GeoWave是由国家地理空间情报局(NGA)与RadiantBlue和Booz ...
- Zabbix 4.2.5 安装部署实践详解
一.安装 1.安装CentOS操作系统,并配置网络 2.安装Zabbix官方源 rpm -ivh http://repo.zabbix.com/zabbix/4.2/rhel/7/x86_64/zab ...
- hadoop2 Ubuntu 下安装部署
搭建Hadoop环境( 我以hadoop 2.7.3 为例, 系统为 64bit Ubuntu14.04 ) hadoop 2.7.3 官网下载 , 选择自己要安装的版本.注意每个版本对应两个下载选项 ...
- 【原创 HadoopSpark 动手实践 6】Spark 编程实例与案例演示
[原创 Hadoop&Spark 动手实践 6]Spark 编程实例与案例演示 Spark 编程实例和简易电影分析系统的编写 目标: 1. 掌握理论:了解Spark编程的理论基础 2. 搭建 ...
- 【原创 HadoopSpark 动手实践 11】Spark Streaming 应用与动手实践
[原创 Hadoop&Spark 动手实践 11]Spark Streaming 应用与动手实践 目标: 1. 掌握Spark Streaming的基本原理 2. 完成Spark Stream ...
- Hadoop2.6.5安装部署及环境变量配置
系统环境: 四台虚拟机 192.168.1.167 vm4.com 192.168.1.31 vm3.com 192.168.1.62 vm2.com 192.168.1.39 vm1.com 系统版 ...
- Istio 1.1安装部署实践
3月20日,Istio 1.1版本正式发布,我们已在<全方位解读 | Istio v1.1正式发布>一文中为大家进行了简单介绍.本文将给大家带来详细的部署过程详解,需要说明的是,本文针对单 ...
- 自动化工具后起之秀Ansible的部署实践
本文转自微信号EAWorld.扫描下方二维码,关注成功后,回复"普元方法+",将会获得热门课堂免费学习机会! 从早期手动加脚本的部署方式,到后来自动化工具(chef, puppet ...
最新文章
- java avl_Java底层实现AVL 平衡二叉树
- mysql+dump+选项_详解MySQL数据备份之mysqldump使用方法
- 求解N个值中最大的k个数,N远大于k
- 从搜索引擎到社交网络的艰难转变
- centos关机与重启命令详解
- 拿下“[warn] (OS 64)指定的网络名不再可用”错误
- 如何把SAP CRM WebClient UI上某个字段高亮加粗显示
- [PHP] 算法-把数组排成最小的数的PHP实现
- 【NLP论文推荐】 掌握实体关系抽取必读的文章
- 【转】项目代码风格要求
- OpenCV3.1.0安装在ubuntu16.40(Linux)步骤(电赛AM5708板子可用)
- 【MATLAB深度学习工具箱】学习笔记--体脂估计算例再分析:拟合神经网络fitnet里面的函数】
- GBase 8c 技术白皮书 六
- C语言每日一练——第133天:打鱼还是晒网
- 点评美国名校的(EE)和(CS)
- 92921-24-9,SULFO-SMCC, Sulfo-SMCC Crosslinker,磺基-SMCC钠盐溶于大部分有机溶剂
- 研华服务器显示不了全屏啊,研华串口服务器设置
- 三极管的工作状态及电流关系
- 分布式存储的架构以及存在的问题和解决办法1
- JS滚轮事件(mousewheel/DOMMouseScroll)了解 (转载)
热门文章
- 二手车电商很热?其实都是平台在自嗨!
- C#的网络适配器操作
- win7下80端口被(Pid=4)占用的解决方法
- 不要担心,我可以一个人走
- 《C++ Primer》 Part V (Advanced Topics)
- [原]为虚拟机添加新的磁盘分区
- 强烈推荐一款完全免费的绿色JRE+Tomcat+MySQL集成开发工具 - JTM
- 机器学习中的数据标准化
- Sentinel v1.4.2 发布,更好用的集群限流功能
- 软件包管理 之 Fedora Extras(Fedora 计划的扩充资源) rpm.livna.org软件仓库的介绍和应用...