大家需要在每个节点上提前装好"expect"工具

expect的使用请看我的另一篇文章：

http://tianxingzhe.blog.51cto.com/3390077/1687661

spawn命令激活一个Unix程序来进行交互式的运行。　

send命令向进程发送字符串。

expect命令等待进程的某些字符串

set timeout 1    设置超时时间  timeout -1 为永不超时

expect eof

只有spawn执行的命令结果才会被expect捕捉到，因为spawn会启动一个进程，只有这个进程的相关信息才会被捕捉到，主要包括：标准输入的提示信息，eof和timeout。
这里，eof是必须去匹配的，在spawn进程结束后会向expect发送eof；如果不去匹配，有时也能运行，比如sleep多少秒后再去spawn下一个命令，但是不要依赖这种行为，很有可能今天还可以，明天就不能用了。

expect  \"#\"   期待返回shell提示符(是#或者$)

interact 命令

执行完成后保持交互状态，把控制权交给控制台，这个时候就可以手工操作了。如果没有这一句登录完成后会退出，而不是留在远程终端上。如果你只是登录过去执行一段命令就退出，可改为 expect eof

id_dsa/ id_dsa.pub：你用openssh工具生成的私钥公钥对
authorized_keys ：你使用ssh连接的linux服务器需要认证你的身份，所以你需要在连接的linux服务器上安装自已的公钥，authorized_keys这里面就是存放你自己的id_dsa.pub的内容

scp是有Security的文件copy，基于ssh登录。操作起来比较方便，比如要把当前一个文件copy到远程另外一台主机上，可以如下命令。

scp /home/daisy/full.tar.gz

大体思路

1、首先在一个文本文件中保存1000台机器的hadoop用户名和密码
2、用shell遍历这个文件 写一个循环用namenode的去循环登陆其他的999个节点，执行生成密钥的工作，然后把生成的公钥写回namenode
3、在namenode上生成密钥 写入这个文件
4、把第三部生成的文件拷贝到剩下的机器上
5、用循环遍历验证免密的效果

本解决方法主要包括两个脚本： sshpass.sh和ssh4slaves

1. sshpass.sh

#!/bin/bash
# Name     : sshpass.sh
# Time     : 17/09/2012
# Author   : simplestone@dbinterest.com
# Purpose  : For fast and easy setup of the SSH Passwordless access among all the nodes
#            in a cluster.
# User     : Any user you are performing the test! Better to settup a separate user from your
#            working env to avoid troubles!!! "root" is used in this example, and you can change it
#            via the export virable "USER=root"
# Attention: The test env is assuming that each $USER on each $HOST is usring the same password!
#            And this likely makes sense as no body want to put more trouble on this.
# Usage    : 1st, make sure the script has the execute permisison "chmod +x ssh_pass.sh"
#            ./ssh_pass.sh password
#          : 2nd, ensure the "ssh4slaves.sh" script is with ssh_pass.sh for all nodes setup!!!
#          : 3rd, "expect" has to be installed on all the nodes for the SSH config
export FILELOC="/root"
export SLAVESFILE="$FILELOC/sshslaves"
export HOSTS=`cat $FILELOC/sshhosts`
export SLAVES=`cat $FILELOC/sshslaves`
export SSH4SLAVESCRIPT="$FILELOC/ssh4slaves.sh"
export MASTER=hdp01
export USER=root
export PASSWD=$1
export SSHLOC="$FILELOC/.ssh/"
export RSAFILE="$FILELOC/.ssh/id_rsa"
export RSAPUBFILE="$FILELOC/.ssh/id_rsa.pub"
export AUTHFILE="$FILELOC/.ssh/authorized_keys"
export EXPECTCHK=`rpm -qa expect | wc -l`
#
if [ $EXPECTCHK != 1 ]thenecho ''echo "########################################################################################"echo "Please install the \"expect\" package first on all nodes to allow the script to run!!!"echo "yum -y install expect"echo "########################################################################################"
elseif [ -e $RSAFILE ]thenecho "########################################################################################"echo "Attention: This is for TEST ONLY, please fully test it before applying it to PROD"echo "environment!!! OR you might get in trouble!!!"echo ''echo "BETTER TO HAVE A NEW USER FOR THE TEST TO AVOID DESTROYING YOUR ENVIRONMENT!"echo ''echo "Please manually delete the ssh related file on each host before executing the script!!!"echo ''for host in $HOSTSdo echo "Please run command on $host: rm -rf $SSHLOC"doneecho "########################################################################################"else# Just generate for host in $HOSTSdoif [ $host = "$MASTER" ]then echo ''echo "###########################################################"echo "Generating RSA keys for MASTER host $MASTER"echo "###########################################################"echo ''expect -c "set timeout 1spawn ssh $USER@$hostexpect \"yes/no\"send -- \"yes\r\"expect \"password:\"send -- \"$PASSWD\r\"expect \"#\"send \"ssh-keygen -t rsa -P '' -f $RSAFILE\r\"expect \"#\"send \"ssh-copy-id -i $RSAPUBFILE $MASTER\r\"expect \"password:\"send -- \"$PASSWD\r\"expect eof"elseecho ''echo "###########################################################"echo "Generating RSA keys for all OTHER hosts..."echo "hostname is $host"echo "###########################################################"echo ''expect -c "set timeout 1spawn ssh $USER@$hostexpect \"yes/no\"send -- \"yes\r\"expect \"password:\"send -- \"$PASSWD\r\"expect \"#\"send \"ssh-keygen -t rsa -P '' -f $RSAFILE\r\"expect \"#\"send \"ssh-copy-id -i $RSAPUBFILE $MASTER\r\"expect \"yes/no\"send -- \"yes\r\"expect \"password:\"send -- \"$PASSWD\r\"expect eof"fidone            ### for host in $SLAVES doecho ''echo "############################################################################"echo "Copying authorized_keys to host $host from the MASTER host $MASTER..."echo "############################################################################"echo ''expect -c "set timeout 1spawn scp $AUTHFILE "$USER@$host:$SSHLOC"expect \"password:\"send -- $PASSWD\rexpect eof"done#for host in $SLAVESdoecho ''echo "############################################################################"echo "Distributing the $SLAVESFILE file to slave host $host..."echo "############################################################################"echo ''scp $SLAVESFILE "$host:$FILELOC"echo ''echo "############################################################################"echo "Distributing the $SSH4SLAVESCRIPT script to slave host $host..."echo "############################################################################"echo ''scp $SSH4SLAVESCRIPT "$host:$FILELOC"donefor host in $SLAVESdoecho ''echo "############################################################################"echo "Working on the slaves node $host to ensure no prompt for the "yes/no" question..."echo "############################################################################"echo ''ssh -q $USER@$host $SSH4SLAVESCRIPTdone### Check whether the Passwordless ssh works ###for host in $HOSTSdoecho ''echo "############################################################################"echo "Check whether the Passwordless SSH works for $host..."echo "############################################################################"echo ''ssh $host uname -a && datedonefi
fi
###
# rm -rf /root/.ssh
# mv /root/.ssh /root/sshlogin
#{ eval "$GET_ID" ; } | ssh $1 "umask 077; test -d .ssh || mkdir .ssh ; cat >> .ssh/authorized_keys; test -x /sbin/restorecon && /sbin/restorecon .ssh .ssh/authorized_keys" || exit 1
#cat /root/.ssh/id_rsa.pub | ssh hdp01 "umask 077; test -d .ssh || mkdir .ssh ; cat >> .ssh/authorized_keys; test -x /sbin/restorecon && /sbin/restorecon .ssh .ssh/authorized_keys" || exit 1
#/root/.ssh/id_rsa.pub

./ssh_pass.sh password （password替换程序里的$1参数），本例中密码为stonetest

2. ssh4slaves

#!/bin/bash
# Name     : ssh4slaves.sh
# Time     : 17/09/2012
# Author   : simplestone@dbinterest.com
# Purpose  : For fast and easy setup of the SSH Passwordless access among all the slave nodes
#            in a cluster. Mainly to ensure no prompt for "yes/no" again!!!
# User     : Any user you are performing the test! Better to settup a separate user from your
#            working env to avoid troubles!!! "root" is used in this example, and you can change it
#            via the export virable "USER=root"
# Attention: The test env is assuming that each $USER on each $HOST is usring the same password!
#            And this likely makes sense as no body want to put more trouble on this.
# Usage    : This script is called by the main script "ssh_pass.sh"
#            1st, make sure the script has the execute permisison "chmod +x ssh4slaves.sh" before
#            distributing it to other slaves node.
#            2nd, Remember to change variable "PASSWORD" before start the main script "sshpass.sh"
export FILELOC="/root"
export SLAVES=`cat $FILELOC/sshslaves`
export USER=root
export PASSWD=stonetest
for host in $SLAVES
doecho ''echo "Ensure ssh passwordless works among all slave nodes..."echo ''expect -c "set timeout 1spawn ssh $USER@$hostexpect \"yes/no\"send -- \"yes\r\"expect eof"done

3. 其他配置

[root@hdp01 ~]# pwd
/root
[root@hdp01 ~]# cat sshhosts
hdp01
hdp02
hdp03
[root@hdp01 ~]# cat sshslaves
hdp02
hdp03
[root@hdp01 ~]# ls -lrth | tail -2
-rwxr-xr-x  1 root root 1.3K Sep 18 02:08 ssh4slaves.sh
-rwxr-xr-x  1 root root 6.5K Sep 18 02:11 ssh_pass.sh

4. 测试输出

[root@hdp01 ~]# ./ssh_pass.sh stonetest
###########################################################
Generating RSA keys for MASTER host hdp01
###########################################################
spawn ssh root@hdp01
The authenticity of host 'hdp01 (192.168.1.121)' can't be established.
RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hdp01,192.168.1.121' (RSA) to the list of known hosts.
root@hdp01's password:
Last login: Tue Sep 18 02:09:29 2012 from hdp02.dbinterest.local
[root@hdp01 ~]# ssh-keygen -t rsa -P '' -f /root/.ssh/id_rsa
Generating public/private rsa key pair.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
3a:c3:98:b3:e4:39:fa:fe:87:c6:22:90:16:57:4e:47 root@hdp01.dbinterest.local
The key's randomart image is:
+--[ RSA 2048]----+
|      .E         |
|     o .         |
|    + .          |
| . . .           |
| .o     S        |
|o.   + .         |
|..  =.=.         |
|  .oo++o.        |
|  .=*=..         |
+-----------------+
[root@hdp01 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01
root@hdp01's password:
Now try logging into the machine, with "ssh 'hdp01'", and check in:.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.
[root@hdp01 ~]#
###########################################################
Generating RSA keys for all OTHER hosts...
hostname is hdp02
###########################################################
spawn ssh root@hdp02
The authenticity of host 'hdp02 (192.168.1.122)' can't be established.
RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hdp02,192.168.1.122' (RSA) to the list of known hosts.
root@hdp02's password:
Last login: Tue Sep 18 02:09:23 2012 from hdp02.dbinterest.local
[root@hdp02 ~]# ssh-keygen -t rsa -P '' -f /root/.ssh/id_rsa
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
a9:89:fe:40:8a:8e:21:55:da:3b:6b:68:4f:3e:8f:fc root@hdp02.dbinterest.local
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|    .            |
|   +     .       |
|  o o   S        |
| o o o o         |
|+ ..* o          |
|+.o=o=           |
|.o oB=E          |
+-----------------+
[root@hdp02 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01
The authenticity of host 'hdp01 (192.168.1.121)' can't be established.
RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hdp01,192.168.1.121' (RSA) to the list of known hosts.
root@hdp01's password:
Now try logging into the machine, with "ssh 'hdp01'", and check in:.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.###########################################################
Generating RSA keys for all OTHER hosts...
hostname is hdp03
###########################################################
spawn ssh root@hdp03
The authenticity of host 'hdp03 (192.168.1.123)' can't be established.
RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hdp03,192.168.1.123' (RSA) to the list of known hosts.
root@hdp03's password:
Last login: Tue Sep 18 02:09:19 2012 from hdp02.dbinterest.local
[root@hdp03 ~]# ssh-keygen -t rsa -P '' -f /root/.ssh/id_rsa
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
a4:3d:dd:54:42:c0:45:ec:ed:ae:d6:bd:14:a0:9b:16 root@hdp03.dbinterest.local
The key's randomart image is:
+--[ RSA 2048]----+
|         ..*= .  |
|          . .o   |
|        .  ..o   |
|       + . oo o  |
|      . S .E.. . |
|         .  + . .|
|           + o o |
|          . . + .|
|           ... ..|
+-----------------+
[root@hdp03 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub hdp01
The authenticity of host 'hdp01 (192.168.1.121)' can't be established.
RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hdp01,192.168.1.121' (RSA) to the list of known hosts.
root@hdp01's password:
Now try logging into the machine, with "ssh 'hdp01'", and check in:.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.
[root@hdp03 ~]#
############################################################################
Copying authorized_keys to host hdp02 from the MASTER host hdp01...
############################################################################
spawn scp /root/.ssh/authorized_keys root@hdp02:/root/.ssh/
root@hdp02's password:
authorized_keys                                                                                            100% 1227     1.2KB/s   00:00
############################################################################
Copying authorized_keys to host hdp03 from the MASTER host hdp01...
############################################################################
spawn scp /root/.ssh/authorized_keys root@hdp03:/root/.ssh/
root@hdp03's password:
authorized_keys                                                                                            100% 1227     1.2KB/s   00:00
############################################################################
Distributing the /root/sshslaves file to slave host hdp02...
############################################################################
sshslaves                                                                                                  100%   12     0.0KB/s   00:00
############################################################################
Distributing the /root/ssh4slaves.sh script to slave host hdp02...
############################################################################
ssh4slaves.sh                                                                                              100% 1277     1.3KB/s   00:00
############################################################################
Distributing the /root/sshslaves file to slave host hdp03...
############################################################################
sshslaves                                                                                                  100%   12     0.0KB/s   00:00
############################################################################
Distributing the /root/ssh4slaves.sh script to slave host hdp03...
############################################################################
ssh4slaves.sh                                                                                              100% 1277     1.3KB/s   00:00
############################################################################
Working on the slaves node hdp02 to ensure no prompt for the yes/no question...
############################################################################Ensure ssh passwordless works among all slave nodes...
spawn ssh root@hdp02
The authenticity of host 'hdp02 (192.168.1.122)' can't be established.
RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hdp02,192.168.1.122' (RSA) to the list of known hosts.
Last login: Tue Sep 18 02:11:54 2012 from hdp01.dbinterest.local
[root@hdp02 ~]#
Ensure ssh passwordless works among all slave nodes...
spawn ssh root@hdp03
The authenticity of host 'hdp03 (192.168.1.123)' can't be established.
RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hdp03,192.168.1.123' (RSA) to the list of known hosts.
Last login: Tue Sep 18 02:11:55 2012 from hdp01.dbinterest.local
[root@hdp03 ~]#
############################################################################
Working on the slaves node hdp03 to ensure no prompt for the yes/no question...
############################################################################Ensure ssh passwordless works among all slave nodes...
spawn ssh root@hdp02
The authenticity of host 'hdp02 (192.168.1.122)' can't be established.
RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hdp02,192.168.1.122' (RSA) to the list of known hosts.
Last login: Tue Sep 18 02:11:58 2012 from hdp02.dbinterest.local
[root@hdp02 ~]#
Ensure ssh passwordless works among all slave nodes...
spawn ssh root@hdp03
The authenticity of host 'hdp03 (192.168.1.123)' can't be established.
RSA key fingerprint is 23:fa:69:0b:a5:b0:c2:80:13:13:ba:2b:7d:b1:5b:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hdp03,192.168.1.123' (RSA) to the list of known hosts.
Last login: Tue Sep 18 02:11:59 2012 from hdp02.dbinterest.local
############################################################################
Check whether the Passwordless SSH works for hdp01...
############################################################################
Linux hdp01.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Tue Sep 18 02:12:05 PDT 2012
############################################################################
Check whether the Passwordless SSH works for hdp02...
############################################################################
Linux hdp02.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Tue Sep 18 02:12:05 PDT 2012
############################################################################
Check whether the Passwordless SSH works for hdp03...
############################################################################
Linux hdp03.dbinterest.local 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Tue Sep 18 02:12:06 PDT 2012

5. 其他节点测试

[root@hdp02 ~]#
[root@hdp02 ~]# ssh hdp02
Last login: Tue Sep 18 02:12:00 2012 from hdp03.dbinterest.local
[root@hdp02 ~]# exit
logout
Connection to hdp02 closed.
[root@hdp02 ~]# ssh hdp03
Last login: Tue Sep 18 02:12:02 2012 from hdp03.dbinterest.local
[root@hdp03 ~]# exit
logout
Connection to hdp03 closed.
[root@hdp02 ~]#
----------
[root@hdp03 ~]#
[root@hdp03 ~]# ssh hdp01
Last login: Tue Sep 18 02:12:22 2012 from hdp02.dbinterest.local
[root@hdp01 ~]# exit
logout
Connection to hdp01 closed.
[root@hdp03 ~]# ssh hdp02
Last login: Tue Sep 18 02:12:25 2012 from hdp02.dbinterest.local
[root@hdp02 ~]# exit
logout
Connection to hdp02 closed.
[root@hdp03 ~]# ssh hdp03
Last login: Tue Sep 18 02:12:30 2012 from hdp02.dbinterest.local
[root@hdp03 ~]# exit
logout
Connection to hdp03 closed.
[root@hdp03 ~]#

代码下载见附件

参考文章：

http://www.cnblogs.com/iloveyoucc/archive/2012/05/11/2496433.html

http://f.dataguru.cn/thread-19920-1-1.html

本文出自 “点滴积累” 博客，请务必保留此出处http://tianxingzhe.blog.51cto.com/3390077/1687600