Hadoop学习之路一 配置hadoop cluster时，必须先打通ssh

     

我们都知道Apache Hadoop 是跑在linux上的(CentOS,RedHat,Ubuntu...)，如果是在开发环境下，配置程pseudo-distributed即可，但如果在上prod环境时，切记务必记得打通NameNode 到DataNode的ssh（Secure Shell），确保NN和DN之间的传输访问不再需要验证.....这一点会烦死你..

 

1. 首先，通过which 命令，查看是否已安装了验证SSH

[hadoop@localhost ~]$ which ssh

/usr/bin/ssh

[hadoop@localhost ~]$ which sshd

/usr/sbin/sshd

[hadoop@localhost ~]$ which ssh-keygen

/usr/bin/ssh-keygen

如果出现其他异常...则表示未安装验证SHH，请访问www.openssh.com，进行安装

 

2.生成SSH key pair

[hadoop@localhost ~]$ ssh-keygen -t rsa

一路敲回车，别输入口令，否则每次访问slave node 都需要输入

注：如key pair 已生成，则出现如下讯息，是否覆盖

[hadoop@localhost hadoop]$ ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 

/home/hadoop/.ssh/id_rsa already exists.

Overwrite (y/n)? n

 

3.查看生成的key pair

[hadoop@localhost hadoop]$ more /home/hadoop/.ssh/id_rsa

 

4.分发到slave node

在master node 输入

[hadoop-user@master]$ scp ~/.ssh/id_rsa.pub hadoop-user@target:~/master_key

在slave node 输入，将master key 作为authorized_keys

[hadoop-user@target]$ mkdir ~/.ssh 

[hadoop-user@target]$ chmod 700 ~/.ssh

[hadoop-user@target]$ mv ~/master_key ~/.ssh/authorized_keys

[hadoop-user@target]$ chmod 600 ~/.ssh/authorized_keys