Hadoop学习之路一 配置hadoop cluster时,必须先打通ssh
我们都知道Apache Hadoop 是跑在linux上的(CentOS,RedHat,Ubuntu...),如果是在开发环境下,配置程pseudo-distributed即可,但如果在上prod环境时,切记务必记得打通NameNode 到DataNode的ssh(Secure Shell),确保NN和DN之间的传输访问不再需要验证.....这一点会烦死你..
1. 首先,通过which 命令,查看是否已安装了验证SSH
[hadoop@localhost ~]$ which ssh
/usr/bin/ssh
[hadoop@localhost ~]$ which sshd
/usr/sbin/sshd
[hadoop@localhost ~]$ which ssh-keygen
/usr/bin/ssh-keygen
如果出现其他异常...则表示未安装验证SHH,请访问www.openssh.com,进行安装
2.生成SSH key pair
[hadoop@localhost ~]$ ssh-keygen -t rsa
一路敲回车,别输入口令,否则每次访问slave node 都需要输入
注:如key pair 已生成,则出现如下讯息,是否覆盖
[hadoop@localhost hadoop]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa):
/home/hadoop/.ssh/id_rsa already exists.
Overwrite (y/n)? n
3.查看生成的key pair
[hadoop@localhost hadoop]$ more /home/hadoop/.ssh/id_rsa
4.分发到slave node
在master node 输入
[hadoop-user@master]$ scp ~/.ssh/id_rsa.pub hadoop-user@target:~/master_key
在slave node 输入,将master key 作为authorized_keys
[hadoop-user@target]$ mkdir ~/.ssh
[hadoop-user@target]$ chmod 700 ~/.ssh
[hadoop-user@target]$ mv ~/master_key ~/.ssh/authorized_keys
[hadoop-user@target]$ chmod 600 ~/.ssh/authorized_keys