欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

Hadoop学习之路一 配置hadoop cluster时,必须先打通ssh

程序员文章站 2022-07-13 10:40:47
...

     

我们都知道Apache Hadoop 是跑在linux上的(CentOS,RedHat,Ubuntu...),如果是在开发环境下,配置程pseudo-distributed即可,但如果在上prod环境时,切记务必记得打通NameNode 到DataNode的ssh(Secure Shell),确保NN和DN之间的传输访问不再需要验证.....这一点会烦死你..

 

1. 首先,通过which 命令,查看是否已安装了验证SSH

[hadoop@localhost ~]$ which ssh

/usr/bin/ssh

[hadoop@localhost ~]$ which sshd

/usr/sbin/sshd

[hadoop@localhost ~]$ which ssh-keygen

/usr/bin/ssh-keygen

如果出现其他异常...则表示未安装验证SHH,请访问www.openssh.com,进行安装

 

2.生成SSH key pair

[hadoop@localhost ~]$ ssh-keygen -t rsa

一路敲回车,别输入口令,否则每次访问slave node 都需要输入

注:如key pair 已生成,则出现如下讯息,是否覆盖

[hadoop@localhost hadoop]$ ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 

/home/hadoop/.ssh/id_rsa already exists.

Overwrite (y/n)? n

 

3.查看生成的key pair

[hadoop@localhost hadoop]$ more /home/hadoop/.ssh/id_rsa

 

4.分发到slave node

在master node 输入

[hadoop-user@master]$ scp ~/.ssh/id_rsa.pub hadoop-user@target:~/master_key

在slave node 输入,将master key 作为authorized_keys

[hadoop-user@target]$ mkdir ~/.ssh 

[hadoop-user@target]$ chmod 700 ~/.ssh

[hadoop-user@target]$ mv ~/master_key ~/.ssh/authorized_keys

[hadoop-user@target]$ chmod 600 ~/.ssh/authorized_keys