欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识！
您现在的位置是： 首页

Hadoop2.7.1+Hbase1.2.1集群环境搭建(10)基于ZK的Hadoop HA集群安装

程序员文章站 2022-06-16 15:12:50
...
(1)hadoop2.7.1源码编译 http://aperise.iteye.com/blog/2246856
(2)hadoop2.7.1安装准备 http://aperise.iteye.com/blog/2253544
(3)1.x和2.x都支持的集群安装 http://aperise.iteye.com/blog/2245547
(4)hbase安装准备 http://aperise.iteye.com/blog/2254451
(5)hbase安装 http://aperise.iteye.com/blog/2254460
(6)snappy安装 http://aperise.iteye.com/blog/2254487
(7)hbase性能优化 http://aperise.iteye.com/blog/2282670
(8)雅虎YCSBC测试hbase性能测试 http://aperise.iteye.com/blog/2248863
(9)spring-hadoop实战 http://aperise.iteye.com/blog/2254491
(10)基于ZK的Hadoop HA集群安装  http://aperise.iteye.com/blog/2305809

 

1.Hadoop集群方式介绍

    1.1 hadoop1.x和hadoop2.x都支持的namenode+secondarynamenode方式

Hadoop2.7.1+Hbase1.2.1集群环境搭建(10)基于ZK的Hadoop HA集群安装 博客分类： Hadoop hadoopHAzkzookeeper
         优点：搭建环境简单，适合开发者模式下调试程序

         缺点：namenode作为很重要的服务，存在单点故障，如果namenode出问题，会导致整个集群不可用

    

    1.2.仅hadoop2.x支持的active namenode+standby namenode方式


Hadoop2.7.1+Hbase1.2.1集群环境搭建(10)基于ZK的Hadoop HA集群安装 博客分类： Hadoop hadoopHAzkzookeeper
       优点：为解决1.x中namenode单节点故障而生，充分保障Hadoop集群的高可用

       缺点：需要zookeeper最少3台，需要journalnode最少三台，目前最多支持2台namenode，不过节点可以复用，但是不建议

 

    1.3 Hadoop官网关于集群方式介绍

        1）单机Hadoop环境搭建

        http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/SingleCluster.html

        2）集群方式

            集群方式一(hadoop1.x和hadoop2.x都支持的namenode+secondarynamenode方式)

            http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/ClusterSetup.html

 

            集群方式二(仅hadoop2.x支持的active namenode+standby namenode方式，也叫HADOOP HA方式)，这种方式又分为HDFS的HA和YARN的HA单独分开讲解。

                     HDFS HA(zookeeper+journalnode)http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html

                     HDFS HA(zookeeper+NFS)http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailability

                     YARN HA(zookeeper)http://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html

 

        生产环境多采用HDFS(zookeeper+journalnode)(active NameNode+standby NameNode+JournalNode+DFSZKFailoverController+DataNode)+YARN(zookeeper)(active ResourceManager+standby ResourceManager+NodeManager)方式，这里我讲解的是仅hadoop2.x支持基于zookeeper的Hadoop HA集群方式，这种方式主要适用于生产环境。

 

2.基于zookeeper的Hadoop HA集群安装

    2.1 安装环境介绍

Hadoop2.7.1+Hbase1.2.1集群环境搭建(10)基于ZK的Hadoop HA集群安装 博客分类： Hadoop hadoopHAzkzookeeper
 

    2.2 安装前准备工作

        1）关闭防火墙

centos7防火墙操作介绍 
#centos7启动firewall
systemctl start firewalld.service
#centos7重启firewall
systemctl restart firewalld.service
#centos7停止firewall
systemctl stop firewalld.service 
#centos7禁止firewall开机启动
systemctl disable firewalld.service 
#centos7查看防火墙状态
firewall-cmd --state
#开放防火墙端口
vi /etc/sysconfig/iptables-config
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 6379 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 6380 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 6381 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 16379 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 16380 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 16381 -j ACCEPT

         这里我关闭防火墙，root下执行如下命令：

systemctl stop firewalld.service 
systemctl disable firewalld.service

 

        2）优化selinux

        作用：Hadoop主节点管理子节点是通过SSH实现的， SELinux不关闭的情况下无法实现，会限制ssh免密码登录。

        编辑/etc/selinux/config，修改前：

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected. 
# mls - Multi Level Security protection.
SELINUXTYPE=targeted

         修改后：

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
#SELINUX=enforcing
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected. 
# mls - Multi Level Security protection.
#SELINUXTYPE=targeted

         执行以下命令使selinux 修改立即生效：

setenforce 0

 

    3）机器名配置

        作用：Hadoop集群中机器IP可能变化导致集群间服务中断，所以在Hadoop中最好以机器名进行配置。

        修改各机器上文件/etc/hostname，配置主机名称如下：

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.185.31 hadoop31
192.168.185.32 hadoop32
192.168.185.33 hadoop33
192.168.185.34 hadoop34
192.168.185.35 hadoop35

         而centos7下各个机器的主机名设置文件为/etc/hostname，以hadoop31节点主机配置为例，配置如下：

#localdomain
hadoop31

 

    4)创建hadoop用户和组

        作用：后续单独以用户hadoop来管理Hadoop集群,防止其他用户误操作关闭Hadoop 集群

#以root用户创建hadoop用户和组创建hadoop用户和组 
groupadd hadoop 
useradd -g hadoop hadoop 
#修改用户密码
passwd hadoop

 

    5）用户hadoop免秘钥登录

        作用：Hadoop中主节点管理从节点是通过SSH协议登录到从节点实现的，而一般的SSH登录，都是需要输入密码验证的，为了Hadoop主节点方便管理成千上百的从节点，这里将主节点公钥拷贝到从节点，实现SSH协议免秘钥登录，我这里做的是所有主从节点之间机器免秘钥登录

#首先切换到上面的hadoop用户,这里我是在hadoop31机器上操作 
ssh hadoop31
su hadoop 
#生成非对称公钥和私钥，这个在集群中所有节点机器都必须执行,一直回车就行 
ssh-keygen -t rsa 
#通过ssh登录远程机器时，本机会默认将当前用户目录下的.ssh/authorized_keys带到远程机器进行验证，这里是/home/hadoop/.ssh/authorized_keys中公钥(来自其他机器上的/home/hadoop/.ssh/id_rsa.pub.pub),以下代码只在主节点执行就可以做到主从节点之间SSH免密码登录 
cd /home/hadoop/.ssh/ 
#首先将Master节点的公钥添加到authorized_keys 
cat id_rsa.pub>>authorized_keys 
#其次将Slaves节点的公钥添加到authorized_keys，这里我是在Hadoop31机器上操作的 
ssh hadoop@192.168.185.32 cat /home/hadoop/.ssh/id_rsa.pub>> authorized_keys 
ssh hadoop@192.168.185.33 cat /home/hadoop/.ssh/id_rsa.pub>> authorized_keys 
ssh hadoop@192.168.185.34 cat /home/hadoop/.ssh/id_rsa.pub>> authorized_keys 
ssh hadoop@192.168.185.35 cat /home/hadoop/.ssh/id_rsa.pub>> authorized_keys 
#必须设置修改/home/hadoop/.ssh/authorized_keys权限 
chmod 600 /home/hadoop/.ssh/authorized_keys 
#这里将Master节点的authorized_keys分发到其他slaves节点 
scp -r /home/hadoop/.ssh/authorized_keys hadoop@192.168.185.32:/home/hadoop/.ssh/ 
scp -r /home/hadoop/.ssh/authorized_keys hadoop@192.168.185.33:/home/hadoop/.ssh/ 
scp -r /home/hadoop/.ssh/authorized_keys hadoop@192.168.185.34:/home/hadoop/.ssh/ 
scp -r /home/hadoop/.ssh/authorized_keys hadoop@192.168.185.35:/home/hadoop/.ssh/

 

    6）JDK安装

        作用：Hadoop需要java环境支撑，而Hadoop2.7.1最少需要java版本1.7，安装如下：

#登录到到到hadoop用户下
su hadoop
#下载jdk-7u65-linux-x64.gz放置于/home/hadoop/java并解压
cd /home/hadoop/java
tar -zxvf jdk-7u65-linux-x64.gz
#编辑vi /home/hadoop/.bashrc,在文件末尾追加如下内容
export JAVA_HOME=/home/hadoop/java/jdk1.7.0_65 
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar 
export PATH=$PATH:$JAVA_HOME/bin 
#使得/home/hadoop/.bashrc配置生效
source /home/hadoop/.bashrc

         很多人是配置linux全局/etc/profile，这里不建议这么做，一旦有人在里面降级了java环境或者删除了java环境，就会出问题，建议的是在管理Hadoop集群的用户下面修改其.bashrc单独配置该用户环境变量

 

    7）zookeeper安装

#1登录hadoop用户并下载并解压zookeeper3.4.6
su hadoop
cd /home/hadoop 
tar -zxvf zookeeper-3.4.6.tar.gz 

#2在集群中各个节点中配置/etc/hosts，内容如下：
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.185.31 hadoop31 
192.168.185.32 hadoop32 
192.168.185.33 hadoop33 
192.168.185.34 hadoop34 
192.168.185.35 hadoop35

#3在集群中各个节点中创建zookeeper数据文件
ssh hadoop31
cd /home/hadoop 
#zookeeper数据存放位置
mkdir -p /opt/hadoop/zookeeper 
ssh hadoop32
cd /home/hadoop 
#zookeeper数据存放位置
mkdir -p /opt/hadoop/zookeeper 
ssh hadoop33
cd /home/hadoop 
#zookeeper数据存放位置
mkdir -p /opt/hadoop/zookeeper 
ssh hadoop34
cd /home/hadoop 
#zookeeper数据存放位置
mkdir -p /opt/hadoop/zookeeper 
ssh hadoop35
cd /home/hadoop 
#zookeeper数据存放位置
mkdir -p /opt/hadoop/zookeeper 

#4配置zoo.cfg
ssh hadoop31
cd /home/hadoop/zookeeper-3.4.6/conf
cp zoo_sample.cfg zoo.cfg
vi zoo.cfg
#内容如下
initLimit=10 
syncLimit=5 
dataDir=/opt/hadoop/zookeeper 
clientPort=2181 
#数据文件保存最近的3个快照，默认是都保存，时间长的话会占用很大磁盘空间
autopurge.snapRetainCount=3
#单位为小时，每小时清理一次快照数据
autopurge.purgeInterval=1
server.1=hadoop31:2888:3888 
server.2=hadoop32:2888:3888 
server.3=hadoop33:2888:3888
server.4=hadoop34:2888:3888 
server.5=hadoop35:2888:3888 
#5在hadoop31上远程复制分发安装文件
scp -r /home/hadoop/zookeeper-3.4.6 hadoop@hadoop32:/home/hadoop/ 
scp -r /home/hadoop/zookeeper-3.4.6 hadoop@hadoop33:/home/hadoop/ 
scp -r /home/hadoop/zookeeper-3.4.6 hadoop@hadoop34:/home/hadoop/ 
scp -r /home/hadoop/zookeeper-3.4.6 hadoop@hadoop35:/home/hadoop/ 

#6在集群中各个节点设置myid必须为数字 
ssh hadoop31 
echo "1" > /opt/hadoop/zookeeper/myid 
ssh hadoop32 
echo "2" > /opt/hadoop/zookeeper/myid 
ssh hadoop33 
echo "3" > /opt/hadoop/zookeeper/myid 

#7.各个节点如何启动zookeeper
ssh hadoop31
/home/hadoop/zookeeper-3.4.6/bin/zkServer.sh start

#8.各个节点如何关闭zookeeper
ssh hadoop31
/home/hadoop/zookeeper-3.4.6/bin/zkServer.sh stop 

#9.各个节点如何查看zookeeper状态
ssh hadoop31
/home/hadoop/zookeeper-3.4.6/bin/zkServer.sh status 

#10.各个节点如何通过客户端访问zookeeper上目录数据
ssh hadoop31
/home/hadoop/zookeeper-3.4.6/bin/zkCli.sh -server hadoop31:2181,hadoop32:2181,hadoop33:2181,hadoop34:2181,hadoop35:2181

 

    2.3 Hadoop HA安装

        1）hadoop-2.7.1.tar.gz

#下载hadoop-2.7.1.tar.gz放置于/home/hadoop下并解压,这里我在hadoop31操作
ssh hadoop31
su hadoop
cd /home/hadoop
tar –zxvf hadoop-2.7.1.tar.gz

      

        2）core-site.xml

        修改配置文件/home/hadoop/hadoop-2.7.1/etc/hadoop/core-site.xml

<?xml version="1.0" encoding="UTF-8"?>  
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>  
<configuration>  
	<!-- 开启垃圾回收站功能,HDFS文件删除后先进入垃圾回收站，垃圾回收站最长保留数据时间为1天，超过一天后就删除 --> 
	<property>
		<name>fs.trash.interval</name>
		<value>1440</value>
	</property>
	<!-- Hadoop HA部署方式下namenode访问地址，bigdatacluster-ha是名字可自定义，后面hdfs-site.xml会用到 --> 
	<property>
		<name>fs.defaultFS</name>  
		<value>hdfs:// bigdatacluster-ha</value>
	</property>
	<!--hadoop访问文件的IO操作都需要通过代码库。因此，在很多情况下，io.file.buffer.size都被用来设置SequenceFile中用到的读/写缓存大小。不论是对硬盘或者是网络操作来讲，较大的缓存都可以提供更高的数据传输，但这也就意味着更大的内存消耗和延迟。这个参数要设置为系统页面大小的倍数，以byte为单位，默认值是4KB，一般情况下，可以设置为64KB（65536byte）,这里设置128K-->  
	<property>  
		<name>io.file.buffer.size</name>  
		<value>131072</value>  
	</property> 
	<!-- 指定hadoop临时目录 --> 
	<property> 
		<name>hadoop.tmp.dir</name> 
		<value>/opt/hadoop/tmp</value> 
	</property> 
	<!-- 指定zookeeper地址 --> 
	<property> 
		<name>ha.zookeeper.quorum</name> 
		<value>hadoop31:2181,hadoop32:2181,hadoop33:2181,hadoop34:2181,hadoop35:2181</value> 
	</property> 
	<property> 
		<name>ha.zookeeper.session-timeout.ms</name> 
		<value>300000</value> 
	</property>
	<!-- 指定Hadoop压缩格式，Apache官网下载的安装包不支持snappy，需要自己编译安装，如何编译安装包我在博客http://aperise.iteye.com/blog/2254487有讲解,不适用snappy的话可以不配置 --> 
	<property>  
		<name>io.compression.codecs</name>  
		<value>org.apache.hadoop.io.compress.SnappyCodec</value>  
	</property>  
</configuration>

 

        3）hdfs-site.xml

        修改配置文件/home/hadoop/hadoop-2.7.1/etc/hadoop/hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?> 
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> 
<configuration> 
	<!--指定hdfs的nameservice为bigdatacluster-ha，需要和core-site.xml中的保持一致 --> 
	<property> 
		<name>dfs.nameservices</name> 
		<value>bigdatacluster-ha</value> 
	</property> 
	<!—指定磁盘预留多少空间，防止磁盘被撑满用完，单位为bytes --> 
	<property>
		<name>dfs.datanode.du.reserved</name>
		<value>107374182400</value>
	</property>
	<!-- bigdatacluster-ha下面有两个NameNode，分别是namenode1，namenode2 --> 
	<property> 
		<name>dfs.ha.namenodes.bigdatacluster-ha</name> 
		<value>namenode1,namenode2</value> 
	</property> 
	<!-- namenode1的RPC通信地址,这里端口要和core-site.xml中fs.defaultFS保持一致 --> 
	<property> 
		<name>dfs.namenode.rpc-address.bigdatacluster-ha.namenode1</name> 
		<value>hadoop31:9000</value> 
	</property> 
	<!-- namenode1的http通信地址 --> 
	<property> 
		<name>dfs.namenode.http-address.bigdatacluster-ha.namenode1</name> 
		<value>hadoop31:50070</value> 
	</property> 
	<!-- namenode2的RPC通信地址,这里端口要和core-site.xml中fs.defaultFS保持一致 --> 
	<property> 
		<name>dfs.namenode.rpc-address.bigdatacluster-ha.namenode2</name> 
		<value>hadoop32:9000</value> 
	</property> 
	<!-- namenode2的http通信地址 --> 
	<property> 
		<name>dfs.namenode.http-address.bigdatacluster-ha.namenode2</name> 
		<value>hadoop32:50070</value> 
	</property> 

	<!-- 指定NameNode的元数据在JournalNode上的存放位置 --> 
	<property> 
		<name>dfs.namenode.shared.edits.dir</name> 
		<value>qjournal://hadoop31:8485;hadoop32:8485;hadoop33:8485;hadoop34:8485;hadoop35:8485/bigdatacluster-ha</value> 
	</property> 

	<!-- 配置失败自动切换实现方式 --> 
	<property> 
		<name>dfs.client.failover.proxy.provider.bigdatacluster-ha</name>
		<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
	</property> 

	<!-- 配置隔离机制,主要用户远程管理监听其他机器相关服务 --> 
	<property> 
		<name>dfs.ha.fencing.methods</name> 
		<value>sshfence</value> 
	</property> 
	<!-- 使用隔离机制时需要ssh免密码登陆 --> 
	<property> 
		<name>dfs.ha.fencing.ssh.private-key-files</name> 
		<value>/home/hadoop/.ssh/id_rsa</value> 
	</property> 

	<!-- 指定NameNode的元数据在JournalNode上的存放位置 --> 
	<property> 
		<name>dfs.journalnode.edits.dir</name> 
		<value>/opt/hadoop/journal</value> 
	</property> 

	<!--指定支持高可用自动切换机制--> 
	<property> 
		<name>dfs.ha.automatic-failover.enabled</name> 
		<value>true</value> 
	</property> 

	<!--指定namenode名称空间的存储地址--> 
	<property> 
		<name>dfs.namenode.name.dir</name>    
		<value>file:/opt/hadoop/hdfs/name</value> 
	</property> 

	<!--指定datanode数据存储地址--> 
	<property> 
		<name>dfs.datanode.data.dir</name> 
		<value>file:/opt/hadoop/hdfs/data</value> 
	</property> 

	<!--指定数据冗余份数--> 
	<property> 
		<name>dfs.replication</name> 
		<value>3</value> 
	</property> 

	<!--指定可以通过web访问hdfs目录--> 
	<property> 
		<name>dfs.webhdfs.enabled</name> 
		<value>true</value> 
	</property> 

	<property> 
		<name>ha.zookeeper.quorum</name> 
		<value>hadoop31:2181,hadoop32:2181,hadoop33:2181,hadoop34:2181,hadoop35:2181</value> 
	</property> 

	
	<property>
		<name>dfs.namenode.handler.count</name>
		<value>600</value>
		<description>The number of server threads for the namenode.</description>
	</property>
	<property>
		<name>dfs.datanode.handler.count</name>
		<value>600</value>
		<description>The number of server threads for the datanode.</description>
	</property>
	<property>
		<name>dfs.client.socket-timeout</name>
		<value>600000</value>
	</property>
	<property>  
		<!--这里设置Hadoop允许打开最大文件数，默认4096，不设置的话会提示xcievers exceeded错误-->  
		<name>dfs.datanode.max.transfer.threads</name>  
		<value>409600</value>  
	</property>   
</configuration>

 

        4）mapred-site.xml

        修改配置文件/home/hadoop/hadoop-2.7.1/etc/hadoop/mapred-site.xml

<?xml version="1.0"?>   
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>   
<configuration>   
    <!-- 配置MapReduce运行于yarn中 -->   
    <property>   
        <name>mapreduce.framework.name</name>   
        <value>yarn</value>   
    </property>    
    <property>  
        <name>mapreduce.job.maps</name>  
        <value>12</value>  
    </property>  
    <property>  
        <name>mapreduce.job.reduces</name>  
        <value>12</value>  
    </property>  
  
    <!-- 指定Hadoop压缩格式，Apache官网下载的安装包不支持snappy，需要自己编译安装，如何编译安装包我在博客http://aperise.iteye.com/blog/2254487有讲解,不适用snappy的话可以不配置 -->   
    <property>  
        <name>mapreduce.output.fileoutputformat.compress</name>  
        <value>true</value>  
        <description>Should the job outputs be compressed?  
        </description>  
    </property>  
    <property>  
        <name>mapreduce.output.fileoutputformat.compress.type</name>  
        <value>RECORD</value>  
        <description>If the job outputs are to compressed as SequenceFiles, how should  
               they be compressed? Should be one of NONE, RECORD or BLOCK.  
        </description>  
    </property>  
    <property>  
        <name>mapreduce.output.fileoutputformat.compress.codec</name>  
        <value>org.apache.hadoop.io.compress.SnappyCodec</value>  
        <description>If the job outputs are compressed, how should they be compressed?  
        </description>  
    </property>  
    <property>  
        <name>mapreduce.map.output.compress</name>  
        <value>true</value>  
        <description>Should the outputs of the maps be compressed before being  
               sent across the network. Uses SequenceFile compression.  
        </description>  
    </property>  
    <property>  
        <name>mapreduce.map.output.compress.codec</name>  
        <value>org.apache.hadoop.io.compress.SnappyCodec</value>  
        <description>If the map outputs are compressed, how should they be   
               compressed?  
        </description>  
    </property>    
</configuration>

 

 

        5）yarn-site.xml

        修改配置文件/home/hadoop/hadoop-2.7.1/etc/hadoop/yarn-site.xml

<?xml version="1.0"?> 
<configuration> 
	<!--日志聚合功能yarn.log start------------------------------------------------------------------------>  
	<property> 
		<name>yarn.log-aggregation-enable</name> 
		<value>true</value> 
	</property> 
	<!--在HDFS上聚合的日志最长保留多少秒。3天-->  
	<property> 
		<name>yarn.log-aggregation.retain-seconds</name> 
		<value>259200</value> 
	</property> 
	<!--日志聚合功能yarn.log end-------------------------------------------------------------------------->  

	<!--resourcemanager失联后重新链接的时间-->  
	<property>  
		<name>yarn.resourcemanager.connect.retry-interval.ms</name>
		<value>2000</value>  
	</property> 

	<!--配置resourcemanager start------------------------------------------------------------------------->
	<property> 
		<name>yarn.resourcemanager.zk-address</name> 
		<value>hadoop31:2181,hadoop32:2181,hadoop33:2181,hadoop34:2181,hadoop35:2181</value>  
	</property> 
	<property>  
		<name>yarn.resourcemanager.cluster-id</name>  
		<value>besttonecluster-yarn</value>  
	</property>  
	<!--开启resourcemanager HA,默认为false-->  
	<property>  
		<name>yarn.resourcemanager.ha.enabled</name>  
		<value>true</value>  
	</property>  
	<property> 
		<name>yarn.resourcemanager.ha.rm-ids</name> 
		<value>rm1,rm2</value> 
	</property> 
	<property> 
		<name>yarn.resourcemanager.hostname.rm1</name> 
		<value>hadoop31</value> 
	</property>     
	<property> 
		<name>yarn.resourcemanager.hostname.rm2</name> 
		<value>hadoop32</value> 
	</property> 
	<!--配置rm1--> 
	<property>
		<name>yarn.resourcemanager.webapp.address.rm1</name>
		<value>hadoop31:8088</value>
	</property>
	<!--配置rm2-->  
	<property>
		<name>yarn.resourcemanager.webapp.address.rm2</name>
		<value>hadoop32:8088</value>
	</property>
	<!--开启故障自动切换-->  
	<property>
		<name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
		<value>true</value>
	</property>
	<property>
		<name>yarn.resourcemanager.ha.automatic-failover.embedded</name>
		<value>true</value>
	</property>
	<property>
		<name>yarn.resourcemanager.ha.automatic-failover.zk-base-path</name>
		<value>/yarn-leader-election</value>
	</property>

	<!--开启自动恢复功能-->  
	<property> 
		<name>yarn.resourcemanager.recovery.enabled</name>  
		<value>true</value>  
	</property> 
	<property>
		<name>yarn.resourcemanager.store.class</name>
		<value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
	</property>
	<!--配置resourcemanager end--------------------------------------------------------------------------->

	<!--配置nodemanager start----------------------------------------------------------------------------->
	<property>  
		<name>yarn.nodemanager.aux-services</name>  
		<value>mapreduce_shuffle</value>  
	</property>  
	<property>  
		<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>  
		<value>org.apache.hadoop.mapred.ShuffleHandler</value>  
	</property>  
	<!--配置nodemanager end------------------------------------------------------------------------------->
</configuration>

 

        6）slaves

        修改配置文件/home/hadoop/hadoop-2.7.1/etc/hadoop/slaves

Hadoop31
Hadoop32
Hadoop33
Hadoop34
Hadoop35

 

        7）hadoop-env.sh和yarn-env.sh

        在/home/hadoop/hadoop-2.7.1/etc/hadoop/hadoop-env.sh和/home/hadoop/hadoop-2.7.1/etc/hadoop/yarn-env.sh中配置JAVA_HOME

export JAVA_HOME=/home/hadoop/java/jdk1.7.0_65

 

        8）bashrc

        当前用户hadoop生效，在用户目录下/home/hadoop/.bashrc增加如下配置

export HADOOP_HOME=/home/hadoop/hadoop2.7.1
export PATH=${HADOOP_HOME}/bin:${PATH}

 

        9）分发安装文件到其他机器

#这里我是在hadoop31上操作
scp -r /home/hadoop/hadoop-2.7.1 hadoop@hadoop32:/home/hadoop/
scp -r /home/hadoop/hadoop-2.7.1 hadoop@ hadoop33:/home/hadoop/
scp -r /home/hadoop/hadoop-2.7.1 hadoop@ hadoop34:/home/hadoop/ 
scp -r /home/hadoop/hadoop-2.7.1 hadoop@ hadoop35:/home/hadoop/

 

    2.4 Hadoop HA初次启动

        1）启动zookeeper

ssh hadoop31
/home/hadoop/zookeeper-3.4.6/bin/zkServer.sh start
ssh hadoop32
/home/hadoop/zookeeper-3.4.6/bin/zkServer.sh start
ssh hadoop33
/home/hadoop/zookeeper-3.4.6/bin/zkServer.sh start
ssh hadoop34
/home/hadoop/zookeeper-3.4.6/bin/zkServer.sh start
ssh hadoop35
/home/hadoop/zookeeper-3.4.6/bin/zkServer.sh start

         #jps查看是否有QuorumPeerMain 进程

        #/home/hadoop/zookeeper-3.4.6/ bin/zkServer.sh status查看zookeeper状态

        #/home/hadoop/zookeeper-3.4.6/ bin/zkServer.sh stop关闭zookeeper

 

        2）格式化zookeeper上hadoop-ha目录

/home/hadoop/hadoop-2.7.1/bin/hdfs zkfc –formatZK
#可以通过如下方法检查zookeeper上是否已经有Hadoop HA目录
# /home/hadoop/zookeeper-3.4.6/bin/zkCli.sh -server hadoop31:2181,hadoop32:2181,hadoop33:2181,hadoop34:2181,hadoop35:2181 
#ls /

 

        3）启动namenode日志同步服务journalnode

ssh hadoop31
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemon.sh start journalnode 
ssh hadoop32
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemon.sh start journalnode 
ssh hadoop33
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemon.sh start journalnode 
ssh hadoop34
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemon.sh start journalnode 
ssh hadoop35
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemon.sh start journalnode

 

        4）格式化namenode

#这步操作只能在namenode服务节点hadoop31或者hadoop32执行中一台上执行
ssh hadoop31
/home/hadoop/hadoop-2.7.1/bin/hdfs namenode -format

 

        5）启动namenode、同步备用namenode、启动备用namenode

#启动namenode
ssh hadoop31
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemon.sh start namenode 
#同步备用namenode、启动备用namenode
ssh hadoop32
/home/hadoop/hadoop-2.7.1/bin/hdfs namenode -bootstrapStandby 
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemon.sh start namenode

 

        6)启动DFSZKFailoverController

ssh hadoop31
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemon.sh start zkfc 
ssh hadoop32
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemon.sh start zkfc

 

        7）启动datanode

#注意hadoop-daemons.sh datanode是启动所有datanode，而hadoop-daemon.sh datanode是启动单个datanode
ssh hadoop31
/home/hadoop/hadoop-2.7.1/sbin/hadoop-daemons.sh start datanode

 

        8）启动yarn

#在hadoop31上启动resouremanager,在hadoop31，hadoop32，hadoop33，hadoop34，hadoop35上启动nodemanager
ssh hadoop31
/home/hadoop/hadoop-2.7.1/sbin/start-yarn.sh 
#在hadoop31上启动备用resouremanager
ssh hadoop32
/home/hadoop/hadoop-2.7.1/sbin/yarn-daemon.sh start resourcemanager

         至此，Hadoop 基于zookeeper的高可用集群就安装成功，并且启动了。

 

  • Hadoop2.7.1+Hbase1.2.1集群环境搭建(10)基于ZK的Hadoop HA集群安装 博客分类： Hadoop hadoopHAzkzookeeper
  • 大小: 23.8 KB
  • Hadoop2.7.1+Hbase1.2.1集群环境搭建(10)基于ZK的Hadoop HA集群安装 博客分类： Hadoop hadoopHAzkzookeeper
  • 大小: 165.7 KB
  • Hadoop2.7.1+Hbase1.2.1集群环境搭建(10)基于ZK的Hadoop HA集群安装 博客分类： Hadoop hadoopHAzkzookeeper
  • 大小: 129.9 KB
相关标签： hadoop HA zk zookeeper

上一篇： 分布式系统ID的生成方法之UUID、数据库、算法、Redis、Leaf方案

下一篇： Hadoop 3.x主要变化(相对于Hadoop 2.x)

推荐阅读