欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  科技

centos7安装CDH6文档

程序员文章站 2022-04-14 10:53:33
1、准备工作 1.1 环境 centos7、jdk8、mysql5.7、python2.7、CDH6 1.2文件下载 1.2.1 cloudmanger地址 https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPMS/x86_64/ clouder ......

1、准备工作

1.1 环境

centos7、jdk8、mysql5.7、python2.7、cdh6

1.2文件下载

1.2.1 cloudmanger地址

https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/rpms/x86_64/ 
  • cloudera-manager-agent-6.0.0-530873.el7.x86_64.rpm
  • cloudera-manager-daemons-6.0.0-530873.el7.x86_64.rpm
  • cloudera-manager-server-6.0.0-530873.el7.x86_64.rpm

    1.2.2 资源下载地址

    https://archive.cloudera.com/cdh6/6.0.0/parcels/
  • cdh-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel
  • cdh-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256
  • manifest.json

2、开始安装

2.1 服务器配置

2.1.1 修改host和主机名

  1. vim /etc/hosts修改host,例如

    198.168.12.1 cdh1

    192.168.12.2 cdh2

    192.168.12.3 cdh3

    不能有大写字母和下划线
  2. vim /etc/hostname修改主机名,修改完后三台机器分别执行hostname ***(当前节点的主机名)

    2.1.2 配置免密登录

    1. ssh-keygen -t rsa -p '' -f ~/.ssh/id_rsa b、ssh-copy-id 每台服务器地址

      2.1.3 关闭防火墙

    2. 临时关闭 systemctl stop firewalld
    3. 禁止开机启动 systemctl disable firewalld

      2.1.4 安装jdk,配置环境变量

      这个大家都会,如果不会可以参考这个教程

      https://www.cnblogs.com/ocean-sky/p/8392444.html

      2.1.5 其他依赖

      每台服务器yum安装依赖包:redhat-lsb、portmap、httpd、mod_ssl、openssl-devel、python-psycopg2、mysql-python、chkconfig、python、chkconfig、bind-utils、libxslt、zlib、sqlite、cyrus-sasl-plain、cyrus-sasl-gssapi、fuse、fuse-libs

      2.2 mysql5.7安装(在主节点安装)

      2.2.1 安装

    4. wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm
    5. yum -y install mysql57-community-release-el7-10.noarch.rpm
    6. yum -y install mysql-community-server
    7. 启动 systemctl start mysqld.service
    8. 查看运行状态 systemctl status mysqld.service
    9. 查看初始密码 grep "password" /var/log/mysqld.log
    10. 登录后修改密码(尽量复杂一点,太简单了验证不通过,也可以查看如何关闭mysql校验规则) alter user 'root'@'localhost' identified by 'new password';
    11. 卸载yum源防止自动更新 yum -y remove mysql57-community-release-el7-10.noarch
    12. 安装mysql-shared-compat或mysql-shared软件包(这是cloudera manager agent软件包安装所必需的)yum install mysql-shared-advanced

      2.2.2 创建cdh相关数据库

  • create database scm default character set utf8 default collate utf8_general_ci;
  • grant all on scm.* to 'scm'@'%' identified by 'password';
  • create database amon default character set utf8 default collate utf8_general_ci;
  • grant all on amon.* to 'amon'@'%' identified by 'password';
  • create database rman default character set utf8 default collate utf8_general_ci;
  • grant all on rman.* to 'rman'@'%' identified by 'password';
  • create database hue default character set utf8 default collate utf8_general_ci;
  • grant all on hue.* to 'hue'@'%' identified by 'password';
  • create database metastore default character set utf8 default collate utf8_general_ci;
  • grant all on metastore.* to 'hive'@'%' identified by 'password';
  • create database sentry default character set utf8 default collate utf8_general_ci;
  • grant all on sentry.* to 'sentry'@'%' identified by 'password';
  • create database nav default character set utf8 default collate utf8_general_ci;
  • grant all on nav.* to 'nav'@'%' identified by 'password';
  • create database oozie default character set utf8 default collate utf8_general_ci;
  • grant all on oozie.* to 'oozie'@'%' identified by 'password';

    2.3 cdh安装

    1. mkdir /opt/cloudera-manager
    2. 将cloudera-manager-agent-6.0.0-530873.el7.x86_64.rpm、cloudera-manager-daemons-6.0.0-530873.el7.x86_64.rpm 、cloudera-manager-server-6.0.0-530873.el7.x86_64.rpm
      复制到/opt/cloudera-manager文件夹中
    3. 按 daemons、server、agent顺序解压
    4. 修改文件/etc/cloudera-scm-agent/config.ini中serverhost指定server节点
    5. 下载mysql-connector-java-5.1.47.jar包并复制为/usr/share/java/mysql-connector-java.jar
    6. 在主节点执行
      /opt/cloudera/cm/schema/scmpreparedatabase.sh mysql -h 主节点ip --scm-host 主节点ip scm scm
    7. mkdir -p /opt/cloudera/parcel-repo
    8. 将cdh-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel、cdh-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256、manifest.json复制到/opt/cloudera/parcel-repo
    9. 将cdh-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256 重命名为 cdh-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha
    10. 在manifest.json文件中,找到对应版本的秘钥,复制到cdh-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha文件中。
    11. 启动systemctl start cloudera-scm-agent、systemctl start cloudera-scm-server

      2.4 集群配置

      2.4.1 禁用透明大页面压缩

      echo never > /sys/kernel/mm/transparent_hugepage/defrag

      echo never > /sys/kernel/mm/transparent_hugepage/enabled

      2.4.1 分配组件

      访问http://192.168.1.1:7180开始配置集群组件
      因为是先安装集群后编写文档,没有截图就没有详细些。