欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

Hue安装部署(Centos 7.2)

程序员文章站 2022-05-17 14:00:11
...

Hue安装部署(Centos 7.2)

2017-09-13 11:17:03 小强签名设计 阅读数 4635更多

分类专栏: 大数据生态圈

版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。

本文链接:https://blog.csdn.net/m0_37739193/article/details/77963240

一,HUE介绍

http://blog.csdn.net/liangyihuai/article/details/54137163

 

二,Hue的安装和部署
1,在root用户下用yum安装所依赖的系统包(能联网)

 

yum -y install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel openldap-devel python-devel sqlite-devel openssl-devel mysql-devel gmp-devel

(1).安装的时候发现并没有libtidy包,网上查了下所这个libtidy只对于单元测试而言,没安装上倒也不影响安装

(2)如果安装mysql-devel报错 ,类似这样Hue安装部署(Centos 7.2)

解决方案参考https://blog.csdn.net/weixin_39689084/article/details/102627542

 

[aaa@qq.com ~]$ tar -zxvf hue-3.9.0-cdh5.5.2.tar.gz
[aaa@qq.com ~]$ cd hue-3.9.0-cdh5.5.2

 

2,编译源码包
[aaa@qq.com hue-3.9.0-cdh5.5.2]$ make apps

 

3,修改配置文件hue.ini
[aaa@qq.com hue-3.9.0-cdh5.5.2]$ vi desktop/conf/hue.ini

 

[desktop]
  # 安全秘钥,存储session的加密处理
  secret_key=dfsahjfhflsajdhfljahl
  # Time zone name
  time_zone=Asia/Shanghai
  # Enable or disable debug mode.
  django_debug_mode=false
  # Enable or disable backtrace for server error
  http_500_debug_mode=false
  # This should be the hadoop cluster admin
  ## default_hdfs_superuser=hdfs
  default_hdfs_superuser=root
  # 不启用的模块
           
  #app_blacklist=impala,security,rdbms,jobsub,pig,hbase,sqoop,zookeeper,metastore,indexer

[[database]]
  # 数据库引擎类型
  engine=mysql
  # 数据库主机地址
  host=hadoop101
  # 数据库端口
  port=3306
  # 数据库用户名
  user=hue
  # 数据库密码
  password=123456
  # 数据库库名
  name=hue

说明:设置一个secret_key的目的是加密你的cookie让你的hue更安全。找到secret_key这个项,然后随便设置一串字符串(官方建议30-60个字符长度)

 

4,启动hue
[aaa@qq.com hue-3.9.0-cdh5.5.2]$ build/env/bin/supervisor

关于hue安装后出现KeyError: "Couldn't get user id for user hue"的解决方法

首先说明出现此问题的原因是因为你使用的root用户安装了hue,然后在root用户下使用的build/env/bin/supervisor,如下图所示那样:Hue安装部署(Centos 7.2)

 

知道了原因,就容易解决问题了。首先要创建个普通用户,并给添加密码。如果密码给的过于简单,会给出提示,忽略就行,如下图:Hue安装部署(Centos 7.2)

 

然后,我们要给刚才解压的hue文件改变拥有者属性,通过 chown -R 用户名 文件地址。如下图:

Hue安装部署(Centos 7.2)

 

最后,我们使用 su 命令切换用户,到hue文件夹下执行运行hue的命令就可以了。

[aaa@qq.com hue-3.9.0]# su hue
[aaa@qq.com hue-3.9.0]$ build/env/bin/supervisor

[INFO] Not running as root, skipping privilege drop
starting server with options:
{'daemonize': False,
 'host': 'h153',
 'pidfile': None,
 'port': 8888,
 'server_group': 'hue',
 'server_name': 'localhost',
 'server_user': 'hue',
 'ssl_certificate': None,
 'ssl_cipher_list': 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA',
 'ssl_private_key': None,
 'threads': 40,
 'workdir': None}


5,浏览器查看(http://192.168.205.153:8888)

页面切换中文版

(1).修改配置文件
vi /opt/soft/hadoop/hue-3.10.0/desktop/core/src/desktop/settings.py

LANGUAGE_CODE = 'zh_CN'
#LANGUAGE_CODE = 'en-us'

LANGUAGES = [
('en-us', _('English')),
('zh_CN', _('Simplified Chinese')),
]

(2).重新编译
make apps

(3).启动hue
build/env/bin/supervisor

 

Hue安装部署(Centos 7.2)

首次登陆设置任意用户名和密码作为超级用户,需要你记住,建议和Hadoop用户名一样

Hue安装部署(Centos 7.2)

 

三,hue集成hadoop2.x
1,hdfs-site.xml配置文件添加如下内容
[aaa@qq.com hadoop-2.6.0-cdh5.5.2]$ vi etc/hadoop/hdfs-site.xml

  1. <property>

  2. <name>dfs.webhdfs.enabled</name>

  3. <value>true</value>

  4. </property>


2,core-site.xml配置文件添加如下内容
[aaa@qq.com hadoop-2.6.0-cdh5.5.2]$ vi etc/hadoop/core-site.xml

  1. <property>

  2. <name>hadoop.proxyuser.hue.hosts</name>

  3. <value>*</value>

  4. </property>

  5. <property>

  6. <name>hadoop.proxyuser.hue.groups</name>

  7. <value>*</value>

  8. </property>

3,修改hue的hue.ini配置文件

###########################################################################
# Settings to configure your Hadoop cluster.
###########################################################################

[hadoop]

  # Configuration for HDFS NameNode
  # ------------------------------------------------------------------------
  [[hdfs_clusters]]
    # HA support by using HttpFs

    [[[default]]]
      # Enter the filesystem uri
      fs_defaultfs=hdfs://h153:8020

      # NameNode logical name.
      ## logical_name=

      # Use WebHdfs/HttpFs as the communication mechanism.
      # Domain should be the NameNode or HttpFs host.
      # Default port is 14000 for HttpFs.
      webhdfs_url=http://h153:50070/webhdfs/v1

      # Change this if your HDFS cluster is Kerberos-secured
      ## security_enabled=false

      # In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
      # have to be verified against certificate authority
      ## ssl_cert_ca_verify=True

      # Directory of the Hadoop configuration
      ## hadoop_conf_dir=$HADOOP_CONF_DIR when set or '/etc/hadoop/conf'
      hadoop_hdfs_home=/home/hadoop/hadoop-2.6.0-cdh5.5.2
      hadoop_bin=/home/hadoop/hadoop-2.6.0-cdh5.5.2/bin
      hadoop_conf_dir=/home/hadoop/hadoop-2.6.0-cdh5.5.2/etc/hadoop

  # Configuration for YARN (MR2)
  # ------------------------------------------------------------------------
  [[yarn_clusters]]

    [[[default]]]
      # Enter the host on which you are running the ResourceManager
      resourcemanager_host=h153

      # The port where the ResourceManager IPC listens on
      ## resourcemanager_port=8032

      # Whether to submit jobs to this cluster
      submit_to=True

      # Resource Manager logical name (required for HA)
      ## logical_name=

      # Change this if your YARN cluster is Kerberos-secured
      ## security_enabled=false

      # URL of the ResourceManager API
      resourcemanager_api_url=http://h153:8088

      # URL of the ProxyServer API
      proxy_api_url=http://h153:8088

      # URL of the HistoryServer API
      history_server_api_url=http://h153:19888

      # In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
      # have to be verified against certificate authority
      ## ssl_cert_ca_verify=True

    # HA support by specifying multiple clusters
    # e.g.

    # [[[ha]]]
      # Resource Manager logical name (required for HA)
      ## logical_name=my-rm-name


4,重新启动hdfs
[aaa@qq.com hadoop-2.6.0-cdh5.5.2]$ sbin/stop-all.sh
[aaa@qq.com hadoop-2.6.0-cdh5.5.2]$ sbin/start-all.sh
[aaa@qq.com hadoop-2.6.0-cdh5.5.2]$ sbin/mr-jobhistory-daemon.sh start historyserver

 

5,重新启动hue服务器
[aaa@qq.com hue-3.9.0-cdh5.5.2]$ build/env/bin/supervisor

 

6,查看测试结果
[aaa@qq.com ~]$ hadoop fs -ls /
drwxr-xr-x   - hadoop supergroup          0 2017-09-11 18:50 /hbase
drwxr-xr-x   - hadoop supergroup          0 2017-09-11 19:07 /input
drwxr-xr-x   - hadoop supergroup          0 2017-09-11 19:08 /output
drwx------   - hadoop supergroup          0 2017-09-11 20:14 /tmp
drwxr-xr-x   - hadoop supergroup          0 2017-09-11 20:14 /user
[aaa@qq.com ~]$ hadoop fs -cat /input/he.txt
hello world
hello hadoop
hello hive
Hue安装部署(Centos 7.2)

 

四,hue集成hive
1,配置hue.ini配置文件(找到[beeswax]段落,为什么叫[beeswax]而不是[hive]这是历史原因)

 

[beeswax]

  # Host where HiveServer2 is running.
  # If Kerberos security is enabled, use fully-qualified domain name (FQDN).
  hive_server_host=h153

  # Port where HiveServer2 Thrift server runs on.
  hive_server_port=10000

  # Hive configuration directory, where hive-site.xml is located
  hive_conf_dir=/home/hadoop/hive-1.1.0-cdh5.5.2/conf

  # Timeout in seconds for thrift calls to Hive service
  server_conn_timeout=120

  # Choose whether to use the old GetLog() thrift call from before Hive 0.14 to retrieve the logs.
  # If false, use the FetchResults() thrift call from Hive 1.0 or more instead.
  ## use_get_log_api=false

  # Set a LIMIT clause when browsing a partitioned table.
  # A positive value will be set as the LIMIT. If 0 or negative, do not set any limit.
  ## browse_partitioned_table_limit=250

  # The maximum number of partitions that will be included in the SELECT * LIMIT sample query for partitioned tables.
  ## sample_table_max_partitions=10

  # A limit to the number of rows that can be downloaded from a query.
  # A value of -1 means there will be no limit.
  # A maximum of 65,000 is applied to XLS downloads.
  ## download_row_limit=1000000

  # Hue will try to close the Hive query when the user leaves the editor page.
  # This will free all the query resources in HiveServer2, but also make its results inaccessible.
  ## close_queries=false

  # Thrift version to use when communicating with HiveServer2.
  # New column format is from version 7.
  ## thrift_version=7


2,修改hive的hive-site.xml文件配置metastore server
[aaa@qq.com hive-1.1.0-cdh5.5.2]$ vi conf/hive-site.xml

  1. <configuration>

  2. <property>

  3. <name>hive.metastore.uris</name>

  4. <value>thrift://h153:9083</value>

  5. </property>

  6. </configuration>


3,修改hdfs文件系统的/tmp权限
[aaa@qq.com hadoop-2.6.0-cdh5.5.2]$ bin/hdfs dfs -chmod -R o+x /tmp

 

4,创建表并插入数据
[aaa@qq.com hui]$ vi student.txt
101 zhangsan
102 lisi
103 laowu
[aaa@qq.com hive-1.1.0-cdh5.5.2]$ ./bin/hive
hive> create  table  student (id  int,  name  string) row  format  delimited  fields  terminated  by ' ' ;
hive> load data local inpath '/home/hadoop/hui/student.txt' into  table  student ;
hive> select  *  from  student ;
OK
101     zhangsan
102     lisi
103     laowu
Time taken: 1.627 seconds, Fetched: 3 row(s)
hive> exit;

 

5,启动metastore server(先启动)和hiveserver2
[aaa@qq.com hive-1.1.0-cdh5.5.2]$ nohup bin/hive --service metastore &
[aaa@qq.com hive-1.1.0-cdh5.5.2]$ ./bin/hiveserver2

 

6,查看配置是否生效
连接hive的时候出现这个错误:

Hue安装部署(Centos 7.2)

解决:yum -y install cyrus-sasl-plain cyrus-sasl-devel cyrus-sasl-gssapi

Hue安装部署(Centos 7.2)

 

五,hue集成Zookeeper
1,修改hue.ini中zookeeper相关配置

 

[zookeeper]

  [[clusters]]

    [[[default]]]
      # Zookeeper ensemble. Comma separated list of Host/Port.
      # e.g. localhost:2181,localhost:2182,localhost:2183
      host_ports=h153:2181,h154:2181,h155:2181


2,重新启动hue服务器
[aaa@qq.com hue-3.9.0-cdh5.5.2]$ build/env/bin/supervisor

3,查看效果(连接不太稳定,多连几次)

Hue安装部署(Centos 7.2)

 

六,hue集成hbase
1,修改hue.ini中hbase相关配置

 

[hbase]
  # Comma-separated list of HBase Thrift servers for clusters in the format of '(name|host:port)'.
  # Use full hostname with security.
  # If using Kerberos we assume GSSAPI SASL, not PLAIN.
  hbase_clusters=(Cluster|h153:9090)


2,启动HBase
[aaa@qq.com hbase-1.0.0-cdh5.5.2]$ bin/start-hbase.sh

 

3,HUE跟Hbase通讯是通过hbase-thrift,启动thrift server
[aaa@qq.com hbase-1.0.0-cdh5.5.2]$ bin/hbase-daemon.sh start thrift

 

4,创建表
[aaa@qq.com hbase-1.0.0-cdh5.5.2]$ bin/hbase shell
hbase(main):002:0> create 'hehe','cf'

 

5,查看测试结果

Hue安装部署(Centos 7.2)

 

参考:
http://blog.csdn.net/youfashion/article/details/51002563
http://blog.csdn.net/eason_oracle/article/details/52153061
http://www.cnblogs.com/zlslch/p/6820387.html

相关标签: hue hadoop