欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

1.hive介绍及安装配置

程序员文章站 2022-07-14 15:09:12
...

1.Hive介绍

  1. 数据库OLTP 在线事务处理
  2. 数据仓库OLAP 在线分析处理 延迟高
  3. 类sql方式(HQL)
  4. 使用sql方式,用来读写,管理位于分布式存储系统上的大型数据集的数据仓库技术
  5. hive是基于Hadoop的一个数据仓库工具,可以将结构化的数据文件映射为一张数据库表,并提供完整的sql查询功能,可以将sql语句转换为MapReduce任务进行运行。其优点是学习成本低,可以通过类SQL语句快速实现简单的MapReduce统计,不必开发专门的MapReduce应用,十分适合数据仓库的统计分析。
  6. hive使用的是hdfs做为存储
  7. 使用maprecude做为计算模型
  8. 用于海量数据计算分析

2.安装

  1. 基于hadoop完全分布式环境(搭建过程略)

  2. 下载hive包

    #wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.1.1/apache-hive-2.1.1-bin.tar.gz
  3. 解压到指定路径和创建软链接

    #tar xf apache-hive-2.1.1-bin.tar.gz -C /soft/
    #ln -s /soft/apache-hive-2.1.1-bin /soft/hive
  4. 配置环境变量

    #vim  /etc/profile
    添加一下内容
    HIVE_HOME=/soft/hive
    PATH=$PATH:$HIVE_HOME/bin
    #source /etc/profile
  5. 测试

    #hive --version

3.配置mysql相关

  1. 安装mysql(过程省略)
  2. 创建相关数据库并授权

    mysql>create database dbhive;
    mysql>use hive;
    mysql>grant all on dbhive.* to "hive"@"%" identified by "123456"; 

5. 配置hive

  1. 修改hive-site.xml(修改mysql相关配置)

    #cd /soft/hive/conf
    #vim hive-site.xml
     <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>123456</value>
    <description>password to use against metastore database</description>
     </property>
     <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    <description>Username to use against metastore database</description>
     </property>
     <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.10.103:3306/dbhive</value>
     </property>
  2. 复制mysql驱动程序

    #cp /root/mysql-connector-java-5.1.38-bin.jar /soft/hive/lib/
  3. 在mysql中初始化hive的schema

    #cd /soft/hive/bin/
    #./schematool -dbType mysql -initSchema
    which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/soft/jdk/bin:/soft/hadoop/bin:/soft/hadoop/sbin:/soft/hive/bin:/root/bin)
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/soft/apache-hive-2.1.1-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/soft/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
    Metastore connection URL:    jdbc:mysql://192.168.10.103:3306/dbhive
    Metastore Connection Driver :    com.mysql.jdbc.Driver
    Metastore connection User:   hive
    Starting metastore schema initialization to 2.1.0
    Initialization script hive-schema-2.1.0.mysql.sql
    Initialization script completed
    schemaTool completed
  4. 查看mysql

    mysql> use dbhive;
    Reading table information for completion of table and column names
    You can turn off this feature to get a quicker startup with -A
    
    Database changed
    mysql> show tables;
    +---------------------------+
    | Tables_in_dbhive          |
    +---------------------------+
    | AUX_TABLE                 |
    | BUCKETING_COLS            |
    | CDS                       |
    | COLUMNS_V2                |
    | COMPACTION_QUEUE          |
    | COMPLETED_COMPACTIONS     |
    | COMPLETED_TXN_COMPONENTS  |
    | DATABASE_PARAMS           |
    | DBS                       |
    | DB_PRIVS                  |
    | DELEGATION_TOKENS         |
    | FUNCS                     |
    | FUNC_RU                   |
    | GLOBAL_PRIVS              |
    | HIVE_LOCKS                |
    | IDXS                      |
    | INDEX_PARAMS              |
    | KEY_CONSTRAINTS           |
    | MASTER_KEYS               |
    | NEXT_COMPACTION_QUEUE_ID  |
    | NEXT_LOCK_ID              |
    | NEXT_TXN_ID               |
    | NOTIFICATION_LOG          |
    | NOTIFICATION_SEQUENCE     |
    | NUCLEUS_TABLES            |
    | PARTITIONS                |
    | PARTITION_EVENTS          |
    | PARTITION_KEYS            |
    | PARTITION_KEY_VALS        |
    | PARTITION_PARAMS          |
    | PART_COL_PRIVS            |
    | PART_COL_STATS            |
    | PART_PRIVS                |
    | ROLES                     |
    | ROLE_MAP                  |
    | SDS                       |
    | SD_PARAMS                 |
    | SEQUENCE_TABLE            |
    | SERDES                    |
    | SERDE_PARAMS              |
    | SKEWED_COL_NAMES          |
    | SKEWED_COL_VALUE_LOC_MAP  |
    | SKEWED_STRING_LIST        |
    | SKEWED_STRING_LIST_VALUES |
    | SKEWED_VALUES             |
    | SORT_COLS                 |
    | TABLE_PARAMS              |
    | TAB_COL_STATS             |
    | TBLS                      |
    | TBL_COL_PRIVS             |
    | TBL_PRIVS                 |
    | TXNS                      |
    | TXN_COMPONENTS            |
    | TYPES                     |
    | TYPE_FIELDS               |
    | VERSION                   |
    | WRITE_SET                 |
    +---------------------------+
    57 rows in set (0.00 sec)
  5. 登录hive

    #hive

转载于:https://www.cnblogs.com/liu-yao/p/1hive-jie-shao-ji-an-zhuang-pei-zhi.html