hive hiveconf 配置 博客分类： Hive hive

hadoop fs -count -q


配置环境变量：
# hadoop env
export HADOOP_HOME=
export HADOOP_CONF_DIR=
# hive env
export HIVE_HOME=
export HIVE_CONF_DIR=
export HIVE_AUX_JARS_PATH=
在HIVE_CONF_DIR下面需要有hive-default.xml

HiveConf    hive-default.xml  hive-site.xml

HiveConf:

hive.exec.mode.local.auto  true  决定 Hive 是否应该自动地根据输入文件大小，在本地运行（在GateWay运行）
hive.auto.convert.join false  是否根据输入小表的大小，自动将 Reduce 端的 Common Join 转化为 Map Join，从而加快大表关联小表的 Join 速度。

metastore相关配置：
hive.metastore.local  true or false   local or remote metastore 
HiveMetaStoreClient里面：
HiveMetaStoreClient(HiveConf conf, HiveMetaHookLoader hookLoader){
    boolean localMetaStore = conf.getBoolean("hive.metastore.local", false);
    if (localMetaStore) { //true,本地metastore
      // instantiate the metastore server handler directly instead of connecting
      // through the network
      client = new HiveMetaStore.HMSHandler("hive client", conf);
      open = true;
      return;
    }
}



Hive Configuration Variables
hive.exec.scratchdir This directory is used by hive to store the plans for different map/reduce stages for the query as well as to stored the intermediate outputs of these stages. /tmp/<user.name>/hive
hive.exec.compress.output Determines whether the output of the final map/reduce job in a query is compressed or not. false
hive.exec.compress.intermediate Determines whether the output of the intermediate map/reduce jobs in a query is compressed or not. false

mapred.reduce.tasks ：reduce数目
The default number of reduce tasks per job.  Typically set to a prime close to the number of available hosts.  Ignored when  mapred.job.tracker is "local". Hadoop set this to 1 by default, whereas hive uses -1 as its default value.  By setting this property to -1, Hive will automatically figure out what should be the number of reducers.


hive.default.fileformat
<property>
  <name>hive.default.fileformat</name>
  <value>SequenceFile</value>
  <description>Default file format for CREATE TABLE statement. Options are TextFile and SequenceFile. Users can explicitly say CREAT
E TABLE ... STORED AS &lt;TEXTFILE|SEQUENCEFILE&gt; to override</description>
</property>
默认创建的文件是SequenceFile，因为textfile在一些压缩算法下文件不可切分。