欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

Hbase-2.0.0_02_常用操作

程序员文章站 2022-10-04 21:19:22
主要是常用的hbase shell命令,包括表的创建与删除,表数据的增删查【hbase没有修改】;以及hbase的导出与导入。 参考教程:HBase教程 参考博客:hbase shell基础和常用命令详解 参考博客:hbase shell常用命令和filter 参考博客:hbase导入导出数据 1. ......

 

       主要是常用的hbase shell命令,包括表的创建与删除,表数据的增删查【hbase没有修改】;以及hbase的导出与导入。

参考教程:hbase教程

参考博客:

参考博客:

参考博客:

 

1. hbase命令

1.1. 命令的登录与退出

1 1.1.    命令的登录与退出
2 [yun@mini03 ~]$ hbase shell  # 登录hbase
3 slf4j: class path contains multiple slf4j bindings.
4 ………………
5 version 2.0.0, r7483b111e4da77adbfc8062b3b22cbe7c2cb91c1, sun apr 22 20:26:55 pdt 2018
6 took 0.0126 seconds
7 hbase(main):001:0> quit   # 退出hbase
8 [yun@mini03 ~]$

 

1.2. 常用命令

名称

命令表达式

创建表

create '表名', '列族名1','列族名2','列族名n'

查看所有表

list

描述表

describe  ‘表名’

判断表存在

exists  '表名'

判断是否禁用启用表

is_enabled '表名'

is_disabled ‘表名’

添加记录     

put  ‘表名’, ‘rowkey’, ‘列族 : 列‘  ,  '值'

查看记录rowkey下的所有数据

get  '表名' , 'rowkey'

查看表中的记录总数

count  '表名'

获取某个列族

get '表名','rowkey','列族'

获取某个列族的某个列

get '表名','rowkey','列族:列’

删除记录

delete  ‘表名’ ,‘行名’ , ‘列族:列'

删除整行

deleteall '表名','rowkey'

删除一张表

先要屏蔽该表,才能对该表进行删除

第一步 disable ‘表名’ ,第二步  drop '表名'

清空表

truncate '表名'

查看所有记录

scan "表名" 

查看某个表某个列中所有数据

scan "表名" , {columns=>'列族名:列名'}

更新记录

就是重写一遍,进行覆盖,hbase没有修改,都是追加

 

说明:

  1、列族里边可以*添加子列很方便。如果列族下没有子列,加不加冒号都是可以的。

hbase> put 't1', 'r1', 'c1', 'value', ts1

  2、t1指表名,r1指行键名,c1指列名,value指单元格值。ts1指时间戳,一般都省略掉了。

 

1.3. 示例1-建表,查看表结构信息,删表

hbase(main):006:0* create 'user','info1','info2','info3'  # 创建表 
created table user
took 2.9927 seconds 
=> hbase::table - user
hbase(main):007:0> list  # 查看所有表
table
user 
1 row(s)
took 0.0468 seconds 
=> ["user"]
hbase(main):008:0> describe 'user' # 描述表
table user is enabled  ##### 当前表可用 
user 
column families description
{name => 'info1', versions => '1', evict_blocks_on_close => 'false', new_version_behavior => 'false', keep_deleted_cells => 'false', cache_data_on_write => 'false', data_block_encoding => 'none', ttl => 'forever', min_versions => '0', replication_scope => '0', bloomfilter => 'row', cache_index_on_write => 'false', in_memory => 'false', cache_blooms_on_write => 'false', prefetch_blocks_on_open => 'false', compression => 'none', blockcache => 'true', blocksize => '65536'} 
{name => 'info2', versions => '1', evict_blocks_on_close => 'false', new_version_behavior => 'false', keep_deleted_cells => 'false', cache_data_on_write => 'false', data_block_encoding => 'none', ttl => 'forever', min_versions => '0', replication_scope => '0', bloomfilter => 'row', cache_index_on_write => 'false', in_memory => 'false', cache_blooms_on_write => 'false', prefetch_blocks_on_open => 'false', compression => 'none', blockcache => 'true', blocksize => '65536'} 
{name => 'info3', versions => '1', evict_blocks_on_close => 'false', new_version_behavior => 'false', keep_deleted_cells => 'false', cache_data_on_write => 'false', data_block_encoding => 'none', ttl => 'forever', min_versions => '0', replication_scope => '0', bloomfilter => 'row', cache_index_on_write => 'false', in_memory => 'false', cache_blooms_on_write => 'false', prefetch_blocks_on_open => 'false', compression => 'none', blockcache => 'true', blocksize => '65536'}  
3 row(s)
took 0.1933 seconds
hbase(main):011:0* exists 'user'  # 判断表是否存在
table user does exist
took 0.0398 seconds 
=> true
hbase(main):026:0> is_enabled 'user'  # 判断表是否可用
true
took 0.0149 seconds
=> true
hbase(main):027:0> is_disabled 'user' # 判断表是否不可用
false
took 0.0148 seconds 
=> 1
hbase(main):015:0> drop 'user'  # 删除表【必须先将表置为不可用】

error: table user is enabled. disable it first.

drop the named table. table must first be disabled:
  hbase> drop 't1'
  hbase> drop 'ns1:t1'

took 0.0342 seconds
hbase(main):018:0* disable 'user'  # 将表置为不可用
took 0.5544 seconds 
hbase(main):019:0> drop 'user'  # 之后删除表
took 0.5143 seconds 
hbase(main):020:0> list
table 
0 row(s)
took 0.0687 seconds 
=> []

  

1.4. 示例2-添加表数据和查询表数据

hbase(main):024:0> create 'user','info1','info2','info3' # 创建表
created table user
took 1.2717 seconds 
=> hbase::table - user
hbase(main):025:0> list  # 查看所有表信息
table 
user  
1 row(s)
took 0.0466 seconds
=> ["user"]
hbase(main):042:0* put 'user','1234','info1:name','zhang'  # 添加数据
took 0.0634 seconds
hbase(main):043:0> scan 'user' # 查看所有记录
row                                              column+cell 
 1234                                            column=info1:name, timestamp=1533740907054, value=zhang
 1 row(s)
took 0.0776 seconds 
hbase(main):044:0> put 'user','1234','info1:name','zhangsan' # 再次添加【实际类似修改】
took 0.0055 seconds
hbase(main):045:0> scan 'user'
row                                              column+cell 
 1234                                            column=info1:name, timestamp=1533740935956, value=zhangsan 
1 row(s)
took 0.0160 seconds   
hbase(main):046:0> put 'user','1234','info2:name','zhang' # 在另一个列族添加数据
took 0.0160 seconds
hbase(main):047:0> put 'user','1234','info2:age','23'  # 在另一个列族的另外一个列添加数据
took 0.0133 seconds
hbase(main):048:0> scan 'user'
row                                              column+cell 
 1234                                            column=info1:name, timestamp=1533740935956, value=zhangsan 
 1234                                            column=info2:age, timestamp=1533741066465, value=23  
 1234                                            column=info2:name, timestamp=1533741052169, value=zhang
1 row(s)
took 0.0171 seconds      
hbase(main):050:0* put 'user','12345','info1:name','lisi'
took 0.0125 seconds 
hbase(main):051:0> put 'user','12345','info2:age','25'
took 0.0143 seconds 
hbase(main):052:0> scan 'user'
row                                              column+cell 
 1234                                            column=info1:name, timestamp=1533740935956, value=zhangsan 
 1234                                            column=info2:age, timestamp=1533741066465, value=23        
 1234                                            column=info2:name, timestamp=1533741052169, value=zhang    
 12345                                           column=info1:name, timestamp=1533741585906, value=lisi     
 12345                                           column=info2:age, timestamp=1533741595725, value=25        
2 row(s)
took 0.0179 seconds   
hbase(main):058:0* count 'user'  # 查看表中的记录总数【根据 row keys 判断】
2 row(s)
took 0.1065 seconds     
=> 2
hbase(main):053:0> get 'user','1234'  # 查看记录rowkey下的所有数据
column                                           cell  
 info1:name                                      timestamp=1533740935956, value=zhangsan  
 info2:age                                       timestamp=1533741066465, value=23        
 info2:name                                      timestamp=1533741052169, value=zhang     
1 row(s)
took 0.0371 seconds      
hbase(main):067:0* get 'user','1234','info2' # 获取某个列族
column                                           cell     
 info2:age                                       timestamp=1533741066465, value=23    
 info2:name                                      timestamp=1533741052169, value=zhang 
1 row(s)
took 0.0305 seconds      
hbase(main):068:0> get 'user','1234','info2:name' # 获取某个列族的某个列
column                                           cell   
 info2:name                                      timestamp=1533741052169, value=zhang 
1 row(s)
took 0.0182 seconds      

  

1.5. 示例3-删除行数据

hbase(main):072:0> get 'user','1234'
column                                           cell                                   
 info1:name                                      timestamp=1533740935956, value=zhangsan
 info2:address                                   timestamp=1533742368985, value=china    
 info2:age                                       timestamp=1533741066465, value=23       
 info2:name                                      timestamp=1533741052169, value=zhang    
1 row(s)
took 0.0146 seconds  
hbase(main):073:0> delete 'user','1234','info2:age' # 删除指定行,指定列记录
#### 注意其中:  delete 'user','1234','info2' 是无效的 ★★★
took 0.0288 seconds                                                                       
hbase(main):074:0> get 'user','1234'
column                                           cell      
 info1:name                                      timestamp=1533740935956, value=zhangsan  
 info2:address                                   timestamp=1533742368985, value=china     
 info2:name                                      timestamp=1533741052169, value=zhang     
1 row(s)
took 0.0140 seconds  
hbase(main):100:0* deleteall 'user','1234' # 删除整行
took 0.0119 seconds                                     
hbase(main):101:0> get 'user','1234'
column                                           cell   
0 row(s)
took 0.0145 seconds  

  

1.6. 示例4-条件scan和truncate

hbase(main):122:0* scan 'user'
row                                              column+cell                                              
 1234                                            column=info2:address, timestamp=1533743416815, value=cn  
 1234                                            column=info2:age, timestamp=1533743407616, value=20      
 1234                                            column=info2:name, timestamp=1533743396872, value=wangwu 
 12345                                           column=info1:name, timestamp=1533741585906, value=lisi   
 12345                                           column=info2:age, timestamp=1533741595725, value=25      
2 row(s)
took 0.0241 seconds
hbase(main):123:0> scan 'user',{columns => 'info2'}
row                                              column+cell  
 1234                                            column=info2:address, timestamp=1533743416815, value=cn 
 1234                                            column=info2:age, timestamp=1533743407616, value=20     
 1234                                            column=info2:name, timestamp=1533743396872, value=wangwu
 12345                                           column=info2:age, timestamp=1533741595725, value=25     
2 row(s)
took 0.0161 seconds    
hbase(main):124:0> scan 'user',{columns => 'info2:age'}
row                                              column+cell  
 1234                                            column=info2:age, timestamp=1533743407616, value=20 
 12345                                           column=info2:age, timestamp=1533741595725, value=25 
2 row(s)
took 0.0158 seconds
hbase(main):128:0* truncate
truncate            truncate_preserve   
hbase(main):128:0* truncate 'user'  # 截断表
truncating 'user' table (it may take a while):
disabling table...
truncating table...
took 2.6156 seconds                                            
hbase(main):129:0> scan 'user'
row                                              column+cell   
0 row(s)
took 0.1305 seconds

  

 

2. hbase导入导出数据

       在实际应用hbase过程中,经常需要将生产环境中的数据备份,或者需要在开发环境中利用生产环境的数据(更加符合实际情况),因此hbase存储的数据的导入导出必不可少。

 

2.1. 表信息准备

hbase(main):002:0> create 'zhang','userinfo','baseinfo','eduinfo','workinfo'
created table zhang
took 1.4311 seconds     
=> hbase::table - zhang
hbase(main):005:0* put 'zhang','12345','userinfo:username','zhangsan'
took 0.3751 seconds                                                  
hbase(main):006:0> put 'zhang','12345','userinfo:password','111111'
took 0.0220 seconds                                                  
hbase(main):007:0> put 'zhang','12345','baseinfo:name','zhangsan'
took 0.0136 seconds                                                  
hbase(main):008:0> put 'zhang','12345','baseinfo:age','22'
took 0.0136 seconds                                                  
hbase(main):009:0> put 'zhang','12345','baseinfo:name','zhangnew'
took 0.0106 seconds                                                  
hbase(main):010:0> put 'zhang','12345','baseinfo:age','25'
took 0.0138 seconds
hbase(main):013:0> put 'zhang','12345','eduinfo:pri_school','star school'
took 0.0106 seconds                                                  
hbase(main):014:0> scan 'zhang'
row                                              column+cell                                                   
 12345                                           column=baseinfo:age, timestamp=1533884261796, value=25                
 12345                                           column=baseinfo:name, timestamp=1533884258020, value=zhangnew         
 12345                                           column=eduinfo:pri_school, timestamp=1533884297216, value=star school 
 12345                                           column=userinfo:password, timestamp=1533884246132, value=111111       
 12345                                           column=userinfo:username, timestamp=1533884241334, value=zhangsan     
1 row(s)
took 0.0179 seconds 

  

2.2. hbase表导出到hdfs

[yun@mini02 ~]$ hbase org.apache.hadoop.hbase.mapreduce.export zhang /zhang/hbase/zhang_tab 
……………………
2018-08-10 15:01:27,354 info  [main] mapreduce.job: the url to track the job: http://mini02:8088/proxy/application_1533865678790_0001/
2018-08-10 15:01:27,355 info  [main] mapreduce.job: running job: job_1533865678790_0001
2018-08-10 15:01:39,564 info  [main] mapreduce.job: job job_1533865678790_0001 running in uber mode : false
2018-08-10 15:01:39,566 info  [main] mapreduce.job:  map 0% reduce 0%
2018-08-10 15:01:52,384 info  [main] mapreduce.job:  map 100% reduce 0%
2018-08-10 15:01:53,416 info  [main] mapreduce.job: job job_1533865678790_0001 completed successfully
2018-08-10 15:01:53,554 info  [main] mapreduce.job: ps in occupied slots (ms)=9661
		total time spent by all reduces in occupied slots (ms)=0
		total time spent by all map tasks (ms)=9661
		total vcore-milliseconds taken by all map tasks=9661
		total megabyte-milliseconds taken by all map tasks=9892864
	map-reduce framework
		map input records=1
		map output records=1
		input split bytes=124
		spilled records=0
		failed shuffles=0
		merged map outputs=0
		gc time elapsed (ms)=493
		cpu time spent (ms)=7670
		physical memory (bytes) snapshot=259244032
		virtual memory (bytes) snapshot=2178433024
		total committed heap usage (bytes)=89653248
	hbase counters
		bytes_in_remote_results=252
		bytes_in_results=252
		millis_between_nexts=1258
		not_serving_region_exception=0
		num_scanner_restarts=0
		num_scan_results_stale=0
		regions_scanned=1
		remote_rpc_calls=1
		remote_rpc_retries=0
		rows_filtered=0
		rows_scanned=1
		rpc_calls=1
		rpc_retries=0
	file input format counters 
		bytes read=0
	file output format counters 
		bytes written=364

  

hdfs查看

1 [yun@mini03 ~]$ hadoop fs -ls /zhang/hbase/zhang_tab
2 found 2 items
3 -rw-r--r--   3 yun supergroup          0 2018-08-10 15:01 /zhang/hbase/zhang_tab/_success
4 -rw-r--r--   3 yun supergroup        364 2018-08-10 15:01 /zhang/hbase/zhang_tab/part-m-00000

 

2.3. hbase表导出到本地集群系统

       具体导出到哪台机器,需要自己去查找

 

[yun@mini02 hbase_data]$ hbase org.apache.hadoop.hbase.mapreduce.export zhang file:///app/software/hbase_data/zhang_tab
……………………
2018-08-10 16:14:17,619 info  [main] mapreduce.jobsubmitter: submitting tokens for job: job_1533865678790_0003
2018-08-10 16:14:18,249 info  [main] impl.yarnclientimpl: submitted application application_1533865678790_0003
2018-08-10 16:14:18,290 info  [main] mapreduce.job: the url to track the job: http://mini02:8088/proxy/application_1533865678790_0003/
2018-08-10 16:14:18,290 info  [main] mapreduce.job: running job: job_1533865678790_0003
2018-08-10 16:14:29,932 info  [main] mapreduce.job: job job_1533865678790_0003 running in uber mode : false
2018-08-10 16:14:29,935 info  [main] mapreduce.job:  map 0% reduce 0%
2018-08-10 16:14:39,958 info  [main] mapreduce.job:  map 100% reduce 0%
2018-08-10 16:14:40,984 info  [main] mapreduce.job: job job_1533865678790_0003 completed successfully
2018-08-10 16:14:41,131 info  [main] mapreduce.job: aps in occupied slots (ms)=7246
		total time spent by all reduces in occupied slots (ms)=0
		total time spent by all map tasks (ms)=7246
		total vcore-milliseconds taken by all map tasks=7246
		total megabyte-milliseconds taken by all map tasks=7419904
	map-reduce framework
		map input records=1
		map output records=1
		input split bytes=124
		spilled records=0
		failed shuffles=0
		merged map outputs=0
		gc time elapsed (ms)=242
		cpu time spent (ms)=5580
		physical memory (bytes) snapshot=263946240
		virtual memory (bytes) snapshot=2176024576
		total committed heap usage (bytes)=94896128
	hbase counters
		bytes_in_remote_results=252
		bytes_in_results=252
		millis_between_nexts=959
		not_serving_region_exception=0
		num_scanner_restarts=0
		num_scan_results_stale=0
		regions_scanned=1
		remote_rpc_calls=1
		remote_rpc_retries=0
		rows_filtered=0
		rows_scanned=1
		rpc_calls=1
		rpc_retries=0
	file input format counters 
		bytes read=0
	file output format counters 
		bytes written=376

  

导出到本地的文件

1 [yun@mini04 task_1533865678790_0003_m_000000]$ pwd
2 /app/software/hbase_data/zhang_tb/_temporary/1/task_1533865678790_0003_m_000000
3 [yun@mini04 task_1533865678790_0003_m_000000]$ ll
4 total 4
5 -rw-r--r-- 1 yun yun 364 aug 10 16:14 part-m-00000

 

2.4. 导入数据到hbase

       把刚才导出的数据,导入到hbase。

 

方式1【该方式导入失败

1 # 把 hbase 中表 zhang 的数据给删掉,然后导入
2 hbase(main):058:0* deleteall 'zhang','12345'
3 took 0.0541 seconds          \
4 hbase(main):059:0> scan 'zhang'
5 row                                              column+cell  
6 0 row(s)
7 took 0.0247 seconds

       结果导入时死活没数据。。。

 

方式2【导入正常

 1 # 对表 zhang 进行truncate,然后导入数据   或者 删除表 zhang,再重建该表
 2 hbase(main):018:0* truncate 'zhang'
 3 truncating 'zhang' table (it may take a while):
 4 disabling table...
 5 truncating table...
 6 took 1.8954 seconds
 7 hbase(main):020:0> scan 'zhang'
 8 row                                              column+cell  
 9 0 row(s)
10 took 1.4118 seconds         

 

导入数据命令

[yun@mini04 task_1533865678790_0003_m_000000]$ hbase org.apache.hadoop.hbase.mapreduce.import zhang file:///app/software/hbase_data/zhang_tb/_temporary/1/task_1533865678790_0003_m_000000/part-m-00000
……………………

  

导入后查看

hbase(main):023:0* scan 'zhang'
row                                              column+cell                                                            
 12345                                           column=baseinfo:age, timestamp=1533884261796, value=25                 
 12345                                           column=baseinfo:name, timestamp=1533884258020, value=zhangnew          
 12345                                           column=eduinfo:pri_school, timestamp=1533884297216, value=star school  
 12345                                           column=userinfo:password, timestamp=1533884246132, value=111111        
 12345                                           column=userinfo:username, timestamp=1533884241334, value=zhangsan      
1 row(s)
took 0.0439 seconds  

  

方式3【导入正常

 1 # 新建表导入  要求:表结构一样
 2 hbase(main):070:0* create 'zhang_test','userinfo','baseinfo','eduinfo','workinfo'
 3 created table zhang_test
 4 took 0.7815 seconds        \
 5 => hbase::table - zhang_test
 6 hbase(main):071:0> list  # 查看所有表
 7 table                                  
 8 scores                                 
 9 user                                   
10 zhang                                  
11 zhang_test                             
12 4 row(s)
13 took 0.0280 seconds                    
14 => ["scores", "user", "zhang", "zhang_test"]

 

导入数据命令

[yun@mini04 task_1533865678790_0003_m_000000]$ hbase org.apache.hadoop.hbase.mapreduce.import zhang_test file:///app/software/hbase_data/zhang_tb/_temporary/1/task_1533865678790_0003_m_000000/part-m-00000
………………

  

导入后查看

1 hbase(main):074:0* scan 'zhang_test'
2 row                                              column+cell                                                          
3  12345                                           column=baseinfo:age, timestamp=1533884261796, value=25               
4  12345                                           column=baseinfo:name, timestamp=1533884258020, value=zhangnew        
5  12345                                           column=eduinfo:pri_school, timestamp=1533884297216, value=star school
6  12345                                           column=userinfo:password, timestamp=1533884246132, value=111111      
7  12345                                           column=userinfo:username, timestamp=1533884241334, value=zhangsan    
8 1 row(s)
9 took 0.0544 seconds