利用ycsb测试hbase性能
程序员文章站
2023-12-25 20:06:51
...
java 、maven、ycsb 的安装及配置见这篇博客： http://blog.csdn.net/hs794502825/article/details/17309845 本篇博客主要介绍 hbase 的安装，以及利用 ycsb 对 hbase 进行基本的测试 步骤一： 在http://mirrors.hust.edu.cn/apache/hbase/下载hbase稳定版：
java 、maven、ycsb 的安装及配置见这篇博客：http://blog.csdn.net/hs794502825/article/details/17309845
本篇博客主要介绍 hbase 的安装，以及利用 ycsb 对 hbase 进行基本的测试
步骤一：
在http://mirrors.hust.edu.cn/apache/hbase/下载hbase稳定版：
hbase-0.94.14.tar.gz 存放于 /home/hs/program 目录下
cd /home/hs/program
tar -zxvf hbase-0.94.14.tar.gz
步骤二：
修改/home/hs/program/YCSB/hbase目录下的pom.xml文件（根据你所使用的hbase版本和hadoop版本，注意红色部分）：
步骤三：
因为更改了步骤二中的文件，所以需要重新编译YCSB项目
cd /home/hs/program/YCSB
mvn clean package | tee mvn-clean-package.dat //将输出同时输出到屏幕和文件
显示与下类似信息，则说明编译成功：
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] YCSB Root ............................................. SUCCESS [7.540s]
[INFO] Core YCSB ............................................. SUCCESS [13.293s]
[INFO] Cassandra DB Binding .................................. SUCCESS [15.861s]
[INFO] HBase DB Binding ...................................... SUCCESS [2:21.346s]
[INFO] Hypertable DB Binding ................................. SUCCESS [5.296s]
[INFO] DynamoDB DB Binding ................................... SUCCESS [7.347s]
[INFO] ElasticSearch Binding ................................. SUCCESS [18.140s]
[INFO] JDBC DB Binding ....................................... SUCCESS [5.495s]
[INFO] Mapkeeper DB Binding .................................. SUCCESS [4.054s]
[INFO] Mongo DB Binding ...................................... SUCCESS [3.368s]
[INFO] OrientDB Binding ...................................... SUCCESS [4.199s]
[INFO] Redis DB Binding ...................................... SUCCESS [3.261s]
[INFO] Voldemort DB Binding .................................. SUCCESS [4.113s]
[INFO] YCSB Release Distribution Builder ..................... SUCCESS [13.764s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4 minutes 8 seconds
[INFO] Finished at: Mon Dec 16 12:43:12 CST 2013
[INFO] Final Memory: 94M/227M
步骤四：
成功编译YCSB之后，在/home/hs/program/YCSB/distribution/target目录下会有ycsb-0.1.4.tar.gz
将该文件解压到/home/hs/program目录下：
tar -zxvf /home/hs/program/YCSB/distribution/target -C /home/hs/program
步骤五：
将/home/hs/program/hbase-0.94.14/lib中的以下文件：
slf4j-api-*.jar
slf4j-log4j12-*.jar
zookeeper-*.jar
拷贝到/home/hs/program/ycsb-0.1.4/hbase-binding/lib中
步骤六：
使用/home/hs/program/hbase-0.94.14/conf中的hbase-site.xml
替换掉/home/hs/program/ycsb-0.1.4/hbase-binding/conf中的hbase-site.xml
步骤七：
执行测试
终端1：
进入hbase的安装目录：
执行：./bin/start-hbase.sh
显示：
starting master, logging to /home/hs/program/hbase-0.94.14/bin/../logs/hbase-hs-master-hs-virtual-machine.out
接着执行：
./bin/hbase shell
create 'usertable','cf'
终端2中：
cd /home/hs/program/ycsb-0.1.4
hs@hs-virtual-machine:~/program/ycsb-0.1.4$ ./bin/ycsb load hbase -P workloads/workloada -p hosts=localhost -p columnfamily=cf > ./my-results/load-hbase-a
hs@hs-virtual-machine:~/program/ycsb-0.1.4$ ./bin/ycsb run hbase -P workloads/workloada -p hosts=localhost -p columnfamily=cf > ./my-results/run-hbase-a
得到如下结果：
YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p hosts=localhost -p columnfamily=cf -t
[OVERALL], RunTime(ms), 5886.0
[OVERALL], Throughput(ops/sec), 169.89466530750934
[UPDATE], Operations, 487
[UPDATE], AverageLatency(us), 9366.180698151951
[UPDATE], MinLatency(us), 33
[UPDATE], MaxLatency(us), 4459758
[UPDATE], 95thPercentileLatency(ms), 0
[UPDATE], 99thPercentileLatency(ms), 0
[UPDATE], Return=0, 486
......
[READ], Operations, 514
[READ], AverageLatency(us), 2482.817120622568
[READ], MinLatency(us), 707
[READ], MaxLatency(us), 176125
[READ], 95thPercentileLatency(ms), 2
[READ], 99thPercentileLatency(ms), 29
[READ], Return=0, 514