利用ycsb测试cassandra性能
程序员文章站
2024-02-05 17:18:46
...
java 、maven、ycsb 的安装及配置见这篇博客: http://blog.csdn.net/hs794502825/article/details/17309845 本篇博客主要介绍 cassandra 的安装,以及利用 ycsb 对 cassandra 进行基本的测试 在 http://cassandra.apache.org/download/ 上面下载了最新版本的
java 、maven、ycsb 的安装及配置见这篇博客:http://blog.csdn.net/hs794502825/article/details/17309845
本篇博客主要介绍 cassandra 的安装,以及利用 ycsb 对 cassandra 进行基本的测试
在 http://cassandra.apache.org/download/ 上面下载了最新版本的 apache-cassandra-2.0.3-bin.tar.gz 存放在 /home/hs/program目录下
cd /home/hs/program tar -zxvf apache-cassandra-2.0.3-bin.tar.gz然后为 cassandra 设置环境变量
sudo gedit /etc/profile
在文件的最后加入:
#set cassandra environment export CASSANDRA_HOME=/home/hs/program/apache-cassandra-2.0.3 export PATH=$PATH:$CASSANDRA_HOME/bin:$CASSANDRA_HOME/lib之后,我就直接以普通用户(hs)执行 cassandra -f
显示了很多错误,大多都是与此相关:
无法生成目录:/var/lib/cassandra/......以及/var/log/cassandra/......
var目录的权限如下:
drwxr-xr-x 13 root root 4096 2013-12-14 21:50
只有所有者root对其有写的权限
cassandra 需要生成数据和日志信息的目录,默认情况下就是
/var/lib/cassandra/ 和 /var/log/cassandra/
然后我就在 hs 用户下执行如下命令:
sudo mkdir /var/lib/cassandra sudo mkdir /var/log/cassandra chown -R hs:hs /var/lib/cassandra chown -R hs:hs /var/log/cassandra 如此一来,hs 就具有写 /var/lib/cassandra/ 和 /var/log/cassandra/ 的权限 在终端1中运行cassandra: cassandra -f 如果有 Listening for thrift clients... 则说明成功启动 cassandra 在终端2中运行cassandra-cli: cassandra-cli 显示:
Connected to: "Test Cluster" on 127.0.0.1/9160 Welcome to Cassandra CLI version 2.0.3 The CLI is deprecated and will be removed in Cassandra 3.0. Consider migrating to cqlsh. CQL is fully backwards compatible with Thrift data; see http://www.datastax.com/dev/blog/thrift-to-cql3
根据提示我就终止了 cassandra-cli,转而去使用 cqlsh
hs@hs-virtual-machine:~$ cqlsh Connected to Test Cluster at localhost:9160. [cqlsh 4.1.0 | Cassandra 2.0.3 | CQL spec 3.1.1 | Thrift protocol 19.38.0]
接下来在创建 keyspace 的时候,出现了如下错误:
cqlsh> create keyspace with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor = '1';
Bad Request: line 1:75 mismatched input ':' expecting '='
上面那一行创建 keyspace 的命令我是从 cqlsh 的官网上 copy 过来的,所以我不知道怎么解决,第一次接触 cqlsh
后来还是去使用 cassandra-cli(注意所有的命令都需要以;结束)
create keyspace usertable; use usertable; create column family data;
在终端3中运行ycsb:
./bin/ycsb load cassandra-10 -P workloads/workloada -p hosts=localhost -p columnfamily=data > ./my-results/load-cassandra-a
得到如下错误:
Loading workload... Starting test. InvalidRequestException(why:unconfigured columnfamily data) at org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20833) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:964) at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:950) at com.yahoo.ycsb.db.CassandraClient10.insert(CassandraClient10.java:477) at com.yahoo.ycsb.DBWrapper.insert(DBWrapper.java:148) at com.yahoo.ycsb.workloads.CoreWorkload.doInsert(CoreWorkload.java:461) at com.yahoo.ycsb.ClientThread.run(Client.java:269)
目测是 column family 的创建有问题
所以我在终端2中删除掉该 column family,然后重建
drop column family data; create column family data with column_type = 'Standard' and comparator = 'UTF8Type';
返回终端3重新运行ycsb:
./bin/ycsb load cassandra-10 -P workloads/workloada -p hosts=localhost -p columnfamily=data > ./my-results/load-cassandra-a
得到如下结果:
YCSB Client 0.1 Command line: -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/workloada -p hosts=localhost -p columnfamily=data -load [OVERALL], RunTime(ms), 2287.0 [OVERALL], Throughput(ops/sec), 437.25404459991256 [INSERT], Operations, 1000 [INSERT], AverageLatency(us), 1670.687 [INSERT], MinLatency(us), 476 [INSERT], MaxLatency(us), 280228 [INSERT], 95thPercentileLatency(ms), 3 [INSERT], 99thPercentileLatency(ms), 12 [INSERT], Return=0, 1000 ......
执行:
./bin/ycsb run cassandra-10 -P workloads/workloada -p hosts=localhost -p columnfamily=data > ./my-results/run-cassandra-a
得到如下结果:
YCSB Client 0.1 Command line: -db com.yahoo.ycsb.db.CassandraClient10 -P workloads/workloada -p hosts=localhost -p columnfamily=data -t [OVERALL], RunTime(ms), 5574.0 [OVERALL], Throughput(ops/sec), 179.4043774668102 [UPDATE], Operations, 475 [UPDATE], AverageLatency(us), 2095.0547368421053 [UPDATE], MinLatency(us), 327 [UPDATE], MaxLatency(us), 143093 [UPDATE], 95thPercentileLatency(ms), 9 [UPDATE], 99thPercentileLatency(ms), 33 [UPDATE], Return=0, 475 ...... [READ], Operations, 525 [READ], AverageLatency(us), 5054.5085714285715 [READ], MinLatency(us), 492 [READ], MaxLatency(us), 674167 [READ], 95thPercentileLatency(ms), 11 [READ], 99thPercentileLatency(ms), 85 [READ], Return=0, 525
下一阶段需要熟悉对 cassandra 的操作,以及使用 cqlsh