试用Cassandra,其写效率太差 博客分类: 技术学习 CassandraApacheMongoDBNoSQLMySQL
程序员文章站
2024-03-15 10:04:53
...
Cassandra是由Facebook贡献的开源分布式数据库。其遵从NoSql理念,是结合了Dynamo与BigTable的产物。最近Twitter和Digg都将其数据库由MySql迁往Cassandra。看到其发展势头不错,我就下载下来,做了个测试。
测试环境:
分别在两台机器上部署cassandra.这里说明下关键配置:
配置文件路径是%Cassandra_Home%\conf\storage-conf.xml
除增加了一个cassandra的服务器外,基本采用默认配置。
测试代码:
测试结果:1 写100条数据,耗时59922毫秒,将近1分钟;
测试环境:
分别在两台机器上部署cassandra.这里说明下关键配置:
配置文件路径是%Cassandra_Home%\conf\storage-conf.xml
<Storage> <!--两台机器的ClusterName必须相同,作为集群标识 --> <ClusterName>BurceServers</ClusterName> <AutoBootstrap>false</AutoBootstrap> <Keyspaces> <Keyspace Name="Keyspace1"> <KeysCachedFraction>0.01</KeysCachedFraction> <ColumnFamily CompareWith="BytesType" Name="Standard1"/> <ColumnFamily CompareWith="UTF8Type" Name="Standard2"/> <ColumnFamily CompareWith="TimeUUIDType" Name="StandardByUUID1"/> <ColumnFamily ColumnType="Super" CompareWith="UTF8Type" CompareSubcolumnsWith="UTF8Type" Name="Super1" Comment="A column family with supercolumns, whose column and subcolumn names are UTF8 strings"/> </Keyspace> </Keyspaces> <Partitioner>org.apache.cassandra.dht.RandomPartitioner</Partitioner> <InitialToken></InitialToken> <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy> <ReplicationFactor>1</ReplicationFactor> <CommitLogDirectory>c:/cassandra/lib/cassandra/commitlog</CommitLogDirectory> <DataFileDirectories> <DataFileDirectory>c:/cassandra/lib/cassandra/data</DataFileDirectory> </DataFileDirectories> <CalloutLocation>c:/cassandra/lib/cassandra/callouts</CalloutLocation> <StagingFileDirectory>c:/cassandra/lib/cassandra/staging</StagingFileDirectory> <!--在这里可以添加多个cassandra服务器--> <Seeds> <Seed>10.219.101.101</Seed> <Seed>10.219.101.121</Seed> </Seeds> <RpcTimeoutInMillis>5000</RpcTimeoutInMillis> <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB> <!--监听地址必须是本机IP--> <ListenAddress>10.219.101.101</ListenAddress> <StoragePort>7000</StoragePort> <ControlPort>7001</ControlPort> <!--基于Thrift的cassandra客户端监听地址--> <ThriftAddress>10.219.101.101</ThriftAddress> <ThriftPort>9160</ThriftPort> <ThriftFramedTransport>false</ThriftFramedTransport> <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB> <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB> <MemtableSizeInMB>64</MemtableSizeInMB> <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions> <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes> <ConcurrentReads>8</ConcurrentReads> <ConcurrentWrites>32</ConcurrentWrites> <CommitLogSync>periodic</CommitLogSync> <CommitLogSyncPeriodInMS>10000</CommitLogSyncPeriodInMS> <GCGraceSeconds>864000</GCGraceSeconds> <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB> </Storage>
除增加了一个cassandra的服务器外,基本采用默认配置。
测试代码:
/** * */ package com.tpri.sis.test; import java.io.UnsupportedEncodingException; import java.nio.charset.Charset; import me.prettyprint.cassandra.service.CassandraClient; import org.apache.cassandra.service.Cassandra; import org.apache.cassandra.service.ColumnPath; import org.apache.cassandra.service.ConsistencyLevel; import org.apache.cassandra.service.InvalidRequestException; import org.apache.cassandra.service.TimedOutException; import org.apache.cassandra.service.UnavailableException; import org.apache.commons.lang.RandomStringUtils; import org.apache.commons.lang.StringUtils; import org.apache.thrift.TException; import org.apache.thrift.protocol.TBinaryProtocol; import org.apache.thrift.protocol.TProtocol; import org.apache.thrift.transport.TSocket; import org.apache.thrift.transport.TTransport; import org.apache.thrift.transport.TTransportException; /** * @author brucepang * */ public class CassandraClientDemo { /** * */ public CassandraClientDemo() { } /** * @param args */ public static void main(String[] args) { try { TTransport tr = new TSocket("10.219.101.101", 9160); TProtocol pro = new TBinaryProtocol(tr); Cassandra.Client cli = new Cassandra.Client(pro); tr.open(); String key = null; String name, age; ColumnPath namePath = new ColumnPath("Standard1", null, "name" .getBytes("UTF-8")); ColumnPath agePath = new ColumnPath("Standard1", null, "age" .getBytes("UTF-8")); String keySpace = "Keyspace1"; long time = 0; long l1 = System.currentTimeMillis(); for (int i = 0; i < 100; i++) { key = String.valueOf(i); name = RandomStringUtils.random(5,"abcdefghefsdf"); time = System.currentTimeMillis(); cli.insert(keySpace, key, namePath, name.getBytes("UTF-8"), time, ConsistencyLevel.ONE); cli.insert(keySpace, key, agePath, key.getBytes("UTF-8"), time, ConsistencyLevel.ONE); } long l2 = System.currentTimeMillis(); long ch = l2 - l1; System.out.println(ch); } catch (TTransportException e) { e.printStackTrace(); } catch (UnsupportedEncodingException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (InvalidRequestException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (UnavailableException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (TimedOutException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (TException e) { // TODO Auto-generated catch block e.printStackTrace(); } } }
测试结果:1 写100条数据,耗时59922毫秒,将近1分钟;