欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

试用Cassandra,其写效率太差 博客分类: 技术学习 CassandraApacheMongoDBNoSQLMySQL 

程序员文章站 2024-03-15 10:04:53
...
Cassandra是由Facebook贡献的开源分布式数据库。其遵从NoSql理念,是结合了Dynamo与BigTable的产物。最近Twitter和Digg都将其数据库由MySql迁往Cassandra。看到其发展势头不错,我就下载下来,做了个测试。

测试环境:
   
    分别在两台机器上部署cassandra.这里说明下关键配置:
配置文件路径是%Cassandra_Home%\conf\storage-conf.xml
<Storage>
<!--两台机器的ClusterName必须相同,作为集群标识 -->
    <ClusterName>BurceServers</ClusterName>
  <AutoBootstrap>false</AutoBootstrap>

    <Keyspaces>
    <Keyspace Name="Keyspace1">
            <KeysCachedFraction>0.01</KeysCachedFraction>
            <ColumnFamily CompareWith="BytesType" Name="Standard1"/>
      <ColumnFamily CompareWith="UTF8Type" Name="Standard2"/>
      <ColumnFamily CompareWith="TimeUUIDType" Name="StandardByUUID1"/>
      <ColumnFamily ColumnType="Super"
                    CompareWith="UTF8Type"
                    CompareSubcolumnsWith="UTF8Type"
                    Name="Super1"
                    Comment="A column family with supercolumns, whose column and subcolumn names are UTF8 strings"/>
    </Keyspace>
  </Keyspaces>

    <Partitioner>org.apache.cassandra.dht.RandomPartitioner</Partitioner>

    <InitialToken></InitialToken>

  <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>

    <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>

    <ReplicationFactor>1</ReplicationFactor>

  <CommitLogDirectory>c:/cassandra/lib/cassandra/commitlog</CommitLogDirectory>
  <DataFileDirectories>
      <DataFileDirectory>c:/cassandra/lib/cassandra/data</DataFileDirectory>
  </DataFileDirectories>
  <CalloutLocation>c:/cassandra/lib/cassandra/callouts</CalloutLocation>
  <StagingFileDirectory>c:/cassandra/lib/cassandra/staging</StagingFileDirectory>

<!--在这里可以添加多个cassandra服务器-->
    <Seeds>
      <Seed>10.219.101.101</Seed>
<Seed>10.219.101.121</Seed>
  </Seeds>


    <RpcTimeoutInMillis>5000</RpcTimeoutInMillis>
    <CommitLogRotationThresholdInMB>128</CommitLogRotationThresholdInMB>

<!--监听地址必须是本机IP-->
    <ListenAddress>10.219.101.101</ListenAddress>
   <StoragePort>7000</StoragePort>
    <ControlPort>7001</ControlPort>
<!--基于Thrift的cassandra客户端监听地址-->
    <ThriftAddress>10.219.101.101</ThriftAddress>
    <ThriftPort>9160</ThriftPort>
    <ThriftFramedTransport>false</ThriftFramedTransport>


    <SlicedBufferSizeInKB>64</SlicedBufferSizeInKB>

   <ColumnIndexSizeInKB>64</ColumnIndexSizeInKB>

    <MemtableSizeInMB>64</MemtableSizeInMB>
  
  <MemtableObjectCountInMillions>0.1</MemtableObjectCountInMillions>
    <MemtableFlushAfterMinutes>60</MemtableFlushAfterMinutes>

    <ConcurrentReads>8</ConcurrentReads>
  <ConcurrentWrites>32</ConcurrentWrites>

    <CommitLogSync>periodic</CommitLogSync>
    <CommitLogSyncPeriodInMS>10000</CommitLogSyncPeriodInMS>
  <GCGraceSeconds>864000</GCGraceSeconds>
  <BinaryMemtableSizeInMB>256</BinaryMemtableSizeInMB>

</Storage>


除增加了一个cassandra的服务器外,基本采用默认配置。

测试代码:

/**
 * 
 */
package com.tpri.sis.test;

import java.io.UnsupportedEncodingException;
import java.nio.charset.Charset;

import me.prettyprint.cassandra.service.CassandraClient;

import org.apache.cassandra.service.Cassandra;
import org.apache.cassandra.service.ColumnPath;
import org.apache.cassandra.service.ConsistencyLevel;
import org.apache.cassandra.service.InvalidRequestException;
import org.apache.cassandra.service.TimedOutException;
import org.apache.cassandra.service.UnavailableException;
import org.apache.commons.lang.RandomStringUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.transport.TTransportException;

/**
 * @author brucepang
 * 
 */
public class CassandraClientDemo {

	/**
	 * 
	 */
	public CassandraClientDemo() {

	}

	/**
	 * @param args
	 */
	public static void main(String[] args) {

		try {
			TTransport tr = new TSocket("10.219.101.101", 9160);

			TProtocol pro = new TBinaryProtocol(tr);
			Cassandra.Client cli = new Cassandra.Client(pro);
			tr.open();

			String key = null;
			String name, age;
			ColumnPath namePath = new ColumnPath("Standard1", null, "name"
					.getBytes("UTF-8"));
			ColumnPath agePath = new ColumnPath("Standard1", null, "age"
					.getBytes("UTF-8"));

			String keySpace = "Keyspace1";
			long time = 0;
			long l1 = System.currentTimeMillis();
			for (int i = 0; i < 100; i++) {
				key = String.valueOf(i);
				name = RandomStringUtils.random(5,"abcdefghefsdf");
				time = System.currentTimeMillis();
				cli.insert(keySpace, key, namePath, name.getBytes("UTF-8"),
						time, ConsistencyLevel.ONE);
				cli.insert(keySpace, key, agePath, key.getBytes("UTF-8"), time,
						ConsistencyLevel.ONE);
			}
			long l2 = System.currentTimeMillis();
			long ch = l2 - l1;
			System.out.println(ch);

		} catch (TTransportException e) {
			e.printStackTrace();
		} catch (UnsupportedEncodingException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (InvalidRequestException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (UnavailableException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (TimedOutException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (TException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

	}
}




 
测试结果:1 写100条数据,耗时59922毫秒,将近1分钟;