kafka初步

1.环境

Windows 7
JDK 1.7.0_72
kafka_2.11-0.10.0.0

2.下载

去官网下载kafka_2.11-0.10.0.0.tgz

kafka原生支持在windows上运行。

3.单机版测试

3.1 启动zookeeper

bin\windows\zookeeper-server-start.bat config/zookeeper.properties

3.2 启动kafka

bin\windows\kafka-server-start.bat config/server.properties

3.3 创建topic

bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

查看有哪些topic

bin\windows\kafka-topics.bat --list --zookeeper localhost:2181

3.4 测试producer和consumer

bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test
This is a message
This is another message

bin\windows\kafka-console-consumer.bat --zookeeper localhost:2181 --topic test --from-beginning

收发正常，测试成功。

4.集群测试
我们将建一个由3个kafka组成的集群，全部跑在同一个windows上。

4.1 修改配置文件

cp config/server.properties config/server-1.properties
cp config/server.properties config/server-2.properties

config/server-1.properties:
    broker.id=1
    listeners=PLAINTEXT://:9093
    log.dir=/tmp/kafka-logs-1

config/server-2.properties:
    broker.id=2
    listeners=PLAINTEXT://:9094
    log.dir=/tmp/kafka-logs-2

4.2 启动zookeeper

bin\windows\zookeeper-server-start.bat config/zookeeper.properties

4.3 启动kafka

bin\windows\kafka-server-start.bat config/server.properties
bin\windows\kafka-server-start.bat config/server-1.properties
bin\windows\kafka-server-start.bat config/server-2.properties

4.4 创建topic并查看

bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replicated-topic

查看topic

bin\windows\kafka-topics.bat --describe --zookeeper localhost:2181 --topic my-replicated-topic

Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3
Configs:
        Topic: my-replicated-topic Partition: 0 Leader:1 Replicas: 1,2,0 Isr: 1,2,0

4.5 测试producer和consumer

bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic my-replicated-topic
This is a message
This is another message

bin\windows\kafka-console-consumer.bat --zookeeper localhost:2181 --from-beginning --topic my-replicated-topic

4.6 杀死4.4查看到的Leader：1，再次运行

bin\windows\kafka-topics.bat --describe --zookeeper localhost:2181 --topic my-replicated-topic

Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3
Configs:
        Topic: my-replicated-topic Partition: 0 Leader: 2 Replicas
: 1,2,0 Isr: 2,0
可以发现Leader变成了2,。然后producer和consumer依旧工作良好。