在windows搭建Flume

flume是一个分布式的支持高并发的日志收集系统。
本文将介绍在windows下的环境搭建。

下载

下载链接

安装

解压下载的包apache-flume-1.9.0-bin.tar.gz,我这里解压在C盘
进入C:\apache-flume-1.9.0-bin\conf

flume-env.ps1.template重命名为flume-env.ps1
修改 flume-conf.properties里的内容：

# Define a memory channel called ch1 on agent1
agent1.channels.ch1.type = memory

# Define an Avro source called avro-source1 on agent1 and tell it
# to bind to 0.0.0.0:41414. Connect it to channel ch1.
agent1.sources.avro-source1.channels = ch1
agent1.sources.avro-source1.type = netcat
agent1.sources.avro-source1.bind = 0.0.0.0
agent1.sources.avro-source1.port = 41414

# Define a logger sink that simply logs all events it receives
# and connect it to the other end of the same channel.
agent1.sinks.log-sink1.channel = ch1
agent1.sinks.log-sink1.type = logger

# Finally, now that we've defined all of our components, tell
# agent1 which ones we want to activate.
agent1.channels = ch1
agent1.sources = avro-source1
agent1.sinks = log-sink1

打开flume-env.ps1，找到$JAVA_OPTS这一行，打开注释，加上
-Dflume.root.logger=DEBUG,console
启动的时候才有日志输出，方便调试

启动

打开cmd，进入C:\apache-flume-1.9.0-bin
执行命令:
bin\flume-ng agent --conf .\conf -f conf\flume-conf.properties -n agent1
命令参数解释:
agent :相当于启动一个服务端
–conf:配置文件夹路径
-f:配置文件路径
-n:启动的代理名称，因为flume-conf.properties定义了名称为agent1，所以这里也要对应上

启动成功后，再打开一个cmd，然后输入telnet localhost 41414
端口号也是已经在flume-conf.properties中定义了
然后随便输入一些文字，回车，再到启动的服务端那边看下是否有对应的文字产生。