ELK-logstash-6.3.2部署

 

　　logstash 是一款强大的数据处理工具，它可以实现数据传输，格式处理，格式化输出，还有强大的插件功能，常用于日志处理。

 

1. logstash部署

1 [yun@mini04 software]$ pwd
2 /app/software
3 [yun@mini04 software]$ tar xf logstash-6.3.2.tar.gz 
4 [yun@mini04 software]$ mv logstash-6.3.2 /app/
5 [yun@mini04 software]$ cd /app/
6 [yun@mini04 ~]$ ln -s logstash-6.3.2/ logstash

 

2. 基本测试

[yun@mini04 bin]$ pwd
/app/logstash/bin
[yun@mini04 bin]$ ./logstash -e 'input { stdin{} } output { stdout{} }'  # 输入什么就打印什么 
sending logstash's logs to /app/logstash/logs which is now configured via log4j2.properties
[2018-08-21t20:42:43,017][warn ][logstash.config.source.multilocal] ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-08-21t20:42:43,912][info ][logstash.runner          ] starting logstash {"logstash.version"=>"6.3.2"}
[2018-08-21t20:42:46,861][info ][logstash.pipeline        ] starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-08-21t20:42:47,143][info ][logstash.pipeline        ] pipeline started successfully {:pipeline_id=>"main", :thread=>"#<thread:0x6c40d7d2 run>"}
the stdin plugin is now waiting for input:
[2018-08-21t20:42:47,325][info ][logstash.agent           ] pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-08-21t20:42:47,693][info ][logstash.agent           ] successfully started logstash api endpoint {:port=>9600}
zhangsan    # 输入
{
          "host" => "mini04",
       "message" => "zhangsan",
    "@timestamp" => 2018-08-21t12:43:08.026z,
      "@version" => "1"
}
lisi   # 输入
{
          "host" => "mini04",
       "message" => "lisi",
    "@timestamp" => 2018-08-21t12:43:11.412z,
      "@version" => "1"
}
[yun@mini04 bin]$ ./logstash -e 'input{ stdin{} } output{ stdout{codec => rubydebug} }'  # 启用debug模式
…………
1111  # 输入
{
       "message" => "1111",
          "host" => "mini04",
    "@timestamp" => 2018-08-21t12:57:09.274z,
      "@version" => "1"
}
oooo  # 输入
{
       "message" => "oooo",
          "host" => "mini04",
    "@timestamp" => 2018-08-21t12:57:55.289z,
      "@version" => "1"
}

　　

3. 使用配置文件

[yun@mini04 config]$ pwd
/app/logstash/config
[yun@mini04 config]$ cat demo.conf 
input{
  stdin{}
}

filter{
}

output{
  # es有3台，随便指定一台即可  也可以是多台如 ["127.0.0.1:9200","127.0.0.2:9200"]
  elasticsearch { 
    hosts => ["mini03:9200"] 
    index => "logstash-%{+yyyy.mm.dd}"
  }
 
  stdout { codec => rubydebug } 
}
[yun@mini04 config]$ /app/logstash/bin/logstash -f /app/logstash/config/demo.conf # 启动
………………
1111
{
          "host" => "mini04",
    "@timestamp" => 2018-08-21t13:40:05.051z,
       "message" => "1111",
      "@version" => "1"
}
222
{
          "host" => "mini04",
    "@timestamp" => 2018-08-21t13:40:08.445z,
       "message" => "222",
      "@version" => "1"
}
zhangsan
{
          "host" => "mini04",
    "@timestamp" => 2018-08-21t13:40:11.333z,
       "message" => "zhangsan",
      "@version" => "1"
}
周八
{
          "host" => "mini04",
    "@timestamp" => 2018-08-21t13:40:15.523z,
       "message" => "周八",
      "@version" => "1"
}

　　

3.1. 浏览器访问