欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

logstash+elasticsearch+kibana

程序员文章站 2024-03-22 10:59:16
...

环境介绍

  • 192.168.17.30 logstash
  • 192.168.17.31 elasticsearch
  • 192.168.17.32 kibana

系统相关优化

vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

vim /etc/sysctl.conf 
fs.file-max=655350
vm.max_map_count=655360

sysctl -p

安装

位置:192.168.17.30 

软件:jdk,nginx,logstash

本文安装nginx来作为日志的生成工具,进行测试,首先安装jdk(可以使用yum源的openjdk)

安装nginx

rpm -ivh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
yum -y install nginx
  • 修改nginx日志的输出,将nginx的日志输出修改为json格式,方便elk的收集
[aaa@qq.com tmp]# vim /etc/nginx/nginx.conf 

    log_format json '{"@timestamp": "$time_iso8601",'
                '"@version": "1",'
                '"client": "$remote_addr",'
                '"url": "$uri", '
                '"status": "$status", '
                '"domain": "$host", '
                '"host": "$server_addr",'
                '"size":"$body_bytes_sent", '
                '"response_time": "$request_time", '
                '"referer": "$http_referer", '
                '"ua": "$http_user_agent" } ';
    access_log  /var/log/nginx/access.log  json;

  • 重启nginx并设置开机自启
systemctl enable nginx
systemctl start nginx

安装logstash

[aaa@qq.com ~]# tar -xf logstash-6.4.0.tar.gz  -C /opt
  • 修改或创建一个配置文件
[aaa@qq.com ~]# cd /opt/logstash-6.4.0/config/
[aaa@qq.com config]# ls
jvm.options  log4j2.properties  logstash-sample.conf  logstash.yml  pipelines.yml  startup.options
[aaa@qq.com config]# cp logstash-sample.conf logstash-nginx.conf
[aaa@qq.com config]# vim logstash-nginx.conf 
# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.
#输入,表示需要采集的日志
input {
  #收集nginx日志
  file{
    type => "nginx-access-log"
    #此处也可以用数组的形式 ["/vat/log/nginx/access.log"] 来代替。
    path => "/var/log/nginx/access.log" 
    start_position => "beginning"
  }
  #收集系统日志
  file {
    type => "syslog"
    tags => ["sys"]
    path => ["/var/log/messages","var/log/maillog"]
    start_position => "beginning"
  }
}
#直接推送给elasticsearch
output {
  elasticsearch {
    hosts => ["http://192.168.17.31:9200"]
  }
}
#入果有redis 可以推送给redis
#output{
#  redis {
#       host => 192.168.17.33
#       key => 'logstash-redis'
#       date_type => 'list'
#       }
#}

#启动服务
[aaa@qq.com bin]# nohup ./logstash -f ../config/logstash-nginx.conf

 安装elasticsearch

位置:192.168.17.31,elasticsearch

  • 首先安装java,同上logstash,可安装openjdk
  • 下载elasticsearch源码包并解压。下载地址:http://artifacts.elastic.co
[aaa@qq.com ~]# yum -y install java-1.8.0-openjdk.x86_64
[aaa@qq.com ~]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.0.tar.gz
[aaa@qq.com ~]# tar -xf elasticsearch-6.4.0.tar.gz -C /opt
  • 创建普通用户elk,并给elasticsearch目录赋权,因为elasticsearch不允许root启动
[aaa@qq.com ~]# groupadd elk
[aaa@qq.com ~]# useradd elk -g elk
[aaa@qq.com ~]# chown -R elk:elk /opt/elasticsearch-6.4.0/
  • 修改elasticsearch配置文件 接受来自logstash的日志
[aaa@qq.com config]# pwd
/opt/elasticsearch-6.4.0/config
[aaa@qq.com config]# vim elasticsearch.yml 
network.host: 192.168.17.31 #声明本节点的ip
http.port: 9200 #http 通信端口
  • 使用elk用户启动服务
  • 启动之后可以使用curl ip:9200来查看是否成功,结果如下
[aaa@qq.com ~]# curl 192.168.17.31:9200
{
  "name" : "CeScCBG",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "BCiy5_yJSQyyP3Po87Rdug",
  "version" : {
    "number" : "6.4.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "595516e",
    "build_date" : "2018-08-17T23:18:47.308994Z",
    "build_snapshot" : false,
    "lucene_version" : "7.4.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

安装kibana

位置:192.168.17.32 Kibana

  • 去官网下载源码包tar,并解压,修改配置文件kibana.yml,启动kibana
[aaa@qq.com ~]# wget https://artifacts.elastic.co/downloads/kibana/kibana-6.4.0-linux-x86_64.tar.gz
[aaa@qq.com ~]# tar xf kibana-6.4.0-linux-x86_64.tar.gz -C /opt/
[aaa@qq.com ~]# cd /opt/kibana-6.4.0-linux-x86_64/config/
[aaa@qq.com config]# vim kibana.yml 
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://192.168.17.31:9200"


[aaa@qq.com bin]# pwd
/opt/kibana-6.4.0-linux-x86_64/bin
[aaa@qq.com bin]# ./kibana

浏览器访问192.168.17.32:5601 即可

logstash+elasticsearch+kibana

 

 

 

 

相关标签: elk