elastic stack(一)软件安装启动

版本：elasticsearch 7.8.0、logstash 7.8.0、kibana 7.8.0(官网下载)   jdk 11

elk的启动账号（必须以非root账号启动）

• 1、检查本地jdk版本是匹配

java -version

 本地jdk环境是1.8（项目需求），此时需要在jdk中重新指定jdk版本

进入elasticsearch下bin目录的启动文件elasticsearch,配置如下：

#!/bin/bash

# CONTROLLING STARTUP:
#
# This script relies on a few environment variables to determine startup
# behavior, those variables are:
#
#   ES_PATH_CONF -- Path to config directory
#   ES_JAVA_OPTS -- External Java Opts on top of the defaults set
#
# Optionally, exact memory values can be set using the `ES_JAVA_OPTS`. Example
# values are "512m", and "10g".
#
#   ES_JAVA_OPTS="-Xms8g -Xmx8g" ./bin/elasticsearch
# 指定jdk11
export JAVA_HOME=/gfkdata/elk/jdk-11.0.8
export PATH=$JAVA_HOME/bin:$PATH

source "`dirname "$0"`"/elasticsearch-env

CHECK_KEYSTORE=true
DAEMONIZE=false
for option in "[e-mail]"; do
  case "$option" in
    -h|--help|-V|--version)
      CHECK_KEYSTORE=false
      ;;
    -d|--daemonize)
      DAEMONIZE=true
      ;;
  esac
done

if [ -z "$ES_TMPDIR" ]; then
  ES_TMPDIR=`"$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.launchers.TempDirectory`
fi

# get keystore password before setting java options to avoid
# conflicting GC configurations for the keystore tools
unset KEYSTORE_PASSWORD
KEYSTORE_PASSWORD=
if [[ $CHECK_KEYSTORE = true ]] \
    && bin/elasticsearch-keystore has-passwd --silent
then
  if ! read -s -r -p "Elasticsearch keystore password: " KEYSTORE_PASSWORD ; then
    echo "Failed to read keystore password on console" 1>&2
    exit 1
  fi
fi

# The JVM options parser produces the final JVM options to start Elasticsearch.
# It does this by incorporating JVM options in the following way:
#   - first, system JVM options are applied (these are hardcoded options in the
#     parser)
#   - second, JVM options are read from jvm.options and jvm.options.d/*.options
#   - third, JVM options from ES_JAVA_OPTS are applied
#   - fourth, ergonomic JVM options are applied
ES_JAVA_OPTS=`export ES_TMPDIR; "$JAVA" "$XSHARE" -cp "$ES_CLASSPATH" org.elasticsearch.tools.launchers.JvmOptionsParser "$ES_PATH_CONF"`

# 添加jdk判断
if [ -x "$JAVA_HOME/bin/java" ]; then
        JAVA="/gfkdata/elk/jdk-11.0.8/bin/java"
else
        JAVA=`which java`
fi

# manual parsing to find out, if process should be detached
if [[ $DAEMONIZE = false ]]; then
  exec \
    "$JAVA" \
    "$XSHARE" \
    $ES_JAVA_OPTS \
    -Des.path.home="$ES_HOME" \
    -Des.path.conf="$ES_PATH_CONF" \
    -Des.distribution.flavor="$ES_DISTRIBUTION_FLAVOR" \
    -Des.distribution.type="$ES_DISTRIBUTION_TYPE" \
    -Des.bundled_jdk="$ES_BUNDLED_JDK" \
    -cp "$ES_CLASSPATH" \
    org.elasticsearch.bootstrap.Elasticsearch \
    "[e-mail]" <<<"$KEYSTORE_PASSWORD"
else
  exec \
    "$JAVA" \
    "$XSHARE" \
    $ES_JAVA_OPTS \
    -Des.path.home="$ES_HOME" \
    -Des.path.conf="$ES_PATH_CONF" \
    -Des.distribution.flavor="$ES_DISTRIBUTION_FLAVOR" \
    -Des.distribution.type="$ES_DISTRIBUTION_TYPE" \
    -Des.bundled_jdk="$ES_BUNDLED_JDK" \
    -cp "$ES_CLASSPATH" \
    org.elasticsearch.bootstrap.Elasticsearch \
    "[e-mail]" \
    <<<"$KEYSTORE_PASSWORD" &
  retval=$?
  pid=$!
  [ $retval -eq 0 ] || exit $retval
  if [ ! -z "$ES_STARTUP_SLEEP_TIME" ]; then
    sleep $ES_STARTUP_SLEEP_TIME
  fi
  if ! ps -p $pid > /dev/null ; then
    exit 1
  fi
  exit 0
fi

exit $?

                                                                                                                                                                                                                     
                                                                                                                                                                                                                  

elasticsearch相关配置

config/elasticsearch.yml        主配置文件

config/jvm.options                 jvm参数配置文件

config/log4j2.properties         日志配置文件

 

1、修改config目录下elasticsearch.yml配置文件（单机搭建）

####集群名称
cluster.name: my-application
####节点名称
node.name: node-1

#### 是否可以成为master节点
#node.master: true
# 是否允许该节点存储数据,默认开启
#node.data: true

####服务ip（支持外网访问）
network.host: 0.0.0.0
####服务端口(默认对外9200)
http.port: 9200

#### 支持跨域访问
#http.cors.enabled: true
#http.cors.allow-origin: "*"


####提供群集中可以成为master
discovery.seed_hosts: ["127.0.0.1"]

####手动指定可成为master的所有节点的name或者ip，这些配置将会在第一次选举中进行计算
cluster.initial_master_nodes: ["127.0.0.1:9300"]

2、更改默认内存配置(默认内存大小1个G)

-Xms256m
-Xmx256m

3、启动报错：max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

解决方法：

vi /etc/sysctl.conf
vm.max_map_count=262144

####修改完 执行以下命令使之生效
sysctl -p

4、启动elasticsearch，bin目录下：./elasticsearch

5、访问启动是否成功 curl http://localhost:9200 (出现下图表示启动成功)

 

logstash相关配置

在config目录下创建自定义配置文件myes.conf，内容如下

nput{
        # 从文件读取日志信息、输送到控制台,以json格式输出
        file{
                path => "/var/log/messages"
                codec =>"json"
                type =>"system"
                start_position =>"beginning"
        }
}

#filter{
#
#}

output{
        # 标准输出
        # stdout{}
        # 输出进行格式化、采用Ruby库来解析日志
        stdout{
                codec=>rubydebug
        }
        elasticsearch{
                ###此处是elasticsearch的ip
                hosts =>"127.0.0.1:9200"
                ##根据每天创建索引
                index =>"system-%{+YYYY.MM.dd}"
        }
}

启动logstash：

启动指定配置文件： ./logstash -f ../config/myes.conf

logstash启动失败时，进入data目录删除.lock文件，重启

查询： ls -alh
删除.lock： rm .lock

kibana相关配置

进入config目录下

####端口
server.port: 5601
####服务ip支持外网访问
server.host: "0.0.0.0"
####服务名称
server.name: "mykibana"
####查询的elasticsearch实例的url
elasticsearch.hosts: ["http://localhost:9200"]

启动kibana，bin目录下 ./kibana

访问url :http://ip:9200/status   或者http://ip:5601/app/kibana

至此，elk单击环境搭建好