linux下ElasticSearch.6.2.2集群安装与head、Kibana、X-Pack..插件的配置安装
程序员文章站
2022-05-09 21:18:39
简介：主要在三台linux服务器上，集群安装ElasticSearch.6.2.1，及其es的插件，各种管理软件
1.集群安装es
1.1 环境
域名...
简介：主要在三台linux服务器上，集群安装ElasticSearch.6.2.1，及其es的插件，各种管理软件
1.集群安装es
1.1 环境
域名 ip biluos.com 192.168.10.173 biluos1.com 192.168.10.174 biluos2.com 192.168.10.175
1.2 三台机器都安装jdk最新版本
[root@biluos es]# java -version openjdk version "1.8.0_161" OpenJDK Runtime Environment (build 1.8.0_161-b14) OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode) [root@biluos es]# [root@biluos1 ~]# java -version java version "1.8.0_121" Java(TM) SE Runtime Environment (build 1.8.0_121-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode) [root@biluos1 ~]# [root@biluos2 ~]# java -version java version "1.8.0_121" Java(TM) SE Runtime Environment (build 1.8.0_121-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode) [root@biluos2 ~]#
1.3 三台机器都统一用户为es
[root@biluos ~]# useradd es You have new mail in /var/spool/mail/root [root@biluos ~]# passwd es # 密码为 qwaszx Changing password for user es. New password: BAD PASSWORD: it is based on a dictionary word BAD PASSWORD: is too simple Retype new password: passwd: all authentication tokens updated successfully. [root@biluos ~]# mkdir /home/es mkdir: cannot create directory `/home/es': File exists [root@biluos ~]# ll /home/ # 注意是不是es用户和用户组 total 4 drwx------ 3 es es 4096 Feb 25 03:51 es [root@biluos ~]# [root@biluos1 ~]# useradd es [root@biluos1 ~]# passwd es # 密码为 qwaszx [root@biluos1 ~]# mkdir /home/es [root@biluos1 ~]# ll /home/ # 注意是不是es用户和用户组 [root@biluos2 ~]# useradd es [root@biluos2 ~]# passwd es # 密码为 qwaszx [root@biluos2 ~]# mkdir /home/es [root@biluos2 ~]# ll /home/ # 注意是不是es用户和用户组
1.4 三台机器都建立/home/es/elasticsearch目录，用来存放es软件包和数据存储，使用es用户
[root@biluos1 ~]# su es [es@biluos1 root]$ cd /home/es/ [es@biluos1 ~]$ mkdir elasticsearch [es@biluos1 ~]$ ll drwxrwxr-x 2 es es 4096 Feb 25 03:59 elasticsearch [es@biluos1 ~]$ 其余两台此处省略
1.5 三台机器都解压安装包到/home/es/elasticsearch
下载：https://www.elastic.co/cn/downloads/elasticsearch
包：elasticsearch-6.2.2.tar.gz
解压：tar -zxvf /home/es/elasticsearch/elasticsearch-6.2.2.tar.gz -C /home/es/elasticsearch
[root@biluos elasticsearch-6.2.0]# pwd /home/es/elasticsearch/elasticsearch-6.2.0 [es@biluos1 elasticsearch-6.2.0]$ pwd /home/es/elasticsearch/elasticsearch-6.2.0 [es@biluos2 elasticsearch-6.2.0]$ pwd /home/es/elasticsearch/elasticsearch-6.2.0
1.6 三台机器都修改es软件包的权限为es用户
因为我是root解压的
[es@biluos2 elasticsearch]$ ll total 4 drwxr-xr-x 9 root root 4096 Feb 25 04:04 elasticsearch-6.2.0 [es@biluos2 elasticsearch]$
使用root用户修改权限
[es@biluos2 ~]$ su root Password: [root@biluos2 es]# chown -R es:es /home/es/ 其余两台此处省略
1.7 三台机器都创建data数据目录和日志目录，使用es用户
[root@biluos2 es]# su es [es@biluos2 ~]$ mkdir -p /home/es/elasticsearch/elasticsearch-6.2.0_data/data/ [es@biluos2 ~]$ mkdir -p /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/ 其余两台此处省略
1.8 三台机器都修改配置
191.168.10.173机器配置
vim /home/es/elasticsearch/elasticsearch-6.2.2/config/elasticsearch.yml
cluster.name: lcc-application # 必须一样 node.name: node-191-168-10-173 # 必须不一样 path.data: /home/es/elasticsearch/elasticsearch-6.2.0_data/data/ path.logs: /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/ bootstrap.memory_lock: false bootstrap.system_call_filter: false network.host: 0.0.0.0 http.port: 9200 # 集群发现 #集群节点ip或者主机 discovery.zen.ping.unicast.hosts: ["192.168.10.173", "192.168.10.174","192.168.10.175"] #设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1，对于大的集群来说，可以设置大一点的值（2-4） discovery.zen.minimum_master_nodes: 3 # 下面两行配置为haad插件配置，三台服务器一致。 http.cors.enabled: true http.cors.allow-origin: "*"
191.168.10.174机器配置
vim /home/es/elasticsearch/elasticsearch-6.2.2/config/elasticsearch.yml
cluster.name: lcc-application # 必须一样 node.name: node-191-168-10-174 # 必须不一样 path.data: /home/es/elasticsearch/elasticsearch-6.2.0_data/data/ path.logs: /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/ bootstrap.memory_lock: false bootstrap.system_call_filter: false network.host: 0.0.0.0 http.port: 9200 # 集群发现 #集群节点ip或者主机 discovery.zen.ping.unicast.hosts: ["192.168.10.173", "192.168.10.174","192.168.10.175"] #设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1，对于大的集群来说，可以设置大一点的值（2-4） discovery.zen.minimum_master_nodes: 3 # 下面两行配置为haad插件配置，三台服务器一致。 http.cors.enabled: true http.cors.allow-origin: "*"
191.168.10.175机器配置
vim /home/es/elasticsearch/elasticsearch-6.2.2/config/elasticsearch.yml
cluster.name: lcc-application # 必须一样 node.name: node-191-168-10-175 # 必须不一样 path.data: /home/es/elasticsearch/elasticsearch-6.2.0_data/data/ path.logs: /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/ bootstrap.memory_lock: false bootstrap.system_call_filter: false network.host: 0.0.0.0 http.port: 9200 # 集群发现 #集群节点ip或者主机 discovery.zen.ping.unicast.hosts: ["192.168.10.173", "192.168.10.174","192.168.10.175"] #设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1，对于大的集群来说，可以设置大一点的值（2-4） discovery.zen.minimum_master_nodes: 3 # 下面两行配置为haad插件配置，三台服务器一致。 http.cors.enabled: true http.cors.allow-origin: "*"
1.9 三台机器都修改 Linux下/etc/security/limits.conf文件设置
更改linux的最大文件描述限制要求
添加或修改如下：
* soft nofile 262144
* hard nofile 262144 更改linux的锁内存限制要求
添加或修改如下：
用户(elasticsearch) soft memlock unlimited
用户(elasticsearch) hard memlock unlimited
最后配置如下
vim /etc/security/limits.conf * soft nofile 262144 * hard nofile 262144 es soft memlock unlimited es hard memlock unlimited
这里es是es用户
1.10 三台机器都修改配置 Linux下/etc/security/limits.d/90-nproc.conf文件设置
1.10 三台机器都修改配置 Linux下/etc/security/limits.d/90-nproc.conf文件设置更改linux的的最大线程数，添加或修改如下：* soft nproc unlimited
vim /etc/security/limits.d/90-nproc.conf * soft nproc unlimited root soft nproc unlimited
1.11 三台机器都修改配置 Linux下/etc/sysctl.conf文件设置
1.11 三台机器都修改配置 Linux下/etc/sysctl.conf文件设置更改linux一个进行能拥有的最多的内存区域要求,添加或修改如下：
vm.max_map_count = 262144 更改linux禁用swapping,添加或修改如下：
vm.swappiness = 1 vim /etc/sysctl.conf vm.max_map_count = 262144 vm.swappiness = 1
1.12 三台机器都启动
1.12 三台机器都启动[es@biluos elasticsearch-6.2.2]$ bin/elasticsearch [es@biluos1 elasticsearch-6.2.2]$ bin/elasticsearch [es@biluos2 elasticsearch-6.2.2]$ bin/elasticsearch
刚刚报错，
[2018-02-25T13:49:33,967][WARN ][o.e.d.z.ZenDiscovery ] [node-191-168-10-173] not enough master nodes discovered during pinging (found [[Candidate{node={node-191-168-10-173}{Wquj4gYuTteo7M5uXzSBiQ}{19L-SAvFRvqvYyjsKtBP_g}{192.168.1.194}{192.168.1.194:9300}, clusterStateVersion=-1}]], but needed [3]), pinging again
我以为是因为没配置ssh，最后发现上面ip写错了，然后验证不配置ssh也没事。
这上面第一个启动的打印日志最多，其他的相同，日志比较少
现在有个问题，我访问
https://192.168.10.173:9200/
https://192.168.10.174:9200/
https://192.168.10.175:9200/
都能打印出版本号，证明都能正确运行，我尝试关闭掉一个174，发现https://192.168.10.174:9200/打不开了，然后，我觉得应该有一个是主master,但是谁是主呢？不好看到，下面我们安装图形化界面head试试
2 安装elasticsearch-head
2 安装elasticsearch-head三台机器只需要一台安装head就可以了
2.1 采用nodejs安装
2.1 采用nodejs安装6.X中，elasticsearch-head
不能放在elasticsearch的 plugins、modules 目录下 不能使用 elasticsearch-plugin install 直接启动elasticsearch即可
2.2 安装 elasticsearch-head
2.2 安装 elasticsearch-head1.修改 elasticsearch/config/elasticsearch.yml
2.添加
http.cors.enabled: true http.cors.allow-origin: "*"
3.下载 elasticsearch-head 或者 git clone 到随便一个文件夹
https://github.com/mobz/elasticsearch-head
我是直接下载到
[es@biluos elasticsearch-head-master]$ pwd /home/es/elasticsearch-head-master
目录的
4.安装nodejs
https://blog.csdn.net/qq_21383435/article/details/79367366
5.cd /home/es/elasticsearch-head-master
6.npm install -g grunt-cli
grunt是一个很方便的构建工具，可以进行打包压缩、测试、执行等等的工作
[es@biluos elasticsearch-head-master]$ npm install -g grunt-cli
npm WARN checkPermissions Missing write access to /opt/moudles/node-v8.9.4-linux-x64/lib/node_modules
npm ERR! path /opt/moudles/node-v8.9.4-linux-x64/lib/node_modules
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall access
npm ERR! Error: EACCES: permission denied, access '/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules'
npm ERR! { Error: EACCES: permission denied, access '/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules'
npm ERR! stack: 'Error: EACCES: permission denied, access \'/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules\'',
npm ERR! errno: -13,
npm ERR! code: 'EACCES',
npm ERR! syscall: 'access',
npm ERR! path: '/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules' }
npm ERR!
npm ERR! Please try running this command again as root/Administrator.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/es/.npm/_logs/2018-02-25T02_49_37_372Z-debug.log
一看就知道是权限问题，因为我的nodejs是root安装的，这里我是es用户
[es@biluos elasticsearch-head-master]$ su root Password: [root@biluos elasticsearch-head-master]# npm install -g grunt-cli /opt/moudles/node-v8.9.4-linux-x64/bin/grunt -> /opt/moudles/node-v8.9.4-linux-x64/lib/node_modules/grunt-cli/bin/grunt + grunt-cli@1.2.0 added 16 packages in 21.663s
7.npm install
[root@biluos elasticsearch-head-master]# npm install
npm WARN deprecated http2@3.3.7: Use the built-in module in node 9.0.0 or newer, instead
npm WARN deprecated coffee-script@1.10.0: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
> phantomjs-prebuilt@2.1.16 install /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt
> node install.js
module.js:540
throw err;
^
Error: Cannot find module '/home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/install.js'
at Function.Module._resolveFilename (module.js:538:15)
at Function.Module._load (module.js:468:25)
at Function.Module.runMain (module.js:684:10)
at startup (bootstrap_node.js:187:16)
at bootstrap_node.js:608:3
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! phantomjs-prebuilt@2.1.16 install: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the phantomjs-prebuilt@2.1.16 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-02-25T03_01_11_887Z-debug.log
这一点是因为我的elasticsearch是es用户，而elasticsearch-head-master文件是root用户
[root@biluos es]# ll total 12 drwxr-xr-x 4 es es 4096 Feb 9 14:55 elasticsearch [root@biluos elasticsearch-head-master]# ll /home/es/elasticsearch-head-master/node_modules total 1028 drwxr-xr-x 2 root root 4096 Feb 25 11:01 abbrev drwxr-xr-x 2 root root 4096 Feb 25 11:01 accepts drwxr-xr-x 3 root root 4096 Feb 25 11:01 after drwxr-xr-x 2 root root 4096 Feb 25 11:01 ansi-regex drwxr-xr-x 2 root root 4096 Feb 25 11:01 ansi-styles
这里我统一用户为es
[root@biluos es]# chown -R es:es /home/es/elasticsearch-head-master/
然后使用es用户，安装成功
[root@biluos elasticsearch-head-master]# su es
[es@biluos elasticsearch-head-master]$ npm install
> phantomjs-prebuilt@2.1.16 install /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt
> node install.js
PhantomJS not found on PATH
Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
Receiving...
[=============---------------------------] 33 [===================================-----] 87%
Received 22866K total.
Extracting tar contents (via spawned process)
Removing /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom
Copying extracted folder /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1519527928056/phantomjs-2.1.1-linux-x86_64 -> /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom
Writing location.js file
Done. Phantomjs binary available at /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
added 70 packages in 84.93s
[es@biluos elasticsearch-head-master]$
8.grunt server
[es@biluos elasticsearch-head-master]$ grunt server (node:6612) ExperimentalWarning: The http2 module is an experimental API. Running "connect:server" (connect) task Waiting forever... Started connect web server on https://localhost:9100
9.https://192.168.10.173:9100/ 成功
10.启动我们的elasticsearch
[es@biluos elasticsearch-6.2.2]$ bin/elasticsearch [es@biluos1 elasticsearch-6.2.2]$ bin/elasticsearch [es@biluos2 elasticsearch-6.2.2]$ bin/elasticsearch
11 .https://192.168.10.173:9100/ 查看
2.3 测试ctrl+c掉一个
2.3 测试ctrl+c掉一个我直接ctrl+c一个174会是什么结果呢？，发现173和175都报错
[2018-02-25T14:54:08,997][WARN ][o.e.d.z.ZenDiscovery ] [node-191-168-10-173] not enough master nodes discovered during pinging (found [[Candidate{node={node-191-168-10-175}{dCejCwUFTnqOrYyrmfD4AQ}{y2SixREcR5STrzmlswaxcA}{192.168.10.175}{192.168.10.175:9300}, clusterStateVersion=8}, Candidate{node={node-191-168-10-173}{Wquj4gYuTteo7M5uXzSBiQ}{xzN6saH5Rp2CPQH5iunHZg}{192.168.1.194}{192.168.1.194:9300}, clusterStateVersion=8}]], but needed [3]), pinging again
而且https://192.168.10.173:9100/无法连接，连接也是红色的
感觉：应该和这个参数有关
## 设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1，对于大的集群来说，可以设置大一点的值（2-4） discovery.zen.minimum_master_nodes: 2 这个的含义是不是 我假设集群有3个节点 ，如果死掉了一个我的集群还能用？如果在死掉一个，节点还剩下一个了，小于2 是不是就不能用了? discovery.zen.minimum_master_nodes: 5 是不是我集群可用的最少节点数限制 参数是最小可用的候选master节点数 一般设置为2，集群里留3个节点作为候选master
3 安装kibana（不需要安装x-pack）
3 安装kibana（不需要安装x-pack）集群的时候，我们只需要在一台机器上安装就可以了，
我开始安装kibana-6.2.1，但是报错如下，后来安装同样版本的kibana-6.2.2，就不报错了
log [07:15:33.047] [warning] You're running Kibana 6.2.1 with some different versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v6.2.2 @ 192.168.10.174:9200 (192.168.10.174), v6.2.2 @ 192.168.1.194:9200 (192.168.1.194), v6.2.2 @ 192.168.10.175:9200 (192.168.10.175)
1.下载https://www.elastic.co/cn/downloads/kibana 版本要和es版本相同
2.直接解压到 /home/es/kibana/kibana-6.2.2-linux-x86_64
3.配置
# vim config/kibana.yml elasticsearch.url: "https://192.168.10.173:9200" # kibana监控哪台es机器 server.host: "192.168.10.173" # kibana运行在哪台机器
4.运行
[root@biluos kibana-6.2.2-linux-x86_64]# bin/kibana log [07:54:52.807] [info][status][plugin:kibana@6.2.2] Status changed from uninitialized to green - Ready log [07:54:52.882] [info][status][plugin:elasticsearch@6.2.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [07:54:53.091] [info][status][plugin:timelion@6.2.2] Status changed from uninitialized to green - Ready log [07:54:53.107] [info][status][plugin:console@6.2.2] Status changed from uninitialized to green - Ready log [07:54:53.116] [info][status][plugin:metrics@6.2.2] Status changed from uninitialized to green - Ready log [07:54:53.152] [info][listening] Server running at https://192.168.10.173:5601 log [07:54:53.197] [info][status][plugin:elasticsearch@6.2.2] Status changed from yellow to green - Ready
5.查看界面https://192.168.10.173:5601 可以直接访问