欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

ElasticSearch2.4.0单机版及集群安装 博客分类: Search elasticsearchelasiicsarch v2.4cluster 

程序员文章站 2024-03-15 10:22:59
...
 
编写不易,转载请注明( http://shihlei.iteye.com/blog/2327971)!

概述

(一)简介
          基于lucene 实现的近实时搜索服务,Restful 接口操作
     
(二)名词解释
Indics:索引库,相当于RDBMS的 数据库,整体控制分片(shard)和副本(Replica),一旦创建,分片不能改变。
 
Document type:索引类型
 
Document:索引记录,由唯一ID区分,ID决定Shard位置
 
Filed:一条索引记录中的组成字段,有类型概念,通过Mapping控制类型,是否分词,索引等
 

一 单机版安装

 
(一)环境
 
环境要求:
     java 7 以上
环境:
    CentOS Linux release 7.2.1511 (Core)
    jre:1.8.0_101
版本:
     elasticsearch-2.4.0.tar.gz
 
核心配置文件:
                elasticsearch-2.4.0/config/elasticsearch.yml : yml协议的默认配置文件
                elasticsearch-2.4.0/config/logging.xml :yml协议格式的日志配置文件
(二)安装
1)创建用户
 
groupadd es
useradd -r -g es -s /bin/bash es
chown -hR es:es /opt/elasticsearch-2.4.0
 
注:由于不能用Root用户启动,否则报java.lang.RuntimeException,所以创建用户
 
Exception in thread "main" java.lang.RuntimeException: don't run elasticsearch as root.
    at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:94)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:160)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details.
 
2)解压缩
    
tar -xvf elasticsearch-2.4.0.tar.gz
 
3)启动
 
cd /opt/elasticsearch-2.4.0
bin/elasticsearch
 
日志:
 
[2016-09-22 08:45:21,503][WARN ][bootstrap                ] unable to install syscall filter: seccomp unavailable: your kernel is buggy and you should upgrade
[2016-09-22 08:45:21,608][INFO ][node                     ] [Deacon Frost] version[2.4.0], pid[205], build[ce9f0c7/2016-08-29T09:14:17Z]
[2016-09-22 08:45:21,608][INFO ][node                     ] [Deacon Frost] initializing ...
[2016-09-22 08:45:21,977][INFO ][plugins                  ] [Deacon Frost] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
[2016-09-22 08:45:21,994][INFO ][env                      ] [Deacon Frost] using [1] data paths, mounts [[/ (none)]], net usable_space [45gb], net total_space [59gb], spins? [possibly], types [aufs]
[2016-09-22 08:45:21,994][INFO ][env                      ] [Deacon Frost] heap size [998.4mb], compressed ordinary object pointers [unknown]
[2016-09-22 08:45:23,147][INFO ][node                     ] [Deacon Frost] initialized
[2016-09-22 08:45:23,148][INFO ][node                     ] [Deacon Frost] starting ...
[2016-09-22 08:45:23,251][INFO ][transport                ] [Deacon Frost] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2016-09-22 08:45:23,255][INFO ][discovery                ] [Deacon Frost] elasticsearch/6TSRp8FFQ3aXBPtkS4sROQ
[2016-09-22 08:45:26,316][INFO ][cluster.service          ] [Deacon Frost] new_master {Deacon Frost}{6TSRp8FFQ3aXBPtkS4sROQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2016-09-22 08:45:26,341][INFO ][http                     ] [Deacon Frost] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2016-09-22 08:45:26,343][INFO ][node                     ] [Deacon Frost] started
[2016-09-22 08:45:26,359][INFO ][gateway                  ] [Deacon Frost] recovered [0] indices into cluster_state
 
     注1 :后台启动方法:bin/elasticsearch -d
     注2 :  这里绑定了两个端口
               9200:默认的Http端口,用于Restful API的调用
               9300:默认TCP端口,用户ES Node 之间的通讯,以及TCP 基本调用 Restful API , Java API TransportClient 需要连接该端口才能访问
     注3:默认绑定127.0.0.1 本机访问,
               远程访问需修改 ${ES_HOME}/config/elasticsearch.yml 的 network.host绑定指定的IP
               
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
 
 
(三)验证
 
浏览器:http://127.0.0.1:9200 返回
 
{
  "name" : "Abomination",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "2.4.0",
    "build_hash" : "ce9f0c7394dee074091dd1bc4e9469251181fc55",
    "build_timestamp" : "2016-08-29T09:14:17Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.2"
  },
  "tagline" : "You Know, for Search"
}
 
由节点名称等,说明安装成功
 
(四)简单操作
 
(1)插入:
 
$ curl -X PUT 'http://localhost:9200/indexdb/type/1' -d '
> {
>    "name":"test",
>    "desc":"test"
> }
> '
{"_index":"indexdb","_type":"type","_id":"1","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":
 
(2)查询:
 
$ curl -XGET 'http://localhost:9200/indexdb/type/1'
{"_index":"indexdb","_type":"type","_id":"1","_version":1,"found":true,"_source":
{
   "name":"test",
   "desc":"test"
}
}
 
(五)关闭
<!--?xml version="1.0" encoding="UTF-8"?-->
for pid in `jps | grep Elasticsearch | awk '{print $1}'`
do
kill -9 $pid
done;
 
(六)常用插件安装
(1)Head:
方便对es进行各种操作的客户端(https://github.com/mobz/elasticsearch-head
 
bin/plugin install mobz/elasticsearch-head
 
访问:
 
http://127.0.0.1:9200/_plugin/head
 

ElasticSearch2.4.0单机版及集群安装
            
    
    博客分类: Search elasticsearchelasiicsarch v2.4cluster 
 
(2)bigdesk:
监控es状态的插件(https://github.com/lukas-vlcek/bigdesk
 
bigdesk 的源码开发在 2015 年夏天就已经停止,所以默认是无法支持 Elasticsearch 2.x 的监控的
 
 

二 集群搭建

 
(一)概述
(1)ES  Node 的角色:
Master Node :  通过选举出来,控制集群元数据和状态,压力较小,CRUD不需要通过 ( node.master=true)
 
Data Node : 拥有数据,并控制数据的CRUD,有查询压力 (node.data=true)
 
Client Node: 只能路由,转发响应的请求给Master Node 或者Data Node ,做聚合操作,可以分担部分Data Node 的压力,成本合适,可以创建。
 
Trib Node : 联合节点,控制到多个集群的访问
 
(2)架构
ES 的所有Node 节点是对等的,可以服务于客户端的查询,索引请求。结点通过选举成为Master,维持Cluster 状态,Master 所有的Node 周知。
 
(二)集群搭建
 
(1)架构规划

<!--?xml version="1.0" encoding="UTF-8"?-->


ElasticSearch2.4.0单机版及集群安装
            
    
    博客分类: Search elasticsearchelasiicsarch v2.4cluster 
 
 
说明:客户端通过 ClientNode 访问集群,ClientNode开放tcp,http访问,
          所有DataNode置于后端,只开发TCP进行通讯
 
(2)目录规划
 
ES_HOME : /opt/elasticsearch-2.4.0
 
结点名称 配置文件目录 数据目录 日志目录 pid目录
ClientNode cluster/config/ClientNode cluster/data/ClientNode cluster/logs/ClientNode cluster/ClientNode.pid
DataNode1 cluster/config/DataNode1 cluster/data/DataNode1 cluster/logs/DataNode1 cluster/DataNode1.pid
DataNode2 cluster/config/DataNode2 cluster/data/DataNode2 cluster/logs/DataNode2 cluster/DataNode2.pid
DataNode3 cluster/config/DataNode3 cluster/data/DataNode3 cluster/logs/DataNode3 cluster/DataNode3.pid
 
 
(3)端口规划
 
结点名称 Http端口 TCP端口
ClientNode 9200 9300
DataNode1 9201 9301
DataNode2 9202 9302
DataNode2 9203 9303
 
(4)安装
步骤1:拷贝配置文件到约定目录:
 
cd /opt/elasticsearch-2.4.0
mkdir -p cluster/config/ClientNode && cp config/logging.yml cluster/config/ClientNode
mkdir -p cluster/config/DataNode1 && cp config/logging.yml cluster/config/DataNode1
mkdir -p cluster/config/DataNode2 && cp config/logging.yml cluster/config/DataNode2
mkdir -p cluster/config/DataNode3 && cp config/logging.yml cluster/config/DataNode3
 
 
步骤2:编辑个结点配置文件:
 
特别注意:ES的集群是通过自发现机制,需要cluster.name配置相同才会自动加入相同的集群,这里集群统一命名为ES_CLUSTER
下面的配置文件已经过删减,可以直接替换elasticsearch.yml
 
ClientNode:
ClientNode :node.master , node.data标志都为false,并开启http访问
 
vi cluster/config/ClientNode/elasticsearch.yml
 
# ---------------------------------- Cluster -----------------------------------
cluster.name: ES_CLUSTER
# ------------------------------------ Node ------------------------------------
node.name: ClientNode
http.enabled: true
node.master: false
node.data: false
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elasticsearch-2.4.0/cluster/data/ClientNode
#
# Path to log files:
#
path.logs: /opt/elasticsearch-2.4.0/cluster/logs/ClientNode
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# Set a custom port for TCP:
transport.tcp.port: 9300  
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
 
DataNode1:
可选举成为Master 的DataNode : node.master,node.data都为true,由于客户端方位从ClientNode口出,http.enabled禁掉了下面同
 
vi cluster/config/DataNode1/elasticsearch.yml
 
# ---------------------------------- Cluster -----------------------------------
cluster.name: ES_CLUSTER
# ------------------------------------ Node ------------------------------------
node.name: DataNode1
http.enabled: false
node.master: true
node.data: true
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elasticsearch-2.4.0/cluster/data/DataNode1
#
# Path to log files:
#
path.logs: /opt/elasticsearch-2.4.0/cluster/logs/DataNode1
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9201
#
# Set a custom port for TCP:
transport.tcp.port: 9301  
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
 
DataNode2
 
vi cluster/config/DataNode2/elasticsearch.yml
 
# ---------------------------------- Cluster -----------------------------------
cluster.name: ES_CLUSTER
# ------------------------------------ Node ------------------------------------
node.name: DataNode2
http.enabled: false
node.master: true
node.data: true
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elasticsearch-2.4.0/cluster/data/DataNode2
#
# Path to log files:
#
path.logs: /opt/elasticsearch-2.4.0/cluster/logs/DataNode2
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9202
#
# Set a custom port for TCP:
transport.tcp.port: 9302  
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
 
DataNode3
 
vi cluster/config/DataNode3/elasticsearch.yml
 
# ---------------------------------- Cluster -----------------------------------
cluster.name: ES_CLUSTER
# ------------------------------------ Node ------------------------------------
node.name: DataNode3
http.enabled: false
node.master: true
node.data: true
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elasticsearch-2.4.0/cluster/data/DataNode3
#
# Path to log files:
#
path.logs: /opt/elasticsearch-2.4.0/cluster/logs/DataNode3
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9203
#
# Set a custom port for TCP:
transport.tcp.port: 9303  
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
 
特别注意:——坑:
 
transport.tcp.port
 
参考:http://knktc.com/2016/06/10/elasticsearch-multiple-instances/?utm_source=tuicool&utm_medium=referral
由于到了2.x版本之后,ES取消了默认的广播模式来发现master节点,需要使用该配置来指定发现master节点。这个配置在单机双实例的配置中需要特别注意下,因为习惯上我们配置时并未指定master节点的tcp端口,如果实例的transport.tcp.port配置为9301,那么实例启动后会认为discovery.zen.ping.unicast.hosts中指定的主机tcp端口也是9301,可能导致这些节点无法找到master节点。因此在该配置中需要指定master节点提供服务的tcp端口。
 
所以配置了
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
 
 
(5)启动
启动命令为:
 
bin/elasticsearch -Des.path.conf=cluster/config/ClientNode  -d -p cluster/ClientNode.pid
bin/elasticsearch -Des.path.conf=cluster/config/DataNode1  -d -p cluster/DataNode1.pid
bin/elasticsearch -Des.path.conf=cluster/config/DataNode2  -d -p cluster/DataNode2.pid
bin/elasticsearch -Des.path.conf=cluster/config/DataNode3  -d -p cluster/DataNode3.pid
 
(6)状态:
 
bash-4.2$ jps -lm
3815 org.elasticsearch.bootstrap.Elasticsearch start -Des.path.conf=cluster/config/DataNode1 -d -p cluster/DataNode1.pid
4152 sun.tools.jps.Jps -lm
4089 org.elasticsearch.bootstrap.Elasticsearch start -Des.path.conf=cluster/config/ClientNode -d -p cluster/ClientNode.pid
3836 org.elasticsearch.bootstrap.Elasticsearch start -Des.path.conf=cluster/config/DataNode2 -d -p cluster/DataNode2.pid
3885 org.elasticsearch.bootstrap.Elasticsearch start -Des.path.conf=cluster/config/DataNode3 -d -p cluster/DataNode3.pid
 
ClientNode
 
{
  "name" : "ClientNode",
  "cluster_name" : "ES_CLUSTER",
  "version" : {
    "number" : "2.4.0",
    "build_hash" : "ce9f0c7394dee074091dd1bc4e9469251181fc55",
    "build_timestamp" : "2016-08-29T09:14:17Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.2"
  },
  "tagline" : "You Know, for Search"
}
 

ElasticSearch2.4.0单机版及集群安装
            
    
    博客分类: Search elasticsearchelasiicsarch v2.4cluster 
 
(7)停止
 
for pid in `jps | grep Elasticsearch | awk '{print $1}'`
do
kill -9 $pid
done;
 
 
 
  • ElasticSearch2.4.0单机版及集群安装
            
    
    博客分类: Search elasticsearchelasiicsarch v2.4cluster 
  • 大小: 95.4 KB
  • ElasticSearch2.4.0单机版及集群安装
            
    
    博客分类: Search elasticsearchelasiicsarch v2.4cluster 
  • 大小: 83.8 KB
  • ElasticSearch2.4.0单机版及集群安装
            
    
    博客分类: Search elasticsearchelasiicsarch v2.4cluster 
  • 大小: 121.2 KB