Spring Boot整合ElasticSearch单/多集群案例

Spring Boot整合ElasticSearch单个集群和多个集群案例分享，本文涉及内容：

• 导入spring boot elasticsearch starter
• 单个es集群案例
• 多个es集群案例

本文内容适合于:

• spring boot 1.x,2.x
• elasticsearch 1.x,2.x,5.x,6.x,+

1.导入spring boot elasticsearch starter
在spring boot项目中导入spring boot elasticsearch starter

maven工程

        <dependency>
            <groupId>com.bbossgroups.plugins</groupId>
            <artifactId>bboss-elasticsearch-spring-boot-starter</artifactId>
            <version>5.2.5</version>
        </dependency>

gradle工程

compile "com.bbossgroups.plugins:bboss-elasticsearch-spring-boot-starter:5.2.5"

2.创建spring boot启动类
新建Application类：

package org.bboss.elasticsearchtest.springboot;


import org.frameworkset.elasticsearch.ElasticSearchHelper;
import org.frameworkset.elasticsearch.client.ClientInterface;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;

/**
 *  @author yinbp [[e-mail]]
 *  
 */

@SpringBootApplication

public class Application {

    private Logger logger = LoggerFactory.getLogger(Application.class);

   
    public static void main(String[] args) {

        SpringApplication.run(Application.class, args);
    }
}

Application类中定义了一个main方法用来启动spring boot elasticsearch测试应用。

Application类将被用于启动下面两个测试用例：

• 单es集群测试用例
• 多es集群测试用例

定义elasticsearch rest client组件实列管理类ServiceApiUtil：

package org.bboss.elasticsearchtest.springboot;
/*
 *  Copyright 2008 biaoping.yin
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 */

import org.frameworkset.elasticsearch.ElasticSearchHelper;
import org.frameworkset.elasticsearch.client.ClientInterface;
import org.springframework.stereotype.Service;

/**
 * 管理es rest client组件实例
 */
@Service
public class ServiceApiUtil {


   /**
    * 获取操作默认的es集群的客户端工具组件
    * @return
    */
   public ClientInterface restClient(){
      return  ElasticSearchHelper.getRestClientUtil();
   }

   /**
    * 获取操作默认的es集群的加载dsl配置文件的客户端工具组件
    * @return
    */
   public ClientInterface restDemoConfigClient(){
      return  ElasticSearchHelper.getConfigRestClientUtil("esmapper/demo.xml");
   }

   /**
    * 获取操作logs的es集群的客户端工具组件
    * @return
    */
   public ClientInterface restClientLogs(){
      return  ElasticSearchHelper.getRestClientUtil("logs");
   }
   /**
    * 获取操作logs的es集群的加载dsl配置文件的客户端工具组件
    * @return
    */
   public ClientInterface restConfigClientLogs(){
      return  ElasticSearchHelper.getConfigRestClientUtil("logs","esmapper/demo.xml");
   }
}

实列管理类ServiceApiUtil将被注入到后续的测试用例中。

3.单es集群配置和使用
3.1 配置单es集群
修改spring boot配置文件application.properties内容（yml格式配置文件参考下面的内容配置）：

##ES集群配置
spring.elasticsearch.bboss.elasticUser=elastic
spring.elasticsearch.bboss.elasticPassword=changeme

#elasticsearch.rest.hostNames=10.1.236.88:9200
#elasticsearch.rest.hostNames=127.0.0.1:9200
#elasticsearch.rest.hostNames=10.21.20.168:9200
spring.elasticsearch.bboss.elasticsearch.rest.hostNames=10.21.20.168:9200
#elasticsearch.rest.hostNames=10.180.211.27:9280,10.180.211.27:9281,10.180.211.27:9282
spring.elasticsearch.bboss.elasticsearch.dateFormat=yyyy.MM.dd
spring.elasticsearch.bboss.elasticsearch.timeZone=Asia/Shanghai
spring.elasticsearch.bboss.elasticsearch.ttl=2d
#在控制台输出脚本调试开关showTemplate,false关闭，true打开，同时log4j至少是info级别
spring.elasticsearch.bboss.elasticsearch.showTemplate=true
spring.elasticsearch.bboss.elasticsearch.discoverHost=false
# dsl配置文件热加载扫描时间间隔，毫秒为单位，默认5秒扫描一次，<= 0时关闭扫描机制
spring.elasticsearch.bboss.dslfile.refreshInterval = -1

##es client http连接池配置
spring.elasticsearch.bboss.http.timeoutConnection = 400000
spring.elasticsearch.bboss.http.timeoutSocket = 400000
spring.elasticsearch.bboss.http.connectionRequestTimeout=400000
spring.elasticsearch.bboss.http.retryTime = 1
spring.elasticsearch.bboss.http.maxLineLength = -1
spring.elasticsearch.bboss.http.maxHeaderCount = 200
spring.elasticsearch.bboss.http.maxTotal = 400
spring.elasticsearch.bboss.http.defaultMaxPerRoute = 200
spring.elasticsearch.bboss.http.soReuseAddress = false
spring.elasticsearch.bboss.http.soKeepAlive = false
spring.elasticsearch.bboss.http.timeToLive = 3600000
spring.elasticsearch.bboss.http.keepAlive = 3600000
spring.elasticsearch.bboss.http.keystore =
spring.elasticsearch.bboss.http.keyPassword =
# ssl 主机名称校验，是否采用default配置，
# 如果指定为default，就采用DefaultHostnameVerifier,否则采用 SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER
spring.elasticsearch.bboss.http.hostnameVerifier =

## 数据库数据源配置，使用db-es数据导入功能时需要配置
#spring.elasticsearch.bboss.db.name = test
#spring.elasticsearch.bboss.db.user = root
#spring.elasticsearch.bboss.db.password = 123456
#spring.elasticsearch.bboss.db.driver = com.mysql.jdbc.Driver
#spring.elasticsearch.bboss.db.url = jdbc:mysql://localhost:3306/bboss
#spring.elasticsearch.bboss.db.usePool = false
#spring.elasticsearch.bboss.db.validateSQL = select 1

单ES集群配置项都是以spring.elasticsearch.bboss开头。

3.2 单集群测试用例
编写es单集群测试用例BBossESStarterTestCase

/*
 * Copyright 1999-2018 Alibaba Group Holding Ltd.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.bboss.elasticsearchtest.springboot;


import org.frameworkset.elasticsearch.client.ClientInterface;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

/**
 * 单集群演示功能测试用例，spring boot配置项以spring.elasticsearch.bboss开头
 * 对应的配置文件为application.properties文件
 * @author  yinbp [[e-mail]]
 */
@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
public class BBossESStarterTestCase {
	 
    @Test
    public void testBbossESStarter() throws Exception {
//        System.out.println(bbossESStarter);

		//验证环境,获取es状态
		String response = serviceApiUtil.restClient().executeHttp("_cluster/state?pretty",ClientInterface.HTTP_GET);

		System.out.println(response);
		//判断索引类型是否存在，false表示不存在，正常返回true表示存在
		boolean exist = serviceApiUtil.restClient().existIndiceType("twitter","tweet");

		//判读索引是否存在，false表示不存在，正常返回true表示存在
		exist = serviceApiUtil.restClient().existIndice("twitter");

		exist = serviceApiUtil.restClient().existIndice("agentinfo");

    }


}

直接通过junit运行上述测试用例即可

4.多ES集群测试用例
4.1 配置多es集群
修改spring boot配置文件application-multi-datasource.properties，内容如下：

##多集群配置样例，如果需要做多集群配置，请将参照本文内容修改application.properties文件内容
spring.elasticsearch.bboss.default.name = default
##default集群配配置
spring.elasticsearch.bboss.default.elasticUser=elastic
spring.elasticsearch.bboss.default.elasticPassword=changeme

#elasticsearch.rest.hostNames=10.1.236.88:9200
#elasticsearch.rest.hostNames=127.0.0.1:9200
spring.elasticsearch.bboss.default.elasticsearch.rest.hostNames=10.21.20.168:9200
#elasticsearch.rest.hostNames=10.180.211.27:9280,10.180.211.27:9281,10.180.211.27:9282
spring.elasticsearch.bboss.default.elasticsearch.dateFormat=yyyy.MM.dd
spring.elasticsearch.bboss.default.elasticsearch.timeZone=Asia/Shanghai
spring.elasticsearch.bboss.default.elasticsearch.ttl=2d
#在控制台输出脚本调试开关showTemplate,false关闭，true打开，同时log4j至少是info级别
spring.elasticsearch.bboss.default.elasticsearch.showTemplate=true
spring.elasticsearch.bboss.default.elasticsearch.discoverHost=false

##default连接池配置
spring.elasticsearch.bboss.default.http.timeoutConnection = 400000
spring.elasticsearch.bboss.default.http.timeoutSocket = 400000
spring.elasticsearch.bboss.default.http.connectionRequestTimeout=400000
spring.elasticsearch.bboss.default.http.retryTime = 1
spring.elasticsearch.bboss.default.http.maxLineLength = -1
spring.elasticsearch.bboss.default.http.maxHeaderCount = 200
spring.elasticsearch.bboss.default.http.maxTotal = 400
spring.elasticsearch.bboss.default.http.defaultMaxPerRoute = 200
spring.elasticsearch.bboss.default.http.keystore =
spring.elasticsearch.bboss.default.http.keyPassword =
# ssl 主机名称校验，是否采用default配置，
# 如果指定为default，就采用DefaultHostnameVerifier,否则采用 SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER
spring.elasticsearch.bboss.default.http.hostnameVerifier =

##logs集群配置
spring.elasticsearch.bboss.logs.name = logs
spring.elasticsearch.bboss.logs.elasticUser=elastic
spring.elasticsearch.bboss.logs.elasticPassword=changeme

#elasticsearch.rest.hostNames=10.1.236.88:9200
spring.elasticsearch.bboss.logs.elasticsearch.rest.hostNames=127.0.0.1:9200
#elasticsearch.rest.hostNames=10.21.20.168:9200
#elasticsearch.rest.hostNames=10.180.211.27:9280,10.180.211.27:9281,10.180.211.27:9282
spring.elasticsearch.bboss.logs.elasticsearch.dateFormat=yyyy.MM.dd
spring.elasticsearch.bboss.logs.elasticsearch.timeZone=Asia/Shanghai
spring.elasticsearch.bboss.logs.elasticsearch.ttl=2d
#在控制台输出脚本调试开关showTemplate,false关闭，true打开，同时log4j至少是info级别
spring.elasticsearch.bboss.logs.elasticsearch.showTemplate=true
spring.elasticsearch.bboss.logs.elasticsearch.discoverHost=false

##logs集群对应的连接池配置
spring.elasticsearch.bboss.logs.http.timeoutConnection = 400000
spring.elasticsearch.bboss.logs.http.timeoutSocket = 400000
spring.elasticsearch.bboss.logs.http.connectionRequestTimeout=400000
spring.elasticsearch.bboss.logs.http.retryTime = 1
spring.elasticsearch.bboss.logs.http.maxLineLength = -1
spring.elasticsearch.bboss.logs.http.maxHeaderCount = 200
spring.elasticsearch.bboss.logs.http.maxTotal = 400
spring.elasticsearch.bboss.logs.http.defaultMaxPerRoute = 200
# https证书配置
spring.elasticsearch.bboss.logs.http.keystore =
spring.elasticsearch.bboss.logs.http.keyPassword =
# ssl 主机名称校验，是否采用default配置，
# 如果指定为default，就采用DefaultHostnameVerifier,否则采用 SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER
spring.elasticsearch.bboss.logs.http.hostnameVerifier =
# dsl配置文件热加载扫描时间间隔，毫秒为单位，默认5秒扫描一次，<= 0时关闭扫描机制
spring.elasticsearch.bboss.dslfile.refreshInterval = -1

配置说明：

上面配置了两个集群：default和logs

每个集群配置项的前缀为：spring.elasticsearch.bboss.集群名字，其中的集群名字是一个自定义的逻辑名称，用来在client api中引用集群。

default集群的配置项前缀为：

spring.elasticsearch.bboss.default
logs集群的配置项前缀为：

spring.elasticsearch.bboss.logs
同时每个集群的配置项目里面必须包含name项目的配置

default集群name配置：

spring.elasticsearch.bboss.default.name = default
logs集群name配置：

##logs集群配置
spring.elasticsearch.bboss.logs.name = logs
4.2 定义加载多es集群配置的spring boot Configuration类
新建类MultiESSTartConfigurer

package org.bboss.elasticsearchtest.springboot;
/*
 *  Copyright 2008 biaoping.yin
 *
 *  Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 *  You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 */

import org.frameworkset.elasticsearch.ElasticSearchHelper;
import org.frameworkset.elasticsearch.boot.BBossESStarter;
import org.frameworkset.elasticsearch.client.ClientInterface;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;

/**
 * 配置多个es集群
 * 指定多es数据源profile：multi-datasource
 */
@Configuration
@Profile("multi-datasource") 
public class MultiESSTartConfigurer {
	@Primary
	@Bean(initMethod = "start")
	@ConfigurationProperties("spring.elasticsearch.bboss.default")
	public BBossESStarter bbossESStarterDefault(){
		return new BBossESStarter();

	}

	@Bean(initMethod = "start")
	@ConfigurationProperties("spring.elasticsearch.bboss.logs")
	public BBossESStarter bbossESStarterLogs(){
		return new BBossESStarter();

	}

}

说明：

MultiESSTartConfigurer通过以下两个方法分别加载default和logs两个es集群的配置

default集群配置加载

@Primary
@Bean(initMethod = "start")
@ConfigurationProperties("spring.elasticsearch.bboss.default")
public BBossESStarter bbossESStarterDefault()
logs集群配置加载

@Bean(initMethod = "start")
@ConfigurationProperties("spring.elasticsearch.bboss.logs")
public BBossESStarter bbossESStarterLogs()

4.3 定义多es集群测试用例
多es集群测试用例MultiBBossESStartersTestCase

package org.bboss.elasticsearchtest.springboot;


import org.frameworkset.elasticsearch.client.ClientInterface;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.junit4.SpringRunner;

/**
 * 多集群演示功能测试用例，spring boot配置项以spring.elasticsearch.bboss.集群名称开头，例如：
 * spring.elasticsearch.bboss.default 默认es集群
 * spring.elasticsearch.bboss.logs  logs es集群
 * 两个集群通过 org.bboss.elasticsearchtest.springboot.MultiESSTartConfigurer加载
 * 对应的配置文件为application-multi-datasource.properties文件
 * 通过ActiveProfiles指定并激活多es集群配置：multi-datasource 
 * @author yinbp [[e-mail]]
 */
@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
@ActiveProfiles("multi-datasource")
public class MultiBBossESStartersTestCase {

    @Test
    public void testMultiBBossESStarters() throws Exception {

		//验证环境,获取es状态
		String response = serviceApiUtil.restClient().executeHttp("_cluster/state?pretty",ClientInterface.HTTP_GET);
		System.out.println(response);


		//判断索引类型是否存在，false表示不存在，正常返回true表示存在
		boolean exist = serviceApiUtil.restClientLogs().existIndiceType("twitter","tweet");
		System.out.println("twitter/tweet:"+exist);
		//判读索引是否存在，false表示不存在，正常返回true表示存在
		exist = serviceApiUtil.restClientLogs().existIndice("twitter");
		System.out.println("twitter:"+exist);
		exist = serviceApiUtil.restClientLogs().existIndice("agentinfo");
		System.out.println("agentinfo:"+exist);
    }


}

直接通过junit运行上述测试用例即可。

5.完整的demo工程
https://gitee.com/bbossgroups/eshelloword-spring-boot-starter

https://github.com/bbossgroups/eshelloword-spring-boot-starter

6 开发交流
elasticsearch技术交流群：166471282

elasticsearch微信公众号：

The best elasticsearch highlevel java rest api-----bboss