SpringBoot 整合 Elasticsearch 實現海量級數據搜索功能

Posted on by

今天給大傢講講 SpringBoot 框架 整合 Elasticsearch 實現海量級數據搜索。

一、簡介

在上篇ElasticSearch 文章中，我們詳細的介紹瞭 ElasticSearch 的各種 api 使用。

實際的項目開發過程中，我們通常基於某些主流框架平臺進行技術開發，比如 SpringBoot，今天我們就以 SpringBoot 整合 ElasticSearch 為例，給大傢詳細的介紹 ElasticSearch 的使用！

SpringBoot 連接 ElasticSearch，主流的方式有以下四種方式

  • 方式一：通過 Elastic Transport Client 客戶端連接 es 服務器，底層基於 TCP 協議通過 transport 模塊和遠程 ES 服務端通信，不過，從 V7.0 開始官方不建議使用，V8.0開始正式移除。

  • 方式二：通過 Elastic Java Low Level Rest Client 客戶端連接 es 服務器，底層基於 HTTP 協議通過 restful API 來和遠程 ES 服務端通信，隻提供瞭最簡單最基本的 API，類似於上篇文章中給大傢介紹的 API 操作邏輯

  • Elastic Java High Level Rest Client
Elastic Java Low Level Rest Client
Elastic Transport Client

  • 方式四：通過 JestClient 客戶端連接 es 服務器，這是開源社區基於 HTTP 協議開發的一款 es 客戶端，官方宣稱接口及代碼設計比 ES 官方提供的 Rest 客戶端更簡潔、更合理，更好用，具有一定的 ES 服務端版本兼容性，但是更新速度不是很快，目前 ES 版本已經出到 V7.9，但是 JestClient 隻支持 V1.0~V6.X 版  本的 ES。

還有一個需要大傢註意的地方，那就是版本號的兼容！

在開發過程中，大傢尤其需要關註一下客戶端和服務端的版本號，要盡可能保持一致，比如服務端 es 的版本號是 6.8.2 ，那麼連接 es 的客戶端版本號，最好也是 6.8.2 ，即使因項目的原因不能保持一致，客戶端的版本號必須在 6.0.0 ～6.8.2 ，不要超過服務器的版本號，這樣客戶端才能保持正常工作，否則會出現很多意想不到的問題，假如客戶端是 7.0.4 的版本號，此時的程序會各種報錯，甚至沒辦法用！

為什麼要這樣做呢？主要原因就是 es 的服務端，高版本不兼容低版本；es6 和 es7 的某些 API 請求參數結構有著很大的區別，所以客戶端和服務端版本號盡量保持一致。

廢話也不多說瞭，直接上代碼！

二、代碼實踐

本文采用的 SpringBoot 版本號是 2.1.0.RELEASE ，服務端 es 的版本號是 6.8.2 ，客戶端采用的是官方推薦的 Elastic Java High Level Rest Client 版本號是 6.4.2 ，方便與 SpringBoot 的版本兼容。

2.1、導入依賴

<!--elasticsearch-->
<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>6.4.2</version>
</dependency>
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-client</artifactId>
    <version>6.4.2</version>
</dependency>
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>6.4.2</version>
</dependency>

2.2、配置環境變量

在 application.properties 全局配置文件中，配置 elasticsearch 自定義環境變量

elasticsearch.scheme=http
elasticsearch.address=127.0.0.1:9200
elasticsearch.userName=
elasticsearch.userPwd=
elasticsearch.socketTimeout=5000
elasticsearch.connectTimeout=5000
elasticsearch.connectionRequestTimeout=5000

2.3、創建 elasticsearch 的 config 類

@Configuration
public class ElasticsearchConfiguration {
    private static final Logger log = LoggerFactory.getLogger(ElasticsearchConfiguration.class);
    private static final int ADDRESS_LENGTH = 2;
    @Value("${elasticsearch.scheme:http}")
    private String scheme;
    @Value("${elasticsearch.address}")
    private String address;
    @Value("${elasticsearch.userName}")
    private String userName;
    @Value("${elasticsearch.userPwd}")
    private String userPwd;
    @Value("${elasticsearch.socketTimeout:5000}")
    private Integer socketTimeout;
    @Value("${elasticsearch.connectTimeout:5000}")
    private Integer connectTimeout;
    @Value("${elasticsearch.connectionRequestTimeout:5000}")
    private Integer connectionRequestTimeout;
    /**
     * 初始化客戶端
     * @return
     */
    @Bean(name = "restHighLevelClient")
    public RestHighLevelClient restClientBuilder() {
        HttpHost[] hosts = Arrays.stream(address.split(","))
                .map(this::buildHttpHost)
                .filter(Objects::nonNull)
                .toArray(HttpHost[]::new);
        RestClientBuilder restClientBuilder = RestClient.builder(hosts);
        // 異步參數配置
        restClientBuilder.setHttpClientConfigCallback(httpClientBuilder -> {
            httpClientBuilder.setDefaultCredentialsProvider(buildCredentialsProvider());
            return httpClientBuilder;
        });
        // 異步連接延時配置
        restClientBuilder.setRequestConfigCallback(requestConfigBuilder -> {
            requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeout);
            requestConfigBuilder.setSocketTimeout(socketTimeout);
            requestConfigBuilder.setConnectTimeout(connectTimeout);
            return requestConfigBuilder;
        });
        return new RestHighLevelClient(restClientBuilder);
    }
    /**
     * 根據配置創建HttpHost
     * @param s
     * @return
     */
    private HttpHost buildHttpHost(String s) {
        String[] address = s.split(":");
        if (address.length == ADDRESS_LENGTH) {
            String ip = address[0];
            int port = Integer.parseInt(address[1]);
            return new HttpHost(ip, port, scheme);
        } else {
            return null;
        }
    }
    /**
     * 構建認證服務
     * @return
     */
    private CredentialsProvider buildCredentialsProvider(){
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(userName,
                userPwd));
        return credentialsProvider;
    }
}

至此，客戶端配置完畢，項目啟動的時候，會自動註入到 Spring 的 ioc 容器裡面。

2.4、索引管理

es 中最重要的就是索引庫，客戶端如何創建呢？請看下文！

  • 創建索引

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class IndexJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 創建索引（簡單模式）
     * @throws IOException
     */
    @Test
    public void createIndex() throws IOException {
        CreateIndexRequest request = new CreateIndexRequest("cs_index");
        CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);
        System.out.println(response.isAcknowledged());
    }
    /**
     * 創建索引（復雜模式）
     * 可以直接把對應的文檔結構也一並初始化
     * @throws IOException
     */
    @Test
    public void createIndexComplete() throws IOException {
        CreateIndexRequest request = new CreateIndexRequest();
        //索引名稱
        request.index("cs_index");
        //索引配置
        Settings settings = Settings.builder()
                .put("index.number_of_shards", 3)
                .put("index.number_of_replicas", 1)
                .build();
        request.settings(settings);
        //映射結構字段
        Map<String, Object> properties = new HashMap();
        properties.put("id", ImmutableBiMap.of("type", "text"));
        properties.put("name", ImmutableBiMap.of("type", "text"));
        properties.put("sex", ImmutableBiMap.of("type", "text"));
        properties.put("age", ImmutableBiMap.of("type", "long"));
        properties.put("city", ImmutableBiMap.of("type", "text"));
        properties.put("createTime", ImmutableBiMap.of("type", "long"));
        Map<String, Object> mapping = new HashMap<>();
        mapping.put("properties", properties);
        //添加一個默認類型
        System.out.println(JSON.toJSONString(request));
        request.mapping("_doc",mapping);
        CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);
        System.out.println(response.isAcknowledged());
    }
}
  • 刪除索引
@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class IndexJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 刪除索引
     * @throws IOException
     */
    @Test
    public void deleteIndex() throws IOException {
        DeleteIndexRequest request = new DeleteIndexRequest("cs_index1");
        AcknowledgedResponse response = client.indices().delete(request, RequestOptions.DEFAULT);
        System.out.println(response.isAcknowledged());
    }
}

  • 查詢索引

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class IndexJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 查詢索引
     * @throws IOException
     */
    @Test
    public void getIndex() throws IOException {
        // 創建請求
        GetIndexRequest request = new GetIndexRequest();
        request.indices("cs_index");
        // 執行請求,獲取響應
        GetIndexResponse response = client.indices().get(request, RequestOptions.DEFAULT);
        System.out.println(response.toString());
    }
}

  • 查詢索引是否存在

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class IndexJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 檢查索引是否存在
     * @throws IOException
     */
    @Test
    public void exists() throws IOException {
        // 創建請求
        GetIndexRequest request = new GetIndexRequest();
        request.indices("cs_index");
        // 執行請求,獲取響應
        boolean response = client.indices().exists(request, RequestOptions.DEFAULT);
        System.out.println(response);
    }
}

  • 查詢所有的索引名稱

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class IndexJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 查詢所有的索引名稱
     * @throws IOException
     */
    @Test
    public void getAllIndices() throws IOException {
        GetAliasesRequest request = new GetAliasesRequest();
        GetAliasesResponse response =  client.indices().getAlias(request,RequestOptions.DEFAULT);
        Map<String, Set<AliasMetaData>> map = response.getAliases();
        Set<String> indices = map.keySet();
        for (String key : indices) {
            System.out.println(key);
        }
    }
}

  • 查詢索引映射字段

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class IndexJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 查詢索引映射字段
     * @throws IOException
     */
    @Test
    public void getMapping() throws IOException {
        GetMappingsRequest request = new GetMappingsRequest();
        request.indices("cs_index");
        request.types("_doc");
        GetMappingsResponse response = client.indices().getMapping(request, RequestOptions.DEFAULT);
        System.out.println(response.toString());
    }
}

  • 添加索引映射字段

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class IndexJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 添加索引映射字段
     * @throws IOException
     */
    @Test
    public void addMapping() throws IOException {
        PutMappingRequest request = new PutMappingRequest();
        request.indices("cs_index");
        request.type("_doc");
        //添加字段
        Map<String, Object> properties = new HashMap();
        properties.put("accountName", ImmutableBiMap.of("type", "keyword"));
        Map<String, Object> mapping = new HashMap<>();
        mapping.put("properties", properties);
        request.source(mapping);
        PutMappingResponse response = client.indices().putMapping(request, RequestOptions.DEFAULT);
        System.out.println(response.isAcknowledged());
    }
}

2.5、文檔管理

所謂文檔，就是向索引裡面添加數據，方便進行數據查詢，詳細操作內容，請看下文！

  • 添加文檔

public class UserDocument {
    private String id;
    private String name;
    private String sex;
    private Integer age;
    private String city;
    private Date createTime;

    //省略get、set...
}
@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class DocJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 添加文檔
     * @throws IOException
     */
    @Test
    public void addDocument() throws IOException {
        // 創建對象
        UserDocument user = new UserDocument();
        user.setId("1");
        user.setName("裡斯");
        user.setCity("武漢");
        user.setSex("男");
        user.setAge(20);
        user.setCreateTime(new Date());
        // 創建索引,即獲取索引
        IndexRequest request = new IndexRequest();
        // 外層參數
        request.id("1");
        request.index("cs_index");
        request.type("_doc");
        request.timeout(TimeValue.timeValueSeconds(1));
        // 存入對象
        request.source(JSON.toJSONString(user), XContentType.JSON);
        // 發送請求
        System.out.println(request.toString());
        IndexResponse response = client.index(request, RequestOptions.DEFAULT);
        System.out.println(response.toString());
    }
}

  • 更新文檔

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class DocJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 更新文檔（按需修改）
     * @throws IOException
     */
    @Test
    public void updateDocument() throws IOException {
        // 創建對象
        UserDocument user = new UserDocument();
        user.setId("2");
        user.setName("程咬金");
        user.setCreateTime(new Date());
        // 創建索引,即獲取索引
        UpdateRequest request = new UpdateRequest();
        // 外層參數
        request.id("2");
        request.index("cs_index");
        request.type("_doc");
        request.timeout(TimeValue.timeValueSeconds(1));
        // 存入對象
        request.doc(JSON.toJSONString(user), XContentType.JSON);
        // 發送請求
        System.out.println(request.toString());
        UpdateResponse response = client.update(request, RequestOptions.DEFAULT);
        System.out.println(response.toString());
    }
}

  • 刪除文檔

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class DocJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 刪除文檔
     * @throws IOException
     */
    @Test
    public void deleteDocument() throws IOException {
        // 創建索引,即獲取索引
        DeleteRequest request = new DeleteRequest();
        // 外層參數
        request.id("1");
        request.index("cs_index");
        request.type("_doc");
        request.timeout(TimeValue.timeValueSeconds(1));
        // 發送請求
        System.out.println(request.toString());
        DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);
        System.out.println(response.toString());
    }
}

  • 查詢文檔是不是存在

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class DocJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 查詢文檔是不是存在
     * @throws IOException
     */
    @Test
    public void exists() throws IOException {
        // 創建索引,即獲取索引
        GetRequest request = new GetRequest();
        // 外層參數
        request.id("3");
        request.index("cs_index");
        request.type("_doc");
        // 發送請求
        System.out.println(request.toString());
        boolean response = client.exists(request, RequestOptions.DEFAULT);
        System.out.println(response);
    }
}

  • 通過 ID 查詢指定文檔

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class DocJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 通過ID，查詢指定文檔
     * @throws IOException
     */
    @Test
    public void getById() throws IOException {
        // 創建索引,即獲取索引
        GetRequest request = new GetRequest();
        // 外層參數
        request.id("1");
        request.index("cs_index");
        request.type("_doc");
        // 發送請求
        System.out.println(request.toString());
        GetResponse response = client.get(request, RequestOptions.DEFAULT);
        System.out.println(response.toString());
    }
}

  • 批量添加文檔

@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = ElasticSearchApplication.class)
public class DocJunit {
    @Autowired
    private RestHighLevelClient client;
    /**
     * 批量添加文檔
     * @throws IOException
     */
    @Test
    public void batchAddDocument() throws IOException {
        // 批量請求
        BulkRequest bulkRequest = new BulkRequest();
        bulkRequest.timeout(TimeValue.timeValueSeconds(10));
        // 創建對象
        List<UserDocument> userArrayList = new ArrayList<>();
        userArrayList.add(new UserDocument("張三", "男", 30, "武漢"));
        userArrayList.add(new UserDocument("裡斯", "女", 31, "北京"));
        userArrayList.add(new UserDocument("王五", "男", 32, "武漢"));
        userArrayList.add(new UserDocument("趙六", "女", 33, "長沙"));
        userArrayList.add(new UserDocument("七七", "男", 34, "武漢"));
        // 添加請求
        for (int i = 0; i < userArrayList.size(); i++) {
            userArrayList.get(i).setId(String.valueOf(i));
            IndexRequest indexRequest = new IndexRequest();
            // 外層參數
            indexRequest.id(String.valueOf(i));
            indexRequest.index("cs_index");
            indexRequest.type("_doc");
            indexRequest.timeout(TimeValue.timeValueSeconds(1));
            indexRequest.source(JSON.toJSONString(userArrayList.get(i)), XContentType.JSON);
            bulkRequest.add(indexRequest);
        }
        // 執行請求
        BulkResponse response = client.bulk(bulkRequest, RequestOptions.DEFAULT);
        System.out.println(response.status());
    }
}

三、小結

本文主要圍繞 SpringBoot 整合 ElasticSearch 接受數據的插入和搜索使用技巧，在實際的使用過程中，版本號尤其的重要，不同版本的 es，對應的 api 是不一樣的。

到此這篇關於SpringBoot 整合 Elasticsearch 實現海量級數據搜索的文章就介紹到這瞭,更多相關SpringBoot 整合 Elasticsearch 數據搜索內容請搜索WalkonNet以前的文章或繼續瀏覽下面的相關文章希望大傢以後多多支持WalkonNet！

推薦閱讀：