java - Elasticsearch 1.5 没有queryapi的curl删除

Question

在我的elasticsearch 1.4上，我曾经使用DeleteByQuery API删除文档，如下所示:

curl -XDELETE http://my_elasticsearch:9200/_all/_query?q=some_field:some_value

这并不完美(因为经常出现 OutOfMemoryError)，但这足以满足我的需求(此时)。

但现在我使用新的 elasticsearch 1.5，并且在文档中我读到了:

Deprecated in 1.5.0.

"Delete by Query will be removed in 2.0: it is problematic since it silently forces a refresh which can quickly cause OutOfMemoryError during concurrent indexing, and can also cause primary and replica to become inconsistent. Instead, use the scroll/scan API to find all matching ids and then issue a bulk request to delete them..

所以我想使用滚动/扫描 API 做同样的事情。但是如何使用这个来删除呢？我不明白怎么办。 documentation API和 documentation Java API对我来说似乎并不完整(缺少删除的示例)。

PS:我正在寻找了解java或curl(无论对我来说，最终我都需要两者)。

Answer 1

我也遇到了这个问题，但找不到好的代码示例。我会告诉你我的想法。我不确定这是否是最好的方法，所以请随时评论如何改进。请注意，我将查询结果的大小设置为 Integer.MAX_VALUE，以便查询将返回所有(或尽可能多)需要删除的结果。

1. 运行查询以获取所有要删除的 ID
2. 将所有 ID 的删除请求添加到批量请求
3. 运行批量请求
4. 重新运行查询以查看是否还有更多记录需要删除

5. 必要时重复

   private void deleteAllByQuery(final String index, final String type, final QueryBuilder query) {
       SearchResponse response = elasticSearchClient.prepareSearch(index)
               .setTypes(type)
               .setQuery(query)
               .setSize(Integer.MAX_VALUE)
               .execute().actionGet();
   
       SearchHit[] searchHits = response.getHits().getHits();
   
       while (searchHits.length > 0) {
           LOGGER.debug("Need to delete " + searchHits.length + " records");
   
           // Create bulk request
           final BulkRequestBuilder bulkRequest = elasticSearchClient.prepareBulk().setRefresh(true);
   
           // Add search results to bulk request
           for (final SearchHit searchHit : searchHits) {
               final DeleteRequest deleteRequest = new DeleteRequest(index, type, searchHit.getId());
               bulkRequest.add(deleteRequest);
           }
   
           // Run bulk request
           final BulkResponse bulkResponse = bulkRequest.execute().actionGet();
           if (bulkResponse.hasFailures()) {
               LOGGER.error(bulkResponse.buildFailureMessage());
           }
   
           // After deleting, we should check for more records
           response = elasticSearchClient.prepareSearch(index)
               .setTypes(type)
               .setQuery(query)
               .setSize(Integer.MAX_VALUE)
               .execute().actionGet();
   
           searchHits = response.getHits().getHits();
       }
   }