elasticsearch - 重新索引 Elasticsearch 不会返回所有文档

Question

我的 flex 搜索大约有150万个文档。我希望为它们重新编制索引，以便每个索引过滤包含某些关键字的文档，以及一个不包含我在其他索引中指定的任何关键字的(null index)文档。我不确定为什么索引返回的文档少于预期的数量。特别是，我预计其中的120万个文档null index，但在新索引中仅返回了约3万个文档。感谢您对我在这里做错的想法!
这就是我重新索引包含多个字段中某些关键字的文档的方式

curl --location --request POST 'http://abcdef2344:9200/_reindex' \
--header 'Content-Type: application/json' \
--data-raw '{
  "source": {
    "index": "mydocs_email_*",
    "query": {
      "bool": {
        "filter": [
          {
            "bool": {
              "should": [
                {
                  "multi_match": {
                    "fields": [
                      "content",
                      "meta.raw.Message:Raw-Header:Subject"
                    ],
                    "query": "keyword1"
                  }
                },
                {
                  "multi_match": {
                    "fields": [
                      "content",
                      "meta.raw.Message:Raw-Header:Subject"
                    ],
                    "query": "keyword2"
                  }
                }
              ]
            }
          }
        ]
      }
    }
  },
  "dest": {
    "index": "analysis_keywords"
  }
}'

然后，我使用must_not创建不包含keyword1和keyword2的另一个索引。

curl --location --request POST 'http://abcdef2344:9200/_reindex' \
--header 'Content-Type: application/json' \
--data-raw '{
  "source": {
    "index": "mydocs_email_*",
    "query": {
      "bool": {
        "filter": [
          {
            "bool": {
              "must_not": [
                {
                  "multi_match": {
                    "fields": [
                      "content",
                      "meta.raw.Message:Raw-Header:Subject"
                    ],
                    "query": "keyword1"
                  }
                },
                {
                  "multi_match": {
                    "fields": [
                      "content",
                      "meta.raw.Message:Raw-Header:Subject"
                    ],
                    "query": "keyword2"
                  }
                }
              ]
            }
          }
        ]
      }
    }
  },
  "dest": {
    "index": "analysis_null"
  }
}'

null index返回了29.7k个文档。从错误消息中看来，我应该期望有128万个文件。它还说，我需要增加索引中的字段数-运行上面的代码后，我也需要这样做。虽然文件数量仍然保持不变。

{"took":53251,"timed_out":false,"total":1277428,"updated":243,"created":29755,"deleted":0,"batches":30,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1.0,"throttled_until_millis":0,"failures":[{"index":"analysis_null","type":"_doc","id":"/email/.......msg","cause":{"type":"illegal_argument_exception","reason":"Limit of total fields [1000] in index [analysis_null] has been exceeded"},"status":400}]

Answer 1

错误的含义与所说的完全一样-重新索引期间超出了字段总数的硬限制。
重新索引之前不更改该设置可以解决问题吗？

DELETE analysis_null

PUT analysis_null
{
  "settings": {
    "index.mapping.total_fields.limit": 10000
  }
}