elasticsearch - 如何使用 Elasticsearch Mapping API 更改现有索引的字段类型

Question

我正在使用 ELK 并具有以下文档结构

 {
  "_index": "prod1-db.log-*",
  "_type": "db.log",
  "_id": "AVadEaq7",
  "_score": null,
  "_source": {
    "message": "2016-07-08T12:52:42.026+0000 I NETWORK  [conn4928242] end connection 192.168.170.62:47530 (31 connections now open)",
    "@version": "1",
    "@timestamp": "2016-08-18T09:50:54.247Z",
    "type": "log",
    "input_type": "log",
    "count": 1,
    "beat": {
      "hostname": "prod1",
      "name": "prod1"
    },
    "offset": 1421607236,
    "source": "/var/log/db/db.log",
    "fields": null,
    "host": "prod1",
    "tags": [
      "beats_input_codec_plain_applied"
    ]
  },
  "fields": {
    "@timestamp": [
      1471513854247
    ]
  },
  "sort": [
    1471513854247
  ]
} 

我想将 message 字段更改为 not_analyzed。我想知道如何使用 Elasticsedarch Mapping API 来实现这一点？例如，如何使用 PUT Mapping API 为现有索引添加新类型？

我正在使用 Kibana 4.5 和 Elasticsearch 2.3。

更新 在 logstash 中尝试了以下 template.json，

 1 {
 2   "template": "logstash-*",
 3   "mappings": {
 4     "_default_": {
 5       "properties": {
 6         "message" : {
 7           "type" : "string",
 8           "index" : "not_analyzed"
 9         }
10       }
11     }
12   }
13 }

启动logstash时出现以下错误，

logstash_1       | {:timestamp=>"2016-08-24T11:00:26.097000+0000", :message=>"Invalid setting for elasticsearch output plugin:\n\n  output {\n    elasticsearch {\n      # This setting must be a path\n      # File does not exist or cannot be opened /home/dw/docker-elk/logstash/core_mapping_template.json\n      template => \"/home/dw/docker-elk/logstash/core_mapping_template.json\"\n      ...\n    }\n  }", :level=>:error}
logstash_1       | {:timestamp=>"2016-08-24T11:00:26.153000+0000", :message=>"Pipeline aborted due to error", :exception=>#<LogStash::ConfigurationError: Something is wrong with your configuration.>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/config/mixin.rb:134:in `config_init'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/outputs/base.rb:63:in `initialize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:74:in `register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/agent.rb:473:in `start_pipeline'"], :level=>:error}
logstash_1       | {:timestamp=>"2016-08-24T11:00:29.168000+0000", :message=>"stopping pipeline", :id=>"main"}

Answer 1

当索引已经存在时，您不能更改它的映射，除非您为对象或多字段创建新字段。

如果您想为此使用 Mapping API，您的请求将如下所示:

PUT /prod1-db.log-*/_mapping/log
{
  "properties": {
    "message": {
      "type": "string",
      "index": "not_analyzed"
    }
  }
}

不过，我建议您使用映射创建一个 JSON 文件并将其添加到您的 logstash 配置中。

模板文件可能如下所示(您需要自定义):

{
  "template": "logstash-*",
  "mappings": {
    "_default_": {
      "properties": {
        "action" : {
          "type" : "string",
          "fields" : {
            "raw" : {
              "index" : "not_analyzed",
              "type" : "string"
            }
          }
        },
        "ad_domain" : {
          "type" : "string"
        },
        "auth" : {
          "type" : "long"
        },
        "authtime" : {
          "type" : "long"
        },
        "avscantime" : {
          "type" : "long"
        },
        "cached" : {
          "type" : "boolean"
        }
      }
    }
  }
}

Logstash 配置中的 elasticsearch 条目如下所示:

elasticsearch {
    template => "/etc/logstash/template/template.json"
    template_overwrite => true
}