我是Elasticsearch的新手。我正在尝试索引包含100,000+个对象的json文件。我的json文件的格式为:
[{"ingredients": [{"text": "Butter"}, {"text": "Strawberries"}, {"text": "Granola"}],
"url": "http://tastykitchen.com/recipes/breakfastbrunch/yogurt-parfaits/",
"title": "Yogurt Parfaits",
"id": "000095fc1d",
"instructions": [{"text": "Layer all ingredients in a serving dish."}]},
{"ingredients":
.....]
这是列表的形式。
我现在使用写来索引文件的python代码是:
from elasticsearch import Elasticsearch
es = Elasticsearch([{'host': 'localhost','port': 9200}])
f = open('data.json')
import json
data = json.load(f)
for i in data:
res = es.index(index='food',doc_type='Recipe',id=i["id"],body=i)
该方法花费大量时间并且效率低下。我阅读的其他方法需要使用以下格式的文件:
{"index": {"_index": "index_name", "_type": "index_type", "_id": "doc_id"}}
{"ingredients:....
您能建议一种索引文件的有效方法吗?
最佳答案
尝试使用Elasticsearch批量Api
Performs multiple indexing or delete operations in a single API call. This reduces overhead and can greatly increase indexing speed.
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#docs-bulk
https://elasticsearch-py.readthedocs.io/en/master/helpers.html
关于python - 如何使用 Elasticsearch 和Python有效地索引json文件?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/61372166/