我正在通过 Elastic Search 收集日志。日志收集如下。
ex.
{
"name" : "John"
"team" : "IT"
"startTime" : "21:00"
"result" : "pass"
},
{
"name" : "James"
"team" : "HR"
"startTime" : "21:04"
"result" : "pass"
},
{
"name" : "Paul"
"team" : "IT"
"startTime" : "21:05"
"result" : "pass"
},
{
"name" : "Jackson"
"team" : "Marketing"
"startTime" : "21:30"
"result" : "fail"
},
{
"name" : "John"
"team" : "IT"
"startTime" : "21:41"
"result" : "pass"
},
.....and so on
如果您对这些收集的日志运行以下查询,GET logData/_search
{
"size": 0,
"aggs": {
"Documents_per_team": {
"terms": {
"field": "team"
}
}
}
}
以下结果将被曝光。"aggregations" : {
"Documents_per_team" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "IT",
"doc_count" : 70
},
{
"key" : "Marketing",
"doc_count" : 55
},
{
"key" : "HR",
"doc_count" : 11
}
]
}
}
}
如果文档名称在此结果中重复,我想要的是消除重复。[原样]
[我想要的结果]
删除重复项后,我可以获得逐个团队的结果吗?
谢谢
最佳答案
你有两个选择:
两种方法都假定
name
每个团队级别的 s 都是唯一的。如果不是,您需要 adjust accordingly .此外,假设 name
映射为 keyword
类型,就像 team
.如果没有,您需要将它们替换为 your_field.keyword
1. 基数{
"size": 0,
"aggs": {
"Documents_per_team": {
"terms": {
"field": "team"
},
"aggs": {
"unique_names_per_team": {
"cardinality": {
"field": "name"
}
}
}
}
}
}
2. 脚本化指标{
"size": 0,
"aggs": {
"Documents_per_team": {
"scripted_metric": {
"init_script": "state.by_department = [:]; state.dept_vs_name = [:];",
"map_script": """
def dept = doc['team'].value;
def name = doc['name'].value;
def name_already_considered = state.by_department.containsKey(dept) && state.dept_vs_name[dept].containsKey(name);
if (name_already_considered) {
return;
}
if (state.by_department.containsKey(dept)) {
state.by_department[dept] += 1;
} else {
state.by_department[dept] = 1
}
if (!state.dept_vs_name.containsKey(dept)) {
// init new map & set is first member
state.dept_vs_name[dept] = [name:true];
} else if (!state.dept_vs_name[dept].containsKey(name)) {
state.dept_vs_name[dept][name] = true;
}
""",
"combine_script": "return state.by_department",
"reduce_script": "return states"
}
}
}
}
备注 :如果您还希望查看基础部门与名称分割,您可以修改 combine_script
返回整个状态,即 return state
.
关于elasticsearch - 我想将 Elasticsearch 中的重复值合并为一个,并使用不同的过滤器查看结果,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/63863754/