json - 使用jq进行多个级别的计数

Question

我们发现了一些与感染有关的域名。现在，我们在.json文件中有了一个DNS名称列表，我想生成一个汇总输出，其中显示:用户列表，他们访问的唯一域，总数。如果我还可以获得每个域名的计数，则可获赠积分。

这是该文件的示例:

{"machine": "possible_victim01", "domain": "evil.com", "timestamp":1435071870}
{"machine": "possible_victim01", "domain": "evil.com", "timestamp":1435071875}
{"machine": "possible_victim01", "domain": "soevil.com", "timestamp":1435071877}
{"machine": "possible_victim02", "domain": "bad.com", "timestamp":1435071877}
{"machine": "possible_victim03", "domain": "soevil.com", "timestamp":1435071879}

理想情况下，我希望输出为:

{"possible_victim01": "total": 3, {"evil.com": 2, "soevil.com": 1}}
{"possible_victim02": "total": 1, {"bad.com": 1}}
{"possible_victim03": "total": 1, {"soevil.com": 1}}

我很乐意满足:

{"possible_victim01": "total": 3, ["evil.com", "soevil.com"]}
{"possible_victim02": "total": 1, ["bad.com"]}
{"possible_victim03": "total": 1, ["soevil.com"]}

我可以获得每个用户的记录总数，但是却丢失了域列表:

cat sample.json | jq -s 'group_by(.machine) | map({machine:.[0].machine,domain:.[0].domain, count:length}) '
[{"machine": "possible_victim01", "domain": "evil.com", "count": 3},  
{"machine": "possible_victim02", "domain": "bad.com", "count": 1},
{"machine": "possible_victim03", "domain": "soevil.com", "count": 1}]

这篇文章描述了如何解决问题的后半部分... JQ Aggregations and Crosstabs。我还没有发现任何描述前半部分的内容，请转到:

{"machine": "possible_victim01", "domain": "evil.com", "count":2}
{"machine": "possible_victim01", "domain": "soevil.com", "count":1}
{"machine": "possible_victim02", "domain": "bad.com", "count":1}
{"machine": "possible_victim03", "domain": "soevil.com", "count":1}

Answer 1

您需要执行两次group_by，一次按机器名称分组，然后再进行一次子分组以获取每个域的子计数。

jQuery查询:

group_by(.machine) | map({
    "machine": .[0].machine, 
    "total":length, 
    "domains": (group_by(.domain) | map({
        "key":.[0].domain, 
        "value":length}) | from_entries
    )
})

输出示例:

{
  "machine": "possible_victim01",
  "total": 3,
  "domains": {
    "evil.com": 2,
    "soevil.com": 1
  }
}
{
  "machine": "possible_victim02",
  "total": 1,
  "domains": {
    "bad.com": 1
  }
}
{
  "machine": "possible_victim03",
  "total": 1,
  "domains": {
    "soevil.com": 1
  }
}