我正在尝试使用 SPARQL 从三元组商店生成一些用户统计信息。请参阅下面的查询。如何改进?我在这里做坏事吗?为什么这会消耗这么多内存? (见本文末尾的背景故事)
我更喜欢在三元组商店内进行聚合和连接。拆分查询意味着我必须在数据库之外“手动”加入结果,从而失去三重存储的效率和优化。无需无缘无故地重新发明轮子。
查询
SELECT
?person
(COUNT(DISTINCT ?sent_email) AS ?sent_emails)
(COUNT(DISTINCT ?received_email) AS ?received_emails)
(COUNT(DISTINCT ?receivedInCC_email) AS ?receivedInCC_emails)
(COUNT(DISTINCT ?revision) AS ?commits)
WHERE {
?person rdf:type foaf:Person.
OPTIONAL {
?sent_email rdf:type email:Email.
?sent_email email:sender ?person.
}
OPTIONAL {
?received_email rdf:type email:Email.
?received_email email:recipient ?person.
}
OPTIONAL {
?receivedInCC_email rdf:type email:Email.
?receivedInCC_email email:ccRecipient ?person.
}
OPTIONAL {
?revision rdf:type vcs:VcsRevision.
?revision vcs:committedBy ?person.
}
}
GROUP BY ?person
ORDER BY DESC(?commits)
背景
问题是我在 AllegroGraph 中收到错误“QUERY MEMORY LIMIT REACHED”(另请参阅我的相关 SO question)。由于存储库仅包含大约 20 万个三元组,这些三元组很容易放入一个(ntriples)输入文件中。 60MB,不知道怎么执行查询结果需要4GB以上的内存,大概高了两个数量级。
最佳答案
尝试在子查询中拆分计算,例如:
SELECT
?person
(MAX(?sent_emails_) AS ?sent_emails_)
(MAX(?received_emails_ AS ?received_emails_)
(MAX(?receivedInCC_emails_ AS ?receivedInCC_emails_)
(MAX(?commits_) AS ?commits)
WHERE {
{
SELECT
?person
(COUNT(DISTINCT ?sent_email) AS ?sent_emails_)
(0 AS ?received_emails_)
(0 AS ?commits_)
WHERE {
?sent_email rdf:type email:Email.
?sent_email email:sender ?person.
?person rdf:type foaf:Person.
} GROUP BY ?person
} union {
(similar pattern for the others)
....
}
}
GROUP BY ?person
ORDER BY DESC(?commits)
目标是:
关于optimization - 具有多个聚合的 SPARQL 查询超出内存限制,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/13937516/