sql - 带有 ORDER BY 的 DISTINCT 非常慢

Question

所以我是第一次使用 postgres，发现运行 distinct 和 group by queries 相当慢，目前我正在尝试查找最新记录以及它是否有效。 这是我想到的第一个查询:

SELECT DISTINCT ON (device_id) c.device_id, c.timestamp, c.working
         FROM call_logs c
         ORDER BY c.device_id, c.timestamp desc

它可以工作，但需要时间才能运行。

 Unique  (cost=94840.24..97370.54 rows=11 width=17) (actual time=424.424..556.253 rows=13 loops=1)
   ->  Sort  (cost=94840.24..96105.39 rows=506061 width=17) (actual time=424.423..531.905 rows=506061 loops=1)
     Sort Key: device_id, "timestamp" DESC
     Sort Method: external merge  Disk: 13272kB
     ->  Seq Scan on call_logs c  (cost=0.00..36512.61 rows=506061 width=17) (actual time=0.059..162.932 rows=506061 loops=1)
 Planning time: 0.152 ms
 Execution time: 557.957 ms
(7 rows)

我已更新查询以使用以下速度更快但非常丑陋的查询:

SELECT c.device_id, c.timestamp, c.working FROM call_logs c
  INNER JOIN (SELECT c.device_id, MAX(c.timestamp) AS timestamp
                                              FROM call_logs c
                                              GROUP BY c.device_id)
                                              newest on newest.timestamp = c.timestamp

和分析:

Nested Loop  (cost=39043.34..39136.08 rows=12 width=17) (actual time=216.406..216.580 rows=15 loops=1)
   ->  HashAggregate  (cost=39042.91..39043.02 rows=11 width=16) (actual time=216.347..216.351 rows=13 loops=1)
     Group Key: c_1.device_id
     ->  Seq Scan on call_logs c_1  (cost=0.00..36512.61 rows=506061 width=16) (actual time=0.026..125.482 rows=506061 loops=1)
   ->  Index Scan using call_logs_timestamp on call_logs c  (cost=0.42..8.44 rows=1 width=17) (actual time=0.016..0.016 rows=1 loops=13)
     Index Cond: ("timestamp" = (max(c_1."timestamp")))
 Planning time: 0.318 ms
 Execution time: 216.631 ms
(8 rows)

即使是 200 毫秒对我来说也确实有点慢，因为我想要的只是每个设备的最高记录(在索引表中)

这是我正在使用的索引:

CREATE INDEX call_logs_timestamp
ON public.call_logs USING btree
(timestamp)
TABLESPACE pg_default;

我已经尝试了以下索引，但根本没有帮助:

CREATE INDEX dev_ts_1
ON public.call_logs USING btree
(device_id, timestamp DESC, working)
TABLESPACE pg_default;

有什么想法是我遗漏了一些明显的东西吗？

Answer 1

200 毫秒对于遍历 50 万行来说确实不坏。但是对于这个查询:

SELECT DISTINCT ON (device_id) c.device_id, c.timestamp, c.working
FROM call_logs c
ORDER BY c.device_id, c.timestamp desc

那么您在 call_logs(device_id, timestamp desc, working) 上的索引应该是最佳索引。

为同一索引编写查询的其他两种方法是:

select c.*
from (select c.device_id, c.timestamp, c.working, c.*,
             row_number() over (partition by device_id order by timestamp desc) as seqnum
      from call_logs c
     ) c
where seqnum = 1;

和:

select c.device_id, c.timestamp, c.working
from call_logs c
where not exists (select 1
                  from call_logs c2
                  where c2.device_id = c.device_id and
                        c2.timestamp > c.timestamp
                 );