sql - 日期之前的最新记录，按类别 : optimising

Question

我在 PostgreSQL 数据库中有一个名为 feeds_up 的表。看起来像:

| feed_url | isup | hasproblems | observed timestamp with tz    | id (pk)|
|----------|------|-------------|-------------------------------|--------|
| http://b.| t    | f           | 2013-02-27 16:34:46.327401+11 | 15235  |
| http://f.| f    | t           | 2013-02-27 16:31:25.415126+11 | 15236  |

它大约有 30 万行，每五分钟增长约 20 行。我有一个经常运行的查询(每次加载页面)

select distinct on (feed_url) feed_url, isUp, hasProblems
    from feeds_up
    where observed <= '2013-02-27T05:38:00.000Z'
    order by feed_url, observed desc;

我在那里放了一个例子时间，那个时间是参数化的。解释分析在 explain.depesz.com 上.大约需要 8 秒。疯了!

feed_url 只有大约 20 个唯一值，所以这看起来效率很低。我以为我会很愚蠢并在函数中尝试 FOR 循环。

CREATE OR REPLACE FUNCTION feedStatusAtDate(theTime timestamp with time zone) RETURNS SETOF feeds_up AS
$BODY$
DECLARE
    url feeds_list%rowtype;
BEGIN
FOR url IN SELECT * FROM feeds_list 
LOOP
    RETURN QUERY SELECT * FROM feeds_up
    WHERE observed <= theTime
    AND feed_url = url.feed_url
    ORDER BY observed DESC LIMIT 1;
END LOOP;
END;
$BODY$ language plpgsql;

select * from feedStatusAtDate('2013-02-27T05:38:00.000Z');

这只需要 307 毫秒!

在 SQL 中使用 FOR 循环让我误入歧途，我怎样才能做出像第一个一样高效的漂亮查询？那可能吗？或者这是 FOR 循环真正最好的事情吗？

预计到达时间

Postgres 版本:PostgreSQL 9.1.5 on i686-pc-linux-gnu，由 gcc (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973] 编译，32 位

feeds_up 上的索引:

CREATE INDEX feeds_up_url
  ON feeds_up
  USING btree
  (feed_url COLLATE pg_catalog."default");

CREATE INDEX feeds_up_url_observed
  ON feeds_up
  USING btree
  (feed_url COLLATE pg_catalog."default", observed DESC);

CREATE INDEX feeds_up_observed
  ON public.feeds_up
  USING btree
  (observed DESC);

Answer 1

假设“id”是连续的并且总是连续的，您可以通过在子查询中找到每个 feed_url 的 MAX(id) 来简化，然后按如下方式提取其余数据:

SELECT fu.feed_url, fu.isup, fu.hasproblems, fu.observed
FROM feeds_up fu
JOIN
(
  SELECT feed_url, max(id)  AS id FROM feeds_up
  WHERE observed <= '2013-03-27T05:38:00.000Z'
  GROUP BY feed_url
) AS q USING (id);
ORDER BY fu.feed_url, fu.observed desc;

我做了一个快速测试，仅使用“观察到”的索引就可以非常有效地工作。

更新:

要使用“observed”而不是“id”(因为记录可能不会按顺序插入)，您可以按如下方式修改上述查询:

SELECT DISTINCT ON (fu.feed_url) fu.feed_url, fu.isup, fu.hasproblems, fu.observed
FROM feeds_up fu
JOIN
(
  SELECT feed_url, max(observed) as observed FROM feeds_up
  WHERE observed <= '2013-03-27T05:38:00.000Z'
  GROUP BY feed_url
) AS q USING (feed_url, observed)
ORDER BY fu.feed_url, fu.observed desc;

在我的系统上，它与“已观察”上的一个索引几乎同时运行。 YMMV