此查询是使用 PostgreSQL 11(最新)之上的 Django ORM 构建的。这里的 PostgreSQL 运行在 docker 容器中。在生产中，我们使用 Cloud SQL，查询时间更快，但总体相当慢。
更新:查询和EXPLAIN ANALYZE也可在此处使用:https://explain.depesz.com/s/yQE7
我已经对所有 ID 列建立了索引，因此当 EXPLAIN 在 orders_query 上显示出过长的全表扫描时，我感到很惊讶。长时间运行的部分位于 EXPLAIN 日志的最顶部，但以下是相关部分(最上面的行):
GroupAggregate (cost=999695.25..19231670646.03 rows=1897315 width=105)
Group Key: users_profile.id, users_user.last_login
-> Sort (cost=999695.25..1007470.17 rows=3109968 width=73)
Sort Key: users_profile.id, users_user.last_login
-> Hash Right Join (cost=738.71..387928.02 rows=3109968 width=73)
Hash Cond: (orders_query.user_id = users_user.id)
-> Seq Scan on orders_query (cost=0.00..345054.24 rows=2943324 width=8)
-> Hash (cost=702.78..702.78 rows=2874 width=73)
-> Hash Right Join (cost=301.35..702.78 rows=2874 width=73)
问题:是否可以创建另一个索引来避免这种会减慢 SQL 查询速度的Seq Scan？
SELECT "users_profile"."id",
"users_profile"."created",
"users_profile"."modified",
"users_profile"."guid",
"users_profile"."user_id",
"users_profile"."charter_role_id",
"users_profile"."owner_id",
"users_profile"."volume_limit",
"users_profile"."vol_limit_ovrd",
"users_profile"."charter_vol_limit_contrib_ovrd",
"users_profile"."sponsored_ovrd",
"users_profile"."view_peers_ovrd",
"users_profile"."inherit_permits",
Count(DISTINCT "orders_order"."id") AS "orders",
(SELECT Count(DISTINCT U0."guid") AS "io"
FROM "orders_order" U0
INNER JOIN "users_user" U1
ON ( U0."user_id" = U1."id" )
INNER JOIN "users_profile" U2
ON ( U1."id" = U2."user_id" )
WHERE U2."owner_id" = "users_profile"."id"
GROUP BY U2."owner_id") AS "indirect_orders",
Count(DISTINCT "orders_query"."id") AS "searches",
(SELECT Count(DISTINCT U0."guid") AS "isch"
FROM "orders_query" U0
INNER JOIN "users_user" U1
ON ( U0."user_id" = U1."id" )
INNER JOIN "users_profile" U2
ON ( U1."id" = U2."user_id" )
WHERE U2."owner_id" = "users_profile"."id"
GROUP BY U2."owner_id") AS "indirect_searches",
(SELECT Count(DISTINCT U0."guid") AS "sg"
FROM "users_profile" U0
WHERE U0."owner_id" = "users_profile"."id"
GROUP BY U0."owner_id") AS "signups",
"users_user"."last_login" AS "last_activity"
FROM "users_profile"
INNER JOIN "users_user"
ON ( "users_profile"."user_id" = "users_user"."id" )
LEFT OUTER JOIN "orders_order"
ON ( "users_user"."id" = "orders_order"."user_id" )
LEFT OUTER JOIN "orders_query"
ON ( "users_user"."id" = "orders_query"."user_id" )
GROUP BY "users_profile"."id",
"users_user"."last_login"
解释分析
EXPLAIN ANALYZE
GroupAggregate (cost=999695.25..19231670646.03 rows=1897315 width=105) (actual time=682829.362..1112031.341 rows=1057 loops=1)
" Group Key: users_profile.id, users_user.last_login"
-> Sort (cost=999695.25..1007470.17 rows=3109968 width=73) (actual time=682711.948..768155.952 rows=222437142 loops=1)
" Sort Key: users_profile.id, users_user.last_login"
Sort Method: external merge Disk: 18873640kB
-> Hash Right Join (cost=738.71..387928.02 rows=3109968 width=73) (actual time=12.263..115509.811 rows=222437142 loops=1)
Hash Cond: (orders_query.user_id = users_user.id)
-> Seq Scan on orders_query (cost=0.00..345054.24 rows=2943324 width=8) (actual time=0.024..6992.146 rows=2945348 loops=1)
-> Hash (cost=702.78..702.78 rows=2874 width=73) (actual time=12.203..12.203 rows=3456 loops=1)
Buckets: 4096 Batches: 1 Memory Usage: 409kB
-> Hash Right Join (cost=301.35..702.78 rows=2874 width=73) (actual time=3.807..10.194 rows=3456 loops=1)
Hash Cond: (orders_order.user_id = users_user.id)
-> Seq Scan on orders_order (cost=0.00..344.96 rows=7396 width=8) (actual time=0.009..1.771 rows=7396 loops=1)
-> Hash (cost=288.13..288.13 rows=1057 width=69) (actual time=3.762..3.762 rows=1057 loops=1)
Buckets: 2048 Batches: 1 Memory Usage: 128kB
-> Hash Join (cost=36.78..288.13 rows=1057 width=69) (actual time=1.137..3.169 rows=1057 loops=1)
Hash Cond: (users_user.id = users_profile.user_id)
-> Seq Scan on users_user (cost=0.00..244.20 rows=2720 width=12) (actual time=0.014..0.610 rows=2720 loops=1)
-> Hash (cost=23.57..23.57 rows=1057 width=57) (actual time=1.022..1.174 rows=1057 loops=1)
Buckets: 2048 Batches: 1 Memory Usage: 113kB
-> Seq Scan on users_profile (cost=0.00..23.57 rows=1057 width=57) (actual time=0.008..0.504 rows=1057 loops=1)
SubPlan 1
-> GroupAggregate (cost=4.96..87.25 rows=14 width=12) (actual time=0.016..0.016 rows=0 loops=1057)
Group Key: u2.owner_id
-> Nested Loop (cost=4.96..86.89 rows=44 width=20) (actual time=0.008..0.013 rows=2 loops=1057)
-> Nested Loop (cost=4.68..71.05 rows=16 width=12) (actual time=0.005..0.008 rows=1 loops=1057)
-> Bitmap Heap Scan on users_profile u2 (cost=4.40..18.29 rows=16 width=8) (actual time=0.003..0.004 rows=1 loops=1057)
Recheck Cond: (owner_id = users_profile.id)
Heap Blocks: exact=300
-> Bitmap Index Scan on users_profile_owner_id_a9029da4 (cost=0.00..4.40 rows=16 width=0) (actual time=0.002..0.002 rows=1 loops=1057)
Index Cond: (owner_id = users_profile.id)
-> Index Only Scan using users_user_pkey on users_user u1 (cost=0.28..3.30 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=1037)
Index Cond: (id = u2.user_id)
Heap Fetches: 0
-> Index Scan using orders_order_user_id_e9b59eb1 on orders_order u0 (cost=0.28..0.89 rows=10 width=20) (actual time=0.003..0.004 rows=2 loops=1037)
Index Cond: (user_id = u1.id)
SubPlan 2
-> GroupAggregate (cost=0.99..10029.94 rows=14 width=12) (actual time=5.109..5.109 rows=0 loops=1057)
Group Key: u2_1.owner_id
-> Nested Loop (cost=0.99..9943.23 rows=17314 width=20) (actual time=0.197..4.366 rows=858 loops=1057)
-> Nested Loop (cost=0.56..110.20 rows=16 width=12) (actual time=0.182..0.196 rows=1 loops=1057)
-> Index Scan using users_profile_user_id_key on users_profile u2_1 (cost=0.28..57.44 rows=16 width=8) (actual time=0.182..0.192 rows=1 loops=1057)
Filter: (owner_id = users_profile.id)
Rows Removed by Filter: 1056
-> Index Only Scan using users_user_pkey on users_user u1_1 (cost=0.28..3.30 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=1037)
Index Cond: (id = u2_1.user_id)
Heap Fetches: 0
-> Index Scan using orders_query_user_id_a0b49874 on orders_query u0_1 (cost=0.43..564.34 rows=5022 width=20) (actual time=0.038..4.010 rows=874 loops=1037)
Index Cond: (user_id = u1_1.id)
SubPlan 3
-> GroupAggregate (cost=4.40..18.51 rows=14 width=12) (actual time=0.005..0.005 rows=0 loops=1057)
Group Key: u0_2.owner_id
-> Bitmap Heap Scan on users_profile u0_2 (cost=4.40..18.29 rows=16 width=20) (actual time=0.002..0.002 rows=1 loops=1057)
Recheck Cond: (owner_id = users_profile.id)
Heap Blocks: exact=300
-> Bitmap Index Scan on users_profile_owner_id_a9029da4 (cost=0.00..4.40 rows=16 width=0) (actual time=0.001..0.001 rows=1 loops=1057)
Index Cond: (owner_id = users_profile.id)
Planning Time: 4.279 ms
Execution Time: 1112994.119 ms
解释
GroupAggregate (cost=999695.25..19231670646.03 rows=1897315 width=105)
Group Key: users_profile.id, users_user.last_login
-> Sort (cost=999695.25..1007470.17 rows=3109968 width=73)
Sort Key: users_profile.id, users_user.last_login
-> Hash Right Join (cost=738.71..387928.02 rows=3109968 width=73)
Hash Cond: (orders_query.user_id = users_user.id)
-> Seq Scan on orders_query (cost=0.00..345054.24 rows=2943324 width=8)
-> Hash (cost=702.78..702.78 rows=2874 width=73)
-> Hash Right Join (cost=301.35..702.78 rows=2874 width=73)
Hash Cond: (orders_order.user_id = users_user.id)
-> Seq Scan on orders_order (cost=0.00..344.96 rows=7396 width=8)
-> Hash (cost=288.13..288.13 rows=1057 width=69)
-> Hash Join (cost=36.78..288.13 rows=1057 width=69)
Hash Cond: (users_user.id = users_profile.user_id)
-> Seq Scan on users_user (cost=0.00..244.20 rows=2720 width=12)
-> Hash (cost=23.57..23.57 rows=1057 width=57)
-> Seq Scan on users_profile (cost=0.00..23.57 rows=1057 width=57)
SubPlan 1
-> GroupAggregate (cost=4.96..87.25 rows=14 width=12)
Group Key: u2.owner_id
-> Nested Loop (cost=4.96..86.89 rows=44 width=20)
-> Nested Loop (cost=4.68..71.05 rows=16 width=12)
-> Bitmap Heap Scan on users_profile u2 (cost=4.40..18.29 rows=16 width=8)
Recheck Cond: (owner_id = users_profile.id)
-> Bitmap Index Scan on users_profile_owner_id_a9029da4 (cost=0.00..4.40 rows=16 width=0)
Index Cond: (owner_id = users_profile.id)
-> Index Only Scan using users_user_pkey on users_user u1 (cost=0.28..3.30 rows=1 width=4)
Index Cond: (id = u2.user_id)
-> Index Scan using orders_order_user_id_e9b59eb1 on orders_order u0 (cost=0.28..0.89 rows=10 width=20)
Index Cond: (user_id = u1.id)
SubPlan 2
-> GroupAggregate (cost=0.99..10029.94 rows=14 width=12)
Group Key: u2_1.owner_id
-> Nested Loop (cost=0.99..9943.23 rows=17314 width=20)
-> Nested Loop (cost=0.56..110.20 rows=16 width=12)
-> Index Scan using users_profile_user_id_key on users_profile u2_1 (cost=0.28..57.44 rows=16 width=8)
Filter: (owner_id = users_profile.id)
-> Index Only Scan using users_user_pkey on users_user u1_1 (cost=0.28..3.30 rows=1 width=4)
Index Cond: (id = u2_1.user_id)
-> Index Scan using orders_query_user_id_a0b49874 on orders_query u0_1 (cost=0.43..564.34 rows=5022 width=20)
Index Cond: (user_id = u1_1.id)
SubPlan 3
-> GroupAggregate (cost=4.40..18.51 rows=14 width=12)
Group Key: u0_2.owner_id
-> Bitmap Heap Scan on users_profile u0_2 (cost=4.40..18.29 rows=16 width=20)
Recheck Cond: (owner_id = users_profile.id)
-> Bitmap Index Scan on users_profile_owner_id_a9029da4 (cost=0.00..4.40 rows=16 width=0)
Index Cond: (owner_id = users_profile.id)
最佳答案
在我看来，您将每个用户的“orders_order”与每个“orders_query”连接起来，在结果中产生太多行(2亿)，然后需要对分组依据进行排序，这需要很长时间。
尝试以与“indirect_orders”和“indirect_searches”相同的方式为“orders”和“searches”编写子查询，以便可以删除这两个左外连接。
关于django - 如何防止此 PostgreSQL 查询中的全表扫描？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/63510144/