sql - 优化我的 Azure SQL PaaS 表和/或查询以提高性能

Question

我正在设计一张具有非常专门的使用模式的 table 。 该表将以有限的流量连续记录 - 每秒约 25 条记录，然后每天晚上我都会运行一个大查询来提取大量数据。

我的表创建脚本当前如下所示:

SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

IF NOT EXISTS (select * from sysobjects where name='records' and xtype='U')
CREATE TABLE [dbo].[records](
    [TripID] varchar(255) NOT NULL,
    [RecordTimeUTC] datetime2(0) NOT NULL,
    [TimeOfDaySeconds] [int] NOT NULL,
    [T0Latitude] [float] NOT NULL,
    [T0Longitude] [float] NOT NULL,
    [T1Latitude] [float] NULL,
    [T1Longitude] [float] NULL,
    [T2Latitude] [float] NULL,
    [T2Longitude] [float] NULL,
    [T3Latitude] [float] NULL,
    [T3Longitude] [float] NULL,
    [T4Latitude] [float] NULL,
    [T4Longitude] [float] NULL,
    [T5Latitude] [float] NULL,
    [T5Longitude] [float] NULL,
    [VehicleID] [int] NULL,
    [ID] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY
) ON [PRIMARY]
GO

IF NOT EXISTS (select * from sys.indexes where name='TripIDRecordTimeIndex' and object_id = OBJECT_ID('dbo.records'))
    CREATE INDEX TripIDRecordTimeIndex ON records (TripID, RecordTimeUTC desc)
GO

IF NOT EXISTS (select * from sys.indexes where name='TripIDIndex' and object_id = OBJECT_ID('dbo.records'))
    CREATE INDEX TripIDIndex ON records (TripID)
GO

IF NOT EXISTS (select * from sys.indexes where name='RecordTimeUTCIndex' and object_id = OBJECT_ID('dbo.records'))
    CREATE INDEX RecordTimeUTCIndex ON records (RecordTimeUTC desc)
GO

IF NOT EXISTS (select  * from sys.objects where name like 'UniqueConstraint2' and parent_object_id = OBJECT_ID('dbo.records'))
    ALTER TABLE [dbo].[records] ADD CONSTRAINT UniqueConstraint2 UNIQUE(VehicleID, RecordTimeUTC desc);
GO

IF NOT EXISTS (select * from sys.indexes where name='VehicleIDIndex' and object_id = OBJECT_ID('dbo.records'))
    CREATE INDEX VehicleIDIndex ON records (VehicleID)
GO

目前我的表中有大约 6000 万条记录，大小不到 50 GB。

提取数据的查询非常耗时。目前需要一个多小时。我不确定是我的表设计还是查询设计才是根本原因(尽管可能两者都是)。

我需要为我指定的一组 TripID 中的每个 TripID 提取最新 X 数量的项目。大约有 10k 个不同的 ID，我通常想要查询其中大约一半。它们之间的 X 也有所不同，因此我目前最好的查询方法是生成一个看起来有点像这样的脚本:

SELECT rs.* FROM (SELECT *, ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) AS Rank FROM records where TripID in (20141000,20441000,30011022,30011021,30011008,30012029,30012028,30012027,30011007,30011019,30011018,30012026,30012025,30012024,30011017,30011016,30012023,30012022,30011015,30011014,30012021,30012020,30011013,30011012,30013000,30013001,30013019,30013009,30011011,30011010,30011009,30013008,30013007,30012010,30012009,30013005,30013004,30013003,30012014,30012019,30013021,30013020,30011006,30011004,30012018,30012017,30012016,30013006,30011003,30011002,30012015,30012013,30013013,30013002,30011001,30011000,30011020,30012012,30012011,30011005,30011030,30012001,30012008,30012007,30011029,30011028,30012006,30012005,30011031,30011027,30012004,30012003,30011026,30011025,30011024,30012002,30012000,30012031,30011023,30012030,30015005,30016006,30016013,30016012,30014020,30014019,30014018,30016011,30016010,30014017,30014016,30016009,30016008,30014015,30014013,30014012,30016005,30016004,30016003,30014010,30014009,30016002,30016001,30014008,30014007,30016000,30016007,30014006,30014005,30014004,30014003,30014002,30014001,30014000,30014023,30014014,30015012,30015004,30015003,30013018,30013017,30015002,30015001,30013016,30013015,30013014,30015000,30015013,30015011,30013012,30013011,30015010,30015009,30013010,30014011,30015008,30015007,30014022,30014021,30015006,33651001,33661006)) rs WHERE Rank <= 690
 UNION 
SELECT rs.* FROM (SELECT *, ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) AS Rank FROM records where TripID in (20431003,20431002,20431001,20432003,20432002,20432001,30221001,33861002,33861003)) rs WHERE Rank <= 855
 UNION 
SELECT rs.* FROM (SELECT *, ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) AS Rank FROM records where TripID in (20171029,20171030,20002002,26122001)) rs WHERE Rank <= 45
 UNION 
...

(上面的查询返回第一个列表中每个行程的 690 个实例，第二个列表中每个行程的 855 个实例，第三个列表中的 45 个实例，依此类推。查询比这个大得多 - 这只是它的一个片段。我总共提取了 10-1500 万行)

正如前面提到的，我的表现很糟糕。是云的东西吗？这是设计的事情吗？我应该使用聚集索引吗？ (尝试过 TripID，但情况更糟)。我可以以某种方式改进我的查询吗？例如为每个ID提取相同数量的实例，然后过滤？

我注意到我有一些额外的索引可能不会在我的查询中使用。我只是尝试添加更多，因为插入性能不是问题。计划是在我的查询中使用 TripIDRecordTimeIndex。

即使将 Azure SQL 中的数据计划扩展到 S7 (800 DPU)，我也无法快速运行。如有任何反馈，我们将不胜感激。

编辑:我最近将 TripID 从 int 更改为 varchar(255) - 这会破坏我的性能吗？

编辑2:执行计划:

Download link to full execution plan

Edit3:发现在我查询的 TripID 周围添加引号 ('') 可以极大地提高性能!

Edit4:我按照 TheGameiswar 的建议添加了索引 - 区别是白天和黑夜!谢谢你!附上新的执行计划。

New Execution plan

Answer 1

SELECT rs.* FROM (SELECT *, 
ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) 
AS Rank FROM records where TripID in (20141000,20441000,30011022,30011021,30011008,30012029,30012028,30012027,30011007,30011019,30011018, 30012026,30012025.....)) rs WHERE Rank <= 690

您拥有的索引对于以下部分的查询没有用...

SELECT *, 
ROW_NUMBER() over (Partition BY TripID ORDER BY RecordTimeUTC DESC ) 
AS Rank FROM records where TripID in

我会创建一个如下所示的索引

create index nci_sometst on table (tripid,recorddatetime)
include(<remaining columsn you are selecting>)

上面的查询可以帮助获取 TripID 的 IN 部分的记录，但是当您使用派生表计算排名时，如果您的内部查询结果集很大，这可能没有多大帮助。

我可能会将其放入临时表中并在排名上创建索引，因此这对于其他联合查询也很有帮助。

还查看了您的执行计划，我可以看到您多次扫描同一个表并每次读取许多行

即使没有数据类型转换警告，您的查询也不会使用您有效拥有的任何索引