我有一个提要,它用统计信息填充表格中的单个文本字段。 我需要将此数据拉入另一个表中的多个字段 但是奇怪的格式使得自动导入变得困难。
文件格式为纯文本,示例如下:
08:34:52 Checksum=180957248,TicketType=6,InitialUserType=G,InitialUserID=520,CommunicationType=Incoming,Date=26-03-2012,Time=08:35:00,Service=ST,Duration=00:00:14,Cost=0.12
实际上它由以下部分组成:
[timestamp] [Field1 name]=[Field1 value],[Field2 name]=[Field2 value],[Field4 name]=[Field4 value]...[CR]
所有字段的顺序始终相同,但并不总是存在。 列总数可以在 5 到 30 之间。
我已经尝试了下面的函数来翻译它,它似乎大部分工作但似乎随机跳过字段:
解析数据:
(SELECT [Data].[dbo].[GetFromTextString] ( 'Checksum=' ,',' ,RAWTEXT)) AS RowCheckSum,
(SELECT [Data].[dbo].[GetFromTextString] ( 'TicketType=' ,',' ,RAWTEXT)) AS TicketType,
和功能:
CREATE FUNCTION [dbo].[GetFromTextString]
-- Input start and end and return value.
(@uniqueprefix VARCHAR(100),
@commonsuffix VARCHAR(100),
@datastring VARCHAR(MAX) )
RETURNS VARCHAR(MAX) -- Picked Value.
AS
BEGIN
DECLARE @ADJLEN INT = LEN(@uniqueprefix)
SET @datastring = @datastring + @commonsuffix
RETURN (
CASE WHEN (CHARINDEX(@uniqueprefix,@datastring) > 0)
AND (CHARINDEX(@uniqueprefix + @commonsuffix,@datastring) = 0)
THEN SUBSTRING(@datastring, PATINDEX('%' + @uniqueprefix + '%',@datastring)+@ADJLEN, CHARINDEX(@commonsuffix,@datastring,PATINDEX('%' + @uniqueprefix + '%',@datastring))- PATINDEX('%' + @uniqueprefix + '%',@datastring)-@ADJLEN) ELSE NULL END
)
END
谁能建议一种更好/更干净的方法来去除数据,或者有人能弄清楚为什么这个公式会跳过行?
非常感谢任何帮助。
最佳答案
注意 - 第一个解决方案是垃圾。由于历史原因,我保留了它,但下面包含了更好的解决方案
我什至不确定这是否会比您当前的方法更快,但这是我处理问题的方式(如果我被迫采用仅 SQL 的解决方案)。首先需要的是一个表值函数,它将执行拆分函数:
CREATE FUNCTION dbo.Split (@TextToSplit VARCHAR(MAX), @Delimiter VARCHAR(MAX))
RETURNS @Values TABLE (Position INT IDENTITY(1, 1) NOT NULL, TextValues VARCHAR(MAX) NOT NULL)
AS
BEGIN
WHILE CHARINDEX(@Delimiter, @TextToSplit) > 0
BEGIN
INSERT @Values
SELECT LEFT(@TextToSplit, CHARINDEX(@Delimiter, @TextToSplit) - 1)
SET @TextToSplit = SUBSTRING(@TextToSplit, CHARINDEX(@Delimiter, @TextToSplit) + 1, LEN(@TextToSplit))
END
INSERT @Values VALUES (@TextToSplit)
RETURN
END
在我的示例中,我使用临时表 @Worklist 工作,您可能需要相应地调整您的临时表,或者您可以将相关数据插入我使用虚拟数据的 @Worklist 中:
DECLARE @WorkList TABLE (ID INT IDENTITY(1, 1) NOT NULL, TextField VARCHAR(MAX))
INSERT @WorkList
SELECT '08:34:52 Checksum=180957248,TicketType=6,InitialUserType=G,InitialUserID=520,CommunicationType=Incoming,Date=26-03-2012,Time=08:35:00,Service=ST,Duration=00:00:14,Cost=0.12'
UNION
SELECT '08:34:52 Checksum=180957249,TicketType=5,InitialUserType=H,InitialUserID=521,CommunicationType=Outgoing,Date=27-03-2012,Time=14:27:00,Service=ST,Duration=00:15:12,Cost=0.37'
查询的主要部分在这里完成。它很长,所以我试着尽可能地评论它。如果需要进一步说明,我可以添加更多评论。
DECLARE @Output TABLE (ID INT IDENTITY(1, 1) NOT NULL, TextField VARCHAR(MAX))
DECLARE @KeyPairs TABLE (WorkListID INT NOT NULL, KeyField VARCHAR(MAX), ValueField VARCHAR(MAX))
-- STORE TIMESTAMP DATA - THIS ASSUMES THE FIRST SPACE IS THE END OF THE TIMESTAMP
INSERT @KeyPairs
SELECT ID, 'TimeStamp', LEFT(TextField, CHARINDEX(' ', TextField))
FROM @WorkList
-- CLEAR THE TIMESTAMP FROM THE WORKLIST
UPDATE @WorkList
SET TextField = SUBSTRING(TextField, CHARINDEX(' ', TextField) + 1, LEN(TextField))
DECLARE @ID INT = (SELECT MIN(ID) FROM @WorkList)
WHILE @ID IS NOT NULL
BEGIN
-- SPLIT THE STRING FIRST INTO ALL THE PAIRS (e.g. Checksum=180957248)
INSERT @Output
SELECT TextValues
FROM dbo.Split((SELECT TextField FROM @WorkList WHERE ID = @ID), ',')
DECLARE @ID2 INT = (SELECT MIN(ID) FROM @Output)
-- FOR ALL THE PAIRS SPLIT THEM INTO A KEY AND A VALUE (USING THE POSITION OF THE SPLIT FUNCTION)
WHILE @ID2 IS NOT NULL
BEGIN
INSERT @KeyPairs
SELECT @ID,
MAX(CASE WHEN Position = 1 THEN TextValues ELSE '' END),
MAX(CASE WHEN Position = 2 THEN TextValues ELSE '' END)
FROM dbo.Split((SELECT TextField FROM @Output WHERE ID = @ID2), '=')
DELETE @Output
WHERE ID = @ID2
SET @ID2 = (SELECT MIN(ID) FROM @Output)
END
DELETE @WorkList
WHERE ID = @ID
SET @ID = (SELECT MIN(ID) FROM @WorkList)
END
-- WE NOW HAVE A TABLE CONTAINING EAV MODEL STYLE DATA. THIS NEEDS TO BE PIVOTED INTO THE CORRECT FORMAT
-- ENSURE COLUMNS ARE LISTED IN THE ORDER YOU WANT THEM TO APPEAR
SELECT *
FROM @KeyPairs p
PIVOT
( MAX(ValueField)
FOR KeyField IN
( [TimeStamp], [Checksum], [TicketType], [InitialUserType],
[InitialUserID], [CommunicationType], [Date], [Time],
[Service], [Duration], [Cost]
)
) AS PivotTable;
编辑(4 年后)
最近的一次投票引起了我的注意,我有点讨厌自己以目前的形式发布这个答案。
更好的分割函数是:
CREATE FUNCTION dbo.Split
(
@List NVARCHAR(MAX),
@Delimiter NVARCHAR(255)
)
RETURNS TABLE
WITH SCHEMABINDING AS
RETURN
( WITH N1 AS (SELECT N FROM (VALUES (1),(1),(1),(1),(1),(1),(1),(1),(1), (1)) n (N)),
N2(N) AS (SELECT 1 FROM N1 a CROSS JOIN N1 b),
N3(N) AS (SELECT 1 FROM N2 a CROSS JOIN N2 b),
N4(N) AS (SELECT 1 FROM N3 a CROSS JOIN N3 b),
cteTally(N) AS
( SELECT 0 UNION ALL
SELECT TOP (DATALENGTH(ISNULL(@List,1))) ROW_NUMBER() OVER (ORDER BY (SELECT NULL))
FROM n4
),
cteStart(N1) AS
( SELECT t.N+1
FROM cteTally t
WHERE (SUBSTRING(@List,t.N,1) = @Delimiter OR t.N = 0)
)
SELECT Item = SUBSTRING(@List, s.N1, ISNULL(NULLIF(CHARINDEX(@Delimiter,@List,s.N1),0)-s.N1,8000)),
Position = s.N1,
ItemNumber = ROW_NUMBER() OVER(ORDER BY s.N1)
FROM cteStart s
);
然后根本不需要循环,您只需通过两次调用 split 函数来获取您的 EAV 样式数据集,就可以得到一个基于集合的正确解决方案:
DECLARE @WorkList TABLE (ID INT IDENTITY(1, 1) NOT NULL, TextField VARCHAR(MAX))
INSERT @WorkList
SELECT '08:34:52 Checksum=180957248,TicketType=6,InitialUserType=G,InitialUserID=520,CommunicationType=Incoming,Date=26-03-2012,Time=08:35:00,Service=ST,Duration=00:00:14,Cost=0.12'
UNION
SELECT '08:34:52 Checksum=180957249,TicketType=5,InitialUserType=H,InitialUserID=521,CommunicationType=Outgoing,Date=27-03-2012,Time=14:27:00,Service=ST,Duration=00:15:12,Cost=0.37';
WITH KeyPairs AS
( SELECT w.ID,
[Timestamp] = LEFT(w.TextField, CHARINDEX(' ', w.TextField)),
KeyField = MAX(CASE WHEN v.ItemNumber = 1 THEN v.Item END),
ValueField = MAX(CASE WHEN v.ItemNumber = 2 THEN v.Item END)
FROM @WorkList AS w
CROSS APPLY dbo.Split(SUBSTRING(TextField, CHARINDEX(' ', TextField) + 1, LEN(TextField)), ',') AS kp
CROSS APPLY dbo.Split(kp.Item, '=') AS v
GROUP BY w.ID, kp.ItemNumber,w.TextField
)
SELECT *
FROM KeyPairs AS kp
PIVOT
( MAX(ValueField)
FOR KeyField IN
( [Checksum], [TicketType], [InitialUserType],
[InitialUserID], [CommunicationType], [Date], [Time],
[Service], [Duration], [Cost]
)
) AS pvt;
关于sql - 将文本解析为多列,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/9905953/