analytics - 如何从Azure数据湖分析文件(txt和CSV等文件)中获取记录计数

Question

adl://rs06ipadl01.azuredatalakestore.net/FIA/RDS/old/BANNER/2018/06/15/old_Banner.csv

我需要从上面的文件中获取记录。

Answer 1

您可以使用.Csv和.Text等内置提取器来获取文件内容，然后使用COUNT来统计记录数。一个简单的例子:

DECLARE @inputFile string = @"input/input124.csv";
DECLARE @outputFile string = @"output/output.csv";

// Get the file
@input =
    EXTRACT col1 string,
            col2 string,
            col3 int
    FROM @inputFile
    USING Extractors.Csv( skipFirstNRows:1 );     // skip header row if you have one


// Count the records
@output = SELECT COUNT(*) AS records FROM @input;


// Output the result
OUTPUT @output
TO @outputFile
USING Outputters.Csv(quoting:false);