c# - 在 csv 文件中查找重复项目

Question

我有一个 CSV 文件

FirstName LastName and ID column, Id is an Unique Column

Chris, Webber, 1 
Chris, Ben, 2
Chris, Dudley, 3
David, Floy, 4
Chris, Ben, 5 
Chris, Webber, 6

我需要在不使用数据库的情况下获得两个列表，我需要从 C# 中的文件中读取它并创建两个列表重复列表和原始列表。

重复列表包含所有重复项

Chris, Webber, 1
Chris, Webber, 6
Chris, Ben, 2
Chris, Ben, 5

原始列表具有唯一的条目和第一次出现的重复条目。

Chris, Webber, 1
Chris, Ben, 2
Chris, Dudley, 3
David, Floy, 4

What is the best way solve this?

Answer 1

var lines = File.ReadLines("yourFile.ext");

// this assumes you can hold the whole file in memory

// uniqueness is defined by the first two columns
var grouped = lines.GroupBy(line => string.Join(", ", line.Split(',').Take(2)))
                   .ToArray();

// "unique entry and first occurrence of duplicate entry" -> first entry in group
var unique = grouped.Select(g => g.First());

var dupes = grouped.Where(g => g.Count() > 1)
                   .SelectMany(g => g);

Console.WriteLine("unique");
foreach (var name in unique)
    Console.WriteLine(name);

Console.WriteLine("\nDupes");
foreach (var name in dupes)
    Console.WriteLine(name);

输出:

unique
Chris, Webber, 1
Chris, Ben, 2
Chris, Dudley, 3
David, Floy, 4

Dupes
Chris, Webber, 1
Chris, Webber, 6
Chris, Ben, 2
Chris, Ben, 5