[编辑]
感谢@VilleKrumlinde，我修复了我之前在尝试避免代码分析警告时不小心引入的错误。我不小心打开了“重叠”文件处理，它一直在重置文件长度。该问题现已修复，您可以针对同一流多次调用 FastWrite() 而不会出现问题。
[结束编辑]
概览
我正在做一些计时测试来比较将结构数组写入磁盘的两种不同方式。我相信人们普遍认为 I/O 成本与其他事物相比如此之高，因此不值得花太多时间优化其他事物。
但是，我的计时测试似乎表明情况并非如此。要么我犯了错误(这完全有可能)，要么我的优化确实非常重要。
历史
首先是一些历史:这个 FastWrite() 方法最初是在几年前编写的，用于支持将结构写入由遗留 C++ 程序使用的文件，我们仍在为此目的使用它。 (还有一个相应的 FastRead() 方法。)编写它主要是为了更容易将可 blittable 结构数组写入文件，其速度是次要问题。
不止一个人告诉我，像这样的优化实际上并不比使用 BinaryWriter 快多少，所以我终于硬着头皮进行了一些计时测试。结果让我大吃一惊...
看来我的FastWrite() 方法比使用BinaryWriter 的等效方法快 30 - 50 倍。这看起来很荒谬，所以我在这里发布我的代码，看看是否有人能找到错误。
系统规范
- 测试了一个 x86 RELEASE 版本，从调试器外部运行。
- 在 Windows 8、x64、16GB 内存上运行。
- 在普通硬盘驱动器(不是 SSD)上运行。
- 将 .Net 4 与 Visual Studio 2012 结合使用(因此安装了 .Net 4.5)
结果
我的结果是:
SlowWrite() took 00:00:02.0747141
FastWrite() took 00:00:00.0318139
SlowWrite() took 00:00:01.9205158
FastWrite() took 00:00:00.0327242
SlowWrite() took 00:00:01.9289878
FastWrite() took 00:00:00.0321100
SlowWrite() took 00:00:01.9374454
FastWrite() took 00:00:00.0316074
如您所见，这似乎表明 FastWrite() 在那次运行中快了 50 倍。
这是我的测试代码。运行测试后，我对两个文件进行了二进制比较，以验证它们确实相同(即 FastWrite() 和 SlowWrite() 生成相同的文件)。
看看你能从中得到什么。 :)
using System;
using System.ComponentModel;
using System.Diagnostics;
using System.IO;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using Microsoft.Win32.SafeHandles;
namespace ConsoleApplication1
{
internal class Program
{
[StructLayout(LayoutKind.Sequential, Pack = 1)]
struct TestStruct
{
public byte ByteValue;
public short ShortValue;
public int IntValue;
public long LongValue;
public float FloatValue;
public double DoubleValue;
}
static void Main()
{
Directory.CreateDirectory("C:\\TEST");
string filename1 = "C:\\TEST\\TEST1.BIN";
string filename2 = "C:\\TEST\\TEST2.BIN";
int count = 1000;
var array = new TestStruct[10000];
for (int i = 0; i < array.Length; ++i)
array[i].IntValue = i;
var sw = new Stopwatch();
for (int trial = 0; trial < 4; ++trial)
{
sw.Restart();
using (var output = new FileStream(filename1, FileMode.Create))
using (var writer = new BinaryWriter(output, Encoding.Default, true))
{
for (int i = 0; i < count; ++i)
{
output.Position = 0;
SlowWrite(writer, array, 0, array.Length);
}
}
Console.WriteLine("SlowWrite() took " + sw.Elapsed);
sw.Restart();
using (var output = new FileStream(filename2, FileMode.Create))
{
for (int i = 0; i < count; ++i)
{
output.Position = 0;
FastWrite(output, array, 0, array.Length);
}
}
Console.WriteLine("FastWrite() took " + sw.Elapsed);
}
}
static void SlowWrite(BinaryWriter writer, TestStruct[] array, int offset, int count)
{
for (int i = offset; i < offset + count; ++i)
{
var item = array[i]; // I also tried just writing from array[i] directly with similar results.
writer.Write(item.ByteValue);
writer.Write(item.ShortValue);
writer.Write(item.IntValue);
writer.Write(item.LongValue);
writer.Write(item.FloatValue);
writer.Write(item.DoubleValue);
}
}
static void FastWrite<T>(FileStream fs, T[] array, int offset, int count) where T: struct
{
int sizeOfT = Marshal.SizeOf(typeof(T));
GCHandle gcHandle = GCHandle.Alloc(array, GCHandleType.Pinned);
try
{
uint bytesWritten;
uint bytesToWrite = (uint)(count * sizeOfT);
if
(
!WriteFile
(
fs.SafeFileHandle,
new IntPtr(gcHandle.AddrOfPinnedObject().ToInt64() + (offset*sizeOfT)),
bytesToWrite,
out bytesWritten,
IntPtr.Zero
)
)
{
throw new IOException("Unable to write file.", new Win32Exception(Marshal.GetLastWin32Error()));
}
Debug.Assert(bytesWritten == bytesToWrite);
}
finally
{
gcHandle.Free();
}
}
[DllImport("kernel32.dll", SetLastError=true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool WriteFile
(
SafeFileHandle hFile,
IntPtr lpBuffer,
uint nNumberOfBytesToWrite,
out uint lpNumberOfBytesWritten,
IntPtr lpOverlapped
);
}
}
跟进
我还测试了@ErenErsönmez 提出的代码，如下(我在测试结束时验证了所有三个文件是相同的):
static void ErenWrite<T>(FileStream fs, T[] array, int offset, int count) where T : struct
{
// Note: This doesn't use 'offset' or 'count', but it could easily be changed to do so,
// and it doesn't change the results of this particular test program.
int size = Marshal.SizeOf(typeof(TestStruct)) * array.Length;
var bytes = new byte[size];
GCHandle gcHandle = GCHandle.Alloc(array, GCHandleType.Pinned);
try
{
var ptr = new IntPtr(gcHandle.AddrOfPinnedObject().ToInt64());
Marshal.Copy(ptr, bytes, 0, size);
fs.Write(bytes, 0, size);
}
finally
{
gcHandle.Free();
}
}
我为该代码添加了一个测试，同时删除了 output.Position = 0; 行，以便文件现在增长到 263K(这是一个合理的大小)。
经过这些更改，结果是:
注意 当您不将文件指针重置回零时，看看FastWrite() 时间有多慢! :
SlowWrite() took 00:00:01.9929327
FastWrite() took 00:00:00.1152534
ErenWrite() took 00:00:00.2185131
SlowWrite() took 00:00:01.8877979
FastWrite() took 00:00:00.2087977
ErenWrite() took 00:00:00.2191266
SlowWrite() took 00:00:01.9279477
FastWrite() took 00:00:00.2096208
ErenWrite() took 00:00:00.2102270
SlowWrite() took 00:00:01.7823760
FastWrite() took 00:00:00.1137891
ErenWrite() took 00:00:00.3028128
因此看起来您可以使用编码(marshal)处理实现几乎相同的速度，根本不必使用 Windows API。唯一的缺点是 Eren 的方法必须复制整个结构数组，如果内存有限，这可能是个问题。
最佳答案
我认为区别与 BinaryWriter 无关。我认为这是因为您在 SlowWrite (10000 * 6) 中执行多个文件 IO 而在 FastWrite 中执行单个 IO。您的 FastWrite 具有准备好写入文件的单个字节 block 的优势。另一方面，您在 SlowWrite 中将结构一个一个地转换为字节数组。
为了验证这个理论，我写了一个小方法来预先构建一个包含所有结构的大字节数组，然后在 SlowWrite 中使用这个字节数组:
static byte[] bytes;
static void Prep(TestStruct[] array)
{
int size = Marshal.SizeOf(typeof(TestStruct)) * array.Length;
bytes = new byte[size];
GCHandle gcHandle = GCHandle.Alloc(array, GCHandleType.Pinned);
var ptr = gcHandle.AddrOfPinnedObject();
Marshal.Copy(ptr, bytes, 0, size);
gcHandle.Free();
}
static void SlowWrite(BinaryWriter writer)
{
writer.Write(bytes);
}
结果:
SlowWrite() took 00:00:00.0360392
FastWrite() took 00:00:00.0385015
SlowWrite() took 00:00:00.0358703
FastWrite() took 00:00:00.0381371
SlowWrite() took 00:00:00.0373875
FastWrite() took 00:00:00.0367692
SlowWrite() took 00:00:00.0348295
FastWrite() took 00:00:00.0373931
请注意 SlowWrite 现在的性能与 FastWrite 相当，我认为这表明性能差异不是由于实际的 IO 性能，而是更多地与二进制文件有关转化过程。
关于c# - Windows API 似乎比 BinaryWriter 快得多 - 我的测试正确吗？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/16298818/