javascript - node.js - 如何调整 child_process.spawn() 的标准输出缓冲区大小(又名 highWaterMark)？

Question

我正在尝试通过 child_process.spawn() 调用调整 stdout 的缓冲区大小 (highWaterMark)，但我很难成功完成此操作。下面是一些代码，说明了我正在寻找的内容。我正在尝试使用各种标准输出缓冲区大小执行一些基准测试，但在我可以实际调整缓冲区大小之前，我将无法这样做。任何建议将不胜感激，谢谢!

// I'm using the `stdout` from a child_process.spawn() call to feed into another stream, and I'm having a hard time increasing the buffer size on stdout.
// The code below illustrates the issue:
const child_process = require('child_process');
const stream = require('stream');
const fs = require('fs');

const compressStream = child_process.spawn("cat", ["testfile_32MB"], {writableHighWaterMark: 1024*1024, highWaterMark: 1024*1024});  // <-- child_process.spawn does not have highWaterMark or writableHighWaterMark options, so including them has no effect, but I'm including it just as an illustration of what I want.
compressStream.stdout.writableLength = 1024*1024; // <-- This does not help.
compressStream.stdout._writableState.writableLength = 1024*1024; // <-- This does not help.
compressStream.stdout._writableState.writableHighWaterMark = 1024*1024; // <-- This does not help.
compressStream.stdout.writableHighWaterMark = 1024*1024; // <-- This does not help.
compressStream.stdout._readableState.highWaterMark = 1024*1024; // <-- This does not help.
compressStream.stdout._writableState.highWaterMark = 1024*1024; // <-- This does not help.
compressStream.stdout.on("data", (data) => {
  console.log("data length [from spawn()]:", data.length); // <-- data.length will always be <= 64KB! How can we get it higher?
});

const readStream = fs.createReadStream("testfile_32MB", {highWaterMark: 1024*1024});
readStream.on("data", (data) => {
  console.log("data length [from createReadStream()]:", data.length); // <-- This prints values <= 1MB, which is what I wish I had for child_process.spawn/stdout...
});

请注意，您需要一个名为“testfile_32MB”的文件来运行此示例代码，并且它的大小至少应为几兆字节。

Answer 1

唉，你不能这样做。 highWaterMark 是一种机制，用于通知写入器停止，因为读取器未读取且缓冲区已满。

Socket 将在数据可用时立即向您发送数据。它将始终使用内部硬编码的 64k 缓冲区。增加 highWaterMark 将使其能够容纳更多数据，但它仍会向您发送 64Kb block - 只不过每次事件循环轮换时您会突发多次调用。

使用 spawn 时，您无权访问 Stream 创建选项 - 您最终可以传递预先构造的命名管道流 - 但这并不能解决您的问题.