node.js - 将 .limit() 与 Firestore 查询结合使用

Question

我有一个使用 Firestore 充当队列的应用程序。数据来自另一个源，并存储在 Firestore 中的 /data/{id} 中，属性 unprocessed 设置为 true。在我的 Node.js 脚本中，我想一次查询一些未处理的记录，以缓慢地处理它们。有数十万条记录，因此尝试将它们全部加载到内存中只会导致进程崩溃。

代码:

firestore.collection('data').where('unprocessed', '==', true).limit(25).onSnapshot((snapshot) => {
    snapshot.forEach((doc) => {
      processItem(doc);
    });
  });

processItem() 函数执行必要的处理，将数据保存回 Firestore，并将 unprocessed 属性设置为 false。

我遇到的问题是，每当我尝试运行此代码时，都会收到以下错误:

Error: Error 3: Order must include __name__
    at sendError (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:254:15)
    at DestroyableTransform.stream.on.proto (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:532:13)
    at emitOne (events.js:115:13)
    at DestroyableTransform.emit (events.js:210:7)
    at addChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:284:12)
    at readableAddChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:271:11)
    at DestroyableTransform.Readable.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:238:10)
    at DestroyableTransform.Transform.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:146:32)
    at afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:102:51)
    at TransformState.afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:79:12)
    at DestroyableTransform.noop [as _transform] (C:\[myApp]\node_modules\through2\through2.js:26:3)
    at DestroyableTransform.Transform._read (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:182:10)
    at DestroyableTransform.Transform._write (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:170:83)
    at doWrite (C:\[myApp]\node_modules\readable-stream\lib\_stream_writable.js:406:64)
    at writeOrBuffer (C:\[myApp]\node_modules\readable-stream\lib\_stream_writable.js:395:5)
    at DestroyableTransform.Writable.write (C:\[myApp]\node_modules\readable-stream\lib\_stream_writable.js:322:11)

从我的代码中删除 .limit() 可以使其运行，但随后我遇到了另一个问题，即处理和存储项目的函数未正确完成，并且我的内存使用情况只是不断增长，直到进程崩溃。

我的第一直觉是认为 .limit() 由于某种原因与 .onSnapshot() 不兼容，但也许有人可以给我一个更好的想法这里发生的事情。

编辑

我还尝试添加.orderBy('__name__')，如https://firebase.google.com/docs/firestore/manage-data/delete-data所示(在“删除集合”部分)，但这只会导致另一个错误:

Error: Trying to compare documents on fields that don't exist. Please include the fields you are ordering on in your select() call.
    at C:\[myApp]\node_modules\@google-cloud\firestore\src\reference.js:1679:19
    at Array.sort (native)
    at computeSnapshot (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:438:20)
    at push (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:469:18)
    at DestroyableTransform.stream.on.proto (C:\[myApp]\node_modules\@google-cloud\firestore\src\watch.js:514:15)
    at emitOne (events.js:115:13)
    at DestroyableTransform.emit (events.js:210:7)
    at addChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:284:12)
    at readableAddChunk (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:271:11)
    at DestroyableTransform.Readable.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_readable.js:238:10)
    at DestroyableTransform.Transform.push (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:146:32)
    at afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:102:51)
    at TransformState.afterTransform (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:79:12)
    at DestroyableTransform.noop [as _transform] (C:\[myApp]\node_modules\through2\through2.js:26:3)
    at DestroyableTransform.Transform._read (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:182:10)
    at DestroyableTransform.Transform._write (C:\[myApp]\node_modules\readable-stream\lib\_stream_transform.js:170:83)

Answer 1

如果您添加 .orderBy(FirebaseFirestore.FieldPath.documentId())，它应该可以解决问题。关于为什么可以解决此问题的简要说明是 here 。这是gist一个例子。