javascript - 如何将异步方法拆分成更小的部分并依次调用它们？

Question

我正在玩 puppeteer 来控制浏览器并进行一些自动化测试。我的测试场景需要在 100 页上执行大量异步操作。我可以一次打开 100 个页面并执行所有操作，但似乎对我的 cpu 来说太多了。所以我想将其拆分并放入队列中，例如一次 20 页。

关于代码: 我有一个模块，其中包含一个对象，该对象由键和数组组成，其中填充了对异步方法的调用。然后我遍历这个对象并在浏览器中为每个键创建一个单独的页面。我调用异步方法并等待解析。

export default {

  async checkAllKeys() {
    const requirementsMap = {
      key_1: [this.method1, this.method2, this.method3],
      key_2: [this.method1, this.method2, this.method3],
      ...
      key_100: [this.method1, this.method2, this.method3],
    };

    const result = {};
    const promises = [];

    Object.keys(requirementsMap).forEach(async (key) => {
      promises.push(new Promise(async (resolve, reject) => {

        const newPage = await browser.newPage();

        await newPage.goto('http://www.example.com/');

        const requirements = requirementsMap[key];
        const promises1 = requirements.map(requirement => requirement.call(this, newPage).catch(() => 'error occurred'));

        result[key] = await Promise.all(promises1);
        await newPage.close();
        resolve();
      }));
    });

    await Promise.all(promises);
    return result;
  },

  async method1(newPage) {
    // do some async actions
  },
  async method2() {
    // and etc....
  },
};

我怎样才能把它分成更小的部分，然后一个接一个地发射它们？

Answer 1

您可以通过构造 X 个请求的批处理来做到这一点，按照以下行:

const batchSize = 20;
const requirementKeysArr = Object.keys(requirementsMap);

while (requirementKeysArr.length > 0){
   //this line does the trick: extracts from the requirementKeysArr the first batchSize elements and puts them in the batch variable
   const batch = requirementKeysArr.splice(0, batchSize);
   await Promise.all(batch);
}

因此，您的代码将类似于:

export default {

  async checkAllKeys() {
    const requirementsMap = {
      key_1: [this.method1, this.method2, this.method3],
      key_2: [this.method1, this.method2, this.method3],
      ...
      key_100: [this.method1, this.method2, this.method3],
    };

    const result = {};
    const promises = [];
    const batchSize = 20;
    const requirementKeysArr = Object.keys(requirementsMap);

    while (requirementKeysArr.length > 0){

       const batch = requirementKeysArr.splice(0, batchSize).map( async (key) => {
        const newPage = await browser.newPage();

        await newPage.goto('http://www.example.com/');

        const requirements = requirementsMap[key];
        const promises1 = requirements.map(requirement => requirement.call(this, newPage).catch(() => 'error occurred'));

        result[key] = await Promise.all(promises1);
        await newPage.close();
        return result[key];

});        

    await Promise.all(batch);
  } // end while
  return result;
  },

  async method1(newPage) {
    // do some async actions
  },
  async method2() {
    // and etc....
  },
};