node.js - 如何通过nodejs jobserver更好地利用CPU

Question

下面给出的是nodejs集群的简单实现，其中创建子进程以充分利用CPU核心。

const cluster = require('cluster'),
      http    = require('http'),
      numCPUs = 4;

if (cluster.isMaster) {
    for (var i = 0; i < numCPUs; i++)
        cluster.fork()
} else {
   function1()
   function2()
   function3()
}

What I have is a Jobserver which processes variable counts of jobs. ie, executes different independent functions once in specific time intervals.

无法使用上述集群模型，因为我不希望每个 fork 的子级都执行所有功能。

How do I distribute these functions among all the forked child, so that each child executes a set of functions and all of my CPU cores are utilised ?

Answer 1

您可以检查worker.id来为每个工作人员运行不同的功能，这记录在here中.

const cluster = require('cluster'),
      http    = require('http'),
      numCPUs = 4;

if (cluster.isMaster) {
    for (var i = 0; i < numCPUs; i++)
        cluster.fork()
} else {
   const { id } = cluster.worker;

   console.log('I am worker #' + id);

   switch (id) {
       case 0: {
         function1();
       }
       case 1: {
         function2();
       }
       case 2: {
         function3();
       }
   }
}

编辑:另一种解决方案

const cluster = require('cluster'),
      http    = require('http'),
      numCPUs = 4;

const coreFunction = {
  0: () => { // core 1 functions
    function1();
    // could add multiple functions per core etc;
  },

  1: () => { // core 2 functions
    function2();
    function22();
  }
};

if (cluster.isMaster) {
    for (var i = 0; i < numCPUs; i++)
        cluster.fork()
} else {
   const { id } = cluster.worker;

   console.log('I am worker #' + id);

   coreFunction[id]();
}

编辑:另一个解决方案#2

const cluster = require('cluster'),
      http    = require('http'),
      os = require('os');

const numCPUs = os.cpus().length;

const functions = [
    function1,
    function2,
    function3,
    function4,
    function5,
];

function chunk(arr, n) {
    return arr.slice(0,(arr.length+n-1)/n|0).
       map(function(c,i) { return arr.slice(n*i,n*i+n); });
}

const functionChunks = chunk(functions, numCPUs);

if (cluster.isMaster) {
    for (var i = 0; i < numCPUs; i++)
        cluster.fork()
} else {
   const { id } = cluster.worker;

   console.log('I am worker #' + id);

   functionChunks[id].forEach(function (func) {
       func();
   })
}