Jenkins 管道和信号量

Question

我正在构建一个 Jenkins 作业，它将连续运行我的所有暂存测试，但不是一次全部运行(它们依赖于共享硬件)。因此，我正在创建并行作业，并使用信号量确保一次仅运行有限数量的作业。
这是重现该问题的管道的简化版本:

import java.util.concurrent.Semaphore

def run(job) {
  return {
    this.limiter.acquire();
    try {
      println "running ${job}"
      build job
      println "finished ${job}"
    } finally {
      this.limiter.release();
    }
  }
}

def getJobs() {
  def allJobs = Jenkins.getInstance().getJobNames()
  def stagingJobs = []
  for(String job : allJobs) {
    if (job.startsWith("staging/temp")) {
      stagingJobs.add(job)
    }
  }
  println "${stagingJobs.size()} jobs were found."
  return stagingJobs
}

this.limiter = new Semaphore(2)
def jobs = [:]
for (job in getJobs()) {
  jobs[job] = run(job)
}
parallel jobs

当我在没有信号量的情况下运行时，一切正常。但是使用上面的代码，除了:

[Pipeline] echo
6 jobs were found.
[Pipeline] parallel
[Pipeline] [staging/temp1] { (Branch: staging/temp1)
[Pipeline] [staging/temp2] { (Branch: staging/temp2)
[Pipeline] [staging/temp3] { (Branch: staging/temp3)
[Pipeline] [staging/temp4] { (Branch: staging/temp4)
[Pipeline] [staging/temp5] { (Branch: staging/temp5)
[Pipeline] [staging/temp6] { (Branch: staging/temp6)

如果我查看管道步骤，我可以看到前两个作业启动，以及它们的日志消息输出。但是，运行者似乎从未收到过暂存作业完成的通知。结果，信号量永远不会释放，其他 4 个作业永远无法启动。这是在下游构建确定完成后的线程转储中期测试:

Thread #7
    at DSL.build(unsure what happened to downstream build)
    at WorkflowScript.run(WorkflowScript:9)
    at DSL.parallel(Native Method)
    at WorkflowScript.run(WorkflowScript:38)
Thread #8
    at DSL.build(unsure what happened to downstream build)
    at WorkflowScript.run(WorkflowScript:9)
Thread #11
    at WorkflowScript.run(WorkflowScript:6)
Thread #12
    at WorkflowScript.run(WorkflowScript:6)

最终它超时了几个 java.lang.InterruptedException错误。

是否可以在管道中使用信号量，或者是否有更好的方法来确保一次只运行一部分作业？我宁愿避免为一个简单的测试运行程序而旋转节点。

Answer 1

Concurrent Step plugin刚刚发布，应该很适合这个用例。

有了这个，你可以简化你的代码:

def semaphore = createSemaphore permit:2

def run(job) {
  return {
    acquireSemaphore (semaphore) {
      println "running ${job}"
      build job
      println "finished ${job}"
    }
  }
}

...