java - 为什么 stash/unstash 在此 Jenkinsfile 中不起作用？

Question

我有一个现场运行的 Jenkins 服务器，它使用 Jenkinsfile 来管理一个管道，该管道使用并行测试执行器插件在多个代理上运行我的所有 JUnit 测试以加快测试速度。我们有自己制造的 Blade 服务器(比购买便宜多了!)，它使我们的测试时间从近 2 小时缩短到 22 分钟。 JUnit 插件非常适 merge 行测试。

但是 Jacoco 插件没有。所以我试图将覆盖率文件 merge 到一个文件中，以便 Jacoco 插件可以发布覆盖率结果。 Stash/unstash 正在存储源代码，但当我尝试存储不同的 Jacoco 输出文件以将它们取消存储在主服务器上时，它不起作用。

有什么想法吗？

这是我的 Jenkins 文件:

#!/usr/bin/env groovy

def branch
def hash

node('remote') {
  sh 'echo starting'

  branch = env.gitlabBranch ?: '**'
  echo "Branch: $branch"

  checkout([$class: 'GitSCM',
        branches: [[name: "$branch"]],
        extensions: [
          [$class: 'PruneStaleBranch'],
          [$class: 'CheckoutOption', timeout: 120],
          [$class: 'CloneOption', depth: 0, noTags: true, shallow: true, timeout: 180]
        ],
        doGenerateSubmoduleConfigurations: false,
        submoduleCfg: [],
        userRemoteConfigs: [[credentialsId: 'gitlabLabptop', url: 'git@gitlab.com:protocase/my_project_url.git']]
       ]
      )

  hash = sh (script: 'git rev-parse HEAD', returnStdout: true).trim()

  ### - this stash works fine -###
  stash name: 'sources', includes: '**', excludes: '**/.git,**/.git/**'
}

def numBranches = 9
def splits = splitTests count(numBranches)
def branches = [:]

for (int i = 0; i < splits.size(); i++) {
  def index = i // fresh variable per iteration; i will be mutated

  branches["split${i}"] = {
    timeout(time: 125, unit: 'MINUTES') {
      node('remote') {
    sh 'echo starting a node'
    deleteDir()

    ### - this unstash works fine - ###
    unstash 'sources'

    def exclusions = splits.get(index);
    writeFile file: 'test/exclusions.txt', text: exclusions.join("\n")

    sh 'ant clean'

    sh 'rm -rf build'

    sh 'ant jar'

    sh 'ant -buildfile build-test.xml buildTests'

    sh 'ant -buildfile build-test.xml jenkinsBatch'

    junit 'build/test/results/*.xml'

    sh "mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco${index}.exec"
    echo "name: coverage$index, unclude jacoco${index}"

       ### - this stash appears to work - ### 
       stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"
       echo "stashed"

      }
    }
  }
}

parallel branches


def branchIndecis = 0..numBranches

node('master') {
  if (currentBuild.result != "ABORTED") {

    echo "collecting exec files"

    branchIndecis.each {
      echo "unstash coverage${it}"

      ### !!! this unstash causes an error !!! ###
      unstash name: "coverage${it}"



      echo "make file name"
      def coverageFileName = "build/test/jacoco/jacoco${it}.exec"
      echo "merge file"
      sh "ant -buildfile build-test.xml -Dfile=${coverageFileName} coverageMerge"
    }

    echo "collected exec files"

    step([$class: 'JacocoPublisher',
      execPattern:'build/test/jacoco/jacoco.exec',
      classPattern: 'build/classes',
      sourcePattern: 'src'])

    echo "finishing ${branch} - ${hash}"

  }
}

我得到的输出是:

[split7] [jdesigner] Running shell script
[split7] + mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco7.exec
[Pipeline] [split7] echo
[split7] name: coverage7, unclude jacoco7
[Pipeline] [split7] stash
[split7] Stashed 1 file(s)
[Pipeline] [split7] echo
[split7] stashed
[Pipeline] [split7] }
[Pipeline] [split7] // node
[Pipeline] [split7] }
[Pipeline] [split7] // timeout
[Pipeline] [split7] }
[Pipeline] // parallel
[Pipeline] node
Running on eightyeight in /var/jenkins/workspace/jdesigner
[Pipeline] {
[Pipeline] echo
collecting exec files
[Pipeline] echo
unstash coverage0
[Pipeline] unstash
[Pipeline] }
[Pipeline] End of Pipeline
Finished: FAILURE

[编辑] coverage0 的存储是

[split0] Recording test results
[Pipeline] [split0] sh
[split0] [jdesigner] Running shell script
[split0] + mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco0.exec
[Pipeline] [split0] echo
[split0] name: coverage0, include jacoco0
[Pipeline] [split0] stash
[split0] Stashed 1 file(s)
[Pipeline] [split0] echo
[split0] stashed
[Pipeline] [split0] }
[Pipeline] [split0] // node
[Pipeline] [split0] }
[Pipeline] [split0] // timeout
[Pipeline] [split0] }
[split3]     [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.737 sec
[split3]     [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.737 sec

注意线

[split0] name: coverage0, include jacoco0

只是我的 echo 语句，我在其中回显了脚本这一部分的名称:

    sh "mv build/test/jacoco/jacoco.exec build/test/jacoco/jacoco${index}.exec"
    echo "name: coverage$index, include jacoco${index}"

    stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"
    echo "stashed"

注意实际的存储不是在节点上完成的，它被列为管道，即使它是在远程节点上完成的。我看到的东西表明存储是在主服务器上完成的，但实际上不是该目录所在的位置。

[[进一步编辑]] - 感谢 eis 的建议。

master 上的 jobs/jdesigner/builds/1639/stashes/目录包含 coverage#.tar.gz 文件，其中包含适当的 jacoco#.exec 文件。当我 try catch unstash 时:

try {
    unstash name: "coverage${it}"
} catch (error) {
    echo "error unstashing: ${error}"
}

我得到的输出是:

collecting exec files
[Pipeline] echo
unstash coverage0
[Pipeline] unstash
[Pipeline] echo
error unstashing: java.io.NotSerializableException: groovy.lang.IntRange
[Pipeline] echo
make file name

Answer 1

TLDR:这是迭代样式导致问题的一个案例，因为 it 使用的键不是 Serializable。

难以调试的原因是错误消息没有正确报告，可能是由于 this issue .在代码中捕获异常并“手动”报告解决了这个问题。

实际问题已通过使用 Serializable 键修复。

---

更长的版本:

因为在你的例子中这是可行的:

node('remote') {
    ### - this stash works fine -###
    stash name: 'sources', includes: '**', excludes: '**/.git,**/.git/**'
}
node('remote') {    
    ### - this unstash works fine - ###
    unstash 'sources'
}

但这不是:

node('remote') {

   ### - this stash appears to work - ### 
   stash name: "coverage$index", includes: "build/test/jacoco/jacoco${index}.exec"
   echo "stashed"

}
node('master') {
   echo "unstash coverage${it}"

   ### !!! this unstash causes an error !!! ###
   unstash name: "coverage${it}"
}

我最初认为工作的是在您的远程节点上 stash 和取消 stash 的，而不工作的是 stash 在您的远程节点上但您尝试在您的主节点上取消 stash (自然不会找到它) .

然而，事实并非如此。根据this ,

When you stash a file on a slave, the files are send to the master. The files will be stored in the Job folder, in the associated build folder under the stash folder. Each stash will be stored as a tar file. These files are deleted at the end of the build.

所以主远程分离应该不会有什么不同。此外，如果是关于未找到 stash ，您可以 see from the sources它会因 "No such saved stash '"+ name + "' 而失败，因为根据 AbortException javadoc “当捕获到此异常时，将报告指定的消息。”。这很清楚没有发生。

相反，应该使用 try-catch block 进行调试，以找出破坏构建的真正异常是什么。

至于默认没有正确报告的原因，有this issue :“构建日志中未正确报告流程结束时的序列化错误，仅 Jenkins 日志”。错误报告声称其“已修复”，但显然只是因为在新版本中，某些 对此行为的测试没有触发问题，因此它可能仍然存在。

捕获到错误消息后，可以看出问题是 this - 我们在传递它时试图序列化一个不可序列化的 key 。