How to report status of a subset of projects

Using Jenkins Ask a question
1

Hi all,

I have several dozen Jenkins jobs, split in to subcategories. For the sake of argument, let’s consider the “develop” and “release” subcategories. Each subcategory is governed by a “scheduler” job that parcels out jobs to various nodes, and (if all jobs in the subcategory are successful), updates a git branch and sends an email.

Everything we do is with the scripted pipeline (we have some complex non-standard requirements for how and when things are run, and hey, we’re developers - we like to write code). In order to make the above mechanism work, we do something like this:

           myBuildJob = build job: buildJob
           recordResult(myBuildJob)

And at the end of the scheduler pipeline we interrogate the storage variables and construct our email.

This works fine for jobs that succeed. However, we’re currently propagating errors from builds triggered by the scheduler. This seems like a robust way to prevent the git branch from updating if there are any problems. However, it appears that when myBuildJob fails, a groovy exception is raised and we never get to the recordResult() line.

(Side note: You can sort-of make this work with try/finally, but it’s really really ugly and appears to break error propagation anyway).

To me, the obvious next step is probably to disable error propagation and interrogate the result of each job in the scheduler pipeline before moving on to dependent jobs. However, this seems suspiciously like a re-implementation of the error propagating functionality, which makes me think I’ve not understood something properly…

I therefore have the following questions:

  1. Is there a whitelisted way to get the last result of an arbitrary job? My googling suggests not. This would allow us to keep propagating errors and just interrogate all the results at the end. (Side note: we’re a few months away from a security audit and I don’t really understand the implications of allowing more groovy signatures, so I’ll need some convincing to use a non-whitelisted method),

  2. Is my idea to manually check job results after failures a sensible one? I’m concerned that I’ve not understood the idiomatic way to deal with the above problem.

Thanks,
Tom

2

Hi Tom,

I think there’s one way to potentially solve your issue that you’ve missed. The build step (documentation) allows you to turn off propagation, but still retrieve the exit status of the downstream job. You can do this by just leaving the wait parameter enabled, which it is by default. Hence, you can probably do something like this:

myBuildJob = build job: buildJob, propagate: false
recordResult(myBuildJob)
if (myBuildJob.result != 'SUCCESS') {
    error "Job ${buildJob} failed with status ${myBuildJob.result}"
}

This will still allow the upstream build to fail if the downstream fails, and you can still record the result. It does mean that the “chain” of exceptions is broken, but since the Pipeline: Build Step plugin generates its own exception in the event that the job fails or is unstable, I don’t think this is a great loss (any exception generated in the downstream job stays there; it’s not propagated up to the upstream job). The code above should be completely safe by default and not generate any script security issues, though I have not specifically tested it as such.

Do note that, according to a quick reading of the code, even in the case where propagate is off, an exception can still be raised if the pipeline is aborted. This should be a non-issue as the pipeline is being aborted anyway and we don’t care about the status of the downstream job.

3

Hi Michael,

Thanks for your prompt reply - that’s essentially what I’ve ended up doing, I just felt like I’d missed something!

Thanks,
Tom