Is it possible to dynamically compose declarative pipeline "blocks"?

Using Jenkins
1

I’m trying to figure out something about declarative pipelines. I know that shared libraries can have entire pipelines as a last statement in call function; however, the pipeline would still have a predefined structure. What I’m wondering is whether it’s possible to compose and return a declarative pipeline from shared library. Eventually I’d like to achieve this:

library 'shared'

myPipeline {
   stages {
       ...
   }
}

myPipeline would then augment the stages block passed to it with other stages and properties, and return a pipeline. Just having scripted statements in stages doesn’t work obviously, I also tried to return parsed AST from shared library but that also doesn’t work. Maybe it can be accomplished by writing a plugin? Ideally this plugin would be just some glue code so that the logic can still be implemented in the shared library.

Would appreciate any thoughts on this!

2

Also tried creating and returning org.jenkinsci.plugins.pipeline.modeldefinition.model.Root from Jenkinsfile – didn’t work

3

One of the main purposes of declarative is to have the entire pipeline parseable at start for flow control.

What you are essentially describing is scripted pieplines which is considered the advanced mode of declarative.

I would recommend looking into scripted, then you can have a compositable pipeline really easily

4

Thank you for the reply! Yes, I already implemented something similar with scripted pipeline, basically:

myPipeline.groovy

def call(Closure body) {
  pipeline {
    stages {
      stage {
        steps {
          script {
            body()
          }
        }
      }
    }
  }
}

Jenkinsfile

myPipeline {
  echo 'test'
}

However, this way some advantages of declarative pipelines are lost, namely:

  1. stages can’t be determined before the pipeline is executed
  2. extended static validation can’t be performed
  3. it’s difficult or impossible to pass blocks like options or parameters
  4. some declarative blocks like matrix or when are not available
  5. developers have to get used to custom API that I would define in call function, and can’t use regular Jenkins declarative constructs

One of the main purposes of declarative is to have the entire pipeline parseable at start for flow control.

That’s what I would like to keep, while also being able to compose the structure of the pipeline.

5

thats declarative btw, if it has pipeline {} its declarative.

That would be parallel and if statements in scripted

I’m not saying your wrong, but declarative doesn’t really support this, you can’t really do super advanced custom things in declarative. The best I could find is like making a fake custom post like step - https://github.com/halkeye/jenkins-shared-library/blob/f2b30f61541b21f345b4677e29bba647e1e63da7/vars/buildDockerfile.groovy#L115-L124

You might want to look into the idea of having jenkinsfile being generated by an external templating like script that updates your SCM, so by the time jenkins sees it, the Jenkinsfile is deterministic, plus it would be in CI if you ever needed to go back to a previous build.

6

Yeah you’re right – I meant it’s a script block within a declarative pipeline.

Yeah I saw that implemented once actually – with a microservice that would render Jenkinsfiles from templates stored in git.

I was just hoping there’s a way to achieve this by leveraging existing APIs and maybe writing some glue code plugin.

Anyway thanks for your answers again, I will probably stick with my example posted above. If you have any suggestions on the general direction to look at (like what code or plugins I could read), would appreciate if you let me know