Hook a post always step from a step in a Jenkins library

Using Jenkins Ask a question
,
1

We are on the latest Jenkins LTS with fresh plugins. Our pipelines are all declarative and as a way to simplify our jenkins files we have a custom step to wrap around calling sh(). We can call it shStep() for this topic.

What our custom shStep() does is call additional pipeline bootstrapping logic, including running top in batch mode. This allows us to have a top.log file which will tell us exactly when a subprocess started, how much memory and CPU was used, etc… This is a very valuable log for us to introspect and tune the CPU and RAM resources to allocate to our kubernetes containers.

Since we collect top.log, and other resources our declarative pipelines always follow the steps {} block with a block to archive artifacts:

post {
    always {
        container('ci') {
            sh stopPipelineReports()
        }
        archiveArtifacts(getPipelineReports())
    }
}

What I’m looking to do is from the shStep() call declare the post { always {} } block so that we no longer need to explicitly include the lines in our declarative pipelines.

This would allow us to declare things much more simply.

Today we have:

stage('CI') {
    agent {
        kubernetes { ... }
    }
    steps {
         shStep('my-script.sh')
    }
    post {
        always {
            container('ci') {
                sh stopPipelineReports()
            }
            archiveArtifacts(getPipelineReports())
        }
    }
}

I know that I can merge the content of the post { always { } } as a single call to a jenkins lib variable, but what I’m looking for is to potentially completely skip declaring that block by declaring/calling post { always { ... } } from inside shStep().

2

I don’t think you can directly declare a post block inside a step function. :thinking:

3

Right, I’m not absolutely looking doing a literal post { always { ... } } but if there is a way to insert a callback to run at the end of the steps {} block that would do the trick.

There are potentially many use cases for doing this.

I know I could do something similar with a withXyz style closure where we call the teardown logic after calling body() but that implies additional nesting which I’m trying to avoid for simplicity.

4

After thinking about this, it might not be as lean as I originally envisioned, but a with*() { block might be the right approach here:

steps{
    withMonitoredSteps() {
        shStep(...)
        ...
    }
}

In withMonitoredSteps.groovy we then have something like:

def call(Closure body) {
    startMonitoring()
    try {
        body()
    } finally {
        stopStepsMonitoring()
        archiveArtifacts(...)
    }
}

We logic in shStep() that calls startMonitoring() but it won’t start the setup logic more than once, and then we can share the same logic to be in an explicit post { always {} } block if we want to, but the benefit of the overall shorter jenkinsfiles.

The result would be first:

stage('CI') {
    agent {
        kubernetes { ... }
    }
    steps {
         shStep('my-script.sh')
    }
    post {
        always {
            stopPipelineReports()
            archiveArtifacts(getPipelineReports())
            }
        }
    }
}

or the equivalent:

stage('CI') {
    agent {
        kubernetes { ... }
    }
    steps {
        withMonitoredSteps() {
            shStep('my-script.sh')
        }
    }
}
5

Just wondering… Will you encounter any errors since the call method is missing the Closure body parameter?

6

The lack of Closure body was an oversight, thanks for the reminder.

7

Have you tried creating a sh global var named sh.groovy with a call method which handles declarative call args?

In scripted pipeline, vars are prioritized over steps when names clash and “pipeline” for declarative pipeline is treated as a var under the hood in Java code.

Because of this, I imagine you’ll be able to override sh with a compatible but custom var of your own for declarative pipelines (must be a global shared lib var).

In scripted, you can try an initial test with

def call(def x) {
    echo 'shell step was overridden'
}
8

That’s an interesting idea, but I assume it would make it hard to call the built-in sh: how will I actually execute the shell script if the real sh is masked?

It does not really matter to me anyways since I’m happy to have a custom name for the wrapper, my main concern was on how to have a custom script always called at the end of all the steps in a stage without having to remember to declare a post { always { ... } } block in the declarative pipelines.

My primary concern is to make our jenkins files behave more consistently across our company, all while making it simpler for our devs to write them: less fluff is always a plus.