Nested parallel in declarative pipeline

Using Jenkins Ask a question
1

i wanted to run code like this , basically i wanted to run two stages in parallel and also the commands in those stages should run in parallel in itself.
is there a way to do this in declarative pipeline, please let me know.

pipeline {
  agent any
  stages {
    stage("ABC") {
      parallel {
        stage("first") {
          steps {
            echo "abc1"
          }
        }
        stage("second") {
          stages {
            stage("one") {
              parallel {
                stage("two") {
                  steps {
                    echo "aaa1"
                  }
                }
                stage("three") {
                  steps {
                    echo "bbb"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}
2

Have you tried it?

Assuming it doesn’t work, it should be doable with the scripted syntax, but not even certain about that as I’ve never needed to go this scenario

1 Like
3

i tried, i got below error, so just wanted to check if anyone has any idea how to make it.

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 5: Expected a block for stages @ line 5, column 5.
parallel (
^

WorkflowScript: 5: No stages specified @ line 5, column 5.
parallel (
^

2 errors

at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
4

To me, in theory, it is possible to run two stages in parallel and have the commands in those stages run in parallel using the Declarative Pipeline syntax. Your example is already using the parallel keyword, which is used to run multiple stages in parallel.

To run commands in parallel within a stage, you could use the parallel keyword again within the steps block of the stage. Here is an updated version of your pipeline that should demonstrate this:

pipeline {
  agent any
  stages {
    stage("ABC") {
      parallel {
        stage("first") {
          steps {
            echo "abc1"
          }
        }
        stage("second") {
          steps {
            parallel (
              "two": {
                echo "aaa1"
              },
              "three": {
                echo "bbb"
              }
            )
          }
        }
      }
    }
  }
}

In this updated version, the second stage now has a steps block, within which there is a parallel keyword that runs two commands ("two" and "three" ) in parallel. Note that the parallel keyword is used twice: once at the top level to run the two stages in parallel, and once within the steps block to run the two commands in parallel within the second stage.

Let us know if this works, I haven’t tried it.

5

Thanks for the response @poddingue ,
tried executing the code you pasted , but seems again same issue, can use parallel only once.
might be in declarative pipeline this is not possible, need to mix some script as well.
need to check.

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 15: Parallel stages or branches can only be included in a top-level stage. @ line 15, column 17.
echo “aaa1”

6

Thanks for your feedback.
My fault, I could have tested before proposing…

Here’s an example of how you could try the same parallelism using scripted pipeline syntax:

node {
  stage("ABC") {
    parallel(
      "first": {
        echo "abc1"
      },
      "second": {
        parallel(
          "one": {
            parallel(
              "two": {
                echo "aaa1"
              },
              "three": {
                echo "bbb"
              }
            )
          }
        )
      }
    )
  }
}

Let us know if that works for you.

7

@poddingue, your solution works but only shows one stage “ABC” in the dashboard. Would it be possible to still make the parallel tasks “stages” that show up in the jenkins dashboard?

When I try to replace:
“first”: {
by:
stage(“first”) {

It breaks with:
java.lang.IllegalArgumentException: Expected named arguments but got [null, {one={two=null, three=null}}]

1 Like
8

Hello and welcome to this community, @hllorens. :wave:

To make the parallel tasks show up as stages in the Jenkins dashboard, you need to use the stage directive inside the parallel block correctly. Each parallel branch should be wrapped in a stage block.

node {
  stage("ABC") {
    parallel(
      "first": {
        stage("first") {
          echo "abc1"
        }
      },
      "second": {
        stage("second") {
          parallel(
            "one": {
              stage("one") {
                parallel(
                  "two": {
                    stage("two") {
                      echo "aaa1"
                    }
                  },
                  "three": {
                    stage("three") {
                      echo "bbb"
                    }
                  }
                )
              }
            }
          )
        }
      }
    )
  }
}

gave me

image
image2556×1182 219 KB

and

Started by user admin

[Pipeline] Start of Pipeline
[Pipeline] node
Still waiting to schedule task
‘docker-ssh-jenkins-agent
’ is offline
Running on docker-ssh-jenkins-agent
 in /home/jenkins/agent/workspace/parallel
[Pipeline] {
[Pipeline] stage
[Pipeline] { (ABC)
[Pipeline] parallel
[Pipeline] { (Branch: first)
[Pipeline] { (Branch: second)
[Pipeline] stage
[Pipeline] { (first)
[Pipeline] stage
[Pipeline] { (second)
[Pipeline] echo
abc1
[Pipeline] }
[Pipeline] parallel
[Pipeline] { (Branch: one)
[Pipeline] stage
[Pipeline] { (one)
[Pipeline] // stage
[Pipeline] }
[Pipeline] parallel
[Pipeline] { (Branch: two)
[Pipeline] { (Branch: three)
[Pipeline] stage
[Pipeline] { (two)
[Pipeline] stage
[Pipeline] { (three)
[Pipeline] echo
aaa1
[Pipeline] }
[Pipeline] echo
bbb
[Pipeline] }
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] }
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
1 Like
9

Thanks @poddingue! That works.

I also tested this variant which includes two sequential stages inside a nested parallel stage (see blue ocean) and also works:

node {
    stage("parallel-stages") {
    parallel(
      "first": {
        stage("first") {
            echo "first"
        }
      },
      "second": {
            stage("second") {
            parallel(
              "second1": {
                stage("second1.1"){
                echo "second1.1"
                }
                stage("second1.2"){
                echo "second1.2"
                }
              },
              "second2": {
                stage("second2.1"){
                echo "second2.1"
                }
              }
            )
            }
      }
    )
    }
}

image
image583×312 9.89 KB

Just what I needed.

2 Likes
10

Nice, thanks a lot for the feedback. :+1:

1 Like