Stage environment variables are set globally

Using Jenkins
1

Jenkins setup:
Jenkins: 2.426.3.3
OS: Linux - 5.10.217-205.860.amzn2.x86_64
Java: 11.0.22 - Red Hat, Inc. (OpenJDK 64-Bit Server VM)

ace-editor:1.1
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
authentication-tokens:1.53.v1c90fd9191a_b_
aws-credentials:218.v1b_e9466ec5da_
aws-java-sdk:1.11.995
aws-java-sdk-ec2:1.12.606-418.vce5b_4cd017c6
aws-java-sdk-elasticbeanstalk:1.12.606-418.vce5b_4cd017c6
aws-java-sdk-minimal:1.12.606-418.vce5b_4cd017c6
azure-credentials:216.ve0b_4a_485ffc2
azure-sdk:84.v53035e83f3c2
blueocean-commons:1.27.9
blueocean-rest:1.27.9
bluesteel-cjoc:1.2.56
bootstrap5-api:5.3.2-3
bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloudbees-administrative-monitors:1.0.13
cloudbees-analytics:1.59
cloudbees-assurance:2.276.0.31
cloudbees-blueocean-default-theme:0.8
cloudbees-casc-client:2.38
cloudbees-casc-items-api:2.50
cloudbees-casc-items-server:2.50
cloudbees-casc-server:2.36
cloudbees-folder:6.879.934.cb-v98a_156fcdd04
cloudbees-folders-plus:3.31
cloudbees-jenkins-advisor:358.v58972d19b_1f0
cloudbees-license:9.79
cloudbees-monitoring:2.16
cloudbees-nodes-plus:1.25
cloudbees-platform-common:1.26
cloudbees-platform-data:1.39
cloudbees-plugin-usage:2.20
cloudbees-prometheus:1.3
cloudbees-quiet-start:1.9
cloudbees-restricted-credentials:0.4
cloudbees-ssh-slaves:2.25
cloudbees-support:3.31
cloudbees-uc-data-api:4.57
cloudbees-unified-ui:1.30
cloudbees-update-center-plugin:4.85
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-95.v22a_d30ee5d36
configuration-as-code:1737.v652ee9b_a_e0d9
credentials:1311.vcf0a_900b_37c2
credentials-binding:642.v737c34dea_6c2
display-url-api:2.200.vb_9327d658781
docker-commons:439.va_3cb_0a_6a_fb_29
durable-task:543.v262f6a_803410
ec2:1648.vf3d852e00486
echarts-api:5.4.3-2
font-awesome-api:6.5.1-1
git:5.2.1
git-client:4.6.0
github:1.37.3.1
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1758.v048414714f5d
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
infradna-backup:3.38.72
instance-identity:185.v303dc7c645f9
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.16.1-373.ve709c6871598
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jjwt-api:0.11.5-77.v646c772fddb_0
jquery3-api:3.7.1-1
jsch:0.2.16-86.v42e010d9484b_
junit:1252.vfc2e5efa_294f
kube-agent-management:1.1.68
kubernetes:4151.v6fa_f0fb_0b_4c9
kubernetes-client-api:6.8.1-224.vd388fca_4db_3b_
kubernetes-credentials:0.11
ldap:711.vb_d1a_491714dc
mailer:463.vedf8358e006b_
mapdb-api:1.0.9-28.vf251ce40855d
controller-provisioning-core:2.6.65
controller-provisioning-kubernetes:3.0.68
matrix-auth:3.2.1
metrics:4.2.18-442.v02e107157925
mina-sshd-api-common:2.11.0-86.v836f585d47fa_
mina-sshd-api-core:2.11.0-86.v836f585d47fa_
mina-sshd-api-scp:2.11.0-86.v836f585d47fa_
mina-sshd-api-sftp:2.11.0-86.v836f585d47fa_
nectar-license:8.42
nectar-rbac:5.88
node-iterator-api:55.v3b_77d4032326
oauth-credentials:0.646.v02b_66dc03d2e
okhttp-api:4.11.0-157.v6852a_a_fa_ec11
operations-center-agent:3.0.68
operations-center-clusterops:3.0.68
operations-center-context:3.0.68
operations-center-ec2-cloud:3.0.68
operations-center-elasticsearch-provider:3.0.68
operations-center-jnlp-controller:3.0.68
operations-center-kubernetes-cloud:3.0.68
operations-center-license:3.0.68
operations-center-monitoring:3.0.68
operations-center-rbac:3.0.68
operations-center-server:3.0.68
operations-center-sso:3.0.68
operations-center-updatecenter:3.0.68
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.8.0
popper-api:1.16.1-3
popper2-api:2.11.6-4
prism-api:1.29.0-8
pubsub-light:1.18
role-strategy:633.v836e5b_3e80a_5
saml:4.429.v9a_781a_61f1da_
scm-api:683.vb_16722fb_b_80b_
script-security:1305.v487433146192
snakeyaml-api:2.2-111.vc6598e30cc65
sse-gateway:1.26
ssh-credentials:308.ve4497b_ccd8f4
sshd:3.312.v1c601b_c83b_0e
structs:325.vcb_307d2a_2782
support-core:1366.v9d076592655d
token-macro:384.vf35b_f26814ec
trilead-api:2.84.v72119de229b_7
unique-id:2.101.v21a_b_6390a_b_04
user-activity-monitoring:1.18
variant:60.v7290fc0eb_b_cd
view-job-filters:369.ve0513a_a_f5524
workflow-api:1283.v99c10937efcb_
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3837.v305192405b_c0
workflow-durable-task-step:1313.vcb_970b_d2a_fb_3
workflow-job:1385.vb_58b_86ea_fff1
workflow-scm-step:415.v434365564324
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:865.v43e78cc44e0d

Hi, I stumbled into an issue when running some steps in parallel. I have some scripts that are run at the start of jobs to provide some basic information so binaries like Docker images are created with the right values based on the repository and the pipeline. This information is added as environment variables using the syntax env.MY_VARIABLE=value by some internal libraries using Groovy scripts.
According to this documentation, environment variables set inside a stage are scopped to that stage. But when using the syntax described above, that’s not the case as the variables are set globally.
I created a simple declarative script (we use the YAML plugin for this, that’s why is written in YAML) where I use the same syntax to prove that updates to environment variables in one stage are visible to other stages

  stages:
    - stage: 'Setup'
      context: build
      agent:
        label: 'kaniko'
      when:
        - "branch '*'"
      beforeAgent: true
      steps:
        - kaniko:
          script:
            - env.TEST_VARIABLE = "first value"
            - println "Start value is ${env.TEST_VARIABLE}"
    - stage: 'Build'
      context: build
      failFast: true
      beforeAgent: true
      when:
        - "branch '*'"
      parallel:
      - stage: test 1
        agent:
          label: "kaniko"
        steps:
          - kaniko:
            script:
              - println "Start test 1, previous value is ${env.TEST_VARIABLE}"
              - env.TEST_VARIABLE = "parallel1"
              - println "test 1, value updated to ${env.TEST_VARIABLE}"
              - sleep(10)
              - println "End test 1, new value is ${env.TEST_VARIABLE}"

      - stage: test 2
        agent:
          label: "kaniko"
        steps:
          - kaniko:
            script:
              - sleep(5)
              - println "Start test 2, previous value is ${env.TEST_VARIABLE}"
              - env.TEST_VARIABLE = "parallel2"
              - println "test 2, value updated to ${env.TEST_VARIABLE}"
              - println "End test 2, new value is ${env.TEST_VARIABLE}"

The result of this is the following

test1 16:57:48 Start test 1, previous value is first value
test1 16:57:48 test 1, value updated to parallel1
test1 16:57:48 Sleeping for 10 sec
test2 16:57:48 Sleeping for 5 sec
test2 16:57:53 Start test 2, previous value is parallel1
test2 16:57:53 test 2, value updated to parallel2
test2 16:57:53 End test 2, new value is parallel2
test1 16:57:58 End test 1, new value is parallel2

If instead, I create the variables inside the environment tag like the following example, values are not shared between jobs:

    - stage: test 1
      agent:
        label: "kaniko"
      environment:
        TEST_VARIABLE: parallel1
      steps:
        - kaniko:
          script:
            - println "Start test 1, previous value is ${env.TEST_VARIABLE}"
    - stage: test 2
      agent:
        label: "kaniko"
      steps:
        - kaniko:
          script:
            - println "Start test 2, previous value is ${env.TEST_VARIABLE}"

This prints
Start test 1, previous value is parallel1
Start test 2, previous value is null

So is this the expected behaviour? And if that’s the case, is there any way to set the variables to the stage scope using a Groovy library?

Thanks.

2

Hello and welcome to this community, @juan-cvega. :wave:

As far as I know, yes, this is the expected behavior.
When you set environment variables using env.VARIABLE = value in a Groovy script, they are set globally and are visible across all stages. However, when you define environment variables within the environment block of a stage, they are scoped to that specific stage.

To ensure that environment variables are scoped to a specific stage using a Groovy library, you can use the withEnv block. This allows you to set environment variables for the duration of the block temporarily.

Here is an untested example of how you could achieve this in your YAML pipeline:

stages:
  - stage: 'Setup'
    context: build
    agent:
      label: 'kaniko'
    when:
      - "branch '*'"
    beforeAgent: true
    steps:
      - kaniko:
        script:
          - env.TEST_VARIABLE = "first value"
          - println "Start value is ${env.TEST_VARIABLE}"

  - stage: 'Build'
    context: build
    failFast: true
    beforeAgent: true
    when:
      - "branch '*'"
    parallel:
    - stage: test 1
      agent:
        label: "kaniko"
      steps:
        - kaniko:
          script:
            - withEnv(["TEST_VARIABLE=parallel1"]) {
                println "Start test 1, previous value is ${env.TEST_VARIABLE}"
                env.TEST_VARIABLE = "parallel1"
                println "test 1, value updated to ${env.TEST_VARIABLE}"
                sleep(10)
                println "End test 1, new value is ${env.TEST_VARIABLE}"
              }

    - stage: test 2
      agent:
        label: "kaniko"
      steps:
        - kaniko:
          script:
            - withEnv(["TEST_VARIABLE=parallel2"]) {
                sleep(5)
                println "Start test 2, previous value is ${env.TEST_VARIABLE}"
                env.TEST_VARIABLE = "parallel2"
                println "test 2, value updated to ${env.TEST_VARIABLE}"
                println "End test 2, new value is ${env.TEST_VARIABLE}"
              }
3

Hi @poddingue, thanks for your answer.
I was looking for a way to define this within a function in our shared library. We currently have a script within our /vars folder that look something like this:

def call(Map args = [:], def body) {
    ... #some variables here

    withCICDVersioning {
        DockerRegistry dockerRegistry = DockerRegistry.ACR_SNAPSHOT
        env.DOCKER_REGISTRY = dockerRegistry as DockerRegistry
        env.DOCKER_ACR_REGISTRY_URI = dockerRegistry.getDns(domain)
        env.DOCKER_IMAGE_NAME = imageName
        env.DOCKER_IMAGE_FULL_NAME = "${env.DOCKER_ACR_REGISTRY_URI}/${env.DOCKER_IMAGE_NAME}:${env.PACKAGE_VERSION}"
        env.DOCKER_CACHE_REGISTRY = cacheRegistry.getDns()
        env.DOCKER_CACHE_REPOSITORY = "${cacheRegistry.getRegistryName()}/${env.DOCKER_IMAGE_NAME}"
...

So, if understand this correctly, to keep the variables to the current stage, I should be able to do like this?:

    ... #some variables here
    withCICDVersioning {
        DockerRegistry dockerRegistry = DockerRegistry.ACR_SNAPSHOT
        withEnv([
                DOCKER_REGISTRY = dockerRegistry as DockerRegistry,
                DOCKER_ACR_REGISTRY_URI = dockerRegistry.getDns(domain),
                DOCKER_IMAGE_NAME = imageName,
                DOCKER_IMAGE_FULL_NAME = "${DOCKER_ACR_REGISTRY_URI}/${DOCKER_IMAGE_NAME}:${PACKAGE_VERSION}", #Not sure if environment variables can be referenced this way or if I need env. as a prefix
                DOCKER_CACHE_REGISTRY = cacheRegistry.getDns(),
                DOCKER_CACHE_REPOSITORY = "${cacheRegistry.getRegistryName()}/${DOCKER_IMAGE_NAME}"
        ])