Retrieving the environment variables from the Docker container and using it in the Jenkins pipeline

Using Jenkins Ask a question
, ,
1
Jenkins setup:

Jenkins 2.435
Ant Plugin (ant): 497.v94e7d9fffa_b_9
OWASP Markup Formatter Plugin (antisamy-markup-formatter): 162.v0e6ec0fcfcf6
Apache HttpComponents Client 4.x API Plugin (apache-httpcomponents-client-4-api): 4.5.14-208.v438351942757
Apache HttpComponents Client 5.x API Plugin (apache-httpcomponents-client-5-api): 5.3-1.0
Authentication Tokens API Plugin (authentication-tokens): 1.53.v1c90fd9191a_b_
Bitbucket Pipeline for Blue Ocean (blueocean-bitbucket-pipeline): 1.27.9
Common API for Blue Ocean (blueocean-commons): 1.27.9
Config API for Blue Ocean (blueocean-config): 1.27.9
Blue Ocean Core JS (blueocean-core-js): 1.27.9
Dashboard for Blue Ocean (blueocean-dashboard): 1.27.9
Display URL for Blue Ocean (blueocean-display-url): 2.4.2
Events API for Blue Ocean (blueocean-events): 1.27.9
Git Pipeline for Blue Ocean (blueocean-git-pipeline): 1.27.9
GitHub Pipeline for Blue Ocean (blueocean-github-pipeline): 1.27.9
i18n for Blue Ocean (blueocean-i18n): 1.27.9
JWT for Blue Ocean (blueocean-jwt): 1.27.9
Personalization for Blue Ocean (blueocean-personalization): 1.27.9
Pipeline implementation for Blue Ocean (blueocean-pipeline-api-impl): 1.27.9
Blue Ocean Pipeline Editor (blueocean-pipeline-editor): 1.27.9
Pipeline SCM API for Blue Ocean (blueocean-pipeline-scm-api): 1.27.9
REST Implementation for Blue Ocean (blueocean-rest-impl): 1.27.9
REST API for Blue Ocean (blueocean-rest): 1.27.9
Web for Blue Ocean (blueocean-web): 1.27.9
Blue Ocean (blueocean): 1.27.9
Bootstrap 5 API Plugin (bootstrap5-api): 5.3.2-3
bouncycastle API Plugin (bouncycastle-api): 2.30.1.77-225.v26ea_c9455fd9
Branch API Plugin (branch-api): 2.1144.v1425d1c3d5a_7
Build Timeout (build-timeout): 1.32
Caffeine API Plugin (caffeine-api): 3.1.8-133.v17b_1ff2e0599
Checks API plugin (checks-api): 2.0.2
Cloud Statistics Plugin (cloud-stats): 336.v788e4055508b_
Bitbucket Branch Source Plugin (cloudbees-bitbucket-branch-source): 856.v04c46c86f911
Folders Plugin (cloudbees-folder): 6.858.v898218f3609d
Command Agent Launcher Plugin (command-launcher): 107.v773860566e2e
commons-lang3 v3.x Jenkins API Plugin (commons-lang3-api): 3.13.0-62.v7d18e55f51e2
commons-text API Plugin (commons-text-api): 1.11.0-95.v22a_d30ee5d36
Credentials Binding Plugin (credentials-binding): 642.v737c34dea_6c2
Credentials Plugin (credentials): 1311.vcf0a_900b_37c2
Declarative Pipeline Migration Assistant API (declarative-pipeline-migration-assistant-api): 1.6.2
Declarative Pipeline Migration Assistant (declarative-pipeline-migration-assistant): 1.6.2
Display URL API (display-url-api): 2.200.vb_9327d658781
Docker Commons Plugin (docker-commons): 439.va_3cb_0a_6a_fb_29
Docker Compose Build Step Plugin (docker-compose-build-step): 1.0
Docker API Plugin (docker-java-api): 3.3.4-86.v39b_a_5ede342c
Docker plugin (docker-plugin): 1.5
Docker Pipeline (docker-workflow): 572.v950f58993843
Durable Task Plugin (durable-task): 543.v262f6a_803410
ECharts API Plugin (echarts-api): 5.4.3-2
Email Extension Plugin (email-ext): 2.103
Favorite (favorite): 2.208.v91d65b_7792a_c
Font Awesome API Plugin (font-awesome-api): 6.5.1-1
Git client plugin (git-client): 4.6.0
Git plugin (git): 5.2.1
GitHub API Plugin (github-api): 1.318-461.v7a_c09c9fa_d63
GitHub Branch Source Plugin (github-branch-source): 1767.va_7d01ea_c7256
GitHub plugin (github): 1.37.3.1
Gradle Plugin (gradle): 2.9
Groovy (groovy): 457.v99900cb_85593
Gson API Plugin (gson-api): 2.10.1-15.v0d99f670e0a_7
Handy Uri Templates 2.x API Plugin (handy-uri-templates-2-api): 2.1.8-30.v7e777411b_148
HTML Publisher plugin (htmlpublisher): 1.32
Instance Identity (instance-identity): 185.v303dc7c645f9
Ionicons API (ionicons-api): 56.v1b_1c8c49374e
Jackson 2 API Plugin (jackson2-api): 2.16.1-373.ve709c6871598
Jakarta Activation API (jakarta-activation-api): 2.0.1-3
Jakarta Mail API (jakarta-mail-api): 2.0.1-3
JavaBeans Activation Framework (JAF) API (javax-activation-api): 1.2.0-6
JavaMail API (javax-mail-api): 1.6.2-9
JAXB plugin (jaxb): 2.3.9-1
Oracle Java SE Development Kit Installer Plugin (jdk-tool): 73.vddf737284550
Design Language (jenkins-design-language): 1.27.9
Java JSON Web Token (JJWT) Plugin (jjwt-api): 0.11.5-77.v646c772fddb_0
Joda Time API Plugin (joda-time-api): 2.12.6-21.vca_fd74418fb_7
JQuery3 API Plugin (jquery3-api): 3.7.1-1
JSON Path API Plugin (json-path-api): 2.8.0-21.v8b_7dc8b_1037b_
JUnit Plugin (junit): 1252.vfc2e5efa_294f
LDAP Plugin (ldap): 711.vb_d1a_491714dc
Mailer Plugin (mailer): 463.vedf8358e006b_
Matrix Authorization Strategy Plugin (matrix-auth): 3.2.1
Matrix Project Plugin (matrix-project): 822.v01b_8c85d16d2
Mina SSHD API :: Common (mina-sshd-api-common): 2.11.0-86.v836f585d47fa_
Mina SSHD API :: Core (mina-sshd-api-core): 2.11.0-86.v836f585d47fa_
OkHttp Plugin (okhttp-api): 4.11.0-157.v6852a_a_fa_ec11
PAM Authentication plugin (pam-auth): 1.10
Pipeline: Build Step (pipeline-build-step): 540.vb_e8849e1a_b_d8
Pipeline: GitHub Groovy Libraries (pipeline-github-lib): 42.v0739460cda_c4
Pipeline Graph Analysis Plugin (pipeline-graph-analysis): 202.va_d268e64deb_3
Pipeline: Groovy Libraries (pipeline-groovy-lib): 689.veec561a_dee13
Pipeline: Input Step (pipeline-input-step): 477.v339683a_8d55e
Pipeline: Milestone Step (pipeline-milestone-step): 111.v449306f708b_7
Pipeline: Model API (pipeline-model-api): 2.2168.vf921b_4e72c73
Pipeline: Declarative (pipeline-model-definition): 2.2168.vf921b_4e72c73
Pipeline: Declarative Extension Points API (pipeline-model-extensions): 2.2168.vf921b_4e72c73
Pipeline: REST API Plugin (pipeline-rest-api): 2.34
Pipeline: Stage Step (pipeline-stage-step): 305.ve96d0205c1c6
Pipeline: Stage Tags Metadata (pipeline-stage-tags-metadata): 2.2168.vf921b_4e72c73
Pipeline: Stage View Plugin (pipeline-stage-view): 2.34
Pipeline Utility Steps (pipeline-utility-steps): 2.16.0
Plain Credentials Plugin (plain-credentials): 143.v1b_df8b_d3b_e48
Plugin Utilities API Plugin (plugin-util-api): 3.8.0
Pub-Sub “light” Bus (pubsub-light): 1.18
Resource Disposer Plugin (resource-disposer): 0.23
SCM API Plugin (scm-api): 683.vb_16722fb_b_80b_
Script Security Plugin (script-security): 1313.v7a_6067dc7087
SnakeYAML API Plugin (snakeyaml-api): 2.2-111.vc6598e30cc65
Server Sent Events (SSE) Gateway Plugin (sse-gateway): 1.26
SSH Credentials Plugin (ssh-credentials): 308.ve4497b_ccd8f4
SSH Build Agents plugin (ssh-slaves): 2.947.v64ee6b_f87b_c1
SSH server (sshd): 3.312.v1c601b_c83b_0e
Structs Plugin (structs): 325.vcb_307d2a_2782
Timestamper (timestamper): 1.26
Token Macro Plugin (token-macro): 400.v35420b_922dcb_
Trilead API Plugin (trilead-api): 2.133.vfb_8a_7b_9c5dd1
Variant Plugin (variant): 60.v7290fc0eb_b_cd
Pipeline (workflow-aggregator): 596.v8c21c963d92d
Pipeline: API (workflow-api): 1283.v99c10937efcb_
Pipeline: Basic Steps (workflow-basic-steps): 1042.ve7b_140c4a_e0c
Pipeline: Groovy (workflow-cps): 3837.v305192405b_c0
Pipeline: Nodes and Processes (workflow-durable-task-step): 1313.vcb_970b_d2a_fb_3
Pipeline: Job (workflow-job): 1385.vb_58b_86ea_fff1
Pipeline: Multibranch (workflow-multibranch): 770.v1a_d0708dd1f6
Pipeline: SCM Step (workflow-scm-step): 415.v434365564324
Pipeline: Step API (workflow-step-api): 639.v6eca_cd8c04a_a_
Pipeline: Supporting APIs (workflow-support): 865.v43e78cc44e0d
Workspace Cleanup Plugin (ws-cleanup): 0.45

Hello everyone!

I have a question regarding accessing sensitive data that are stored as environment variables in a Docker container to use those in a Jenkins pipeline.

Generally, I have built a custom image in Docker based on the jenkins/jenkins official image. Initially, when I was running the docker image to create a docker container, I passed environment variables there and I’m able to view them in the container via Docker CLI.

I’ve attempted to do so far:

pipeline {
    agent any
    tools {
        'org.jenkinsci.plugins.docker.commons.tools.DockerTool' 'docker'
    }

stage('Run Docker Container') {
                steps {
                    script {
                        def myContainer = docker.image('my_custom_image:0.1')
                        def var1Value = myContainer.inside {
                            sh(script: 'echo $Key', returnStdout: true).trim()
                        }
                        echo "The value of Key is: ${var1Value}"
                    }
                }
    }

Although I am aware that this script does not provide much value in a pipeline. I assume that it should be something with docker(my_custom_image).run, but I couldn’t find the correct script.

However, I found a way of retrieving the value of environment variables by using global credentials, particularly secret text which has been working for me, I still looking for a solution to implement the initial idea there though.

I would appreciate any advice or tips that help me to figure this out.

2

Hello @vitaliizghonnik,

I’m not entirely sure if I understood everything correctly, but let’s give it a try. You want to retrieve the environment variable from the Docker container and use it in your Jenkins pipeline, correct?

One approach could be to run a command inside the Docker container that prints the value of the environment variable, and then capture the output of that command. However, I’m not entirely comfortable with this approach.

Here’s a possible modification you could make to your pipeline script:

pipeline {
    agent any
    tools {
        'org.jenkinsci.plugins.docker.commons.tools.DockerTool' 'docker'
    }

    stages {
        stage('Run Docker Container') {
            steps {
                script {
                    def myContainer = docker.image('my_custom_image:0.1')
                    myContainer.run()
                    def var1Value = sh(script: 'docker exec <container_id> /bin/sh -c "echo $Key"', returnStdout: true).trim()
                    echo "The value of Key is: ${var1Value}"
                }
            }
        }
    }
}
3

Hello @poddingue,
Thank you for your prompt response and willingness to assist me in making it work.

Yes, you understood me correctly. Sorry, I should mention it initially, but I didn’t install Docker inside my custom Docker image but instead was supposed to use the Docker Plugin available in Jenkins for that reason.

Also, the way how I passed environment variables to the Docker container was the following command: docker run -e Key=$Key

I’m right now writing that message as I tried to implement the proposed solution, and it gave me the following response:

+ docker run -d my_custom_image:0.1

/var/jenkins_home/workspace/Jenkins_Pipeline@2@tmp/durable-dcf0766d/script.sh.copy: 1: docker: not found

script returned exit code 127

I tried to modify the script a bit to solve the problem, but none were successful.
Could you please share any idea what the reason for that error message maybe?

4

I suppose that’s because my solution was using the infamous Docker in Docker… And you don’t have Docker in your Docker container (which is a good thing, don’t get me wrong).

5

Not at all, I get it. I will rebuild the Docker image containing Docker and try the solution.

6

Hello @poddingue,
After a long gap since our last conversation, I have implemented Docker inside Docker and adjusted the custom image using the following instructions:

RUN curl -fsSLo /usr/share/keyrings/docker-archive-keyring.asc \
    https://download.docker.com/linux/debian/gpg
RUN echo "deb [arch=$(dpkg --print-architecture) \
    signed-by=/usr/share/keyrings/docker-archive-keyring.asc] \
    https://download.docker.com/linux/debian \
    $(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker.list
RUN apt-get update && apt-get install -y docker-ce-cli

However, while I ran the build in the stage of running a docker container I got the following notification:

+ docker run -d my_custom_image:0.1
Failed to initialize: unable to resolve docker endpoint: open /certs/client/ca.pem: no such file or directory
script returned exit code 1

I’ve tried to find the location of certs but I haven’t.

Could you please assist me with solving this issue above?
I would greatly appreciate any suggestions.

7

Hi @vitaliizghonnik,

Failed to initialize: unable to resolve docker endpoint: open /certs/client/ca.pem: no such file or directory typically indicates that Docker is trying to find the certificate files for a Docker daemon running with TLS, but it can’t find them. :thinking:

When you run Docker inside Docker, the inner Docker client tries to connect to the outer Docker daemon. By default, it tries to do this using a secure connection, which requires certificate files. If these files are not found, you get the error message you’re seeing.

Even if security-wise, that’s not a good idea, one way to solve this issue would to disable TLS for the Docker daemon.
This should be feasible by setting the DOCKER_TLS_CERTDIR environment variable to an empty string in your Dockerfile.

ENV DOCKER_TLS_CERTDIR=""

Please note that disabling TLS for the Docker daemon can have security implications, as it allows unencrypted and unauthenticated communication with the Docker daemon. If your Docker daemon is exposed to a network, this could potentially allow unauthorized access. Therefore, this solution should only be used in a trusted environment.

1 Like
8

Thank you, @poddingue , for your response and help in resolving the issue.

The main focus of the current discussion is the work on my pet project which I’ve built from scratch, and I have been pushing to a public repository. I needed to grasp all the technologies I had recently started using for the first time.

It was the last thing I wanted to modify and finish with it. Obviously, I’ll not publish the last solution due to probable security consequences along the way for someone who may watch the repository.
But it’s fine because I’ve learned much through the process.

Thanks one more time for your help.

1 Like
9

Thank you so much for your feedback and kind words. :hugs:

Best of luck with your project.