Jenkins Pipeline - Docker downloading Maven dependencies each time even though volume is supplied

Using Jenkins
1

Hello,

In this official documentation, it is mentioned that, if I provide args '-v $HOME/.m2:/root/.m2' then earlier dependencies will be cached and will not be downloaded each time the code is built. Also, I’ve used reuseNode true to make use of same workspace.

Still the old dependencies are being downloaded every time I build my code.
Please let me know what wrong I’m doing here.

Below is my sample Jenkinsfile.

pipeline {
    agent any

    stages {
        stage('Build war') {
            when {
                anyOf {
                    branch 'preprod'
                }
            }

            agent {
                docker {
                    image 'maven:3.9.8-eclipse-temurin-8'

                    // caching data on the agent between Pipeline runs.
                    args '-v $HOME/.m2:/root/.m2'

                    // Run the container on the node specified at the
                    // top-level of the Pipeline, in the same workspace,
                    // rather than on a new node entirely:
                    reuseNode true
                }
            }

            steps {
                sh 'java -version'
                sh 'mvn clean package'
            }
        }

        stage('Build Docker Image') {
            // Use the "parent" agent for the Docker specific steps
            when {
                anyOf {
                    branch 'preprod'
                }
            }
            steps {
                sh 'ls -l'
                sh 'docker ps'
            }
        }
    }
}

Thank you.

2

I do not use ‘docker’ directly from Jenkins, we use the kubernetes plugin so things are not the same and my knowledge is limited on your issue.

The args '-v $HOME/.m2:/root/.m2' line just passes -v $HOME/.m2:/root/.m2 to the docker command line. This is the --volume option of the docker cli with details here: Volumes | Docker Docs

So in your jenkinsfile it will take the $HOME/.m2 directory of your host machine and mount it as /root/.m2 in the running container.

I tested this:

> docker run -it -v $HOME/test:/tmp/test alpine
Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
43c4264eed91: Download complete 
Digest: sha256:beefdbd8a1da6d2915566fde36db9db0b524eb737fc57cd1367effd16dc0d06d
Status: Downloaded newer image for alpine:latest
/ # ls /tmp/test/
/ # touch /tmp/test/created.empty
/ # exit

> ls ~/test/
created.empty

To confirm that the mounted folder does get created on both sides, as long as the permissions are correct.

I also checked the manifest of the maven container:
https://hub.docker.com/layers/library/maven/3.9.8-eclipse-temurin-8/images/sha256-0db6bee82730be63b74ea4234741e69c9fe4106b4072f0fa79858fbd6349e1ba?context=explore

It clearly sets ENV MAVEN_CONFIG=/root/.m2 which is the directory that you actually mount.

What I would do in your jenkinsfile under the first steps is to add a check to make sure that the mounting works:

            steps {
                sh '''
set +e  # to avoid errors if ls, du, cat are failing.
echo $MAVEN_CONFIG  # to ensure that it is what we expect
ls -lah /root/.m2
du -shc /root/.m2
cat /root/.m2/previous
date > /root/.m2/previous  # persist a file that we can re-read.
set -e  # we want to fail if maven fails.
java -version
mvn clean package
set +e  # debug mode again, no need to cause errors for the rest of these commands.
ls -lah /root/.m2
du -shc /root/.m2
'''
            }

This should allow you to test that the mounted volume did the persistence and get some insight into what is in there and if files actually get written into the .m2 folder.

I have not used maven in about 10y, but I would look into the maven docs to make sure that caching is indeed done under that folder. There might be a maven config file that configures the cache to a different location.

Note that each sh call is extra workload on the controller so rather than calling sh multiple times, it is much more efficient for the server to combine all the calls in a single shell script.

1 Like
3

Thank you, @sodul, for the detailed explanation and proof of concept. :+1:

@Padma, could you please let us know if the proposed solution was successful for you?

4

I’ll check this tomorrow and will let you know.

Thank you for the response @sodul .