Migrating Jenkins from a Server to a Container

Using Jenkins Ask a question
7

This is more of a docker concepts question; I have been glossing over a lot of deep details with short descriptions so I’ll expand a bit. This is going to be a long one because I will try not to assume much with this reply.

What you point out with volumes jenkins_home:/var/lib/jenkins/ is an internal volume because that’s what your docker-compose.yml file says.

When you have volumes listed it is the equivalent of docker run -v jenkins_home:/var/lib/jenkins/ and in your case jenkins_home is the Volume ID.

I know it is a volume ID because of this section of your docker-compose file at the root keys

volumes:
  jenkins_home:

This is the equivalent of docker volume create jenkins_home. So your compose file is saying:

  • Create a volume and
  • Create a container mounting the volume I just created to path /var/lib/jenkins

You have existing files on an EBS volume. Meaning you have files already available outside of Docker. So you don’t want to use an internal Docker volume; instead you want to mount a folder-based volume. For example, if you want to mount your EBS volume to /mnt/jenkins_home folder path then you would create your container with docker run -v /mnt/jenkins_home:/var/lib/jenkins

In your compose file, it would be

services:
  jenkins:
    ...
    volumes:
      - /mnt/jenkins_home:/var/lib/jenkins

You would also remove the top level volumes key because you no longer need an internal docker volume.

Permissions

Another problem you’ll encounter is permissions. The permissions of your jenkins_home EBS volume must match the user and group running inside of the Jenkins container even if that same user does not exist on the host.

For example, I run my Jenkins images with alpine instead of official Docker because I like how I do docker better (my opinion of coarse). Because of that; I must match the jenkins user inside of the container (uid 100 and gid 101) with the file structure. In my specific case, I would chown with numeric IDs with the find command. For example,

find /mnt/jenkins_home \( \! -uid 100 -o \! -gid 101 \) -exec chown 100:101 {} +

Matching permissions in your case

In your case, you’re using the official Jenkins docker image so you must first discover the UID and GID of Jenkins. You can do this by running the official Docker image by overriding the ENTRYPOINT and the CMD. Here’s the docker command.

$ docker run --entrypoint '' --rm jenkins/jenkins:2.319.1-jdk11 id
uid=1000(jenkins) gid=1000(jenkins) groups=1000(jenkins)

The official Jenkins docker image uses UID 1000 and GID 1000. So you must make sure that your Jenkins files permissioned for UID/GID 1000 on your EBS volume before starting Jenkins.

Learn about the official Docker image

You first need to inspect the Jenkins official image to learn about how it is laid out. You can use dive utility but in my case I’ll give you examples that only use available Docker commands requiring no extra software.

docker inspect jenkins/jenkins:2.319.1-jdk11

Relevant sections include

volumes section

"Volumes": {
    "/var/jenkins_home": {}
},

env section

"Env": [
    "PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "LANG=C.UTF-8",
    "JENKINS_HOME=/var/jenkins_home",
    "JENKINS_SLAVE_AGENT_PORT=50000",
    "REF=/usr/share/jenkins/ref",
    "JENKINS_VERSION=2.319.1",
    "JENKINS_UC=https://updates.jenkins.io",
    "JENKINS_UC_EXPERIMENTAL=https://updates.jenkins.io/experimental",
    "JENKINS_INCREMENTALS_REPO_MIRROR=https://repo.jenkins-ci.org/incrementals",
    "COPY_REFERENCE_FILE_LOG=/var/jenkins_home/copy_reference_file.log",
    "JAVA_HOME=/opt/java/openjdk"
],

cmd section

We know it is unused because of

"Cmd": null,

entrypoint section

Which shows you the shell script used to launch Jenkins.

"Entrypoint": [
    "/sbin/tini",
    "--",
    "/usr/local/bin/jenkins.sh"
],

How to inspect the shell script

You must override the entrypoint and pass a shell to CMD when launching an interactive terminal so that you can inspect the contents of the Docker image. This lets you use standard GNU utils to browse it.

docker run -it --rm --entrypoint '' jenkins/jenkins:2.319.1-jdk11 /bin/bash

Not all Docker images have /bin/bash or even utilities available. However, the official Jenkins docker image does.

Inspect the shell script and a support script that gets sourced:

less /usr/local/bin/jenkins.sh
less /usr/local/bin/jenkins-support

So in a nutshell how this works is if you have files in /usr/share/jenkins/ref then it will copy them to your JENKINS_HOME on startup.

What have we learned through inspection?

  • JENKINS_HOME in official docker image is /var/jenkins_home. You must mount your files to this location.
  • When baking plugins you can put your own set of plugins into /usr/share/jenkins/ref/plugins/*.jpi

Finishing up

Create your plugins.tar

cd /to/your/jenkins_home
tar -cf /tmp/plugins.tar plugins/*.jpi

Create your Dockerfile

FROM jenkins/jenkins:2.319.1-jdk11
ADD plugins.tar /usr/share/jenkins/ref/

Move your plugins.tar into current dir

Move your tar file into the current directory:

mv /tmp/plugins.tar ./

Create your new docker-compose.yml file.

services:
  jenkins:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8080:8080"
    volumes:
      - /mnt/your/ebs/volume:/var/jenkins_home

Starting your service

First verify your files are in place.

$ ls -1
Dockerfile
docker-compose.yml
plugins.tar

Start your service

docker-compose up -d

What will happen is this will build a new Docker image packaging your plugins into a new Docker image before starting your Jenkins service. It will mount your EBS volume (assuming proper EBS filesystem permissions UID 1000 GID 1000) to the JENKINS_HOME inside of the container.

Summary

This exhaustive explanation is the same suggestion I gave in my first post but in more detail with fewer assumptions of docker familiarity. Rather than following guides it is best to inspect Docker images directly to check your own assumptions. You can’t assume just because a guide mentions the official Jenkins docker image that it will work. Jenkins changes fast and sometimes that means the infrastructure.

Why /var/jenkins_home and not /var/lib/jenkins. Jenkins is multi-platform and multi-OS. /var/lib/jenkins is where the RedHat RPM installs Jenkins HOME. However, this is not the same location as Jenkins on Debian or Ubuntu. And as we’ve discovered it is also not the same as Jenkins running in the official Docker image. So really you need to keep in mind Jenkins concepts (like the JENKINS_HOME) and don’t assume you know where things are organized. I walked through how to do the discovery process for the official Jenkins docker image.

I’ve never used the official Jenkins docker image (only my own). So a lot of my explanation is just reading the source as I’ve explained it; I don’t operate it.

1 Like