I’m trying to use containers on remote machines as build agents.
I have a recent version (2.346) of Jenkins controller running thanks to Docker, and I want to launch a container agent on another (Linux) machine thanks to Docker.
I have followed this tutorial, and my build agent test connection succeeds. The plugin that is used is called Docker. I’m almost at the end of the tutorial; I have my “cloud” visible
The problem I have is linked to the “Restrict where this project can be run” choice.
In the tutorial, there is a Label textfield when describing a new “docker cloud”, and I don’t have it with the installed version of the docker plugin.
So when it’s time to choose the right “cloud” for my project build, well… I’m not able to choose the right one:
I then decided to follow the documentation from Darin which is pretty recent, and which uses another plugin.
Tt looks like Darin’s tutorial aims at getting Docker container running on the same machine as the controller, but that’s not my goal. Anyhow, the docker pipeline plugin he mentions can’t be found. It looks like the name is Docker Pipeline, but the ID is docker-workflow, and the last release is pretty recent.
Hello @poddingue , you are in the correct direction and you did right!
First Case: Docker plugin providing a "Docker Cloud"
A “Cloud” object in Jenkins maps a remote API that provide a computing service (used to provide on-demand agents).
Once the connection is set up to a remote cloud, you have to define a set of “templates” which are the definitions of the “agent kinds” in this cloud.
Given that labels helps to categorize the type of agent expected to run a pipeline, then it means that you will find the “label” setup field in these template.
Second Case: Docker Pipeline
This plugin has a different goal. It allows to execute some of your pipeline steps inside a Docker container.
But this plugin requires an agent where there is a Docker Engine already running, as the Docker commands are executed by the agent.jar process.
The “workflow” is usually the following:
Jenkins controller retrieve the pipeline and parse it (as usual)
Then it selects one of its agents (by default with the label docker)
It allocates an executor on this selected agent (which is already launched, usually on a remote VM)
It delegates to the agent the tasks of creating a workspace (e.g. on the remote VM agent filesystem) and clones the code in this workspace
Then, it delegates to the agent the task of running a serie of docker build / docker run commands
=> I personnaly recommend against this plugin because:
It is confusing (as you can see, it does NOT allows to spawn agent on demand)
It ties the pipeline execution to Jenkins and make it not portable, which goes against the basics of Continuous Integration (how would I repeate the same process on a developer machine). Clearly more sustainable to use a set of sh 'docker build <...>' and sh 'docker run <...>' steps instead in your pipeline.
Despite not requiring a Docker image with Java/the agent.jar (since it already need an agent), it still makes it hard to reuse the entrypoint of your images as it overrides it. A nightmare to use images providing an embeded CLI.
It uses a set of docker run flags (--user, --volumes) that make it hard to be portable and deterministic, which goes against the near idea of Docker (bind mounting a volume is mandatory to share the workspace but permissions management is terrible to diagnose)
So… I did everything from scratch again with the #1 method, and this time, it worked.
I haven’t been able to use Docker through SSH with this plugin ( java.lang.IllegalArgumentException: Unsupported protocol scheme: ssh://user@machine at com.github.dockerjava.netty.NettyDockerCmdExecFactory.init(NettyDockerCmdExecFactory.java:112)), but using the remote docker from my machine with ssh does work.
Anyway, I am now out of the rut, thanks a lot for the help.
,
