Ephemeral Docker Agents Disconnect After a Few Minutes (ChannelClosedException)

Using Jenkins Ask a question
1

Environment:

  • Jenkins controller on Linux

  • Remote Docker host (TLS via tcp://...:2376)

  • Docker plugin (also tried using latest)

  • Connection method: Connect with Docker Container

  • Agent image: based on jenkins/inbound-agent:2.2.6

  • Java: OpenJDK 21

Behavior:
Containers start and connect successfully, jobs begin running, but after ~2 minutes Jenkins reports:

ChannelClosedException: Channel ... failed. The channel is closing down or has closed down

Then Jenkins removes the container, even while the job is still active.

Verified:

  • Docker connection OK (Test Connection works)

  • Happens with or without volume mounts

  • Idle timeout changes don’t help

  • Reproducible every time, even with long-running jobs

Expected:
Agent container should stay connected until the job finishes or idle timeout triggers.

Suspected causes:

  • Docker plugin falsely detecting agent as offline

  • Keepalive or timeout issue in “Connect with Docker Container” mode

  • Possible network or plugin regression

Request:
Need help understanding why Jenkins disconnects active Docker agents and how to keep them connected for the full job duration. The interesting thing is that it connects to another agent in the same way as expected but not to the one we are trying to connec to, could someone please put some light on this and help me in this situation?
Thanks in Advance

Jenkins setup:
Jenkins: 2.504
OS: Linux - 5.14.0-570.51.1.el9_6.x86_64
Java: 21.0.6 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)

Office-365-Connector:5.0.0
ansicolor:1.0.5
ant:511.v0a_a_1a_334f41b_
antisamy-markup-formatter:173.v680e3a_b_69ff3
apache-httpcomponents-client-4-api:4.5.14-269.vfa_2321039a_83
apache-httpcomponents-client-5-api:5.5-157.vb_43ef2f6845e
artifactory:4.0.8
asm-api:9.7.1-97.v4cc844130d97
authentication-tokens:1.144.v5ff4a_5ec5c33
authorize-project:1.8.1
bitbucket-push-and-pull-request:3.1.3
bootstrap5-api:5.3.7-1
bouncycastle-api:2.30.1.81-264.v95c79c0e772c
branch-api:2.1200.v4b_a_3da_2eb_db_4
build-timeout:1.33
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:373.vfe7645102093
cloud-stats:377.vd8a_6c953e98e
cloudbees-bitbucket-branch-source:936.4.3
cloudbees-disk-usage-simple:239.v6a_f1a_c1c8046
cloudbees-folder:6.999.v42253c105443
command-launcher:118.v72741845c17a_
commons-compress-api:1.26.1-2
commons-httpclient3-api:3.1-3
commons-lang3-api:3.18.0-98.v3a_674c06072d
commons-text-api:1.13.1-176.v74d88f22034b_
config-file-provider:980.v88956a_a_5d6a_d
credentials:1419.v2337d1ceceef
credentials-binding:687.v619cb_15e923f
customizable-header:143.v6cb_d4e85f9d5
data-tables-api:2.1.8-1
display-url-api:2.209.v582ed814ff2f
docker-commons:445.v6b_646c962a_94
docker-java-api:3.5.3-122.v156e51f30c0a_
docker-plugin:1274.vc0203fdf2e74
docker-workflow:580.vc0c340686b_54
durable-task:581.v299a_5609d767
echarts-api:5.6.0-5
eddsa-api:0.3.0.1-19.vc432d923e5ee
email-ext:1861.vdb_d991590994
favorite:2.221.v19ca_666b_62f5
flatpickr-api:4.6.13-5.v534d8025a_a_59
font-awesome-api:6.7.2-1
git:5.7.0
git-client:6.1.0
git-server:126.v0d945d8d2b_39
github:1.40.0
github-api:1.321-478.vc9ce627ce001
github-branch-source:1807.v50351eb_7dd13
global-build-stats:316.vf8870f424d78
google-oauth-plugin:1.330.vf5e86021cb_ec
google-play-android-publisher:4.2
gradle:2.13.1
gson-api:2.12.1-113.v347686d6729f
handy-uri-templates-2-api:2.1.8-36.v85e4cb_234a_13
instance-identity:203.v15e81a_1b_7a_38
ionicons-api:88.va_4187cb_eddf1
jackson2-api:2.18.3-402.v74c4eb_f122b_2
jakarta-activation-api:2.1.3-2
jakarta-mail-api:2.1.3-1
javadoc:280.v050b_5c849f69
javax-activation-api:1.2.0-8
javax-mail-api:1.6.2-11
jaxb:2.3.9-133.vb_ec76a_73f706
jdk-tool:83.v417146707a_3d
jersey2-api:2.44-151.v6df377fff741
jjwt-api:0.11.5-112.ve82dfb_224b_a_d
jnr-posix-api:3.1.19-2
job-dsl:1.90
jobConfigHistory:1330.vcb_8320ea_cdb_e
joda-time-api:2.14.0-127.v7d9da_295a_d51
jquery3-api:3.7.1-3
jsch:0.2.16-95.v3eecb_55fa_b_78
json-api:20250107-125.v28b_a_ffa_eb_f01
json-path-api:2.9.0-148.v22a_7ffe323ce
jsoup:1.21.1-52.v96e4041b_60fd
junit:1335.v6b_a_a_e18534e1
ldap:770.vb_455e934581a_
lockable-resources:1327.ved786b_a_197e0
mac:1.6.1
mailer:489.vd4b_25144138f
matrix-auth:3.2.6
matrix-project:847.v88a_f90ff9f20
maven-plugin:3.26
mercurial:1260.vdfb_723cdcc81
metrics:4.2.21-458.vcf496cb_839e4
mina-sshd-api-common:2.14.0-143.v2b_362fc39576
mina-sshd-api-core:2.14.0-143.v2b_362fc39576
msbuild:1.35
nodejs:1.6.2
nuget:1.1
oauth-credentials:0.653.v14cf2088e950
okhttp-api:4.11.0-181.v1de5b_83857df
oss-symbols-api:388.v1e168e8f0d76
pam-auth:1.11
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:61.v629f2cc41d83
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-groovy-lib:744.v5b_556ee7c253
pipeline-input-step:495.ve9c153f6067b_
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2247.va_423189a_7dff
pipeline-model-definition:2.2218.v56d0cda_37c72
pipeline-model-extensions:2.2247.va_423189a_7dff
pipeline-rest-api:2.34
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2218.v56d0cda_37c72
pipeline-stage-view:2.34
pipeline-utility-steps:2.18.0
plain-credentials:199.v9f8e1f741799
plugin-util-api:6.1.0
prometheus:801.v98e119d8eeda_
rebuild:332.va_1ee476d8f6d
resource-disposer:0.25
robot:5.0.0
role-strategy:743.v142ea_b_d5f1d3
run-condition:1.7
saferestart:102.v4dc1b_9636a_ee
scm-api:704.v3ce5c542825a_
script-security:1373.vb_b_4a_a_c26fa_00
snakeyaml-api:2.3-123.v13484c65210a_
snyk-security-scanner:4.1.0
sonar:2.17.3
ssh-credentials:349.vb_8b_6b_9709f5b_
ssh-slaves:3.1031.v72c6b_883b_869
ssh-steps:2.0.68.va_d21a_12a_6476
sshd:3.353.v2b_d33c46e970
structs:343.vdcf37b_a_c81d5
supplychain:0.1.29
timestamper:1.28
token-macro:444.v52de7e9c573d
trilead-api:2.192.vc50a_d147e369
variant:60.v7290fc0eb_b_cd
wix:1.12
workflow-aggregator:600.vb_57cdd26fdd7
workflow-api:1373.v7b_813f10efa_b_
workflow-basic-steps:1058.vcb_fc1e3a_21a_9
workflow-cps:4080.va_15b_44a_91525
workflow-durable-task-step:1398.vf6c9e89e5988
workflow-job:1508.v9cb_c3a_a_89dfd
workflow-multibranch:795.ve0cb_1f45ca_9a_
workflow-scm-step:437.v05a_f66b_e5ef8
workflow-step-api:700.v6e45cb_a_5a_a_21
workflow-support:968.v8f17397e87b_8
ws-cleanup:0.48

2

I don’t see a tag on hub.docker.com for jenkins/inbound-agent:2.2.6. That seems like an outdated version of the inbound agent container image.

We switched to continuous delivery of the container agent images 3 years ago. Version numbers of the inbound agent have been 3000+ for 3 years.

The inbound agent configuration that I’m using currently is jenkins/inbound-agent:3345.v03dee9b_f88fc-2-alpine3.22-jdk21 as described in this configuration as code entry:

jenkins:
  clouds:
  - docker:
      containerCap: 3
      dockerApi:
        dockerHost:
          credentialsId: "docker-host-docker-daemon-x.509"
          uri: "tcp://docker-host.example.com:2376"
      name: "fedora-a"
      templates:
      - connector:
          jnlp:
            jenkinsUrl: "http://jenkins.example.com:8080/"
            user: "1000"
        dockerTemplateBase:
          image: "jenkins/inbound-agent:3345.v03dee9b_f88fc-2-alpine3.22-jdk21"
        labelString: "git-2.49 alpine alpine-jdk21 container"
        mode: EXCLUSIVE
        name: "alpine-jdk21"
        remoteFs: "/home/jenkins/agent"