Jenkins setup:
Jenkins: 2.426.2
OS: Linux - 6.1.61-85.141.amzn2023.x86_64
Java: 17.0.9 - Amazon.com Inc. (OpenJDK 64-Bit Server VM)
ansicolor:1.0.4
ant:497.v94e7d9fffa_b_9
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
apache-httpcomponents-client-5-api:5.3.1-1.0
artifactory:4.0.6
atlassian-jira-software-cloud:2.0.14
authentication-tokens:1.113.v81215a_241826
aws-credentials:231.v08a_59f17d742
aws-java-sdk-ec2:1.12.730-457.v3403b_37d2170
aws-java-sdk-minimal:1.12.730-457.v3403b_37d2170
aws-java-sdk-secretsmanager:1.12.730-457.v3403b_37d2170
aws-secrets-manager-credentials-provider:1.214.va_0a_d8268d068
bitbucket-pullrequest-builder:1.5.0
blueocean:1.27.11
blueocean-autofavorite:1.2.5
blueocean-bitbucket-pipeline:1.27.11
blueocean-commons:1.27.11
blueocean-config:1.27.11
blueocean-core-js:1.27.11
blueocean-dashboard:1.27.11
blueocean-display-url:2.4.2
blueocean-events:1.27.11
blueocean-git-pipeline:1.27.11
blueocean-github-pipeline:1.27.11
blueocean-i18n:1.27.11
blueocean-jira:1.27.11
blueocean-jwt:1.27.11
blueocean-personalization:1.27.11
blueocean-pipeline-api-impl:1.27.11
blueocean-pipeline-editor:1.27.11
blueocean-pipeline-scm-api:1.27.11
blueocean-rest:1.27.11
blueocean-rest-impl:1.27.11
blueocean-web:1.27.11
bootstrap5-api:5.3.2-3
bouncycastle-api:2.30.1.78.1-233.vfdcdeb_0a_08a_a_
branch-api:2.1152.v6f101e97dd77
build-timeout:1.32
build-token-root:151.va_e52fe3215fc
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloud-stats:336.v788e4055508b_
cloudbees-bitbucket-branch-source:886.v44cf5e4ecec5
cloudbees-folder:6.858.v898218f3609d
cobertura:1.17
code-coverage-api:4.99.0
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-108.v2c41b_8b_2b_269
conditional-buildstep:1.4.3
config-file-provider:973.vb_a_80ecb_9a_4d0
configuration-as-code:1805.v1455f39c04cf
configuration-as-code-groovy:1.1
configurationslicing:548.ve92d48e66b_f8
copy-data-to-workspace-plugin:1.0
copyartifact:722.v0662a_9b_e22a_c
coverage:1.10.0
credentials:1319.v7eb_51b_3a_c97b_
credentials-binding:677.vdc9d38cb_254d
dark-theme:416.v535839b_c4e88
data-tables-api:1.13.8-3
datadog:7.0.1
display-url-api:2.204.vf6fddd8a_8b_e9
docker-commons:439.va_3cb_0a_6a_fb_29
docker-java-api:3.3.4-86.v39b_a_5ede342c
docker-plugin:1.5
docker-workflow:580.vc0c340686b_54
durable-task:555.v6802fe0f0b_82
ec2:1688.v8c07e01d657f
echarts-api:5.4.3-3
email-ext:2.105
envinject:2.908.v66a_774b_31d93
envinject-api:1.199.v3ce31253ed13
extended-read-permission:53.v6499940139e5
external-monitor-job:215.v2e88e894db_f8
fail-the-build-plugin:5.v153b_2c826ef0
favorite:2.208.v91d65b_7792a_c
font-awesome-api:6.5.1-2
forensics-api:2.3.0
git:5.2.1
git-client:4.7.0
git-server:126.v0d945d8d2b_39
github:1.39.0
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1789.v5b_0c0cea_18c3
gradle:2.12
gson-api:2.10.1-15.v0d99f670e0a_7
handy-uri-templates-2-api:2.1.8-30.v7e777411b_148
htmlpublisher:1.34
instance-identity:185.v303dc7c645f9
ionicons-api:74.v93d5eb_813d5f
jackson2-api:2.17.0-379.v02de8ec9f64c
jacoco:3.3.5
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javadoc:243.vb_b_503b_b_45537
javax-activation-api:1.2.0-7
javax-mail-api:1.6.2-10
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jenkins-design-language:1.27.11
jersey2-api:2.42-147.va_28a_44603b_d5
jira:3.13
jira-integration:5.3.0-37.v13c5b_5fb_e71b_
jjwt-api:0.11.5-112.ve82dfb_224b_a_d
jnr-posix-api:3.1.19-1
job-dsl:1.87
joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
jquery3-api:3.7.1-1
jsch:0.2.16-86.v42e010d9484b_
json-api:20240303-41.v94e11e6de726
json-path-api:2.9.0-33.v2527142f2e1d
junit:1265.v65b_14fa_f12f0
ldap:725.v3cb_b_711b_1a_ef
mailer:470.vc91f60c5d8e2
mask-passwords:173.v6a_077a_291eb_5
matrix-auth:3.2.2
matrix-project:822.824.v14451b_c0fd42
maven-plugin:3.23
metrics:4.2.21-451.vd51df8df52ec
metrics-datadog:1.0
mina-sshd-api-common:2.12.1-101.v85b_e08b_780dd
mina-sshd-api-core:2.12.1-101.v85b_e08b_780dd
multi-branch-project-plugin:0.7
multibranch-build-strategy-extension:48.v3dc306525d0c
nested-view:1.33
node-iterator-api:55.v3b_77d4032326
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
pam-auth:1.11
parameterized-scheduler:262.v00f3d90585cc
parameterized-trigger:787.v665fcf2a_830b_
permissive-script-security:0.7
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:61.v629f2cc41d83
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-groovy-lib:710.v4b_94b_077a_808
pipeline-input-step:477.v339683a_8d55e
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2198.v41dd8ef6dd56
pipeline-model-declarative-agent:1.1.1
pipeline-model-definition:2.2198.v41dd8ef6dd56
pipeline-model-extensions:2.2198.v41dd8ef6dd56
pipeline-multibranch-defaults:2.1
pipeline-rest-api:2.34
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2198.v41dd8ef6dd56
pipeline-stage-view:2.34
pipeline-utility-steps:2.16.2
plain-credentials:182.v468b_97b_9dcb_8
plugin-util-api:3.8.0
prism-api:1.29.0-11
pubsub-light:1.18
read-only-configurations:1.10
rebuild:332.va_1ee476d8f6d
resource-disposer:0.23
role-strategy:689.v731678c3e0eb_
run-condition:1.7
saml:4.464.vea_cb_75d7f5e0
schedule-build:550.v8eb_e2fcf7b_1c
scm-api:683.vb_16722fb_b_80b_
script-security:1341.va_2819b_414686
simple-theme-plugin:176.v39740c03a_a_f5
slack:684.v833089650554
snakeyaml-api:2.2-111.vc6598e30cc65
sse-gateway:1.26
ssh-credentials:337.v395d2403ccd4
ssh-slaves:2.948.vb_8050d697fec
sshd:3.322.v159e91f6a_550
stash-pullrequest-builder:1.17
stashNotifier:1.484.v6c879a_e09e0c
structs:337.v1b_04ea_4df7c8
test-stability:2.3
theme-manager:215.vc1ff18d67920
timestamper:1.27
token-macro:400.v35420b_922dcb_
trilead-api:2.142.v748523a_76693
variant:60.v7290fc0eb_b_cd
view-job-filters:377.v66f4b_796e5fa_
windows-slaves:1.8.1
workflow-aggregator:596.v8c21c963d92d
workflow-api:1291.v51fd2a_625da_7
workflow-basic-steps:1058.vcb_fc1e3a_21a_9
workflow-cps:3894.3896.vca_2c931e7935
workflow-cps-global-lib:612.v55f2f80781ef
workflow-cps-global-lib-http:2.48.0
workflow-durable-task-step:1331.vc8c2fed35334
workflow-job:1385.vb_58b_86ea_fff1
workflow-multibranch:773.vc4fe1378f1d5
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:657.v03b_e8115821b_
workflow-support:907.v6713a_ed8a_573
ws-cleanup:0.46
We are migrating to Jenkins 2.426.2 from a fairly old version (2.235.5). Our setup (old and new) uses the ec2-plugin heavily to provision worker agents and has been working without any problem on the old version. However, we are noticing stability issues especially with our multibranch pipelines on the new version. It appears every often the controller loses connection and throws something like -
"Jun 03, 2024 3:13:51 AM INFO org.jenkinsci.plugins.workflow.support.concurrent.Timeout lambda$ping$0
org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep [#65]: checking /home/ec2-user/workspace/registration-pipeline_develop on EC2 (AWS-Node) - Jenkins-agent (i-0f1cbf6341983be85) unresponsive for 35 sec
"
Under the pipeline, there’s a corresponding -
“Cannot contact EC2 (AWS-Node) - Jenkins-agent (i-0f1cbf6341983be85): java.lang.InterruptedException”
This throws the pipeline in an infinite loop, to a point where we have to restart the Jenkins server on certain occasions. We never faced this issue with the old instance which was running on much lesser resources.
Also, our pipelines are mostly executing steps within docker containers on these ec2 workers. This problem is only occuring with multibranch pipelines, single branch pipelines have been executing fine.
I would appreciate any feedback on this.
Thanks