Issue upgrading Jenkins to latest version 2.414.1

Hello!

I have followed the upgrade steps in the documentation, and upgraded all the required plugins that might be affected, but when I’m trying to update to the latest LTS version (2.414.1) I basically get no update/no obvious errors.

The steps I’m following are the same I have done before:

  • Update the plugins
  • Download the latest .war in the machine I’m using jenkins
  • systemctl restart jenkins

These are the logs I’m getting from the journal, everything seems good:

-- The job identifier is 83392.
Aug 23 14:40:06 devops jenkins[1378441]: Running from: /usr/share/java/jenkins.war
Aug 23 14:40:07 devops jenkins[1378441]: 2023-08-23 12:40:07.359+0000 [id=1]        INFO        winstone.Logger#logInternal: Beginning extraction from war file
Aug 23 14:40:07 devops jenkins[1378441]: 2023-08-23 12:40:07.442+0000 [id=1]        WARNING        o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath
Aug 23 14:40:07 devops jenkins[1378441]: 2023-08-23 12:40:07.548+0000 [id=1]        INFO        org.eclipse.jetty.server.Server#doStart: jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8-post-Ubuntu-1ubuntu120.04
Aug 23 14:40:07 devops jenkins[1378441]: 2023-08-23 12:40:07.927+0000 [id=1]        INFO        o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
Aug 23 14:40:08 devops jenkins[1378441]: 2023-08-23 12:40:08.002+0000 [id=1]        INFO        o.e.j.s.s.DefaultSessionIdManager#doStart: Session workerName=node0
Aug 23 14:40:08 devops jenkins[1378441]: 2023-08-23 12:40:08.552+0000 [id=1]        INFO        hudson.WebAppMain#contextInitialized: Jenkins home directory: /var/lib/jenkins found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
Aug 23 14:40:08 devops jenkins[1378441]: 2023-08-23 12:40:08.689+0000 [id=1]        INFO        o.e.j.s.handler.ContextHandler#doStart: Started w.@c1fca1e{Jenkins v2.401.2,/,file:///var/cache/jenkins/war/,AVAILABLE}{/var/cache/jenkins/war}
Aug 23 14:40:08 devops jenkins[1378441]: 2023-08-23 12:40:08.703+0000 [id=1]        INFO        o.e.j.server.AbstractConnector#doStart: Started ServerConnector@679b62af{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
Aug 23 14:40:08 devops jenkins[1378441]: 2023-08-23 12:40:08.713+0000 [id=1]        INFO        org.eclipse.jetty.server.Server#doStart: Started Server@3f200884{STARTING}[10.0.13,sto=0] @1936ms
Aug 23 14:40:08 devops jenkins[1378441]: 2023-08-23 12:40:08.715+0000 [id=24]        INFO        winstone.Logger#logInternal: Winstone Servlet Engine running: controlPort=disabled
Aug 23 14:40:09 devops jenkins[1378441]: 2023-08-23 12:40:09.191+0000 [id=30]        INFO        jenkins.InitReactorRunner$1#onAttained: Started initialization
Aug 23 14:40:09 devops jenkins[1378441]: 2023-08-23 12:40:09.303+0000 [id=35]        INFO        jenkins.InitReactorRunner$1#onAttained: Listed all plugins
Aug 23 14:40:09 devops jenkins[1378441]: 2023-08-23 12:40:09.963+0000 [id=29]        INFO        o.j.h.p.backup.BackupPluginImpl#loadConfiguration: Loading configuration...
Aug 23 14:40:15 devops jenkins[1378441]: 2023-08-23 12:40:15.720+0000 [id=31]        INFO        jenkins.InitReactorRunner$1#onAttained: Prepared all plugins
Aug 23 14:40:15 devops jenkins[1378441]: 2023-08-23 12:40:15.774+0000 [id=32]        INFO        jenkins.InitReactorRunner$1#onAttained: Started all plugins
Aug 23 14:40:15 devops jenkins[1378441]: 2023-08-23 12:40:15.812+0000 [id=31]        INFO        jenkins.InitReactorRunner$1#onAttained: Augmented all extensions
Aug 23 14:40:16 devops jenkins[1378441]: WARNING: An illegal reflective access operation has occurred
Aug 23 14:40:16 devops jenkins[1378441]: WARNING: Illegal reflective access by org.codehaus.groovy.vmplugin.v7.Java7$1 (file:/var/cache/jenkins/war/WEB-INF/lib/groovy-all-2.4.21.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
Aug 23 14:40:16 devops jenkins[1378441]: WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.vmplugin.v7.Java7$1
Aug 23 14:40:16 devops jenkins[1378441]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
Aug 23 14:40:16 devops jenkins[1378441]: WARNING: All illegal access operations will be denied in a future release
Aug 23 14:40:16 devops jenkins[1378441]: 2023-08-23 12:40:16.282+0000 [id=32]        INFO        jenkins.model.Jenkins#setBuildsAndWorkspacesDir: Using non default workspaces directories: ${JENKINS_HOME}/workspace/${ITEM_FULLNAME}.
Aug 23 14:40:17 devops jenkins[1378441]: 2023-08-23 12:40:17.109+0000 [id=34]        INFO        h.p.b.g.GlobalTimeOutConfiguration#load: global timeout not set
Aug 23 14:40:17 devops jenkins[1378441]: 2023-08-23 12:40:17.591+0000 [id=34]        INFO        j.a.PrioritySorterPlugin#init1: Configuring the Priority Sorter ...
Aug 23 14:40:17 devops jenkins[1378441]: 2023-08-23 12:40:17.681+0000 [id=32]        INFO        jenkins.InitReactorRunner$1#onAttained: System config loaded
Aug 23 14:40:17 devops jenkins[1378441]: 2023-08-23 12:40:17.687+0000 [id=31]        INFO        jenkins.InitReactorRunner$1#onAttained: System config adapted
Aug 23 14:40:17 devops jenkins[1378441]: 2023-08-23 12:40:17.957+0000 [id=32]        WARNING        c.c.h.p.folder.AbstractFolder#loadChildren: could not find file /var/lib/jenkins/jobs/Pipelines/jobs/Frontend/jobs/ui-builder-merger/config.xml
Aug 23 14:40:17 devops jenkins[1378441]: 2023-08-23 12:40:17.969+0000 [id=32]        WARNING        c.c.h.p.folder.AbstractFolder#loadChildren: could not find file /var/lib/jenkins/jobs/Pipelines/jobs/Frontend/jobs/adm-builder-in-progress/config.xml
Aug 23 14:40:20 devops jenkins[1378441]: 2023-08-23 12:40:20.955+0000 [id=30]        INFO        jenkins.InitReactorRunner$1#onAttained: Loaded all jobs
Aug 23 14:40:20 devops jenkins[1378441]: 2023-08-23 12:40:20.956+0000 [id=37]        INFO        j.a.PrioritySorterPlugin#init2: Sorting existing Queue ...
Aug 23 14:40:20 devops jenkins[1378441]: 2023-08-23 12:40:20.983+0000 [id=37]        INFO        j.a.sorter.AdvancedQueueSorter#init: Initialized the QueueSorter with 0 Buildable Items
Aug 23 14:40:20 devops jenkins[1378441]: 2023-08-23 12:40:20.983+0000 [id=29]        INFO        jenkins.InitReactorRunner$1#onAttained: Configuration for all jobs updated
Aug 23 14:40:21 devops jenkins[1378441]: 2023-08-23 12:40:21.101+0000 [id=30]        INFO        jenkins.InitReactorRunner$1#onAttained: Completed initialization
Aug 23 14:40:21 devops jenkins[1378441]: 2023-08-23 12:40:21.163+0000 [id=23]        INFO        hudson.lifecycle.Lifecycle#onReady: Jenkins is fully up and running
Aug 23 14:40:21 devops systemd[1]: Started Jenkins Continuous Integration Server.
-- Subject: A start job for unit jenkins.service has finished successfully
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- A start job for unit jenkins.service has finished successfully.
-- 
-- The job identifier is 83392.
Aug 23 14:40:21 devops jenkins[1378441]: 2023-08-23 12:40:21.632+0000 [id=56]        WARNING        h.m.DownloadService$Downloadable#updateNow: No tool installer metadata found for jenkins.plugins.nodejs.tools.MirrorNodeJSInstaller

But jenkins starts again at version 2.401.2. Any help please? Thanks!!

My Jenkins setup:

Jenkins: 2.401.2
OS: Linux - 5.4.0-153-generic
Java: 11.0.20 - Ubuntu (OpenJDK 64-Bit Server VM)
---
PrioritySorter:5.0.0
ace-editor:1.1
ansicolor:1.0.3
ant:497.v94e7d9fffa_b_9
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-150.v7a_b_9d17134a_5
authentication-tokens:1.53.v1c90fd9191a_b_
authorize-project:1.7.1
backup:1.6.1
bitbucket:223.vd12f2bca5430
blueocean:1.27.5.1
blueocean-autofavorite:1.2.5
blueocean-bitbucket-pipeline:1.27.5.1
blueocean-commons:1.27.5.1
blueocean-config:1.27.5.1
blueocean-core-js:1.27.5.1
blueocean-dashboard:1.27.5.1
blueocean-display-url:2.4.2
blueocean-events:1.27.5.1
blueocean-git-pipeline:1.27.5.1
blueocean-github-pipeline:1.27.5.1
blueocean-i18n:1.27.5.1
blueocean-jira:1.27.5.1
blueocean-jwt:1.27.5.1
blueocean-personalization:1.27.5.1
blueocean-pipeline-api-impl:1.27.5.1
blueocean-pipeline-editor:1.27.5.1
blueocean-pipeline-scm-api:1.27.5.1
blueocean-rest:1.27.5.1
blueocean-rest-impl:1.27.5.1
blueocean-web:1.27.5.1
bootstrap4-api:4.6.0-6
bootstrap5-api:5.3.0-1
bouncycastle-api:2.29
branch-api:2.1122.v09cb_8ea_8a_724
build-timeout:1.31
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.0
cloudbees-bitbucket-branch-source:825.va_6a_dc46a_f97d
cloudbees-folder:6.848.ve3b_fd7839a_81
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.10.0-68.v0d0b_c439292b_
config-file-provider:953.v0432a_802e4d2
configurationslicing:548.ve92d48e66b_f8
credentials:1271.v54b_1c2c6388a_
credentials-binding:631.v861c06d062b_4
data-tables-api:1.13.5-1
display-url-api:2.3.9
docker-commons:439.va_3cb_0a_6a_fb_29
docker-workflow:572.v950f58993843
durable-task:523.va_a_22cf15d5e0
dynamic_extended_choice_parameter:1.0.1
echarts-api:5.4.0-5
email-ext:2.100
envinject:2.908.v66a_774b_31d93
envinject-api:1.199.v3ce31253ed13
extended-choice-parameter:376.v2e02857547b_a_
external-monitor-job:207.v98a_a_37a_85525
favorite:2.4.3
font-awesome-api:6.4.0-2
generic-webhook-trigger:1.87.0
git:5.2.0
git-client:4.4.0
git-parameter:0.9.19
git-server:99.va_0826a_b_cdfa_d
github:1.37.3
github-api:1.314-431.v78d72a_3fe4c3
github-branch-source:1732.v3f1889a_c475b_
handlebars:3.0.8
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
htmlpublisher:1.32
instance-identity:173.va_37c494ec4e5
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.15.2-350.v0c2f3f8fc595
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javadoc:243.vb_b_503b_b_45537
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.8-1
jdk-tool:73.vddf737284550
jenkins-design-language:1.27.5.1
jenkins-jira-issue-updater:1.18
jersey2-api:2.40-1
jira:3.10
jjwt-api:0.11.5-77.v646c772fddb_0
jobConfigHistory:1227.v7a_79fc4dc01f
jquery:1.12.4-1
jquery-detached:1.2.1
jquery3-api:3.7.0-1
jsch:0.2.8-65.v052c39de79b_2
junit:1217.v4297208a_a_b_ce
ldap:694.vc02a_69c9787f
lockable-resources:1185.v0c528656ce04
mailer:463.vedf8358e006b_
mapdb-api:1.0.9-28.vf251ce40855d
matrix-auth:3.2
matrix-project:808.v5a_b_5f56d6966
maven-plugin:3.23
mercurial:1260.vdfb_723cdcc81
metrics:4.2.18-442.v02e107157925
mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_
mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_
momentjs:1.1.1
multibranch-scan-webhook-trigger:1.0.9
nodejs:1.6.1
nvm-wrapper:0.1.7
okhttp-api:4.11.0-157.v6852a_a_fa_ec11
pam-auth:1.10
pipeline-build-step:505.v5f0844d8d126
pipeline-config-history:1.6
pipeline-github-lib:42.v0739460cda_c4
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:671.v07c339c842e8
pipeline-input-step:477.v339683a_8d55e
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2144.v077a_d1928a_40
pipeline-model-definition:2.2144.v077a_d1928a_40
pipeline-model-extensions:2.2144.v077a_d1928a_40
pipeline-rest-api:2.33
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2144.v077a_d1928a_40
pipeline-stage-view:2.33
pipeline-utility-steps:2.16.0
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.3.0
popper-api:1.16.1-3
popper2-api:2.11.6-2
postbuild-task:1.9
postbuildscript:3.2.0-460.va_fda_0fa_26720
promoted-builds:892.vd6219fc0a_efb
pubsub-light:1.17
resource-disposer:0.23
role-strategy:680.v3a_6a_1698b_864
scm-api:676.v886669a_199a_a_
script-security:1271.vdede89739a_81
slack:684.v833089650554
snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4
sonar:2.15
sse-gateway:1.26
ssh-credentials:308.ve4497b_ccd8f4
ssh-slaves:2.916.vd17b_43357ce4
sshd:3.312.v1c601b_c83b_0e
structs:325.vcb_307d2a_2782
subversion:2.17.3
thinBackup:1.18
timestamper:1.26
token-macro:384.vf35b_f26814ec
trilead-api:2.84.v72119de229b_7
variant:59.vf075fe829ccb
windows-slaves:1.8.1
workflow-aggregator:596.v8c21c963d92d
workflow-api:1259.vb_47f14fffc8a_
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3769.v8b_e595e4d40d
workflow-cps-global-lib:609.vd95673f149b_b
workflow-durable-task-step:1284.v4fcd365b_75b_e
workflow-job:1326.ve643e00e9220
workflow-multibranch:756.v891d88f2cd46
workflow-scm-step:415.v434365564324
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:848.v5a_383b_d14921
ws-cleanup:0.45

I am having the same problem. I am running Jenkins in a Docker container from the jenkins/jenkins:lts image. Pulling the new LTS image today (2023-09-07) caused the container to go into a restart loop (I’m using --restart always on when running the container). I tried running jenkins/jenkins:4.422, same problem. It takes about 50 seconds to finish “restarting” (the status from docker ps) then is up for about 1 second before restarting again.

Rolling back to the jenkins/jenkins:2.401.3 image works just fine for me.

Jenkins upgrades should be done with the operating system package manager on machines that run systemd. If you installed with the operating system package manager, be sure that you upgrade with the same operating system package manager.

Based on your later description, you’re having a completely different problem. While @jluzon is running with an installation controlled by systemd, you’re running an installation controlled by a Docker container. They behave differently and are upgraded differently.

You’ll need to read the Docker log files to find the message that is causing the container to restart. It might be restarted due to memory exhaustion. It might be restarted due to a health check that you’ve added. It might be restarted for some other reason.

Thank you Mark! Until this date I had been upgrading jenkins using the .war file, but now I was able to do it with the system package manager. Thanks for your help.

A post was split to a new topic: Cannot open global configuration after upgrade from 2.346 to 2.414