Accidentally upgraded from Jenkins version 2.492.3 to 2.528.1. Failed to start Jenkins

Hello,

After accidentally upgraded Jenkins from 2.492.3 to 2.528.1 version, Jenkins it’s failing to start (seeing also 502 bad gateway in UI, so I’m not able to even get into the interface). Getting the following errors:

/var/lib/jenkins$ sudo systemctl status jenkins
× jenkins.service - Jenkins Continuous Integration Server
     Loaded: loaded (/lib/systemd/system/jenkins.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2025-11-10 16:42:55 UTC; 32min ago
    Process: 31687 ExecStart=/usr/bin/jenkins (code=exited, status=5)
   Main PID: 31687 (code=exited, status=5)
     Status: "Jenkins stopped"
        CPU: 22.952s

Nov 10 16:42:55  systemd[1]: jenkins.service: Scheduled restart job, restart counter is at 5.
Nov 10 16:42:55  systemd[1]: Stopped Jenkins Continuous Integration Server.
Nov 10 16:42:55 systemd[1]: jenkins.service: Consumed 22.952s CPU time.
Nov 10 16:42:55  systemd[1]: jenkins.service: Start request repeated too quickly.
Nov 10 16:42:55 3 systemd[1]: jenkins.service: Failed with result 'exit-code'.
Nov 10 16:42:55  systemd[1]: Failed to start Jenkins Continuous Integration Server.

sudo systemctl start jenkins
Job for jenkins.service failed because the control process exited with error code.
See "systemctl status jenkins.service" and "journalctl -xeu jenkins.service" for details.


sudo systemctl status jenkins.service
● jenkins.service - Jenkins Continuous Integration Server
     Loaded: loaded (/lib/systemd/system/jenkins.service; enabled; vendor preset: enabled)
     Active: activating (start) since Mon 2025-11-10 17:16:08 UTC; 3s ago
   Main PID: 32131 (java)
      Tasks: 48 (limit: 18847)
     Memory: 322.5M
        CPU: 12.007s
     CGroup: /system.slice/jenkins.service
             └─32131 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/java/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080

Nov 10 17:16:11  jenkins[32131]:         at hudson.PluginManager$2$1$1.run(PluginManager.java:590)
Nov 10 17:16:11  jenkins[32131]:         at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
Nov 10 17:16:11 jenkins[32131]:         at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:304)
Nov 10 17:16:11  jenkins[32131]:         at jenkins.model.Jenkins$5.runTask(Jenkins.java:1151)
Nov 10 17:16:11  jenkins[32131]:         at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
Nov 10 17:16:11  jenkins[32131]:         at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
Nov 10 17:16:11  jenkins[32131]:         at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
Nov 10 17:16:11  jenkins[32131]:         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
Nov 10 17:16:11  jenkins[32131]:         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Nov 10 17:16:11  jenkins[32131]:         at java.base/java.lang.Thread.run(Thread.java:833)

Jenkins has the right filesystem permissions and already verify that my java version is compatible:

java version "17.0.6" 2023-01-17 LTS Java(TM) SE Runtime Environment (build 17.0.6+9-LTS-190) Java HotSpot(TM) 64-Bit Server VM (build 17.0.6+9-LTS-190, mixed mode, sharing)

I verified if there was a /var/log/jenkins/jenkins.log but it doesn’t exist, suspect this is because it’s not even getting far enough to create the log directory.

I ran Jenkins manually to check on mismatch errors:

:/var/lib/jenkins$ sudo -u jenkins env JENKINS_HOME=/var/lib/jenkins /usr/bin/jenkins
Running from: /usr/share/java/jenkins.war
webroot: /var/lib/jenkins/war
2025-11-07 21:45:37.927+0000 [id=1]	INFO	winstone.Logger#logInternal: Beginning extraction from war file
2025-11-07 21:45:37.974+0000 [id=1]	WARNING	o.e.j.ee9.nested.ContextHandler#setContextPath: Empty contextPath
2025-11-07 21:45:38.027+0000 [id=1]	INFO	org.eclipse.jetty.server.Server#doStart: jetty-12.0.25; built: 2025-08-11T23:52:37.219Z; git: a862b76d8372e24205765182d9ae1d1d333ce2ea; jvm 17.0.6+9-LTS-190
2025-11-07 21:45:38.424+0000 [id=1]	INFO	o.e.j.e.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.ee9.jsp.JettyJspServlet
2025-11-07 21:45:38.471+0000 [id=1]	INFO	o.e.j.s.DefaultSessionIdManager#doStart: Session workerName=node0
2025-11-07 21:45:38.891+0000 [id=1]	INFO	hudson.WebAppMain#contextInitialized: Jenkins home directory: /var/lib/jenkins found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
2025-11-07 21:45:39.002+0000 [id=1]	INFO	o.e.j.s.handler.ContextHandler#doStart: Started oeje9n.ContextHandler$CoreContextHandler@74f7d1d2{Jenkins v2.528.1,/,b=file:///var/lib/jenkins/war/,a=AVAILABLE,h=oeje9n.ContextHandler$CoreContextHandler$CoreToNestedHandler@4b4dd216{STARTED}}
2025-11-07 21:45:39.018+0000 [id=1]	INFO	o.e.j.server.AbstractConnector#doStart: Started ServerConnector@35038141{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2025-11-07 21:45:39.030+0000 [id=1]	INFO	org.eclipse.jetty.server.Server#doStart: Started oejs.Server@60dcc9fe{STARTING}[12.0.25,sto=0] @1537ms
2025-11-07 21:45:39.034+0000 [id=25]	INFO	winstone.Logger#logInternal: Winstone Servlet Engine running: controlPort=disabled
2025-11-07 21:45:39.159+0000 [id=24]	INFO	jenkins.model.Jenkins#<init>: Starting version 2.528.1
2025-11-07 21:45:39.240+0000 [id=30]	INFO	hudson.PluginManager#loadDetachedPlugins: Upgrading Jenkins. The last running version was 2.492.3. This Jenkins is version 2.528.1.
2025-11-07 21:45:39.242+0000 [id=31]	INFO	jenkins.InitReactorRunner$1#onAttained: Started initialization
2025-11-07 21:45:39.277+0000 [id=30]	INFO	hudson.PluginManager#loadDetachedPlugins: Upgraded Jenkins from version 2.492.3 to version 2.528.1. Loaded detached plugins (and dependencies): []
2025-11-07 21:45:39.471+0000 [id=32]	INFO	jenkins.InitReactorRunner$1#onAttained: Listed all plugins
2025-11-07 21:45:39.745+0000 [id=31]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Mina SSHD API :: Core v2.16.0-167.va_269f38cc024 (mina-sshd-api-core)
java.io.IOException: Failed to load: Mina SSHD API :: Core (mina-sshd-api-core 2.16.0-167.va_269f38cc024)
[LF]>  - Update required: SSH Credentials Plugin (ssh-credentials 308.ve4497b_ccd8f4) to be updated to 355.v9b_e5b_cde5003 or higher
	at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1020)
	at hudson.PluginManager$2$1$1.run(PluginManager.java:590)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:304)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1151)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
2025-11-07 21:45:39.747+0000 [id=36]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin SSH server v3.374.v19b_d59ce6610 (sshd)
java.io.IOException: Failed to load: SSH server (sshd 3.374.v19b_d59ce6610)
[LF]>  - Failed to load: Mina SSHD API :: Core (mina-sshd-api-core 2.16.0-167.va_269f38cc024)
	at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1020)
	at hudson.PluginManager$2$1$1.run(PluginManager.java:590)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:304)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1151)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
2025-11-07 21:45:39.836+0000 [id=31]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Jenkins Git client plugin v4.6.0 (git-client)
java.io.IOException: Failed to load: Jenkins Git client plugin (git-client 4.6.0)
[LF]>  - Failed to load: Mina SSHD API :: Core (mina-sshd-api-core 2.16.0-167.va_269f38cc024)
	at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1020)
	at hudson.PluginManager$2$1$1.run(PluginManager.java:590)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:304)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1151)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
2025-11-07 21:45:39.846+0000 [id=34]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Git plugin v5.2.1 (git)
java.io.IOException: Failed to load: Git plugin (git 5.2.1)
[LF]>  - Failed to load: Jenkins Git client plugin (git-client 4.6.0)
	at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1020)
	at hudson.PluginManager$2$1$1.run(PluginManager.java:590)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:304)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1151)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
2025-11-07 21:45:39.882+0000 [id=36]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Matrix Authorization Strategy Plugin v3.2.8 (matrix-auth)
java.io.IOException: Failed to load: Matrix Authorization Strategy Plugin (matrix-auth 3.2.8)
[LF]>  - Update required: Folders Plugin (cloudbees-folder 6.858.v898218f3609d) to be updated to 6.1026.ve06dfa_cf31c3 or higher
	at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1020)
	at hudson.PluginManager$2$1$1.run(PluginManager.java:590)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:304)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1151)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
2025-11-07 21:45:39.883+0000 [id=33]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Microsoft Entra ID (previously Azure AD) Plugin v449.v92b_39a_d8e523 (azure-ad)
java.io.IOException: Failed to load: Microsoft Entra ID (previously Azure AD) Plugin (azure-ad 449.v92b_39a_d8e523)
[LF]>  - Failed to load: Matrix Authorization Strategy Plugin (matrix-auth 3.2.8)
	at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:1020)
	at hudson.PluginManager$2$1$1.run(PluginManager.java:590)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:304)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1151)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

Logs show more of SEVERE plugins errors but these are the first ones to show.

Looks like main problem is incompatibility of plugins with the newest version, specially Microsoft Entra ID and probably Matrix Authorizaton Strategy, since is not allowing me to even access to login and we have configured Azure AD to login to Jekins.

My question here is how to fix this plugin? or would it be better to downgrade to my former version? If that’s the case, please point out on the steps to follow.

Your help is really appreciated.

You have 2 plugins that are too old
SSH Credentials and Folders
You can download the plugins manually from the release tab, rename the .hpi to .jpi and copy them into the plugins folder of your instance. That should probably get you in a state that allows to login. From there you can then update the remaining plugins.

1 Like

Thanks for your response, also something that I forgot to mentioned is that when I ran manually jenkins besides the errors on the logs that I already posted I see this in the UI:

Error
com.thoughtworks.xstream.mapper.CannotResolveClassException: com.microsoft.jenkins.azuread.AzureSecurityRealm
	at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(DefaultMapper.java:81)
	at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
	at com.thoughtworks.xstream.mapper.DynamicProxyMapper.realClass(DynamicProxyMapper.java:55)
	at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
	at com.thoughtworks.xstream.mapper.PackageAliasingMapper.realClass(PackageAliasingMapper.java:88)
	at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
	at com.thoughtworks.xstream.mapper.ClassAliasingMapper.realClass(ClassAliasingMapper.java:79)
	at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
	at com.thoughtworks.xstream.mapper.ArrayMapper.realClass(ArrayMapper.java:74)
	at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
	at com.thoughtworks.xstream.mapper.SecurityMapper.realClass(SecurityMapper.java:71)
	at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
	at hudson.util.XStream2$CompatibilityMapper.realClass(XStream2.java:452)
	at hudson.util.xstream.MapperDelegate.realClass(MapperDelegate.java:46)
	at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
	at com.thoughtworks.xstream.mapper.CachingMapper.realClass(CachingMapper.java:47)
	at hudson.util.RobustReflectionConverter.determineType(RobustReflectionConverter.java:551)
	at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:349)
Caused: jenkins.util.xstream.CriticalXStreamException: 
---- Debugging information ----
cause-exception     : com.thoughtworks.xstream.mapper.CannotResolveClassException
cause-message       : com.microsoft.jenkins.azuread.AzureSecurityRealm
class               : hudson.model.Hudson
required-type       : hudson.model.Hudson
converter-type      : hudson.util.RobustReflectionConverter
path                : /hudson/securityRealm
line number         : 171
version             : 2.528.1
-------------------------------
	at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:384)
	at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:291)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
	at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1468)
	at hudson.util.XStream2.unmarshal(XStream2.java:230)
	at hudson.util.XStream2.unmarshal(XStream2.java:201)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1445)
	at hudson.XmlFile.unmarshal(XmlFile.java:196)
Caused: java.io.IOException: Unable to read /var/lib/jenkins/config.xml
	at hudson.XmlFile.unmarshal(XmlFile.java:199)
	at hudson.XmlFile.unmarshal(XmlFile.java:179)
	at jenkins.model.Jenkins.load(Jenkins.java:3367)
	at jenkins.model.Jenkins$13.run(Jenkins.java:3486)
	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:304)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1151)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused: org.jvnet.hudson.reactor.ReactorException
	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:290)
	at jenkins.InitReactorRunner.run(InitReactorRunner.java:49)
	at jenkins.model.Jenkins.executeReactor(Jenkins.java:1186)
	at jenkins.model.Jenkins.<init>(Jenkins.java:985)
	at hudson.model.Hudson.<init>(Hudson.java:100)
	at hudson.model.Hudson.<init>(Hudson.java:85)
	at hudson.WebAppMain$3.run(WebAppMain.java:249)
Caused: hudson.util.HudsonFailedToLoad
	at hudson.WebAppMain$3.run(WebAppMain.java:277)

That error is a direct cause that the azure-ad plugin couldn’t be loaded.

I’m having the same issue. Did you resolve it?

Hello! I was able to fix it by temporarily modify the security realm and authorization strategy sections from the config.xml. Here are the steps:

Find the line <useSecurity>true</useSecurity> and change it to:

<useSecurity>false</useSecurity>

Find and delete these entire sections:

  • Everything from <securityRealm> through </securityRealm>

  • Everything from <authorizationStrategy> through </authorizationStrategy>

    Example of what to remove:

<securityRealm class="hudson.security.HudsonPrivateSecurityRealm">

  <!-- entire section -->

</securityRealm>

<authorizationStrategy class="hudson.security.AuthorizationStrategy$Unsecured"/>

Make sure to FIRST create a copy of the config.xml as a backup. After modifying the file, try:

sudo systemctl start jenkins

Once it’s up, go to the UI and fix the plugins errors from there, in my case by updating the first plugins from the list of Jenkins errors solved all the rest of the issues. Once everything is working, go back to your original config.xml and restart jenkins again.

After that you should be good to go! Hope this helps and thank you @mawinter69 for the help!