Jenkins not starting after upgrade, Required plugin is disabled errors

Hi Folks,

I attempted an upgrade from Jenkins 2.319.3 to 2.414.1, and I’m running into some errors. I have a backup and I’m going to roll back, but I’m wondering if I have been doing something wrong so that I can get it right next time.

I upgraded jenkins (latest LTS) on my linux server using yum. The package installed fine.

The server was running java 8, so I also installed java 11 (also using yum) and did the following:

systemctl edit jenkins
-Added an environment entry for JAVA_HOME and JENKINS_JAVA_CMD, both pointing to the java 11 location (at present I’d like to leave the alternatives java link pointing to java 8 as there are other dependencies on the server)

-Started jenkins

The server fails to start.

Looking at the logs, I’m seeing this on startup:

Sep 08 19:42:09 jenkins[317022]: 2023-09-08 09:42:09.633+0000 [id=29] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Authentication Tokens API Plugin v>
Sep 08 19:42:09 jenkins[317022]: java.io.IOException: Failed to load: Authentication Tokens API Plugin (authentication-tokens 1.4)
Sep 08 19:42:09 jenkins[317022]: - Required plugin is disabled: Credentials Plugin (credentials)

…and then a lot of similar entries, many of them saying a Required plugin is disabled.

In the end the service jenkins start command times out and the service says it isn’t starting.

I can get to the Jenkins console via a browser, but it throws java errors:

com.thoughtworks.xstream.mapper.CannotResolveClassException: hudson.security.ProjectMatrixAuthorizationStrategy
at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(DefaultMapper.java:81)
at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
at com.thoughtworks.xstream.mapper.DynamicProxyMapper.realClass(DynamicProxyMapper.java:55)
at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
at com.thoughtworks.xstream.mapper.PackageAliasingMapper.realClass(PackageAliasingMapper.java:88)
at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
at com.thoughtworks.xstream.mapper.ClassAliasingMapper.realClass(ClassAliasingMapper.java:79)
at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
at com.thoughtworks.xstream.mapper.ArrayMapper.realClass(ArrayMapper.java:74)
at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
at com.thoughtworks.xstream.mapper.SecurityMapper.realClass(SecurityMapper.java:71)
at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
at hudson.util.XStream2$CompatibilityMapper.realClass(XStream2.java:452)
at hudson.util.xstream.MapperDelegate.realClass(MapperDelegate.java:46)
at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
at com.thoughtworks.xstream.mapper.CachingMapper.realClass(CachingMapper.java:47)
at hudson.util.RobustReflectionConverter.determineType(RobustReflectionConverter.java:521)
at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:346)
Caused: jenkins.util.xstream.CriticalXStreamException:
---- Debugging information ----
cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException
cause-message : hudson.security.ProjectMatrixAuthorizationStrategy
class : hudson.model.Hudson
required-type : hudson.model.Hudson
converter-type : hudson.util.RobustReflectionConverter
path : /hudson/authorizationStrategy
line number : 11
version : 2.422

at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:381)
at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:289)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1464)
at hudson.util.XStream2.unmarshal(XStream2.java:230)
at hudson.util.XStream2.unmarshal(XStream2.java:201)
at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1441)
at hudson.XmlFile.unmarshal(XmlFile.java:196)

Caused: java.io.IOException: Unable to read /opt/jenkins/config.xml
at hudson.XmlFile.unmarshal(XmlFile.java:199)
at hudson.XmlFile.unmarshal(XmlFile.java:179)
at jenkins.model.Jenkins.loadConfig(Jenkins.java:3379)
at jenkins.model.Jenkins$12.run(Jenkins.java:3491)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:177)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1170)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused: org.jvnet.hudson.reactor.ReactorException
at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:290)
at jenkins.InitReactorRunner.run(InitReactorRunner.java:49)
at jenkins.model.Jenkins.executeReactor(Jenkins.java:1205)
at jenkins.model.Jenkins.(Jenkins.java:992)
at hudson.model.Hudson.(Hudson.java:86)
at hudson.model.Hudson.(Hudson.java:82)
at hudson.WebAppMain$3.run(WebAppMain.java:247)
Caused: hudson.util.HudsonFailedToLoad
at hudson.WebAppMain$3.run(WebAppMain.java:264)

I figure I might need to upgrade plugins, but there’s no interface to do this. Is there a step I’m missing?

Cheers,

Brendan

Is /opt/jenkins set as your JENKINS_HOME? Does the /opt/jenkins/config.xml exist?

You should analyze the complete systemd log with
journalctl -u jenkins and look at the very first error. This is the root cause for all following problems in loading the plugins.

1 Like

Thanks Alex and Markus. The first output was from journalctl -u jenkins and the config.xml did exist. In the end I decided to roll back to my snapshot (no time to troubleshoot with developers needing to use the server!) and I will regroup and attempt again.

I notice that quite a few of the plugins that are installed need updates and many of them are saying that they won’t after update due to version dependencies, so I might raise a separate question about how best to handle that - my original post in this one will probably muddy the waters too much.



If you still have the image.
Check permissions on /opt/config/jenkins.xml including if it's a mount point and that mount point is writeable.

Also check that Jenkins is run by user "jenkins and that it's readable Nay writeable by that user


Caused: java.io.IOException: Unable to read /opt/jenkins/config.xml
If you still have the image.
Check the permissions on /opt/je

Caused: java.io.IOException: Unable to read /opt/jenkins/config.xml