ERROR upgrading windows Jenkins from 2.263.4 to 2.277.1

Using Jenkins Ask a question
1

We tried Upgrading Jenkins from v2.222.1 to the latest incrementally, till the v2.263.4 upgrade went smooth later we are facing the following issue on the Windows Jenkins server
java.lang.IllegalStateException: An attempt to save the global configuration was made before it was loaded at jenkins.model.Jenkins.save(Jenkins.java:3499) at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3526) at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2723) at jenkins.model.Jenkins$15.run(Jenkins.java:3461) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1151) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused: org.jvnet.hudson.reactor.ReactorException at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:282) at jenkins.InitReactorRunner.run(InitReactorRunner.java:49) at jenkins.model.Jenkins.executeReactor(Jenkins.java:1184) at jenkins.model.Jenkins.(Jenkins.java:976) at hudson.model.Hudson.(Hudson.java:85) at hudson.model.Hudson.(Hudson.java:81) at hudson.WebAppMain$3.run(WebAppMain.java:298) Caused: hudson.util.HudsonFailedToLoad at hudson.WebAppMain$3.run(WebAppMain.java:315)

Unable to upgrade plugins in the Jenkins Dashboard. we tried disabling a few of the plugins that the error message mentioned was causing the problem, however, the problem persisted after removing/disabling the plugins. we are planning to disable most of the plugins as it seems to be the issue.

2

Hello @MWO_users and welcome to this community :wave:

I have read there that one should downgrade to 2.263.1, then restart the service, upgrade all the plugins, the upgrade Jenkins.

3

@poddingue , We have tried downgrade of the jenkins to v2.277.1 and tried to upgrade plugins but they got failed.

note: We are unable to upgrade plugins, following is the error message while upgrading plugins

sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.provider.certpath.SunCertPathBuilder.build(Unknown Source)
	at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source)
	at java.security.cert.CertPathBuilder.build(Unknown Source)
Caused: sun.security.validator.ValidatorException: PKIX path building failed
	at sun.security.validator.PKIXValidator.doBuild(Unknown Source)
	at sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
	at sun.security.validator.Validator.validate(Unknown Source)
	at sun.security.ssl.X509TrustManagerImpl.validate(Unknown Source)
	at sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
	at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
Caused: javax.net.ssl.SSLHandshakeException
	at sun.security.ssl.Alerts.getSSLException(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
	at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
	at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
	at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown Source)
	at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
	at sun.security.ssl.Handshaker.processLoop(Unknown Source)
	at sun.security.ssl.Handshaker.process_record(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
	at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(Unknown Source)
	at java.net.URLConnection.getHeaderFieldLong(Unknown Source)
	at java.net.URLConnection.getContentLengthLong(Unknown Source)
	at java.net.URLConnection.getContentLength(Unknown Source)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getContentLength(Unknown Source)
	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1264)
Caused: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.reflect.GeneratedConstructorAccessor114.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1280)
Caused: java.io.IOException: Failed to load https://updates.jenkins.io/download/plugins/authentication-tokens/1.4/authentication-tokens.hpi to C:\Program Files (x86)\Jenkins\plugins\authentication-tokens.jpi.tmp
	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1287)
Caused: java.io.IOException: Failed to download from https://updates.jenkins.io/download/plugins/authentication-tokens/1.4/authentication-tokens.hpi
	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1321)
	at hudson.model.UpdateCenter$DownloadJob._run(UpdateCenter.java:1869)
	at hudson.model.UpdateCenter$InstallationJob._run(UpdateCenter.java:2153)
	at hudson.model.UpdateCenter$DownloadJob.run(UpdateCenter.java:1843)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:118)
	at java.lang.Thread.run(Unknown Source)
4

are you running a very old version of java? Lets encrypt has been allowed by java for a few years now.

Otherwise maybe your proxy (if you have one) is returning a self signed certificate?

5

How do we check the version of java that we are running in Jenkins? I’m guessing that you are referring to Java Plugin in Jenkins. We currently don’t have java running on the Windows VM that is housing Jenkins.

6

you absolutely do, jenkins requires java to run

java -version is probably the easiest way, but you’ll also want to look at the service to see if JAVA_HOME is set, or it has a different PATH, etc

7

Thanks and we are currently running Java Se Development Kit 8 update 162 v8.0.1620.12 and are about to install Zulu jdk18.0.2 (we need to uninstall Oracle Java because of a licensing issue). Is this version of currently installed Java a bit old and might the reason for the issue that we’re having upgrading?

8

2.277 is pretty old. (361-277 is 84, so more than a year old)

I highly doubt it’ll be able to use java 18.

2.361.1 (which is the latest stable version) now requires Java 11, and has been somewhat tested in Java 17.

At this point. I suggest you try to upgrade to 2.361.1, and use java 11 or 17 from https://adoptium.net/ as its known to be pretty good.

9

Thank you, We Installed zulu18 and we tried to upgrade plugins first but we are unable to upgrade plugins and getting the below error message

Structs	
 Failure -
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.provider.certpath.SunCertPathBuilder.build(Unknown Source)
	at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source)
	at java.security.cert.CertPathBuilder.build(Unknown Source)
Caused: sun.security.validator.ValidatorException: PKIX path building failed
	at sun.security.validator.PKIXValidator.doBuild(Unknown Source)
	at sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
	at sun.security.validator.Validator.validate(Unknown Source)
	at sun.security.ssl.X509TrustManagerImpl.validate(Unknown Source)
	at sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
	at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
Caused: javax.net.ssl.SSLHandshakeException
	at sun.security.ssl.Alerts.getSSLException(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
	at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
	at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
	at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown Source)
	at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
	at sun.security.ssl.Handshaker.processLoop(Unknown Source)
	at sun.security.ssl.Handshaker.process_record(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
	at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(Unknown Source)
	at java.net.URLConnection.getHeaderFieldLong(Unknown Source)
	at java.net.URLConnection.getContentLengthLong(Unknown Source)
	at java.net.URLConnection.getContentLength(Unknown Source)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getContentLength(Unknown Source)
	at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:1264)
Caused: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
10

@MWO_users ,

First, go back to your 2.263.4 setup, install OpenJdk 8 Temurin latest, (https://adoptium.net/marketplace?version=8), and you may as well also grab and install 11 for later.

Start Jenkins using jdk 8 as JAVA_HOME.

Consult the list of incompatible plugins from the dashboard (Tables to divs - Jenkins Jira) and disable (or remove) any incompatible as indicated in the 2.277.1 upgrade notes (Upgrading to Jenkins LTS 2.277.x ).

Before upgrading Core, upgrade all possible available plugins to latest compatible, restart. Validate.

NB: if you installed Jenkins a via a package manager, like rpm, DO NOT proceed to upgrade using this approach. Upgrade the package using the pacakge manager, as @halkeye answered well in a separate post.

If OK, then upgrade to 2.277.1, restart. Check the logs for any errors, especially relating to plugins on the restart. You may have to disable or remove those.

If you get the PKiX error again, this is well documented; google jenkins PKIX error

If running OK then immediately upgrade all the plugins again, restart and check again.

Repeat: read change log (LTS Changelog) and follow upgrade notes (Jenkins LTS Upgrade Guide) upgrade core to next LTS, restart, check logs, upgrade plugins, restart, validating as you go.

Once you get to 2.346.x successfully, stop, switch JAVA_HOME to OpenJdk 11, restart.

Now you can move to 2.361.x+, which requires Java 11. Then you are done.

Jenkins will not start w/anything other than 8 (up to 2.346), 11 or 17. Check the changelog to see what newer releases were OK"d for 11/17 if you want.

2 Likes