Multiple git related errors when updating Jenkins

Using Jenkins Ask a question
1

We are building our Jenkins server from docker.io/jenkins/jenkins:jdk17@sha256:75c908c34bf45fcf50cae779155262fa6a2d85dfd7c14b5d6b5e664521c766bf

Installing the following plugins using the CLI tool:

amazon-ecs
artifactory
atlassian-jira-software-cloud
aws-lambda
aws-sam
basic-branch-build-strategies
bitbucket-approval-filter
bitbucket-build-status-notifier
bitbucket-scm-trait-commit-skip
bitbucket-push-and-pull-request
blueocean
blueocean-jira
blueocean-executor-info
build-history-metrics-plugin
build-timeout
build-timestamp
buildtriggerbadge
command-launcher
datadog
docker-workflow
docker-java-api
docker-compose-build-step
emailext-template
envinject
extended-read-permission
external-monitor-job
extra-columns
font-awesome-api
generic-webhook-trigger
greenballs
jackson2-api
job-import-plugin
jqs-monitoring
kiuwanJenkinsPlugin
ldap
linenumbers
metrics
nodejs
Office-365-Connector
pam-auth
workflow-aggregator
pipeline-utility-steps
pipeline-aws
role-strategy
saml
sonar
snyk-security-scanner
startup-trigger-plugin
thinBackup
timestamper
warnings-ng
windows-slaves
ws-cleanup

Updating from 2.431 to 2.445. After updating we are running into issues being able to pull from git.

When we try to configure a pipeline (not multibranch) and add the repo we get
An internal error occurred during form field validation (HTTP 500). Please reload the page and if the problem persists, ask the administrator for help.
in the UI and

Error while serving REMOEDURL/job/jenkins-testing/descriptorByName/hudson.plugins.git.UserRemoteConfig/checkUrl
java.lang.NullPointerException: Cannot invoke "java.util.Map.size()" because "map" is null
	at java.base/java.util.TreeMap.putAll(TreeMap.java:314)
	at hudson.slaves.EnvironmentVariablesNodeProperty.buildEnvVars(EnvironmentVariablesNodeProperty.java:87)
	at hudson.model.Computer.buildEnvironment(Computer.java:1225)
	at hudson.model.Job.getEnvironment(Job.java:392)
	at hudson.plugins.git.UserRemoteConfig$DescriptorImpl.doCheckUrl(UserRemoteConfig.java:193)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:732)
	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:397)
Caused: java.lang.reflect.InvocationTargetException
	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:401)
	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:409)
	at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:78)
	at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:207)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:140)
	at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:558)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900)
	at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:289)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900)
	at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:289)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:698)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:248)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)
	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1665)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:163)
	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:248)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:60)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
	at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
	at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:64)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
	at jenkins.util.HttpServletFilter$1.doFilter(HttpServletFilter.java:76)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:166)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
	at jenkins.ErrorAttributeFilter.doFilter(ErrorAttributeFilter.java:29)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:154)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:94)
	at jenkins.security.AcegiSecurityExceptionFilter.doFilter(AcegiSecurityExceptionFilter.java:52)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:54)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:110)
	at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:101)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:227)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:221)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:97)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:117)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:63)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:111)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:172)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:53)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:86)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
	at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:38)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:569)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1553)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
	at org.eclipse.jetty.server.Server.handle(Server.java:563)
	at org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
	at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
	at java.base/java.lang.Thread.run(Thread.java:840

In the logs

If I try to do a multibranch pipeline configured with Bitbucket I am not able to see any new logs relevant to that, i don’t get any errors in the console but when it tries to discover branches it just sits there. Nothing is ever printed

We have also noticed the following 2 logs over and over again in the logs:


Unexpected node monitoring termination: Free Disk Space
java.lang.ClassCastException: class java.lang.Integer cannot be cast to class hudson.slaves.NodeProperty (java.lang.Integer is in module java.base of loader 'bootstrap'; hudson.slaves.NodeProperty is in unnamed module of loader org.eclipse.jetty.webapp.WebAppClassLoader @6d4d66d2)
	at hudson.model.Node.getNodeProperty(Node.java:508)
	at hudson.node_monitors.DiskSpaceMonitor.getThresholdBytes(DiskSpaceMonitor.java:60)
	at hudson.node_monitors.DiskSpaceMonitorDescriptor.markNodeOfflineOrOnline(DiskSpaceMonitorDescriptor.java:80)
	at hudson.node_monitors.DiskSpaceMonitorDescriptor.monitor(DiskSpaceMonitorDescriptor.java:72)
	at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:324)


java.lang.ClassCastException: class java.lang.Integer cannot be cast to class hudson.slaves.NodeProperty (java.lang.Integer is in module java.base of loader 'bootstrap'; hudson.slaves.NodeProperty is in unnamed module of loader org.eclipse.jetty.webapp.WebAppClassLoader @6d4d66d2)
	at hudson.model.Node.canTake(Node.java:431)
	at hudson.model.Queue.makeFlyWeightTaskBuildable(Queue.java:1748)
	at hudson.model.Queue.makeBuildable(Queue.java:1710)
	at hudson.model.Queue.maintain(Queue.java:1579)
	at hudson.model.Queue$1.call(Queue.java:332)
	at hudson.model.Queue$1.call(Queue.java:329)
	at jenkins.util.AtmostOneTaskExecutor$1.call(AtmostOneTaskExecutor.java:109)
	at jenkins.util.AtmostOneTaskExecutor$1.call(AtmostOneTaskExecutor.java:99)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:121)
	at java.base/java.lang.Thread.run(Thread.java:840)

We noticed 2 other weird things. First is that on a bunch of jobs, several of the recent job runs had their run date changed too Jan 1, 1940, 12:00AM

The second is that when we try to run one of the jobs affected by the above, it seems to never even try to run. I don’t see a failed run pop up or even that it sitting waiting to run. Just nothing.

2

Is that problem reproducible after a Jenkins restart? It looks like there is a node property in your agent that is just an integer which should be impossible.

3

Yeah it is, I tried restarting and also tried doing an upgrade again before posting this (to the version noted in the post) and the issue persisted.

Something worth noting is we don’t have any long running agents. The main server does have one executer that is never used instead all of the nodes are created using Amazon Elastic Container Service (ECS) / Fargate and these errors are being thrown while we don’t have any exeucuters.

I was able to confirm however that it will properly create an executer when I do a freestyle project, restrict it to the right node name so it spins one up, and I just don’t try to add a git repo.

I am not sure why it would be complaining about a node that doesnt exist?

4

So that did give me the idea to remove the executer on the head node. I am no longer seeing errors in the logs, I am not sure if this happened because of this or if it quited down at some point for some reason. I am not sure.

I could try having it spin up a node and see if those errors come back.

I am not seeing either of the above errors now.

The error I am seeing now (since I can narrow it down properly) is the following:

 Error while serving https://REMOVEURL/job/jenkins-testing/descriptorByName/hudson.plugins.git.UserRemoteConfig/checkUrl
 java.lang.NullPointerException: Cannot invoke "java.util.Map.size()" because "map" is null
 	at java.base/java.util.TreeMap.putAll(TreeMap.java:314)
 	at hudson.slaves.EnvironmentVariablesNodeProperty.buildEnvVars(EnvironmentVariablesNodeProperty.java:87)
 	at hudson.model.Computer.buildEnvironment(Computer.java:1225)
 	at hudson.model.Job.getEnvironment(Job.java:392)
 	at hudson.plugins.git.UserRemoteConfig$DescriptorImpl.doCheckUrl(UserRemoteConfig.java:193)
 	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:732)
 	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:397)
 Caused: java.lang.reflect.InvocationTargetException
 	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:401)
 	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:409)
 	at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:78)
 	at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26)
 	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:207)
 	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:140)
 	at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:558)
 	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
 	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770)
 	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900)
 	at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:289)
 	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
 	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770)
 	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900)
 	at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:289)
 	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
 	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770)
 	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900)
 	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:698)
 	at org.kohsuke.stapler.Stapler.service(Stapler.java:248)
 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
 	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)
 	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1665)
 	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:163)
 	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:248)
 	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
 	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:60)
 	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
 	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
 	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
 	at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125)
 	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
 	at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:64)
 	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
 	at jenkins.util.HttpServletFilter$1.doFilter(HttpServletFilter.java:76)
 	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:160)
 	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:166)
 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
 	at jenkins.ErrorAttributeFilter.doFilter(ErrorAttributeFilter.java:29)
 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
 	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:154)
 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
 	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:94)
 	at jenkins.security.AcegiSecurityExceptionFilter.doFilter(AcegiSecurityExceptionFilter.java:52)
 	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
 	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:54)
 	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
 	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)
 	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)
 	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
 	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100)
 	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
 	at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:110)
 	at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:101)
 	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
 	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:227)
 	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:221)
 	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
 	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:97)
 	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
 	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:117)
 	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
 	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:63)
 	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
 	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:111)
 	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:172)
 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
 	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:53)
 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
 	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:86)
 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
 	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
 	at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:38)
 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
 	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:569)
 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)
 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580)
 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)
 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1553)
 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306)
 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
 	at org.eclipse.jetty.server.Server.handle(Server.java:563)
 	at org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598)
 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753)
 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501)
 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287)
 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)
 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
 	at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
 	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)
 	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)
 	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)
 	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)
 	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)
 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
 	at java.base/java.lang.Thread.run(Thread.java:840)

This happens on a pipeline (not mutlibranch, but that gives similar problems of not beinig able to find branches) under “Pipeline script from SCM” >> “SCM” set to Git >> “Repositories” >> “Repository URL”
When I put in the URL I get “An internal error occurred during form field validation (HTTP 500). Please reload the page and if the problem persists, ask the administrator for help.” below it

I can save, but then the build won’t run.

5

you might have a corrupted config.xml that leads to missing injection of variables. Check the config.xml for the nodeProperties tag if that looks ok.

6

So I can’t find an example of what either of these should look like, I can only find examples of the config.xml per job. Not jenkins itself config one

But looking at the file I found the following:

<nodeProperties>
    <hudson.slaves.EnvironmentVariablesNodeProperty/>
    <int>1</int>
    <string>MAVEN_OPTS</string>
    <string></string>
  </nodeProperties>
  <globalNodeProperties>
    <hudson.slaves.EnvironmentVariablesNodeProperty>
      <envVars serialization="custom">
        <unserializable-parents/>
        <tree-map>
          <default>
            <comparator class="java.lang.String$CaseInsensitiveComparator"/>
          </default>
          <int>2</int>
          <string>ECR</string>
          <string>REMOVEDACCOUNTID.dkr.ecr.us-east-1.amazonaws.com</string>
          <string>JenkinsEnvironment</string>
          <string>prod</string>
        </tree-map>
      </envVars>
    </hudson.slaves.EnvironmentVariablesNodeProperty>
  </globalNodeProperties>

I don’t want to try to just start deleting things on my own since I don’t want to end up with jenkins not being able to start and not even being able to get to this file since it is sitting within a container (at least not easily)

Is <int>1</int> just the part that I need to delete?

Edit:
So I compared this to the Jenkins instance that is still working (this is our test jenkins where we test updates before. rolling them out)
That one just has <nodeProperties/> so nothing within that key
That is on a slightly older version, 2.431.

Still not quite sure what I should delete or not? I don’t want to risk this jenkins not being accessible at all and no container to go into since it just keeps failing to start if I mess up this file

7

yes I meant Jenkins config.xml in JENKINS_HOME
the nodeProperties are wrong. Basically this will try to set 4 node properties, but only the first will create a nodeproperty. The others will be instantiated as int and string and when Jenkins loops over them it fails with error you saw in the logs when your controller still had 1 executor. And the EnvironmentVariablesNodeProperty is not properly initialized which leads to the error you still see.

Assuming you want to set MAVEN_OPTS in environment to the empty string this should look like this:

  <nodeProperties>
    <hudson.slaves.EnvironmentVariablesNodeProperty>
      <envVars serialization="custom">
        <unserializable-parents/>
        <tree-map>
          <default>
            <comparator class="java.lang.String$CaseInsensitiveComparator" reference="../../../../../../views/listView/jobNames/comparator"/>
          </default>
          <int>1</int>
          <string>MAVEN_OPTS</string>
          <string></string>
        </tree-map>
      </envVars>
    </hudson.slaves.EnvironmentVariablesNodeProperty>
  </nodeProperties>
8

But as you don’t run anything on the controller you can also just set it to <nodeProperties/>

9

Thank you very much, that worked perfectly

I just set it to <nodeProperties/>
Which I would think even if I needed that setup, doing that is the safest and then just reconfigure the node properties in the gui to avoid a typo.

I am trying to figure out how this might have happened before we roll this upgrade to the main jenkins. We did have an issue where we had a deployment that was missing all of its plugins due to a problem with the plugin installer cli tool.

Could that have caused this or something else?