Connection problem: secondary (agent) J. cannot bind to primary J

Dear community,

we’ve connection exceptions from secondary J. (agent) to master J., after migrating our primary J. to 2.334 and using the agent.jar on the secondary J.

The secondary J. uses

java -jar agent.jar -jnlpUrl "http://FQDN_of_our_Primary_J:8080/computer/jenkin-name-bla-bla-02/slave-agent.jnlp" -secret some_alpha_numeric_stuff_as_secret -workDir D:\Jenkins\

The primary J. → “Manage Jenkins”/“Configure Global Security”/“TCP port for inbound agents” →
a) a fixed value was already there configured, but hasn’t worked anymore
b) have set it to 50000, restarted both J’s, no changes, same exceptions
c) have set it to 50001, restarted both J’s, no changes, same exceptions
d) have set it to “Random”, restarted both J’s, no changes, same exceptions

Our secondary J. tries to connect to the primary J. and throws this exception:

Feb. 15, 2022 10:08:28 VORM. hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: jenkin-name-bla-bla-02
Feb. 15, 2022 10:08:28 VORM. hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Feb. 15, 2022 10:08:28 VORM. hudson.remoting.Engine startEngine
INFO: Using Remoting version: 4.11.2
Feb. 15, 2022 10:08:28 VORM. org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
INFO: Using D:\Jenkins\remoting as a remoting work directory
Feb. 15, 2022 10:08:28 VORM. hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://FQDN_of_our_Primary_J/]
Feb. 15, 2022 10:08:29 VORM. hudson.remoting.jnlp.Main$CuiListener error
SEVERE: Failed to connect to http://FQDN_of_our_Primary_J/tcpSlaveAgentListener/: Connection refused: connect
java.io.IOException: Failed to connect to http://FQDN_of_our_Primary_J/tcpSlaveAgentListener/: Connection refused: connect
	at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:214)
	at hudson.remoting.Engine.innerRun(Engine.java:724)
	at hudson.remoting.Engine.run(Engine.java:540)
Caused by: java.net.ConnectException: Connection refused: connect
	at java.base/java.net.PlainSocketImpl.waitForConnect(Native Method)
	at java.base/java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107)
	at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
	at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
	at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
	at java.base/java.net.Socket.connect(Socket.java:609)
	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
	at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1253)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1015)
	at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:211)
	... 2 more

Our primary J. throws the following exception:

Feb. 15, 2022 10:08:28.099+0000 [id=74300]	WARNING	jenkins.model.Jenkins#launchTcpSlaveAgentListener: Failed to listen to incoming agent connections through port 8080. Change the port number
java.net.BindException: Address already in use: bind
	at java.base/sun.nio.ch.Net.bind0(Native Method)
	at java.base/sun.nio.ch.Net.bind(Net.java:459)
	at java.base/sun.nio.ch.Net.bind(Net.java:448)
	at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
	at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
	at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:73)
	at hudson.TcpSlaveAgentListener.<init>(TcpSlaveAgentListener.java:97)
Caused: java.net.BindException: Failed to listen on port 8080 because it's already in use.
	at hudson.TcpSlaveAgentListener.<init>(TcpSlaveAgentListener.java:99)
	at jenkins.model.Jenkins.launchTcpSlaveAgentListener(Jenkins.java:1324)
	at jenkins.model.Jenkins.forceSetSlaveAgentPort(Jenkins.java:1233)
	at jenkins.model.Jenkins.setSlaveAgentPort(Jenkins.java:1227)
	at hudson.security.GlobalSecurityConfiguration.configure(GlobalSecurityConfiguration.java:132)
	at hudson.security.GlobalSecurityConfiguration.doConfigure(GlobalSecurityConfiguration.java:105)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710)
	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:398)
	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:410)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:208)
	at org.kohsuke.stapler.SelectionInterceptedFunction$Adapter.invoke(SelectionInterceptedFunction.java:36)
	at org.kohsuke.stapler.verb.HttpVerbInterceptor.invoke(HttpVerbInterceptor.java:48)
	at org.kohsuke.stapler.SelectionInterceptedFunction.bindAndInvoke(SelectionInterceptedFunction.java:26)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:141)
	at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:558)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:59)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
	at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:475)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:766)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:898)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:694)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:240)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:157)
	at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:129)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
	at jenkins.security.ResourceDomainFilter.doFilter(ResourceDomainFilter.java:81)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
	at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:64)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:160)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:154)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:94)
	at jenkins.security.AcegiSecurityExceptionFilter.doFilter(AcegiSecurityExceptionFilter.java:52)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:54)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:102)
	at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:219)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:213)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:97)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:63)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:99)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:111)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:172)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:53)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:86)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:38)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:516)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
	at java.base/java.lang.Thread.run(Thread.java:829)
2022-02-14 11:03:00.521+0000 [id=75124]	INFO	................................................................

Have checkt the HTTP-Headers to the primary J., couldn’t find the Header “X-Jenkins-JNLP-Port” (just to see, which value is used).

Any idea, what goes wrong after the migration to 2.334 ?

I appreciate your support.

Best regards
D.

Same behaviour in 2.335 and 2.336.