JNLP4-connect connection from *** is disconnected

Using Jenkins Ask a question
1

I’m currently using:
Linux 3.10.0-1160.92.1.el7.x86_64 x86_64
Jenkins 2.426.1
Running on a kubernetes environment. The nodes are virtualized windows machines most of them windows 11.
What happens is that during some builds (freestyle projects) the communication channel with the controller breaks and the build is aborted.
This is something we were would see occasionally but lately it have become much more frequent, impacting several nodes in the same day. It appears to be some networking issue, although through ping we have never seen any loss of connectivity at the time these issues happened. The stack is shown at the end of the post.

We have tried tried disabling the ping from the nodes: (-Dhudson.slaves.ChannelPinger.pingTimeoutSeconds=-1, -Dhudson.slaves.ChannelPinger.pingIntervalSeconds=-1) but it didn’t help.

I have seen JIRAs still open for this issue so it appears to be a jenkins bug. But does anyone know of a workaround to prevent this timeout from happening?

Here are the logs:

java.nio.channels.ClosedChannelException
        at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:241)
        at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:221)
        at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:825)
        at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:289)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:177)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:279)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:501)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:244)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:196)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:209)
        at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:793)
        at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:172)
        at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:343)
        at hudson.remoting.Channel.close(Channel.java:1494)
        at hudson.remoting.Channel.close(Channel.java:1447)
        at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:939)
        at hudson.slaves.SlaveComputer.kill(SlaveComputer.java:905)
        at hudson.model.AbstractCIBase.killComputer(AbstractCIBase.java:97)
        at jenkins.model.Jenkins.lambda$_cleanUpDisconnectComputers$11(Jenkins.java:3751)
        at hudson.model.Queue._withLock(Queue.java:1397)
        at hudson.model.Queue.withLock(Queue.java:1271)
        at jenkins.model.Jenkins._cleanUpDisconnectComputers(Jenkins.java:3747)
        at jenkins.model.Jenkins.cleanUp(Jenkins.java:3628)
        at hudson.WebAppMain.contextDestroyed(WebAppMain.java:374)
        at org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:1059)
        at org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:636)
        at org.eclipse.jetty.server.handler.ContextHandler.contextDestroyed(ContextHandler.java:1016)
        at org.eclipse.jetty.servlet.ServletHandler.doStop(ServletHandler.java:306)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:97)
        at org.eclipse.jetty.security.SecurityHandler.doStop(SecurityHandler.java:411)
        at org.eclipse.jetty.security.ConstraintSecurityHandler.doStop(ConstraintSecurityHandler.java:413)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:97)
        at org.eclipse.jetty.server.session.SessionHandler.doStop(SessionHandler.java:497)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:97)
        at org.eclipse.jetty.server.handler.ContextHandler.stopContext(ContextHandler.java:1039)
        at org.eclipse.jetty.servlet.ServletContextHandler.stopContext(ServletContextHandler.java:399)
        at org.eclipse.jetty.webapp.WebAppContext.stopContext(WebAppContext.java:1311)
        at org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:1087)
        at org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:312)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:97)
        at org.eclipse.jetty.server.Server.doStop(Server.java:517)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at winstone.Launcher.shutdown(Launcher.java:442)
        at winstone.ShutdownHook.run(ShutdownHook.java:28)
Caused: java.io.IOException: Backing channel 'JNLP4-connect connection from **************' is disconnected.
        at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
        at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
        at com.sun.proxy.$Proxy92.isAlive(Unknown Source)
        at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1212)
        at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1204)
        at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:195)
        at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:145)
        at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
        at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
        at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
        at hudson.model.Build$BuildExecution.build(Build.java:199)
        at hudson.model.Build$BuildExecution.doRun(Build.java:164)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
        at hudson.model.Run.execute(Run.java:1900)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
        at hudson.model.ResourceController.execute(ResourceController.java:101)
        at hudson.model.Executor.run(Executor.java:442)
FATAL: Unable to delete script file C:\Users\CRUISE~1\AppData\Local\Temp\jenkins5205634626611760213.bat
java.nio.channels.ClosedChannelException
        at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:241)
        at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:221)
        at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:825)
        at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:289)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:177)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:279)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:501)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:244)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:196)
        at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:209)
        at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:793)
        at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:172)
        at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:343)
        at hudson.remoting.Channel.close(Channel.java:1494)
        at hudson.remoting.Channel.close(Channel.java:1447)
        at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:939)
        at hudson.slaves.SlaveComputer.kill(SlaveComputer.java:905)
        at hudson.model.AbstractCIBase.killComputer(AbstractCIBase.java:97)
        at jenkins.model.Jenkins.lambda$_cleanUpDisconnectComputers$11(Jenkins.java:3751)
        at hudson.model.Queue._withLock(Queue.java:1397)
        at hudson.model.Queue.withLock(Queue.java:1271)
        at jenkins.model.Jenkins._cleanUpDisconnectComputers(Jenkins.java:3747)
        at jenkins.model.Jenkins.cleanUp(Jenkins.java:3628)
        at hudson.WebAppMain.contextDestroyed(WebAppMain.java:374)
        at org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:1059)
        at org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:636)
        at org.eclipse.jetty.server.handler.ContextHandler.contextDestroyed(ContextHandler.java:1016)
        at org.eclipse.jetty.servlet.ServletHandler.doStop(ServletHandler.java:306)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:97)
        at org.eclipse.jetty.security.SecurityHandler.doStop(SecurityHandler.java:411)
        at org.eclipse.jetty.security.ConstraintSecurityHandler.doStop(ConstraintSecurityHandler.java:413)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:97)
        at org.eclipse.jetty.server.session.SessionHandler.doStop(SessionHandler.java:497)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:97)
        at org.eclipse.jetty.server.handler.ContextHandler.stopContext(ContextHandler.java:1039)
        at org.eclipse.jetty.servlet.ServletContextHandler.stopContext(ServletContextHandler.java:399)
        at org.eclipse.jetty.webapp.WebAppContext.stopContext(WebAppContext.java:1311)
        at org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:1087)
        at org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:312)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:97)
        at org.eclipse.jetty.server.Server.doStop(Server.java:517)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132)
        at winstone.Launcher.shutdown(Launcher.java:442)
        at winstone.ShutdownHook.run(ShutdownHook.java:28)
Caused: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@7716a117:JNLP4-connect connection from **************": Remote call on JNLP4-connect connection from ************** failed. The channel is closing down or has closed down
        at hudson.remoting.Channel.call(Channel.java:993)
        at hudson.FilePath.act(FilePath.java:1192)
        at hudson.FilePath.act(FilePath.java:1181)
        at hudson.FilePath.delete(FilePath.java:1728)
        at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:163)
        at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
        at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
        at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
        at hudson.model.Build$BuildExecution.build(Build.java:199)
        at hudson.model.Build$BuildExecution.doRun(Build.java:164)
        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
        at hudson.model.Run.execute(Run.java:1900)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
        at hudson.model.ResourceController.execute(ResourceController.java:101)
        at hudson.model.Executor.run(Executor.java:442)
Build step 'Execute Windows batch command' marked build as failure
1 Like
2

Anyone? Exact same problem, tried different JDK, no luck

3

I’ve encountered similar problems with a controller and workers running on kubernetes (GKE), and I’ve seen my share of the hudson.remoting.ChannelClosedException errors. Simply put, this means that the Jenkins controller is no longer able to communicate with the worker/node because it’s been terminated for one reason or another. It’s hard to pinpoint exactly what the issue is from within Jenkins. I’m not sure where your kubernetes cluster is deployed, but have you checked the logs for the workers?

The biggest issue we had was with nodes downscaling (whether due to low usage or because they were ephemeral nodes being reallocated by our cloud service provider).

Again, I’m not sure where your kubernetes cluster(s) are hosted, but it would be good to take a look at the logs and see if they can provide you with better insight into why the nodes are getting terminated. I hope that helps point you in the right direction.