Jenkins controller <---> agent communication with Docker Swarm

Jenkins setup:

  • Jenkins 2.440.2-lts w/ JDK17
  • Agents using JDK11
  • Docker Swarm cluster w/ 3 masters and 6 workers
  • Everything runs in AWS on the same VPC and connectivity has been successfully tested

Please help! My last post was hidden for some reason. We have a pretty big production issue where our controller node(s) are unable to communciate with our worker nodes after having to restore the system after a botched upgrade of the controller to 2.479.1. We had to create 3 new controller nodes and were able to get the controller up and running, but anytime a job is scheduled, it says that any agents are offline. This is the error:

[4:36:23 PM] Creating Service with Name : agt-_portal_PR_69_2-1246
java.net.SocketException: Broken pipe
at java.base/sun.nio.ch.NioSocketImpl.implWrite(Unknown Source)
at java.base/sun.nio.ch.NioSocketImpl.write(Unknown Source)
at java.base/sun.nio.ch.NioSocketImpl$2.write(Unknown Source)
at java.base/java.net.Socket$SocketOutputStream.write(Unknown Source)
at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeChangeCipherSpec(Unknown Source)
at java.base/sun.security.ssl.OutputRecord.changeWriteCiphers(Unknown Source)
at java.base/sun.security.ssl.ChangeCipherSpec$T10ChangeCipherSpecProducer.produce(Unknown Source)
at java.base/sun.security.ssl.Finished$T12FinishedProducer.onProduceFinished(Unknown Source)