Hey Experts,
Using:
Jenkins Core: 2.426.3
Java Version: 11.0.16.1
Maven: 3.8.4
Recently we had upgraded our Jenkins from 2.303.3 to 2.426.3. The service was up and running until 18+hours until we saw issues with the JNLP connections to Jenkins
SEVERE [Executor #-1 for Built-In Node] hudson.model.Executor.run Executor #-1 for Built-In Node: Unexpected executordeath
SEVERE [Executor #-1 for Built-In Node] hudson.model.Executor.run Executor #-1 for Built-In Node: Unexpected executordeath
After which we also observed Jenkins rejecting further incoming requests
SEVERE [TCP agent listener port=50000] hudson.TcpSlaveAgentListener.run Failed to accept TCP connections
SEVERE [TCP agent listener port=50000] hudson.TcpSlaveAgentListener.run Failed to accept TCP connections
We switched to the inbound-agent image with 2.426.3:
inbound-agent:latest-alpine3.19-jdk11
Initially, we were using the jnlp-agent:3.35-5-alpine with older jenkins version
Is there any suggestions to use the jnlp-agent images instead of inbound-agents.
Given, there is a dependency on the remoting version to be used from 2.426.3 (Needs remoting versions > 4.13), I tried adding the suppression available on the Upgrade Changelogs (Minimum required Remoting version updated to 4.13) to include hudson.slaves.SlaveComputer.allowUnsupportedRemotingVersions in the JAVA_OPTS, which still gives me
SEVERE: Agent version 4.13 or newer is required.
java.io.IOException: Agent version 4.13 or newer is required.
at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:221)
at hudson.remoting.Engine.innerRun(Engine.java:527)
at hudson.remoting.Engine.run(Engine.java:488)
Note:
Adding allowUnsupportedRemotingVersions did work with jnlp-agent:latest-jdk11, jenkins was able to make JNLP connections with remoting version 4.9.
02:47:48 Created Pod: phx-oke-farm jenkins/docker-bench-oci-scan-knrbx-89sz7
02:47:48 [Normal][jenkins/docker-bench-oci-scan-knrbx-89sz7][Scheduled] Successfully assigned jenkins/docker-bench-oci-scan-knrbx-89sz7 to 100.105.153.199
02:47:49 [Normal][jenkins/docker-bench-oci-scan-knrbx-89sz7][Pulling] Pulling image "phx.ocir.io/oraclegbudevcorp/cn-farm-farm-infra/jnlp-slave:latest-jdk11"
02:47:54 Still waiting to schedule task
02:47:54 ‘docker-bench-oci-scan-knrbx-89sz7’ is offline
02:47:56 [Normal][jenkins/docker-bench-oci-scan-knrbx-89sz7][Pulled] Successfully pulled image "phx.ocir.io/oraclegbudevcorp/cn-farm-farm-infra/jnlp-slave:latest-jdk11" in 7.554227019s (7.554242137s including waiting)
02:47:57 [Normal][jenkins/docker-bench-oci-scan-knrbx-89sz7][Created] Created container jnlp
02:47:57 [Normal][jenkins/docker-bench-oci-scan-knrbx-89sz7][Started] Started container jnlp
02:47:57 [Normal][jenkins/docker-bench-oci-scan-knrbx-89sz7][Pulling] Pulling image "phx.ocir.io/oraclegbudevcorp/cn-shared/sdaas/governance/staticscan-docker-bench:latest"
02:47:57 [Normal][jenkins/docker-bench-oci-scan-knrbx-89sz7][Pulled] Successfully pulled image "phx.ocir.io/oraclegbudevcorp/cn-shared/sdaas/governance/staticscan-docker-bench:latest" in 208.589372ms (208.600423ms including waiting)
02:47:57 [Normal][jenkins/docker-bench-oci-scan-knrbx-89sz7][Created] Created container docker-bench
02:47:57 [Normal][jenkins/docker-bench-oci-scan-knrbx-89sz7][Started] Started container docker-bench
02:48:03 Agent docker-bench-oci-scan-knrbx-89sz7 is provisioned from template docker-bench-oci-scan-knrbx
02:48:03 ---
02:48:03 apiVersion: "v1"
02:48:03 kind: "Pod"
02:48:03 metadata:
02:48:03 annotations:
02:48:03 buildUrl: "https://hostname:8484/jenkins/job/SDAAS/job/SDAAS_DockerBenchOciScan_generated/295927/"
02:48:03 runUrl: "job/SDAAS/job/SDAAS_DockerBenchOciScan_generated/295927/"
02:48:03 labels:
02:48:03 jenkins: "slave"
02:48:03 jenkins/label-digest: "28455f696d97bde773cc7cc34ac6d7de69c5c531"
02:48:03 jenkins/label: "sdaas-docker-bench-scan-pod"
02:48:03 name: "docker-bench-oci-scan-knrbx-89sz7"
02:48:03 namespace: "jenkins"
02:48:03 spec:
02:48:03 containers:
02:48:03 - env:
02:48:03 - name: "JENKINS_PROTOCOLS"
02:48:03 value: "JNLP4-connect"
02:48:03 - name: "JENKINS_SECRET"
02:48:03 value: "********"
02:48:03 - name: "JENKINS_AGENT_NAME"
02:48:03 value: "docker-bench-oci-scan-knrbx-89sz7"
02:48:03 - name: "JENKINS_DIRECT_CONNECTION"
02:48:03 value: "host:50000"
02:48:03 - name: "JENKINS_INSTANCE_IDENTITY"
02:48:03 value: "some_value"
02:48:03 - name: "JENKINS_NAME"
02:48:03 value: "docker-bench-oci-scan-knrbx-89sz7"
02:48:03 - name: "JENKINS_AGENT_WORKDIR"
02:48:03 value: "/home/jenkins/agent"
02:48:03 image: "phx.ocir.io/oraclegbudevcorp/cn-farm-farm-infra/jnlp-slave:latest-jdk11"
02:48:03 imagePullPolicy: "Always"
02:48:03 name: "jnlp"
02:48:03 resources:
02:48:03 limits: {}
02:48:03 requests: {}
02:48:03 tty: true
02:48:03 volumeMounts:
02:48:03 - mountPath: "/home/jenkins/agent"
02:48:03 name: "workspace-volume"
02:48:03 readOnly: false
02:48:03 workingDir: "/home/jenkins/agent"
02:48:03 - args:
02:48:03 - "cat"
02:48:03 image: "phx.ocir.io/oraclegbudevcorp/cn-shared/sdaas/governance/staticscan-docker-bench:latest"
02:48:03 imagePullPolicy: "Always"
02:48:03 name: "docker-bench"
02:48:03 resources:
02:48:03 limits: {}
02:48:03 requests: {}
02:48:03 tty: true
02:48:03 volumeMounts:
02:48:03 - mountPath: "/home/jenkins/agent"
02:48:03 name: "workspace-volume"
02:48:03 readOnly: false
02:48:03 workingDir: "/home/jenkins/agent"
02:48:03 dnsConfig:
02:48:03 options:
02:48:03 - name: "ndots"
02:48:03 value: "2"
02:48:03 hostNetwork: false
02:48:03 nodeSelector:
02:48:03 runner_type: "governance"
02:48:03 restartPolicy: "Never"
02:48:03 serviceAccountName: "jenkins-user"
02:48:03 volumes:
02:48:03 - emptyDir:
02:48:03 medium: ""
02:48:03 name: "workspace-volume"
02:48:03
02:48:04 Running on docker-bench-oci-scan-knrbx-89sz7 in /home/jenkins/agent/workspace/SDAAS/SDAAS_DockerBenchOciScan_generated
[Pipeline] {
[Pipeline] container
[Pipeline] {
[Pipeline] dir
02:48:04 Running in /home/jenkins/agent/workspace/SDAAS/SDAAS_DockerBenchOciScan_generated/dockerBenchScan_295927
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Scan)
[Pipeline] sh
02:48:12 + mkdir -p /home/jenkins/agent/reports
[Pipeline] sh
02:48:15 + set +x
...
Other part of the Code
But, the same-image fails on unsupported remoting version today (not sure what changed).
Is there any other alternate image that is compatible with the Jenkins to achieve seamless JNLP connections? We don’t see any issue with the current jnlp-agent:3.35-5-alpine image with 2.303.3.
The inbound-agent image somehow is not able to handle the load, and piles up huge number of jobs in queue trying to serve requests.
Please let me know if there is a way to fix this.
