Jenkins cli handshake error

Using Jenkins Ask a question
1

I have multiple jenkins Server running but I cannot access the via Jenkins CLI
The jenkins Server is working OK, I downloaded the jenkins CLI jar to my windows host and and executed the executed the command to tun in as shown in jenkins. But I got this error

PS D:\Bender\jenkins-cli> java -jar jenkins-cli.jar -s https://abmrndplje002.pass-consulting.com/ help
io.jenkins.cli.shaded.jakarta.websocket.DeploymentException: Handshake error.
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager$3$1.run(ClientManager.java:658)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager$3.run(ClientManager.java:696)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager$SameThreadExecutorService.execute(ClientManager.java:849)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:123)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager.connectToServer(ClientManager.java:493)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager.connectToServer(ClientManager.java:337)
        at hudson.cli.CLI.webSocketConnection(CLI.java:364)
        at hudson.cli.CLI._main(CLI.java:324)
        at hudson.cli.CLI.main(CLI.java:103)
Caused by: io.jenkins.cli.shaded.org.glassfish.tyrus.core.HandshakeException: Response code was not 101: 400.
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.TyrusClientEngine.processResponse(TyrusClientEngine.java:301)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.ClientFilter.processRead(ClientFilter.java:167)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:111)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:113)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.SslFilter.handleRead(SslFilter.java:402)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.SslFilter.processRead(SslFilter.java:365)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:111)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.Filter.onRead(Filter.java:113)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$4.completed(TransportFilter.java:295)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.TransportFilter$4.completed(TransportFilter.java:279)
        at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:129)
        at java.base/sun.nio.ch.Invoker$2.run(Invoker.java:221)
        at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:113)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)

The jenkins server is running behind an apache running as reverse proxy


ProxyRequests     Off
ProxyPreserveHost On
AllowEncodedSlashes NoDecode


<VirtualHost *:80>
  Redirect permanent / https://abmrndplje002.pass-consulting.com/
</VirtualHost>


<VirtualHost *:443>

    SSLEngine on

    SSLCertificateFile      "/etc/pki/tls/certs/STAR_pass-consulting_com.crt"
    SSLCertificateKeyFile   "/etc/pki/tls/private/STAR_pass-consulting_com.key"
    SSLCertificateChainFile "/etc/pki/tls/certs/STAR_pass-consulting_com.ca-bundle.crt"
    SSLCACertificatePath    "/etc/pki/tls/certs"

    ProxyPass / http://localhost:8080/   nocanon
    ProxyPassReverse /  http://localhost:8080/
    ProxyPassReverse /  http://abmrndplje002.pass-consulting.com/

    RequestHeader set X-Forwarded-Proto "https"
    RequestHeader set X-Forwarded-Port "443"

    SSLProtocol All -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
    SSLCipherSuite HIGH:!3DES:!aNULL:!MD5:!SEED:!IDEA:!RC4:!EXPORT:!PSK:!SHA

</VirtualHost>

How can I determine what is the problem and how to fix it?

Jenkins setup:

Jenkins: 2.414.1
OS: Linux - 3.10.0-1160.95.1.el7.x86_64
Java: 11.0.20 - Red Hat, Inc. (OpenJDK 64-Bit Server VM)

active-directory:2.33
analysis-model-api:11.6.0
ansible:253.v4fe719ffdd8a_
ansicolor:1.0.3
ant:497.v94e7d9fffa_b_9
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
authentication-tokens:1.53.v1c90fd9191a_b_
blueocean:1.27.6
blueocean-autofavorite:1.2.5
blueocean-bitbucket-pipeline:1.27.6
blueocean-commons:1.27.6
blueocean-config:1.27.6
blueocean-core-js:1.27.6
blueocean-dashboard:1.27.6
blueocean-display-url:2.4.2
blueocean-events:1.27.6
blueocean-git-pipeline:1.27.6
blueocean-github-pipeline:1.27.6
blueocean-i18n:1.27.6
blueocean-jira:1.27.6
blueocean-jwt:1.27.6
blueocean-personalization:1.27.6
blueocean-pipeline-api-impl:1.27.6
blueocean-pipeline-editor:1.27.6
blueocean-pipeline-scm-api:1.27.6
blueocean-rest:1.27.6
blueocean-rest-impl:1.27.6
blueocean-web:1.27.6
bootstrap5-api:5.3.0-1
bouncycastle-api:2.29
branch-api:2.1122.v09cb_8ea_8a_724
build-timeout:1.31
build-user-vars-plugin:1.9
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.0
cloudbees-bitbucket-branch-source:832.v43175a_425ea_6
cloudbees-folder:6.848.ve3b_fd7839a_81
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.10.0-68.v0d0b_c439292b_
conditional-buildstep:1.4.3
config-file-provider:953.v0432a_802e4d2
copyartifact:722.v0662a_9b_e22a_c
credentials:1271.v54b_1c2c6388a_
credentials-binding:631.v861c06d062b_4
data-tables-api:1.13.5-1
declarative-pipeline-migration-assistant:1.6.1
declarative-pipeline-migration-assistant-api:1.6.1
display-url-api:2.3.9
docker-commons:439.va_3cb_0a_6a_fb_29
docker-workflow:572.v950f58993843
durable-task:523.va_a_22cf15d5e0
echarts-api:5.4.0-5
email-ext:2.100
embeddable-build-status:412.v09da_db_1dee68
extensible-choice-parameter:1.8.1
external-monitor-job:207.v98a_a_37a_85525
favorite:2.4.3
font-awesome-api:6.4.0-2
forensics-api:2.3.0
generic-webhook-trigger:1.87.0
git:5.2.0
git-client:4.4.0
git-parameter:0.9.19
git-server:99.va_0826a_b_cdfa_d
github:1.37.3
github-api:1.314-431.v78d72a_3fe4c3
github-branch-source:1732.v3f1889a_c475b_
gitlab-api:5.3.0-91.v1f9a_fda_d654f
gitlab-branch-source:671.v67b_7169092ca_
gitlab-plugin:1.7.15
gradle:2.8.2
h2-api:11.1.4.199-12.v9f4244395f7a_
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
htmlpublisher:1.32
instance-identity:173.va_37c494ec4e5
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.15.2-350.v0c2f3f8fc595
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javadoc:243.vb_b_503b_b_45537
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.8-1
jdk-tool:73.vddf737284550
jenkins-design-language:1.27.6
jersey2-api:2.40-1
jira:3.10
jjwt-api:0.11.5-77.v646c772fddb_0
jnr-posix-api:3.1.17-1
job-import-plugin:3.6
jquery:1.12.4-1
jquery3-api:3.7.0-1
jsch:0.2.8-65.v052c39de79b_2
junit:1217.v4297208a_a_b_ce
ldap:694.vc02a_69c9787f
lockable-resources:1185.v0c528656ce04
mailer:463.vedf8358e006b_
mapdb-api:1.0.9-28.vf251ce40855d
matrix-auth:3.2
matrix-project:808.v5a_b_5f56d6966
maven-artifact-choicelistprovider:1.14
maven-plugin:3.23
mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_
mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_
monitoring:1.95.0
okhttp-api:4.11.0-157.v6852a_a_fa_ec11
openJDK-native-plugin:1.7
pam-auth:1.10
parameterized-trigger:2.46
pipeline-build-step:505.v5f0844d8d126
pipeline-github-lib:42.v0739460cda_c4
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:685.v8ee9ed91d574
pipeline-input-step:477.v339683a_8d55e
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2144.v077a_d1928a_40
pipeline-model-definition:2.2144.v077a_d1928a_40
pipeline-model-extensions:2.2144.v077a_d1928a_40
pipeline-multibranch-defaults:2.1
pipeline-rest-api:2.33
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2144.v077a_d1928a_40
pipeline-stage-view:2.33
pipeline-utility-steps:2.16.0
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.3.0
prism-api:1.29.0-7
publish-over:0.22
pubsub-light:1.17
repository-connector:2.2.1
resource-disposer:0.23
rest-list-parameter:1.7.0
run-condition:1.7
saml:4.429.v9a_781a_61f1da_
scm-api:676.v886669a_199a_a_
script-security:1275.v23895f409fb_d
select2-api:4.0.13-8
snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4
sonar:2.15
sse-gateway:1.26
ssh-credentials:308.ve4497b_ccd8f4
ssh-slaves:2.916.vd17b_43357ce4
ssh-steps:2.0.65.vd26b_5b_9b_de4d
sshd:3.312.v1c601b_c83b_0e
structs:325.vcb_307d2a_2782
subversion:2.17.3
throttle-concurrents:2.14
timestamper:1.26
token-macro:384.vf35b_f26814ec
trilead-api:2.84.v72119de229b_7
variant:59.vf075fe829ccb
warnings-ng:10.4.0
workflow-aggregator:596.v8c21c963d92d
workflow-api:1267.vd9b_a_ddd9eb_47
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3774.v4a_d648d409ce
workflow-durable-task-step:1289.v4d3e7b_01546b_
workflow-job:1342.v046651d5b_dfe
workflow-multibranch:756.v891d88f2cd46
workflow-scm-step:415.v434365564324
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:848.v5a_383b_d14921
ws-cleanup:0.45

2

I checked the apache access log and found the following entries

10.9.0.64 - - [04/Sep/2023:11:49:52 +0200] "GET /cli/ws HTTP/1.1" 400 519 "-" "-"
10.9.0.64 - - [04/Sep/2023:11:50:16 +0200] "GET /cli/ws HTTP/1.1" 400 519 "-" "-"

and wehen I tried to access this via curl I got
the error Error 400 only WS connections accepted here


[grafra1969@jenkins-rnd etc]$ curl -v https://abmrndplje002.pass-consulting.com/cli/ws
* About to connect() to abmrndplje002.pass-consulting.com port 443 (#0)
*   Trying 10.0.5.8...
* Connected to abmrndplje002.pass-consulting.com (10.0.5.8) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
*       subject: CN=*.pass-consulting.com
*       start date: Oct 05 00:00:00 2022 GMT
*       expire date: Nov 05 23:59:59 2023 GMT
*       common name: *.pass-consulting.com
*       issuer: CN=Sectigo RSA Domain Validation Secure Server CA,O=Sectigo Limited,L=Salford,ST=Greater Manchester,C=GB
> GET /cli/ws HTTP/1.1
> User-Agent: curl/7.29.0
> Host: abmrndplje002.pass-consulting.com
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Date: Mon, 04 Sep 2023 09:53:22 GMT
< Server: Jetty(10.0.15)
< X-Content-Type-Options: nosniff
< Cache-Control: must-revalidate,no-cache,no-store
< Content-Type: text/html;charset=iso-8859-1
< Content-Length: 519
< Connection: close
<
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 400 only WS connections accepted here</title>
</head>
<body><h2>HTTP ERROR 400 only WS connections accepted here</h2>
<table>
<tr><th>URI:</th><td>/cli/ws</td></tr>
<tr><th>STATUS:</th><td>400</td></tr>
<tr><th>MESSAGE:</th><td>only WS connections accepted here</td></tr>
<tr><th>SERVLET:</th><td>Stapler</td></tr>
</table>
<hr/><a href="https://eclipse.org/jetty">Powered by Jetty:// 10.0.15</a><hr/>

</body>
</html>
* Closing connection 0
3

I have a similar error while using Jenkins CLI. Did you get a fix for it?

4

Hello up to now no reaction

5

Tried some more experiments this time with authentication.

When I try with

java -jar jenkins-cli.jar -webSocket -s https://server.pass-consulting.com/ -auth @auth.txt help

I got the above error

When I try using http -http instead of -webSocket there is no error but also nit the expected result, just an empt line

6

I have the exact same problem and error message with Jenkins 2.431 after our production environment moved from a virtual machine to a new environment. This new environment is probably Kubernetes.(I have to ask our sysadmins)

My guess is, that this problem is caused by some “network magic” like redirections, port mappings etc., i.e. the Jenkins CLI has no “direct access” to the running Jenkins instance.

@grafra-pass , did you try with a Jenkins instance not running behind apache?

7

@ricfos No my jenkins server are all running behind an apache being used as a reverse-proxy

8

I checked again, and jenkins tells me my reverse proxy is broken.
However when checking what is recommended as fix it seems everything is according to spec.

9

Did some research and found this:

After taking a closer look to the stack trace and the source code involved (class CLI line 364) I found, that Jenkins CLI starts with http protocol from the URL given in command line, but switches to websocket protocol. So the above stacktrace shows that the handshake during the switch from http to ws protocol (see Wikipedia article for Websocket) fails.

Using websockets with a reverse proxy seems possible if the proxy configuration is extended to handle websockets, e.g. look at this example how to configure Apache for websockets

1 Like
10

Sounds good,#
I will give it a try

11

Just found this:

The official Jenkins documentation for using Jenkins with Apache as reverse proxy states, that it’s not possible to use CLI. At the bottom of the page they write:

Using the plain CLI protocol with the HTTP(S) transport to access Jenkins through an Apache reverse proxy does not work. See JENKINS-47279 - “Full-duplex HTTP(S) transport with plain CLI protocol does not work with Apache reverse proxy” for more details. As a workaround, you can use the CLI over SSH.

1 Like
12

OK

Adding the upgrade=websocket to ProxPASS directive did not help.

When triying to access the server via jenkins-cli I got a different error.
It says “Timeout for the semaphore”

Also i cann no longer reache the jenkins server


 D:\Bender\jenkins-cli>java -jar jenkins-cli.jar  -webSocket  -s https://abmbubdlje001.pass-consulting.com:8080 -auth @auth.txt help
io.jenkins.cli.shaded.jakarta.websocket.DeploymentException: Connection failed.
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.JdkClientContainer$1.call(JdkClientContainer.java:187)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.JdkClientContainer$1.call(JdkClientContainer.java:107)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.container.jdk.client.JdkClientContainer.openClientSocket(JdkClientContainer.java:192)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager$3$1.run(ClientManager.java:647)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager$3.run(ClientManager.java:696)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager$SameThreadExecutorService.execute(ClientManager.java:849)
        at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:123)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager.connectToServer(ClientManager.java:493)
        at io.jenkins.cli.shaded.org.glassfish.tyrus.client.ClientManager.connectToServer(ClientManager.java:337)
        at hudson.cli.CLI.webSocketConnection(CLI.java:364)
        at hudson.cli.CLI._main(CLI.java:324)
        at hudson.cli.CLI.main(CLI.java:103)
Caused by: java.io.IOException: Das Zeitlimit fĂĽr die Semaphore wurde erreicht
        at java.base/sun.nio.ch.Iocp.translateErrorToIOException(Iocp.java:299)
        at java.base/sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:389)
        at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:113)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
13

I will give the version cli over SSH a try but tomorrow

14

We solved it by configuring websocket

NGINX as a WebSocket Proxy

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
1 Like
15

Does this CLI over ssh worked for you ?

16

yes the version via SSH worked

17

hello there, can you explain what “the version via ssh” means ?

18

I refer to this connection mode described here

1 Like
19

Thank, ssh is blocked by security team on my side… I abandoned Jenkins CLI.
Since my goal was only to launch a job Jenkins, I will use a basic CURL (with api token and url).
If you have an update I will be delighted to know the final solution for websocket/http connection through Jenkins CLI …