We are currently having issues connecting to our customer servers through SSH with Jenkins, we are always getting the “Auth fail for methods ‘publickey,password’” error message.
We have a Virtual Machine that is running our local Docker containers (including Jenkins) that I freshly set up to have a baseline with a current version of Jenkins. We also have another Virtual Machine (running Ubuntu Server 22.04) that acts as the Agent.
Until now we used a generated SSH key to connect to the servers (including putty, which still works) but apparently, an Update flagged the way that key and how it was generated as insecure and we got the stated error message.
We then generated a new SSH key with ed25519 and added that to the customer servers. Using that key we can successfully connect to the customer system from the Agent without any issues.
I created a test “Freestyle Project” in Jenkins which just has
- Send files or execute commands over SSH
- SSH Server
- Name: Customer Server
- Username: root
- Key: the ed25519 private key
- exec command: ls -l
- SSH Server
When I now Click “Test Configuration”, it returns “Success”. When I then save and run the Project it turns UNSTABLE
Running as SYSTEM
Building remotely on jenkins-agent in workspace /home/jenkins/sharedspace
SSH: Connecting from host [jenkins-agent]
SSH: Connecting with configuration [Customer Server] ...
SSH: Creating session: username [root], hostname [customer IP], port [22]
SSH: Connecting session ...
ERROR: Exception when publishing, exception message [Failed to connect and initialize SSH connection. Message: [Failed to connect session for config [Customer Server]. Message [Auth fail for methods 'publickey,password']]]
Build step 'Send files or execute commands over SSH' changed build result to UNSTABLE
Finished: UNSTABLE
Initially I created a Pipeline for this which turned up the following error:
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins-agent
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1787)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
at hudson.remoting.Channel.call(Channel.java:1003)
at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution.run(CommandStep.java:72)
at org.jenkinsci.plugins.sshsteps.util.SSHStepExecution.lambda$start$0(SSHStepExecution.java:84)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Also: org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: fb709b9a-b348-4367-9abc-011856405c64
com.jcraft.jsch.JSchException: Auth fail for methods 'publickey,password'
at com.jcraft.jsch.Session.connect(Session.java:521)
at com.jcraft.jsch.Session.connect(Session.java:199)
at com.jcraft.jsch.Session$connect$1.call(Unknown Source)
at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:107)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:352)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:68)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:193)
at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy:85)
at org.hidetake.groovy.ssh.connection.ConnectionManager$_connectInternal_closure1.doCall(ConnectionManager.groovy)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:41)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
at org.hidetake.groovy.ssh.util.Utility.retry(Utility.groovy:52)
at org.hidetake.groovy.ssh.util.Utility$retry.callStatic(Unknown Source)
at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy:83)
at org.hidetake.groovy.ssh.connection.ConnectionManager.connectInternal(ConnectionManager.groovy)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:169)
at org.hidetake.groovy.ssh.connection.ConnectionManager.connect(ConnectionManager.groovy:59)
at org.hidetake.groovy.ssh.connection.ConnectionManager$connect.call(Unknown Source)
at org.hidetake.groovy.ssh.session.SessionTask.wetRun(SessionTask.groovy:61)
at org.hidetake.groovy.ssh.session.SessionTask.call(SessionTask.groovy:48)
at java_util_concurrent_Callable$call.call(Unknown Source)
at org.hidetake.groovy.ssh.core.Service.run(Service.groovy:81)
at org.hidetake.groovy.ssh.core.Service$run$0.call(Unknown Source)
at org.jenkinsci.plugins.sshsteps.SSHService.executeCommand(SSHService.groovy:177)
at org.jenkinsci.plugins.sshsteps.steps.CommandStep$Execution$CommandCallable.execute(CommandStep.java:84)
at org.jenkinsci.plugins.sshsteps.util.SSHMasterToSlaveCallable.call(SSHMasterToSlaveCallable.java:32)
at hudson.remoting.UserRequest.perform(UserRequest.java:211)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:377)
at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Finished: FAILURE
The Pipeline looks like this:
def remote = [:]
remote.name = "Customer Server"
remote.host = "IP of Customer Server"
remote.allowAnyHosts = true
node {
withCredentials([sshUserPrivateKey(credentialsId: 'live-ecs-key',
keyFileVariable: 'KEYFILE',
passphraseVariable: '',
usernameVariable: 'USERNAME')]) {
remote.user = USERNAME
remote.identityFile = KEYFILE
stage("SSH Steps Rocks!") {
sshCommand remote: remote, command: 'ls -l'
}
}
}
live-ecs-key is configured to set the username root and the ed25519 private key.
I have no idea what else I should check or where the problem is. The only thing I haven’t tried yet is to completely start from a fresh Jenkins installation and I would only like to do it as a last resort.
Jenkins setup:
Jenkins: 2.446
OS: Linux - 5.15.0-97-generic
Java: 17.0.10 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
ant:497.v94e7d9fffa_b_9
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
bootstrap5-api:5.3.2-3
bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
branch-api:2.1148.vce12cfcdf090
build-timeout:1.32
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloudbees-folder:6.921.vfb_b_224371fb_4
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-95.v22a_d30ee5d36
credentials:1319.v7eb_51b_3a_c97b_
credentials-binding:657.v2b_19db_7d6e6d
cvs:2.19.1
data-tables-api:1.13.8-2
display-url-api:2.200.vb_9327d658781
durable-task:550.v0930093c4b_a_6
echarts-api:5.4.3-2
email-ext:2.104
font-awesome-api:6.5.1-2
git:5.2.1
git-client:4.6.0
git-parameter:0.9.19
git-server:114.v068a_c7cc2574
github:1.38.0
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1772.va_69eda_d018d4
gson-api:2.10.1-15.v0d99f670e0a_7
htmlpublisher:1.32
instance-identity:185.v303dc7c645f9
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.16.1-373.ve709c6871598
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jjwt-api:0.11.5-77.v646c772fddb_0
jnr-posix-api:3.1.19-1
joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
jquery:1.12.4-1
jquery3-api:3.7.1-1
jsch:0.2.16-86.v42e010d9484b_
json-path-api:2.9.0-33.v2527142f2e1d
junit:1259.v65ffcef24a_88
ldap:711.vb_d1a_491714dc
locale:431.v3435fa_8f8445
lockable-resources:1243.v346d600eea_24
mailer:463.vedf8358e006b_
matrix-auth:3.2.1
matrix-project:822.824.v14451b_c0fd42
mina-sshd-api-common:2.12.0-90.v9f7fb_9fa_3d3b_
mina-sshd-api-core:2.12.0-90.v9f7fb_9fa_3d3b_
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
pam-auth:1.10
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:42.v0739460cda_c4
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:704.vc58b_8890a_384
pipeline-input-step:491.vb_07d21da_1a_fb_
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2175.v76a_fff0a_2618
pipeline-model-definition:2.2175.v76a_fff0a_2618
pipeline-model-extensions:2.2175.v76a_fff0a_2618
pipeline-rest-api:2.34
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2175.v76a_fff0a_2618
pipeline-stage-view:2.34
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.8.0
publish-over:0.22
publish-over-ssh:1.25
resource-disposer:0.23
scm-api:683.vb_16722fb_b_80b_
script-security:1326.vdb_c154de8669
shared-workspace:1.0.2
snakeyaml-api:2.2-111.vc6598e30cc65
ssh-agent:346.vda_a_c4f2c8e50
ssh-credentials:308.ve4497b_ccd8f4
ssh-slaves:2.948.vb_8050d697fec
ssh-steps:2.0.68.va_d21a_12a_6476
sshd:3.322.v159e91f6a_550
structs:337.v1b_04ea_4df7c8
timestamper:1.26
token-macro:400.v35420b_922dcb_
trilead-api:2.133.vfb_8a_7b_9c5dd1
variant:60.v7290fc0eb_b_cd
workflow-aggregator:596.v8c21c963d92d
workflow-api:1291.v51fd2a_625da_7
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3867.v535458ce43fd
workflow-durable-task-step:1331.vc8c2fed35334
workflow-job:1400.v7fd111b_ec82f
workflow-multibranch:783.va_6eb_ef636fb_d
workflow-scm-step:415.v434365564324
workflow-step-api:657.v03b_e8115821b_
workflow-support:865.v43e78cc44e0d
ws-cleanup:0.45