Debugging GIT SSH: problem with withCreddentials

Using Jenkins
1

Having problem with occasional (=random errors few times a day) issues with git connection to Bitbucket and would like to enable GIT/SSH Debug.

Found instructions here to prepare job environment variables:

GIT_TRACE=1
GIT_TRACE_PERFORMANCE=1
GIT_CURL_VERBOSE=1
GIT_SSH_COMMAND=‘ssh -vvv’

However problem here is the GIT_SSH_COMMAND seems to override the credentials set by checkout plugin:

checkout([
            $class           : 'GitSCM',
            branches         : [[name: "*/${gitBranch}"]],
            extensions       : [],
            userRemoteConfigs: [[credentialsId: '****', url: 'git@bitbucket.org:XXXX']]
        ])

Always ends up with Host key verification failed.

I probably need to set GIT_SSH_COMMAND to ssh -v -i $VARIABLE but dont know what the variable is used by the checkout plugin where the creds are set.

Is there sane way to enable the debug without refactoring code / doing stuff on the actual agents and manually setting up some ssh stuff? We are using docker agents, everything is configured from Jenkins nothing is manually maintained outside Jenkins Creds Vault / code base.

1 Like
2

Yes it certainly will override. However you can counter that with further environment work.

From one of my own pipelines

  environment {
    IDENTITY_FILE = credentials('git-jenkins-master')
    GIT_SSH_COMMAND = "ssh -i ${IDENTITY_FILE} -vvv"
  }

I happened to need this because my project calls out to Git on its own to run tests and also needed the same credentials that the job is using.

unrelated: just how old is your Jenkins system? that $class: 'GitSCM' formatting is archaic. Modern usage would be

checkout scmGit(
    branches         : [[name: "*/${gitBranch}]],
    extensions       : [],
    userRemoteConfigs: [[credentialsId: '****',  url: 'git@bitbucket.org:XXXX']]
  )
1 Like
3

After tinkering I found one source of confusion: the ssh -vvv output is present ONLY if something fails (e.g. provide invalid/no key). If the git pulls ok, the output is not present – I treated this state as if I failed to configure verbosity before and was hunting ghosts.

Nevertheless, it seems that the debug still cannot be configured as desired. The pipeline

node("node") {
    stage('stage') {
        withCredentials([sshUserPrivateKey(credentialsId: "deployer", keyFileVariable: 'IDENTITY_FILE')]) {
            withEnv([
                    "GIT_SSH_COMMAND=ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${IDENTITY_FILE} -vvv",
                    "GIT_TRACE=1",
                    "GIT_TRACE_PERFORMANCE=1",
                    "GIT_CURL_VERBOSE=1",

            ]) {
                sh "printenv"
                checkout([
                        $class           : 'GitSCM',
                        branches         : [[name: '*/master']],
                        userRemoteConfigs: [[url: 'git@bitbucket.org:XXX.git']]
                ])
            }
        }
    }
}

totally ignores the GIT_SSH_COMMAND even though it is aparrently set (confirmed by included printenv). Also verified by intentionally malforing the ssh override – this does not yield any error and is ignored.

The only thing that seems to work is to set the GIT_SSH_COMMAND in the “Prepare an environment for the run” option in the Job configuration. However this require to provide key in the agent somehow (meaning provide the key file on the agent VM filesystem bypassing the Jenkins creds vault / cred injection). I failed to extract the credentials from Jenkins via Groovy script in the “Prepare an environment for the run”.

O hope I will succeed in reproduction the issue this crude way in dedicated Job that just pulls Bitbucket every 5 minutes in hopes that issue will present itself. But it would be nice to have the ability to enable the debug globally for all jobs. Not mentioning for jobs that do checkout the pipeline code from SCM before actually executing it (this checkout totally ignores any ssh overrides it seems).

Ad git checkout syntax, not sure where it came from TBH, probably copied from somewhere. Does not seem to make difference however, both seem to work. Maybe it is more suitable for scripted pipeline style (as opposed to declarative)?