How to debug GIT Checkout Authentication Failure?

I have a Jenkins Configuration that uses a Groovy Pipeline script to checkout repo. The job has worked for years and now started failing and not aware of any changes. Credentials are username and password kept in Jenkins.(Use the same user in an Azure DevOps CI build with no error.)

How do I debug in Jenkins?

Script:

   checkout([$class: 'GitSCM',
                              branches: [[name: "${params.BRANCH_NETMARK_TAG}"]],
                              extensions: [[$class: 'CloneOption', timeout: 60]],
                              gitTool: 'Default',
                              userRemoteConfigs: [[credentialsId: 'sa-netmark', url: 'https://fa-devops@dev.azure.com/fadv-devops/ProdDev/_git/Netmark']]                              
                            ])

Error


 > git config remote.origin.url https://fa-devops@dev.azure.com/fa-devops/ProdDev/_git/Netmark# timeout=10
Fetching upstream changes from https://fa-devops@dev.azure.com/fadv-devops/ProdDev/_git/Netmark
 > git --version # timeout=10
 > git --version # 'git version 2.30.1'
using GIT_ASKPASS to set credentials sa-netmark | dev.azure.com | Netmark
 > git fetch --tags --force --progress -- https://fa-devops@dev.azure.com/fa-devops/ProdDev/_git/Netmark+refs/heads/*:refs/remotes/origin/* # timeout=60
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://fa-devops@dev.azure.com/fa-devops/ProdDev/_git/Netmark
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:1000)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1241)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1301)
	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://fa-devops@dev.azure.com/fa-devops/ProdDev/_git/Netmark+refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: fatal: Authentication failed for 'https://dev.azure.com/fadv-devops/ProdDev/_git/Netmark/'

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2677)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2102)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:86)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:624)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:160)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:153)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:376)
	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to Linux-Slave-01
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:1001)
		at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:145)
		at sun.reflect.GeneratedMethodAccessor948.invoke(Unknown Source)
		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.lang.reflect.Method.invoke(Method.java:498)
		at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:131)
		at com.sun.proxy.$Proxy121.execute(Unknown Source)
		at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:998)
		at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1241)
		at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1301)
		at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
		at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
		at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at ja

Really appreciate the help.

The most likely issue is that the credential has expired.

Some alternatives to try:

  • Create another credential (personal access token), register that credential in Jenkins, and use it in a test job
  • Create an ssh private key, register it with the git provider, register it in Jenkins, and use it in a test job
  • Clone the repository from a command line with the username and password embedded in the repository URL to confirm that the username and password are correct
  • Create a test job that embeds the username and password in the URL of the repository to confirm the credential is correct