Intermitten Git checkout failures - Failed to exec spawn helper

For a while now I’m seeing errors with git checkout:

19:25:48  ERROR: Error cloning remote repo 'origin'
19:25:48  hudson.plugins.git.GitException: Could not init /home/ubuntu/workspace/pr-unit-tests/62
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:1073)
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:819)
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:170)
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
19:25:48  	at hudson.remoting.UserRequest.perform(UserRequest.java:225)
19:25:48  	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
19:25:48  	at hudson.remoting.Request$2.run(Request.java:391)
19:25:48  	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:81)
19:25:48  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
19:25:48  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
19:25:48  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
19:25:48  	at java.base/java.lang.Thread.run(Thread.java:840)
19:25:48  	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to EC2 - Generic agent(i-094888a474ca27285)
19:25:48  		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1885)
19:25:48  		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:384)
19:25:48  		at hudson.remoting.Channel.call(Channel.java:1074)
19:25:48  		at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:153)
19:25:48  		at jdk.internal.reflect.GeneratedMethodAccessor614.invoke(Unknown Source)
19:25:48  		at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
19:25:48  		at java.base/java.lang.reflect.Method.invoke(Method.java:568)
19:25:48  		at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:138)
19:25:48  		at PluginClassLoader for git-client/jdk.proxy130/jdk.proxy130.$Proxy241.execute(Unknown Source)
19:25:48  		at PluginClassLoader for git//hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1222)
19:25:48  		at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1305)
19:25:48  		at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136)
19:25:48  		at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:101)
19:25:48  		at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:88)
19:25:48  		at PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
19:25:48  		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
19:25:48  		at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
19:25:48  		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
19:25:48  		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
19:25:48  		at java.base/java.lang.Thread.run(Thread.java:840)
19:25:48  Caused by: hudson.plugins.git.GitException: Error performing git command: /usr/bin/git init /home/ubuntu/workspace/pr-unit-tests/62
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2862)
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2766)
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2761)
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:2051)
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:1071)
19:25:48  	... 11 more
19:25:48  Caused by: java.io.IOException: Cannot run program "/usr/bin/git" (in directory "/home/ubuntu/workspace/pr-unit-tests/62"): error=0, Failed to exec spawn helper: pid: 26792, exit value: 1
19:25:48  	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
19:25:48  	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
19:25:48  	at Jenkins v2.464//hudson.Proc$LocalProc.<init>(Proc.java:252)
19:25:48  	at Jenkins v2.464//hudson.Proc$LocalProc.<init>(Proc.java:221)
19:25:48  	at Jenkins v2.464//hudson.Launcher$LocalLauncher.launch(Launcher.java:994)
19:25:48  	at Jenkins v2.464//hudson.Launcher$ProcStarter.start(Launcher.java:506)
19:25:48  	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2839)
19:25:48  	... 15 more
19:25:48  Caused by: java.io.IOException: error=0, Failed to exec spawn helper: pid: 26792, exit value: 1
19:25:48  	at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
19:25:48  	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
19:25:48  	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
19:25:48  	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
19:25:48  	... 21 more
19:25:48  ERROR: Error cloning remote repo 'origin'

I don’t understand what causes this issue. Any help will be appreciated.

Jenkins v2.464.

Refer to the answer in:

@MarkEWaite thanks for the link. This gave me an idea to try and upgrade java from v17 to v21.

Upgraded java to v21 on jenkins controller and on all agents. So far so good. I don’t know what could have caused an unattended upgrade. If this doesn’t work, I will try isolating all jobs in docker to prevent them from messing up jenkins java on servers.

1 Like