steps.archiveArtifacts artifacts crashes for unknown reasons

Community
1

Platform Linux Redhat.

When we run the Jenkins pipeline we sometimes get an exception in the Archive step. Logs shows us that all output files are closed before the archive function is called. Also there are no processes running at the moment the Jenkins step is executed:
steps.archiveArtifacts artifacts: "${artifactPath}/", excludes: '/rt_seq/,**/seq/’

The problem is that this happens for 50% of the times. A work around for us is the tar all output and call after this the archiveArtifacts, however that is not how we would like to do this.
We examined all scenarios and despite all efforts (more than a man month of investigation) we cannot get Jenkins to work. One other thing is that we added a steps.sh “sleep 1800” which gave us time to validat the VM in AWS before this was closed. We used all possible tooling and system functions to examine the log files. All were closed and there were no crashes. We are running out of ideas and have no glue where to look.
Question: is there a log option in the archive function which tells which file is in trouble?

The following is the error we get time after time

java.io.IOException: Truncated TAR archive
	at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:743)
	at java.base/java.io.InputStream.read(Unknown Source)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1309)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:978)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1282)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:953)
	at hudson.util.IOUtils.copy(IOUtils.java:52)
	at hudson.FilePath.readFromTar(FilePath.java:2982)
Also:   	Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.1.23/10.244.1.23:40766
			at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
			at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
			at hudson.remoting.Channel$2.adapt(Channel.java:1034)
			at hudson.remoting.Channel$2.adapt(Channel.java:1030)
			at hudson.remoting.FutureAdapter.get(FutureAdapter.java:66)
			at hudson.FilePath.copyRecursiveTo(FilePath.java:2753)
			at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:73)
			at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:258)
			at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
			at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
			at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
			at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
			at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
			at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
			at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
			at java.base/java.lang.Thread.run(Unknown Source)
java.io.IOException: This archive contains unclosed entries.
		at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.finish(TarArchiveOutputStream.java:291)
		at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:309)
		at hudson.util.io.TarArchiver.close(TarArchiver.java:124)
		at hudson.FilePath.writeToTar(FilePath.java:2953)
		at hudson.FilePath$CopyRecursiveRemoteToLocal.invoke(FilePath.java:2916)
		at hudson.FilePath$CopyRecursiveRemoteToLocal.invoke(FilePath.java:2901)
		at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3502)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:376)
		at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
		at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
		at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:122)
		at java.base/java.lang.Thread.run(Unknown Source)
	Caused: java.util.concurrent.ExecutionException
		at hudson.remoting.Channel$2.adapt(Channel.java:1036)
		at hudson.remoting.Channel$2.adapt(Channel.java:1030)
		at hudson.remoting.FutureAdapter.get(FutureAdapter.java:66)
		at hudson.FilePath.copyRecursiveTo(FilePath.java:2753)
Caused: java.io.IOException: Failed to extract /home/jenkins/agent/workspace/dib_cloud_run/transfer of 417 files
	at hudson.FilePath.readFromTar(FilePath.java:2992)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2750)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:73)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:258)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
2

Hi

Well, I’m using this step without problem but could be nice to have:

  • Size of files
  • Number of files

This help to reproduce

3

Found the issue, one of the log files was not properly closed. This should have been done by the OS when the application exits, however for one or another reason this was not the case. The close( file descriptor ) at the end of the log session on another place in the code solved the issue.