steps.archiveArtifacts artifacts crashes for unknown reasons

Community
1

Platform Linux Redhat.

When we run the Jenkins pipeline we sometimes get an exception in the Archive step. Logs shows us that all output files are closed before the archive function is called. Also there are no processes running at the moment the Jenkins step is executed:
steps.archiveArtifacts artifacts: "${artifactPath}/", excludes: '/rt_seq/,**/seq/’

The problem is that this happens for 50% of the times. A work around for us is the tar all output and call after this the archiveArtifacts, however that is not how we would like to do this.
We examined all scenarios and despite all efforts (more than a man month of investigation) we cannot get Jenkins to work. One other thing is that we added a steps.sh “sleep 1800” which gave us time to validat the VM in AWS before this was closed. We used all possible tooling and system functions to examine the log files. All were closed and there were no crashes. We are running out of ideas and have no glue where to look.
Question: is there a log option in the archive function which tells which file is in trouble?

The following is the error we get time after time

java.io.IOException: Truncated TAR archive
	at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:743)
	at java.base/java.io.InputStream.read(Unknown Source)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1309)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:978)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1282)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:953)
	at hudson.util.IOUtils.copy(IOUtils.java:52)
	at hudson.FilePath.readFromTar(FilePath.java:2982)
Also:   	Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.1.23/10.244.1.23:40766
			at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
			at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
			at hudson.remoting.Channel$2.adapt(Channel.java:1034)
			at hudson.remoting.Channel$2.adapt(Channel.java:1030)
			at hudson.remoting.FutureAdapter.get(FutureAdapter.java:66)
			at hudson.FilePath.copyRecursiveTo(FilePath.java:2753)
			at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:73)
			at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:258)
			at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
			at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
			at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
			at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
			at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
			at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
			at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
			at java.base/java.lang.Thread.run(Unknown Source)
java.io.IOException: This archive contains unclosed entries.
		at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.finish(TarArchiveOutputStream.java:291)
		at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:309)
		at hudson.util.io.TarArchiver.close(TarArchiver.java:124)
		at hudson.FilePath.writeToTar(FilePath.java:2953)
		at hudson.FilePath$CopyRecursiveRemoteToLocal.invoke(FilePath.java:2916)
		at hudson.FilePath$CopyRecursiveRemoteToLocal.invoke(FilePath.java:2901)
		at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3502)
		at hudson.remoting.UserRequest.perform(UserRequest.java:211)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:376)
		at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
		at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
		at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:122)
		at java.base/java.lang.Thread.run(Unknown Source)
	Caused: java.util.concurrent.ExecutionException
		at hudson.remoting.Channel$2.adapt(Channel.java:1036)
		at hudson.remoting.Channel$2.adapt(Channel.java:1030)
		at hudson.remoting.FutureAdapter.get(FutureAdapter.java:66)
		at hudson.FilePath.copyRecursiveTo(FilePath.java:2753)
Caused: java.io.IOException: Failed to extract /home/jenkins/agent/workspace/dib_cloud_run/transfer of 417 files
	at hudson.FilePath.readFromTar(FilePath.java:2992)
	at hudson.FilePath.copyRecursiveTo(FilePath.java:2750)
	at jenkins.model.StandardArtifactManager.archive(StandardArtifactManager.java:73)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:258)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
2

Hi

Well, I’m using this step without problem but could be nice to have:

  • Size of files
  • Number of files

This help to reproduce

4

Although we thought the problem was gone with the close of the files, the crash appears again. We have around 500 files with a size of 1 to 5 kb. They all have the .log extension. When we access the VM where the files are they are not in use and no uniface processes are running. All files are complete and have the session: “start” and “end”, which tells us that there is no early exit due to crashes or other unforseen circumstances. The question is: how can be see on which file the problem occurs? Is there a logging feature or trace option in Jenkins which can give us more information?