As I am relatively new to Jenkins, I am working to understand certain behaviors within the platform.
I have a virtual machine where Jenkins is installed directly, not via Docker. Multiple jobs run within Jenkins, each utilizing a Docker container on the same system.
Question:
I’ve encountered an issue where certain builds fail due to the ‘@’ character in folder/workspace names. My understanding was that when two jobs run on the same machine, the job executed by executor number 2 would use a folder ending in ‘@2’. However, I recently observed a scenario where two jobs were running in parallel—both on the same machine/agent—with the second job starting after the first. Surprisingly, the second job, executed by executor 2, did not use the ‘@2’ folder but instead used the main/original workspace folder.
This behavior is confusing, and I’m trying to understand how Jenkins handles this situation. Could someone please explain how Jenkins manages workspaces and folders in such cases?
The number after @ is not determined by the executor number. An @2 (or a higher number) is only used when the original workspace (without @) is already occupied by a build. So what might have happened is that you have a first build, then the second build started while the first was still in progress. Now the first build ended and a third build started. This third build will again get a workspace without @ in the name because it is free. On which executor the builds happen doesn’t matter.
That explanation makes more sense. As you mentioned, the second build will use the folder with ‘@2’ only if the original workspace is already occupied by another build. However, in my case, two jobs were being executed, so the workspaces should have been different, correct?
Based on your explanation, if two jobs are running simultaneously, both should be using their own original workspace (without the ‘@’)—is that correct?
Correct.
when 2 different jobs are running simultaneously, then both will get a workspace without @ in the name.
The @2 is only appened when the same job is executed in parallel.
The workspace is usually the jobs full name, i.e. when the job is inside a folder you have the folder in the path. E.g. a job with name myJob in folder myFolder will result in the workspace myFolder/myJob. There are exceptions to this when using multibranch jobs or orginzation job types that use generated folders.
PS there is a java property you can set at Jenkins startup that allows to change the separator from @ to something different. Some tools have problems with the @.
If you start Jenkins via java "-Dhudson.slaves.WorkspaceList=---" -jar jenkins.war, the it would use --- instead of @
I have considered changing the ‘@’ character to something else. However, my main concern is whether this change might impact the existing jobs. From what I understand, it should not affect the current jobs, as new folders would be created, and I could later clear the folders with ‘@’ in their names, to save some space.
It should not affect finished builds. But it might influence long running pipelines jobs, that are running when you restart Jenkins to apply the new separator.
The thing is that this separator is also used to create temporary directories with the pattern <workspace>@tmp, this will change to <workspace>---tmp as well.
One thing to consider is that Jenkins is cleaning up unused workspaces itself after 30 days. When the separator is changed Jenkins would no longer consider a <workspace>@2 as something that is tied to a job and hence never delete it.
So cleaning the those after renaming the separator is something that should be done, just to be sure that you don’t run into a full disk.