Limiting the maximum number of parallel stages

Using Jenkins Ask a question
1

Hi,
I’m using parallel commands to run multiple stages of a pipline in parallel.
Since the stages are almost 100, I’ve limited the maximum number of stages that can actually run to max_parallel.
I’ve done this using a code as the following: to start doing the actual processing the stage need to find a free slot, otherwise it will sleep for 20 seconds and then will retry to acquire a free slot.

def slot_number = -1
  
while(slot_number < 0)
{
    lock("test")
    {
        for (int i = 0; i < max_parallel; i++)
        {
            if(!slot_used[i])
            {
                slot_number = i
                slot_used[i] = true
                break
            }
            
        }
    }
    if(slot_number < 0)
        sleep 20
}

I don’t like this approach at all for several reason:

  • sleep command print a line in the log every time it is executed for every stage
  • all the stages seems to be running at the same time and it’s very hard to find the ones that are running for real
  • the time taken to finish the stage takes into account also the time spent sleeping, making the number completely useless

Does anyone has found a more elegant way to do this?
There is at least a way to inform the GUI that the stage is paused and show it to the user?
I couldn’t find anything on the internet.

Environment:
Ubuntu 22.04 VM
Jenkins 2.479.1
OpenJDK 17