Using threads in shared library doesn't do anything

Using Jenkins Ask a question
1

Hii all,
I’m trying to use threads or Executor Service in my Jenkins shared library. While executing, it doesn’t throw any errors, but it doesn’t seem like any operations are performed by the threads.

    void checkMetaFilters(String datacenter, LinkedList<String> clientIDs, HashMap filters) {
        if (!filters) {
            return
        }
        ConcurrentHashMap<String, HashMap> nodesData = new ConcurrentHashMap<String, HashMap>()
        ExecutorService executorService = Executors.newFixedThreadPool(NOMAD_MAX_THREADS)
        for (int i = 0; i < clientIDs.size(); i++) {
            final String id = clientIDs.get(i)
            Logger.info(id)
            executorService.submit ({ ->
                Logger.debug(id)
                HashMap nodeData = getNomadNodeData(datacenter, id)
                Logger.debug(nodeData)
                if (isMetaFilterMatching(nodeData, filters)) {
                    nodesData.put(id, nodeData)
                }
            })
        }
        Logger.debug("Shutting down threads...")
        sleep(5000)
        executorService.shutdown()
        try {
            Logger.debug("Awaiting termination...")
            executorService.awaitTermination(20, TimeUnit.SECONDS)
        }
        catch (InterruptedException e) {
            Logger.printStackTrace(e)
        }
        Logger.debug(nodesData)
        clientIDs.clear()
        clientIDs.addAll(nodesData.keySet())
    }

I’m not sure if I’m missing something or if threads simply doesn’t work.

Thanks in advance

2

Jenkins Pipeline (including shared libraries) runs on the Groovy language with a specific CPS (Continuation Passing Style) transformation which makes it possible to interrupt scripts at any step and then resume them, even after Jenkins has been restarted.

This is what makes Jenkins Pipelines so resilient.

However, this CPS transformation doesn’t play well with some Groovy and Java language features, including native Java/Groovy threads, because those threads are not able to be paused and resumed in the same way.

This is likely why your threading code is not working as expected. :person_shrugging:

Instead of using native threads, you could maybe use Jenkins Pipelines’ parallel step.
It should allow you to run multiple steps concurrently.

3

I’m using also an ExecutorService in a pipeline (actually a shared library). It works when annotating the method with @NonCPS
The limitation is that you can’t call other pipeline steps from within a method that is annotated with @NonCPS

4

not sure if that would help but Using parallel Step for Parallel Execution
Instead of using threads or ExecutorService, consider using the parallel step provided by Jenkins for parallel execution. there is a discussion here

multithreading - Jenkins pipeline script - Thread programming - Stack Overflow.