Managing a shared resource across a [scripted pipeline] Jenkinsfile

Using Jenkins
1

In our test system we have test peers which are effectively shared resources across nodes. The test peers must be powered up when a node (preferably a stage on a node) is active and should be powered down again when no-one is using them anymore, so when the node is no longer occupied by any Jenkinsfile or, preferably, a given stage on a node is no longer running. Note that this is, of course, pan-Jenkinsfile, i.e. it must work on a node/stage basis with multiple instances of a Jenkinsfile being executed, can’t have the shared resource getting powered down just because one Jenkinsfile instance has finished.

I’ve seen something about “shared libraries” which might do the trick, assuming a shared library can be a script that powers something up and then powers it down again when the shared library has completed.

Or is there another, or a better, way to do this?

[Answers for scripted pipeline only please]

2

Did you look at the lockable resource plugin?

2 Likes
3

Though this is probably not exactly what you need I think. I’m not aware of another plugin, that does what you need, so you would need to implement your own plugin I guess.
Not sure if you can do this via a shared library, a library is always loaded in the context of a pipeline execution

1 Like
4

Hi, and thanks for replying. I do use the Lockable Resources plugin for the “boring” case of multiple pipelines needing to own a single thing. I’ve had a look at the API in more detail now; maybe I could do…

[EDITED]

…no, I’ve edited this post a few times now and I can’t think of a way to use it.

5

an idea with a shared library

// vars/withExternalResource.groovy
def call(Closure body) {
lock('my-resource-name') {
  node('built-in') { 
    ws('fixed-folder') {
        readfilewith counter
        if (counter == 0) {
          sh 'start resource'
        }
        increment counter 
        write counter to file
    }
  }
}
try {
  body()
} finally {
  lock('my-resource-name') {
    node('built-in') { 
      ws('fixed-folder') {
        readfilewith counter
        decrement counter
        if (counter == 0) {
          sh 'stop resource'
        }
        write counter to file
      }
    }
  }
}
}

You then call it with

// load the shared lib
library 'my-external-resource@master'
withExternalResource {
  // use the external resource
  sh 'run test'
}

would need of course more safeguarding against failures but in principle this could work

1 Like
6

Coo, that’s got rather more Groovy fu than I’m capable of :-).

Let me give it a spin…

7

Yup, that works. Here’s the actual code I used, simplified for my C-oriented brain:

counter_file = "counter"
shared_resource_dir = "shared_resource"

// Read a string representing an integer value from a file and return
// it as an integer, 0 if there is no file or it does not contain a string
// which can be converted to an integer
def readIntegerFromFile(file_path) {
    def value_int = 0
    try {
        def value_str = readFile(file: "${file_path}")
        value_int =  value_str as int
    } catch(e) {
    }
    return value_int
}

// Write a single integer value to a file as a string
def writeIntegerToFile(file_path, value_int) {
    def value_str = value_int as String
    writeFile(file: "${file_path}", text: value_str)
}

// Acquire any shared resources
def sharedResourceAcquire() {
    powered_up = false

    lock("shared_resource") {
        node("built-in") { 
            ws(shared_resource_dir) {
                def counter = readIntegerFromFile(counter_file)
                if (counter == 0) {
                    // TODO
                    powered_up = true
                }
                counter++
                writeIntegerToFile(counter_file, counter)
            }
        }
    }

    return powered_up
}

// Release any shared resources
def sharedResourceRelease() {
    powered_down = false

    lock("shared_resource") {
        node("built-in") { 
            ws(shared_resource_dir) {
                def counter = readIntegerFromFile(counter_file)
                if (counter > 0) {
                    counter--
                }
                if (counter == 0) {
                    // TODO
                    powered_down = true
                }
                writeIntegerToFile(counter_file, counter)
            }
        }
    }

    return powered_down
}

Thanks!