Hello everyone. I’m trying to create a Docker compose template to spin up an orchestrator and some agents on the fly for each project. I want to have different instances for each project because I don’t want to mix the jobs etc. In my opinion is a little bit cleaner.
Said so, I want to have the following approach, you can correct me if not possible. The code is hosted in GitHub. In GitHub I’m creating a Deploy key for the project and I’m adding it as a Credential in Jenkins.
I’m using JobDSL in order to provision the Jobs that are needed by the dev team and one of those is also the Multibranch one. I want to start building and testing the branches when they have changes, so I created this Pipeline
multibranchPipelineJob('multibranch_build_and_unit_test') {
description('Runs compile and unit tests for each branch')
branchSources {
// Using the shared config without explicit id since it's generated automatically
GitConfig.getMultibranchGitConfig(delegate)
}
configure {
def traits = it / sources / data / 'jenkins.branch.BranchSource' / source / traits
traits << 'jenkins.plugins.git.traits.BranchDiscoveryTrait' {}
}
factory {
workflowBranchProjectFactory {
scriptPath('backend/etc/jenkins/pipelines/pipeline_multibranch_build_and_unit_test_commit.groovy')
}
}
orphanedItemStrategy {
discardOldItems {
numToKeep(20)
}
}
}
Here you can find the GitConfig.getMultibranchGitConfig(delegate) method implementation
// Add method for multibranch configuration
static def getMultibranchGitConfig(def branchSource) {
branchSource.with {
git {
id('unique-id') // Must be set and unique. We don't have multiple repos in same Jenkins but whatever.
remote(DEFAULT_REPO)
credentialsId(DEFAULT_CREDENTIALS)
}
}
}
This seems to work just fine when manually triggering through the GUI, but it doesn’t work through the Webhook notification. I’m able to receive it in the github-webhook endpoint and can see it in the logs, but it isn’t being triggered.
I also can’t use the GitHub branch source since it wants username and password credentials but I’d like to give a deploy key instead of providing my own credentials in there.
Is the only way through the notifyCommit endpoint, where I also need to pass the Jenkins token into that? I would like to avoid passing the plain Jenkins token in the Webhook url field in GitHub. With the notifyCommit endpoint the jobs is being triggered just fine.
This whole thing because I just want to give the power to the developers on creating jobs and maintaining the pipelines through the code so the changes can be tracked, instead of doing the things through the UI and being error-prone. Also this way it is just a matter of copy-pasting the initial configuration and running the seed job
job('jenkins_seed') {
description('Seed job to create all Jenkins jobs from configuration')
scm {
git {
remote {
url("<github_url>")
credentials("github-credentials")
}
branch("develop")
}
}
steps {
systemGroovyCommand('''
def build = Thread.currentThread().executable
def workspace = build.workspace
// Use Jenkins FilePath API to read the file
def configFile = workspace.child("backend/etc/jenkins/global_configs/GitConfig.groovy")
if (configFile.exists()) {
def content = configFile.readToString()
def gitConfig = new GroovyShell().parse(content)
binding.setVariable('GitConfig', gitConfig)
}
''')
jobDsl {
targets('backend/etc/jenkins/jobs/**/*.groovy') // Include all job files under jenkins/jobs
removedJobAction('DELETE')
removedViewAction('DELETE')
lookupStrategy('SEED_JOB')
additionalClasspath('backend/etc/jenkins/global_configs')
}
}
}
Jenkins setup:
Jenkins: 2.479.3
OS: Linux - 6.8.0-1013-oracle
Java: 17.0.13 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
ant:511.v0a_a_1a_334f41b_
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
asm-api:9.7.1-97.v4cc844130d97
bootstrap5-api:5.3.3-1
bouncycastle-api:2.30.1.80-256.vf98926042a_9b_
branch-api:2.1208.vf528356feca_4
build-timeout:1.35
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.2.1
cloudbees-folder:6.980.v5a_cc0cb_25881
commons-lang3-api:3.17.0-84.vb_b_938040b_078
commons-text-api:1.13.0-153.v91dcd89e2a_22
credentials:1405.vb_cda_74a_f8974
credentials-binding:687.v619cb_15e923f
dark-theme:524.vd675b_22b_30cb_
display-url-api:2.209.v582ed814ff2f
durable-task:581.v299a_5609d767
echarts-api:5.5.1-5
eddsa-api:0.3.0-4.v84c6f0f4969e
email-ext:1866.v14fa_6d201654
font-awesome-api:6.6.0-2
git:5.7.0
git-client:6.1.1
github:1.40.0
github-api:1.321-478.vc9ce627ce001
github-branch-source:1810.v913311241fa_9
gradle:2.14
groovy:457.v99900cb_85593
gson-api:2.11.0-85.v1f4e87273c33
instance-identity:201.vd2a_b_5a_468a_a_6
ionicons-api:74.v93d5eb_813d5f
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javax-activation-api:1.2.0-7
javax-mail-api:1.6.2-10
jaxb:2.3.9-1
jjwt-api:0.11.5-112.ve82dfb_224b_a_d
job-dsl:1.90
joda-time-api:2.13.0-93.v9934da_29b_a_e9
jquery3-api:3.7.1-2
json-api:20250107-125.v28b_a_ffa_eb_f01
json-path-api:2.9.0-118.v7f23ed82a_8b_8
junit:1312.v1a_235a_b_94a_31
ldap:770.vb_455e934581a_
mailer:489.vd4b_25144138f
matrix-auth:3.2.3
matrix-project:845.vffd7fa_f27555
metrics:4.2.21-461.v881e35d8fa_b_a_
mina-sshd-api-common:2.14.0-138.v6341ee58e1df
mina-sshd-api-core:2.14.0-138.v6341ee58e1df
okhttp-api:4.11.0-183.va_87fc7a_89810
pam-auth:1.11
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:61.v629f2cc41d83
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-graph-view:402.va_9d108235846
pipeline-groovy-lib:749.v70084559234a_
pipeline-input-step:508.v584c0e9a_2177
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2218.v56d0cda_37c72
pipeline-model-definition:2.2218.v56d0cda_37c72
pipeline-model-extensions:2.2218.v56d0cda_37c72
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2218.v56d0cda_37c72
plain-credentials:183.va_de8f1dd5a_2b_
plugin-util-api:5.1.0
resource-disposer:0.25
scm-api:703.v72ff4b_259600
script-security:1369.v9b_98a_4e95b_2d
snakeyaml-api:2.3-123.v13484c65210a_
ssh-credentials:349.vb_8b_6b_9709f5b_
ssh-slaves:3.1021.va_cc11b_de26a_e
structs:338.v848422169819
theme-manager:278.v2e3c063e42cc
timestamper:1.28
token-macro:400.v35420b_922dcb_
trilead-api:2.147.vb_73cc728a_32e
variant:60.v7290fc0eb_b_cd
workflow-aggregator:600.vb_57cdd26fdd7
workflow-api:1358.vfb_5780da_64cb_
workflow-basic-steps:1058.vcb_fc1e3a_21a_9
workflow-cps:4009.v0089238351a_9
workflow-durable-task-step:1400.v7a_fd50a_091de
workflow-job:1498.v33a_0c6f3a_4b_4
workflow-multibranch:795.ve0cb_1f45ca_9a_
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:683.va_885a_76415f9
workflow-support:944.v5a_859593b_98a_
ws-cleanup:0.48