Gitlab to Jenkins integration (webhook or integration) issues with the push of the code to Jenkins

Jenkins setup:

Jenkins: 2.440.3
OS: Linux - 6.8.0-1019-aws
Java: 17.0.13 - Ubuntu (OpenJDK 64-Bit Server VM)

ansible:307.va_1f3ef06575a_
ant:497.v94e7d9fffa_b_9
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
apache-httpcomponents-client-5-api:5.3.1-1.0
asm-api:9.7-33.v4d23ef79fcc8
authentication-tokens:1.53.v1c90fd9191a_b_
aws-credentials:231.v08a_59f17d742
aws-java-sdk-api-gateway:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-cloudformation:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-cloudfront:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-codedeploy:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-ec2:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-ecr:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-elasticbeanstalk:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-elasticloadbalancingv2:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-iam:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-lambda:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-minimal:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-organizations:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-sns:1.12.696-451.v0651a_da_9ca_ec
aws-java-sdk-sqs:1.12.696-451.v0651a_da_9ca_ec
bootstrap5-api:5.3.3-1
bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
branch-api:2.1152.v6f101e97dd77
build-timeout:1.32
build-timestamp:1.0.3
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.2.0
cloud-stats:336.v788e4055508b_
cloudbees-folder:6.901.vb_4c7a_da_75da_3
command-launcher:107.v773860566e2e
commons-httpclient3-api:3.1-3
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-95.v22a_d30ee5d36
config-file-provider:973.vb_a_80ecb_9a_4d0
configurationslicing:548.ve92d48e66b_f8
credentials:1337.v60b_d7b_c7b_c9f
credentials-binding:657.v2b_19db_7d6e6d
dark-theme:439.vdef09f81f85e
display-url-api:2.200.vb_9327d658781
docker-commons:439.va_3cb_0a_6a_fb_29
docker-java-api:3.3.4-86.v39b_a_5ede342c
docker-plugin:1.6
docker-workflow:572.v950f58993843
durable-task:555.v6802fe0f0b_82
echarts-api:5.5.0-1
email-ext:1806.v856a_01a_fa_39a_
font-awesome-api:6.5.1-3
git:5.2.1
git-client:4.7.0
github:1.38.0
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1787.v8b_8cd49a_f8f1
github-pullrequest:0.7.0
gitlab-plugin:1.8.2
gradle:2.11
gson-api:2.10.1-15.v0d99f670e0a_7
instance-identity:185.v303dc7c645f9
ionicons-api:70.v2959a_b_74e3cf
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javadoc:243.vb_b_503b_b_45537
javax-activation-api:1.2.0-7
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jersey2-api:2.44-151.v6df377fff741
jjwt-api:0.11.5-112.ve82dfb_224b_a_d
joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
jquery3-api:3.7.1-2
jsch:0.2.16-86.v42e010d9484b_
json-api:20240303-41.v94e11e6de726
json-path-api:2.9.0-58.v62e3e85b_a_655
junit:1265.v65b_14fa_f12f0
ldap:719.vcb_d039b_77d0d
mailer:472.vf7c289a_4b_420
matrix-auth:3.2.2
matrix-project:822.824.v14451b_c0fd42
maven-plugin:3.23
mina-sshd-api-common:2.12.1-101.v85b_e08b_780dd
mina-sshd-api-core:2.12.1-101.v85b_e08b_780dd
nexus-artifact-uploader:2.14
nodejs:1.6.1
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
pam-auth:1.10
pipeline-aws:1.45
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:42.v0739460cda_c4
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-groovy-lib:704.vc58b_8890a_384
pipeline-input-step:495.ve9c153f6067b_
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2198.v41dd8ef6dd56
pipeline-model-definition:2.2198.v41dd8ef6dd56
pipeline-model-extensions:2.2198.v41dd8ef6dd56
pipeline-rest-api:2.34
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2198.v41dd8ef6dd56
pipeline-stage-view:2.34
pipeline-utility-steps:2.16.2
plain-credentials:179.vc5cb_98f6db_38
plugin-util-api:4.1.0
resource-disposer:0.23
saferestart:0.7
scm-api:689.v237b_6d3a_ef7f
script-security:1335.vf07d9ce377a_e
slack:684.v833089650554
snakeyaml-api:2.2-111.vc6598e30cc65
sonar:2.17.2
ssh-credentials:337.v395d2403ccd4
ssh-slaves:2.948.vb_8050d697fec
sshd:3.322.v159e91f6a_550
structs:337.v1b_04ea_4df7c8
theme-manager:215.vc1ff18d67920
timestamper:1.26
token-macro:400.v35420b_922dcb_
trilead-api:2.142.v748523a_76693
variant:60.v7290fc0eb_b_cd
versioncolumn:243.vda_c20eea_a_8a_f
workflow-aggregator:596.v8c21c963d92d
workflow-api:1291.v51fd2a_625da_7
workflow-basic-steps:1049.v257a_e6b_30fb_d
workflow-cps:3894.vd0f0248b_a_fc4
workflow-durable-task-step:1336.v768003e07199
workflow-job:1400.v7fd111b_ec82f
workflow-multibranch:773.vc4fe1378f1d5
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:657.v03b_e8115821b_
workflow-support:896.v175a_a_9c5b_78f
ws-cleanup:0.45

=================

Problem details:

Problem to solve

I am trying to get Gitlab to Jenkins integration working. I have tried both the Gitlab integration approach and the gitlab webhook approach.
All connections from Gitlab to Jenkins and Jenkins to Gitlab work fine when running the “test connection” (I am using Maintainer role tokens for this)

The push from my VSCode to gitlab does result in the pipeline on Jenkins starting but no code is being pushed to Jenkins so the maven build stage fails right off.

I verified this in the /var/lib/jenkins/workspace/xxxx on the jenkins controller
There is no code in there.

The VSCode push to github (not gitlab) to Jenkins via webhook is working fine. (same Jenkins server). The parallel Jenkins project has all of the github repository source code in the directory (separate directory from the one above)

I did try editing the Repository setting on gitlab for the protected branch and allowed force push. I also changed the branch settings to allow force push but it did not resolve the issue.

I do think this is related to the respository settings because the connections from gitlab to Jenkins and Jenkins to gitlab are testing ok.

Currently i am just trying to get a simple push to work (later will try MR, etc)

I have researched this online and still stumped on this one.

The gitlab is a docker container on a VPS, but the firewall rules and allowlist on the reverse proxy are all letting the traffic from Jenkins to gitlab through

The Jenkins server is an EC2 instance on AWS. For now i have opened up the security groups on the Jenkins server so that is not the issue.

Since the pipeline is starting on Jenkins when pushing VSCode to gitlab it is just the code push to Jenkins that is failing (or does Jenkins pull the repo from gitlab?) . Both directions are “test connection” 200 ok.

The gitlab branch main (the one i am trying to push) is protected branch and role is Maintainers.

I did try unprotecting the branch but the same issue.

Does anyone have any idea why the source code is not being pushed from gitlab to Jenkins (or Jenkins pulling code from gitlab, whichever is the case)?

I followed this document to the letter when setting this up

I also made the gitlab project public as part of the troubleshooting

If i change something in the Jenkinsfile pipeline configuration (add a stage for example) it is reflected in the running pipeline on Jenkins but since no code is pushed the stages all fail anyways

Gitlab is self managed as a docker container on a VPS but communication with webhooks etc is 200ok.

Versions

  • GitLab (Web: /help or self-managed system information sudo gitlab-rake gitlab:env:info): 17.6.1
  • GitLab Runner, if self-hosted (Web /admin/runners or CLI gitlab-runner --version):
    17.4.0 docker executor runner

A Jenkins to gitlab connection has been configured on Jenkins with a token and the test is 200ok.

Jenkins:

The jenkins project is configured to run when it detects a gitlab push, etc…and a secret token was created and this was used in the gitlab webhook (and that connection from gitlab to jenkins is 200ok). And as noted above the Jenkins pipeline does start but it is just that the source code is not pushed from gitlab to Jenkins (or pulled from gitlab to Jenkins).

In Manage Jenkins—> Conifguration —> Gitlab section i do have “Enable authentication for ‘/project’ end-point” checked off and this is working fine once the jenkins secret token was added to the gitlab webhook.

This is the jenkins error log in the pipeline console due to the source code not being pushed:

[e[1;31mERRORe[m] The goal you specified requires a project to execute but there is no POM in this directory (/var/lib/jenkins/workspace/Gitlab_to_jenkins_project20c_kops). Please verify you invoked Maven from the correct directory. → e[1m[Help 1]e[m

thanks and warm regards
Dave

The gitlab link above is thoroughly insufficient for configuring this type of thing. Use this link instead on the gitlab plugin from the jenkins site

It is working now with changes incorporated from this document

warm regards
Dave