Polling doesn't get updated according to the configuration

Using Jenkins Ask a question
1

Jenkins version 2.346.1
I’ve been experiencing quite often an issue where Jenkins jobs’ polling behavior doesn’t get updated after configuration changes. That has diverse consequences, as builds getting triggered in loop and some not triggered at all.

Actual example:
Job using ‘Pipeline from SCM’ from a Git repository, Global Shared Library from another Git repository, and project source from IBM RTC.
At first, the Global Shared Lib is configured for a specific tag, and has the option ‘Include @Library changes in job recent changes’ enabled. This option supposedly causes builds to start when Lib changes are detected. But that didn’t happen until earlier, because, I thought, I’ve configured it for a ‘tag’. All good!
One day, out of blue, the job started getting triggered in loop because it says there are changes in the library.

So, I disabled the option ‘Include @Library changes in job recent changes’, expecting that the Job’s polling would no longer look for library changes.
However, it still does… See below:

image
image737×918 36 KB 

Started on Aug 5, 2022 12:37:00 PM
Using strategy: SpecificRevisionBuildChooser
[poll] Last Built Revision: Revision 569f04be88ce924557168c9b3ffb27a596198887 (release)
The recommended git tool is: git
using credential bitbucket-pat
 > git --version # timeout=10
 > git --version # 'git version 2.30.2'
using GIT_ASKPASS to set credentials Personal Access Token for BitBucket (Created on 12.04.22, expires in 180 days)
 > git ls-remote -h -- https://host/hidden.git # timeout=10
Found 6 remote heads on https://host/hidden.git
[poll] Latest remote head revision on refs/heads/feature/Implement-experimental-Actions is: d2e450960605ebedde3092e7fe9b09c2415580d3
RTC : checking for changes...
Determining if there are incoming changes using the build toolkit
Checking incoming changes for "Job_Dev_SeamlessSDT"
Reusing facade for /var/jenkins_home/tools/com.ibm.team.build.internal.hjplugin.RTCBuildToolInstallation/RTC_Build_Toolkit_7.0.2-20220301/RCM-EWM-BuildSystem-Toolkit-Win64-7.0.2-202203010930-FINALGA/buildsystem/buildtoolkit
RTC : Changes detected
Done. Took 10 sec
Changes found

In my experience, I learned that each Job’s polling config takes one extra build to refresh. That means, the next polling will behave as per the old config, and only after the next build happens, it will be updated.
In this case, however, builds are happening over and over again due to lib polling even after I removed the option ‘Include @Library changes in job recent changes’.

I also tried cleaning up the Job’s Workspace in the master node (to remove traces of Library), to no avail.
Last time I had a similar issue, I could only get around it by deleting the job and recreating it. But that’s not sustainable…

Any idea how to fix this?

2

Just in case, here’s another actual example, with another job, another Jenkins instance:

I used to have a Global Shared Library with ‘Project repository’ configure to a local repo in the Jenkins host:
/var/jenkins_home/workspace/DevOps/RepoMirror/jenkins-shared-library/jenkins-shared-library.git

I recently removed this Library config and added another one coming from GitHub instead.
https://github.companycloud.com/bios-cc-tools/shared-jenkins-library.git

The pipeline was reconfigured accordingly to use the new Library Name (from Global config).
There are several jobs using the same pipeline, different build parameters.

Now, most jobs work fine: their pipeline checks out that Library from GitHub, and polling doesn’t care at all about Library updates because the option ‘Include @Library changes in job recent changes’ isn’t even checked.

Some few jobs, on the other hand, are behaving well in their builds, but misbehaving in their polling.

Here’s a build log from a build triggered manually (because polling is not working, see later).
See that’s checking out the library from GitHub, as expected:

Started by user Rezende Rafael
Obtained components/proj/Jenkinsfile from git https://github.companycloud.com/bios-cc-tools/shared-jenkins-pipelines.git
Resume disabled by user, switching to high-performance, low-durability mode.
Loading library shared-jenkins-library@latest
Attempting to resolve latest from remote references...
 > git --version # timeout=10
 > git --version # 'git version 2.30.2'
using GIT_ASKPASS to set credentials  Personal Access Token at BDC GitHub (DO NOT DELETE; DO NOT UPDATE)
Setting http proxy: proxy.company.com:8080
 > git ls-remote -- https://github.companycloud.com/bios-cc-tools/shared-jenkins-library.git # timeout=10
Found match: refs/tags/latest revision 8d6fdf17cd7d2dc11cecd3b2fddc88477fcc6a07
Resolving tag commit... (remote references may be a lightweight tag or an annotated tag)
 > git rev-parse --resolve-git-dir /var/jenkins_home/caches/git-5832cedf78c9ad89d7f71c46dc2bb7c1/.git # timeout=10
Setting origin to https://github.companycloud.com/bios-cc-tools/shared-jenkins-library.git
 > git config remote.origin.url https://github.companycloud.com/bios-cc-tools/shared-jenkins-library.git # timeout=10
Fetching origin...
Fetching upstream changes from origin
 > git --version # timeout=10
 > git --version # 'git version 2.30.2'
 > git config --get remote.origin.url # timeout=10
using GIT_ASKPASS to set credentials  Personal Access Token at BDC GitHub (DO NOT DELETE; DO NOT UPDATE)
Setting http proxy: proxy.company.com:8080
 > git fetch --tags --force --progress -- origin +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/tags/latest^{commit} # timeout=10
Resolved tag latest revision 8d6fdf17cd7d2dc11cecd3b2fddc88477fcc6a07
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential github-personal-access-token
Cloning the remote Git repository
Cloning with configured refspecs honoured and with tags
Cloning repository https://github.companycloud.com/bios-cc-tools/shared-jenkins-library.git
 > git init /var/jenkins_home/workspace/cust/cust_Mainstream_proj_1_Delivery@libs/16320839ee7f13c701c82373c27bb922182c0bda5529d843aef58eed6834f12f # timeout=10
Fetching upstream changes from https://github.companycloud.com/bios-cc-tools/shared-jenkins-library.git
 > git --version # timeout=10
 > git --version # 'git version 2.30.2'
using GIT_ASKPASS to set credentials  Personal Access Token at BDC GitHub (DO NOT DELETE; DO NOT UPDATE)
Setting http proxy: proxy.company.com:8080
 > git fetch --tags --force --progress -- https://github.companycloud.com/bios-cc-tools/shared-jenkins-library.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.companycloud.com/bios-cc-tools/shared-jenkins-library.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision 8d6fdf17cd7d2dc11cecd3b2fddc88477fcc6a07 (latest)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 8d6fdf17cd7d2dc11cecd3b2fddc88477fcc6a07 # timeout=10
Commit message: "Merge pull request #12 from bios-cc-tools/MemoryConsumptionReport-fails-for-old-baselines"
[Pipeline] Start of Pipeline

And here is the polling log for the same Job. It’s looking for a Lib that’s long gone, and it’s not polling for the project as we configured in the pipeline.

Started on Aug 5, 2022 2:36:00 PM
Using strategy: SpecificRevisionBuildChooser
[poll] Last Built Revision: Revision d132cd35f27c1c7a7d25cd63eff652eeb46175db (release)
The recommended git tool is: git
No credentials specified
 > git --version # timeout=10
 > git --version # 'git version 2.30.2'
 > git ls-remote -h -- /var/jenkins_home/workspace/DevOps/RepoMirror/jenkins-shared-library/jenkins-shared-library.git # timeout=10
Found 6 remote heads on /var/jenkins_home/workspace/DevOps/RepoMirror/jenkins-shared-library/jenkins-shared-library.git
Done. Took 0.1 sec
No changes

This polling is just completely outdated to the current Job and Global configurations.

3

This is pretty confusing. Last time i got this work when we move repository from bitbucket to github with followed steps.

  1. After repository transfer make one commit to new repo.
  2. Change Repository URL at jenkins job configuration
  3. Go jenkins host and current job workspace folder and change remote url using git remote command
  4. Start build manually via jenkins ui
  5. After build is complete, remove current job workplace folder
  6. Check polling log if polling is done to new repository.
  7. If polling work correctly, add second commit to new repo and check if polling start build with correct commit.
1 Like
4

Hi,

Was this issue ever fixed?

I am on Jenkins 2.401.1, and still face this issue.

5

@rafaelrezend I recently faced this issue , my jobs were triggering on the commit of the shared libs.

But i noticed even the “Include @Library changes in job recent changes” checkbox is not taking effect for the existing pipelines, so i just recreated shared-lib configuration with the “Include @Library changes in job recent changes” unchecked and it took one more build to take effect of new shared lib configuration.

This solved my issue.

1 Like
6

I’m marking this as a solution, because I see how it could potentially help and be way better that recreating the jobs themselves, if someone has many.

Months later, I’m having the issue again, but my setup is different: I use the library function inline:

library identifier: 'internal-library@version', retriever: legacySCM(libraryPath: '.pipelines/', scm: scm)

So, recreating it means recreating the job itself, in which case your workaround is pretty much what I used to do earlier.

Still, I’m happy that you took your time to report what worked for you :slight_smile: Thanks.

(Note: I’ve asked it in StackOverflow. No answers yet as I write. But if you visit it later, you might be luckier)