Different polling ways between Freestyle and Pipeline

[leessang01 (Aloha)]

• Jenkins Version 2.492.2
• the number of executors = 1
• In Pipeline:

  • triggers {
        pollSCM('H/15 * * * *')
    }
    

Hi, I have recently changed my projects from Freestyle to Pipeline on Jenkins. The number of projects is four and each build trigger is polling every 15 minute. The build time with tests is around 1 hour.

In Freestyle, there are no problems about each build of each project. However, in Pipeline, redundant build is triggered. Here is a scenario in my understanding.

• There are two executors in Jenkins: heavy-weight vs. light weight
• The former is used to build a project in real.
• The latter is used only to poll a project, trigger a build, hand over the build job to a heavy-weight executor if changes are found.
• Scenario: When all the heavy-weight executors are occupied in other items, a pending job in a pipeline item is created after one job is triggered and running on a light-weight executor, unlike a freestyle item. In turn, this light-weight executor in a pipeline item sometimes makes a duplicated build. A freestyle item seems to always use a heavy-weight executor and no duplicated build occurs like a pipeline item.
• You could see a stack of a build list as below where # of executors=1 (i.e. heavy-weight executor).
  • Build Queue (2)
    • Job #2 in Pipeline Item B → PENDING and waiting for previously queued and running jobs - like Job #5, Job #1 - to complete. However, it is quite likely the build is duplicated.
    • Job #1 in Pipeline Item B → RUNNING on a light-weight executor but waiting for Job #5 to complete.
  • Build Executor Status
    • Job #5 - (Pipeline or Freestyle) Item A → RUNNING on a single heavy-weight executor

Feel free to comment your opinions about the above scenario. This problem only occurs in Pipeline. Freestyle has no problem.

[Artalus]

I don’t exactly understand what you mean by “redundant” or “duplicated” build in this scenario, since you only mention #2 once in your build queue/executor-status example. Can you elaborate a bit?

---

• The former is used to build a project in real.
• The latter is used only to poll a project, trigger a build, hand over the build job to a heavy-weight executor if changes are found.

Close, but not quite right. Generally speaking, the lightweight executor is used to run anything in your Pipeline that happens outside of a node(label) {…} block. Once your pipeline reaches what’s inside a node block, it will land on a regular executor at some node selected by the label expression, or wait in the queue until such executor is available.

For example with this job:

stage("1") {
    echo "hi"
}
stage("2") {
    node("kai") {
        echo "zzz"
        sleep 100500
    }
}

, assuming that my kai node has only 1 executor, I still can run two builds one after another, and the later build would still be able to print hi before saying Waiting for next available executor on ‘kai’

(in Declarative Pipelines the counterpart of node {} would be a stage with agent { label "something" }block defined)

It’s been a while since I closely worked with Freestyle jobs, but IIRC you virtually cannot run anything outside a heavyweight executor there. So the above pipeline code would be equivalent to something like

node("kai") {
    stage("1") {
        echo "hi"
    }
    stage("2") {
        echo "zzz"
        sleep 100500
    }
}

---

Also, regarding that PENDING thing – if you hover over its name in the queue, what does it actually say? I don’t encounter the pending status that often. This is a long shot and I might be mistaken, but maybe you’ve configured the job with Do not allow concurrent builds, and it’s waiting for the previous build of the same #2 job to complete?

[mawinter69 (Markus Winter)]

One important difference in behaviour is a pipeline will always start immediately unless you explicitly disable concurrent execution, so the default is to allow this.
For freestyle jobs it is the other way round. By default they do not allow concurrent execution and you must enable it explicitly. But even with concurrent execution a freestyle job will end up in the queue when there no executor is available.
You also have to consider that Jenkins will remove duplicates in the queue, so your freestyle job might be added 3 times to the queue in that 1h where a previous build runs but as the parameters are identical this will not lead to 3 executions. But the pipeline (with concurrent execution) will start immediately so it will be executed 3 times, even when in between the pipeline run will be in the queue to wait for an executor.

If possible I would suggest to change from polling the SCM to sending events from the SCM to Jenkins and react on the event to trigger the build.

[leessang01 (Aloha)]

Thank you for your opinions. Let me show you my pipeline script and the above scenario in details.

TL;DR

• Jenkins Configuration: # of executors = 1
• I have not configured any node options. I mean only Built-in Node is used.
• Pipeline Definition: Pipeline script (as below)

pipeline {
  agent any

  options {
    disableConcurrentBuilds()
    timeout(time: 2, unit: 'HOURS')
  }

  triggers {
    pollSCM('H/15 * * * *') // polling every 15 minute
  }

  tools {
    jdk 'oraclejdk-17.0.12'
    git 'Default'
  }

  stages {
    stage('Checkout') {
      deleteDir()
      checkout([$class: 'GitSCM',
        branches: [[name: "main"]],
        extensions: [[
          [$class: 'CloneOption', honorRefspec: true, noTags: true, reference: '', shallow: false, timeout:60],
          [$class: 'LocalBranch', localBranch: '**'], ... ]])
    }
    stage('Build') { ... }
  }

  post { ... }
}

• Let’s say there are TWO identical items–itemA and itemB, each of which has different remote URLs.
• The build process takes 1-hour
• The scenario with timeline what I addressed:

PART I. The first build trigger (ItemA #5: RUNNING)

1. [08:55] A developer push some commits to a remote repository where ItemA is polling.
2. [09:00] ItemA checks the remote repository, finds changes, triggers its build.
   1. Git Polling Log:
      1. Started on Aug 11, 2025, 09:00:00 AM
      2. [poll] Last Built Revision: Revision cc38a540.. (origin/main)
      3. …
      4. Polling for changes in
      5. > git rev-parse origin/main^{commit} # timeout=10
      6. > git log --full-history --no-abbrev --format=raw -M -m --raw
      7. cc38a540..dac11c17
      8. Done. Took 2.1 sec
      9. Changes Found
3. [09:00] ItemA build starts (let’s say #5)
   1. Console Output:
      1. Started by an SCM change
      2. [Pipeline] Start of Pipeline
      3. [Pipeline] node
      4. Running on Jenkins..
      5. (This build will have finished in 1-hour)

[Jenkins Dashboard]
Build Queue (0)
(No builds in the queue.)

Build Executor Status
(1 of 1 executor busy)
-ItemA  #5

PART II. The second build trigger (ItemB #2: RUNNING)

1. [09:05] A developer push some commits to a remote repository where ItemB is polling.
2. [09:15] ItemB checks the remote repository, finds changes, triggers its build.
   1. Git Polling Log:
      1. Started on Aug 11, 2025, 09:15:00 AM
      2. [poll] Last Built Revision: Revision 68932d5d.. (origin/main)
      3. …
      4. Polling for changes in
      5. > git rev-parse origin/main^{commit} # timeout=10
      6. > git log --full-history --no-abbrev --format=raw -M -m --raw
      7. 68932d5d..dc0ec631
      8. Done. Took 2.1 sec
      9. Changes Found
3. [09:15] ItemB build starts (let’s say #2)
   1. Console Output:
      1. Started by an SCM change
      2. [Pipeline] Start of Pipeline
      3. [Pipeline] node
      4. Still waiting to schedule task
      5. Waiting for next available executor

[Jenkins Dashboard]
Build Queue (1)
(part of ItemB #2)

Build Executor Status
(1 of 1 executor busy)
-ItemA  #5

[ItemB Dashboard]
Builds
( ✓ ) #2 AM 09:15 -> the pop-up says 'In progress'

PART III. The third build trigger (ItemB #3: PENDING)

1. [09:30] Once again ItemB checks the remote repository because the next polling schedule arrives. So far, so good. However, changes are found although there are no changes in the remote repository. Finally, it triggers a build.
   1. Git Polling Log:
      1. Started on Aug 11, 2025, 09:30:00 AM
      2. [poll] Last Built Revision: Revision 68932d5d.. (origin/main)
      3. …
      4. Polling for changes in
      5. > git rev-parse origin/main^{commit} # timeout=10
      6. > git log --full-history --no-abbrev --format=raw -M -m --raw
      7. 68932d5d..dc0ec631
      8. Done. Took 2.1 sec
      9. Changes Found
2. [09:30] ItemB build becomes pending (let’s say #3)
   1. This job has no console output yet.
   2. This job will start sequentially, but the build is very likely duplicated with the build of ItemB #2.

[Jenkins Dashboard]
Build Queue (2)
(ItemB)
(part of ItemB #2)

Build Executor Status
(1 of 1 executor busy)
-ItemA  #5

[ItemB Dashboard]
Builds
(...) #3 -> pending-Build #2 is already in progress
( ✓ ) #2 AM 09:15 -> the pop-up says 'In progress'

Discussion

There is something that strikes me as odd. Only Pipeline style has such trigger in the Part III. At 09:30, the job of ItemA #5 is still running, which was triggered in the Part I. The job of ItemB #2 is not only queued, it hasn’t built yet, but the pop-up says ‘In progress’ as mentioned in the Part II. Finally, the job of ItemB #3 is triggered because of changes, which are the same as those in ItemB #2.

The job of ItemB #3 will start when the previous two jobs are done, which are IteamA #5 and ItemB #2 in sequence. However, since ItemB #2 will build the latest project, this job, ItemB #3, will be ‘duplicated’. That is why I say ‘redundant’.

What’s interesting about this redundant build, so called ‘pending’, is that it prevents another job in the current item from being queued after polling. When Freestyle was used in the above scenario, ItemB #2 in the Part II was immediately in a pending state unlike Pipeline and ItemB #3 would not happen. That is why I got to think about thread types, heavy-weight or light-weight, to try to understand how Jenkins handles a polling between Freestyle and Pipeline.

Thank you.

[leessang01 (Aloha)]

Thank you for sharing your opinions. Please check my detailed scenario in the above comment to #Artalus.

One important difference in behaviour is a pipeline will always start immediately unless you explicitly disable concurrent execution, so the default is to allow this

I already set “Do not allow concurrent builds” as well as options{ disableConcurrentBuilds() }. So I expected the same result of Freestyle.

You also have to consider that Jenkins will remove duplicates in the queue, so your freestyle job might be added 3 times to the queue in that 1h where a previous build runs but as the parameters are identical this will not lead to 3 executions.

With a single configured executor, in Freestyle, the first job runs and occupies the single executor. In my understanding, the second job in the same item would not be triggered by the same changes because the first job updated some parameters to detect changes. If the second job in another item, this job will be immediately in a pending state. Any events triggered by polling, as you said 3 times added, will not lead to 3 executions. According to my observation, it just leads to stacking ‘Started by an SCM changes’ sentence in Console Output every polling period until the pending job starts and updates the parameters.

But the pipeline (with concurrent execution) will start immediately so it will be executed 3 times, even when in between the pipeline run will be in the queue to wait for an executor.

My pipeline with disableConcurrentBuilds() will start immediately so it will be executed till 2 times. The first is in progress but waiting for next available executor. The second is in a pending state. Each execution triggered after the third is being stacked in the second job’s Console Output, saying ‘Started by an SCM changes’.

[Artalus]

This is much clearer, thanks.

@leessang01

The job of ItemB #2 is not only queued, it hasn’t built yet, but the pop-up says ‘In progress’ as mentioned in the Part II.

I believe this is because Jenkins assigns statuses like “pending”, “in progress” to the entire build and they only change “from left to right”, roughly speaking; so once something became in progress it cannot revert back to pending. Plus the pipeline does not discern between its parts waiting for a node or doing something — technically you can have a parallel{} block where one branch is waiting for a node and another is frantically calculating something without a node. It only shows on the main page in the build queue as (part of ItemB #2), like you mentioned.

@leessang01

The job of ItemB #3 will start when the previous two jobs are done

Well just to be perfectly clear, it waits solely for ItemB #2 to complete due to the setting of disableConcurrentBuilds() in the pipeline. But since you only have a built-in node with a single executor, yeah, that build won’t land until ItemA #5 finishes, so…

@leessang01

What’s interesting about this redundant build, so called ‘pending’, is that it prevents another job in the current item from being queued after polling. When Freestyle was used in the above scenario, ItemB #2 in the Part II was immediately in a pending state unlike Pipeline and ItemB #3 would not happen

This is more related to the fact that Freestyle cannot run at all without allocating a node, plus the disableConcurrentBuilds options. As Markus said, Jenkins will optimize away the queue by preventing new PENDING builds of the same job being queued, unless they have different build parameters. So theoretically I assume that Freestyle still might be triggeringItemB #3 and then it would get obliterated since #2 is already PENDING.

---

From this part being identical (unless you’ve miscopied) both in ItemB #2 & #3:

[poll] Last Built Revision: Revision 68932d5d.. (origin/main)
…
Polling for changes in
> git rev-parse origin/main^{commit} # timeout=10
> git log --full-history --no-abbrev --format=raw -M -m --raw
68932d5d..dc0ec631

I would assume that for some reason Jenkins only remembers an older build starting (and probably finishing) from 68932d5d, and does not remember that #2 was already started from dc0ec631. This honestly looks like a bug to me.

Is it the same with any job? For example if you were to disable ItemB but keep ItemC, would it also trigger two builds with the identical poll logs?

[mawinter69 (Markus Winter)]

The thing is that time based polling will not store the actual git hash as a parameter. Also as long as Itemb #2 hasn’t started doing anything in an executor Jenkins will not know what hash it will actually checkout. In a freestyle job when ItemA #5 is running with based on hash a1b2c3 then it polls and detects a change then for hash d4e5f6, again 15 minutes later it polls and someone has pushed another change with hash g7h8i9. The freestyle job will then effectively have never built hash d4e5f6.

[leessang01 (Aloha)]

The thing is that time based polling will not store the actual git hash as a parameter. Also as long as ItemB #2 hasn’t started doing anything in an executor Jenkins will not know what hash it will actually checkout

That’s exactly what I am saying. ItemA #5 is running and occupying a single executor; ItemB #2 is triggered, in progress and waiting for next available executor; THE BASELINE GIT HASH used for ItemB to detect changes has not been yet updated. So ItemB’s polling to trigger #3 will end up detecting the same changes as before.

In a freestyle job when ItemA #5 is running with based on hash a1b2c3 then it polls and detects a change then for hash d4e5f6, again 15 minutes later it polls and someone has pushed another change with hash g7h8i9. The freestyle job will then effectively have never built hash d4e5f6.

The above context seems to say only ItemA without ItemB unless I misunderstand them. In Freestyle, when ItemA #5 is running based on a1b2c3 and occupying a single executor, its next poll will be able to detect a change for hash d4e5f6. However, ItemA will immediately add a pending state job (maybe #6).

After someone has pushed another change with hash g7h8i9, the next poll can detect it. If ItemA #5 is still running, another job will NOT be added to a queue in Freestyle. I think a pending state job prevents it from being queued. I don’t know exactly when a baseline git hash is updated. Nevertheless, in most instances, the freestyle job doesn’t lose the git hash g7h8i9 because the job previously triggered by hash d4e5f6 clones the latest hash including g7h8i9.

If possible I would suggest to change from polling the SCM to sending events from the SCM to Jenkins and react on the event to trigger the build.

I agree with your suggestion. If what I found turns out to be a bug, I think it would be better to change it from polling to webhook as you said. Thank you for your suggestion.

[leessang01 (Aloha)]

Plus the pipeline does not discern between its parts waiting for a node or doing something

I agree with this sentence. According to a console output in the Part II, ItemB #2 was triggered and started of pipeline and then only was able to discern that there was no executor available.

So theoretically I assume that Freestyle still might be triggering ItemB #3 and then it would get obliterated since #2 is already PENDING.

Yes, it would. The trigger record will remain in a console output of pending status ItemB #2. So the console output may have one more sentence, “Started by an SCM change”.

I would assume that for some reason Jenkins only remembers an older build starting (and probably finishing) from 68932d5d, and does not remember that #2 was already started from dc0ec631. This honestly looks like a bug to me.

I’ve copied right . IMO, the difference comes from whether or not a pending status is immediately added.

As mawinter69 said, as long as ItemB #2 hasn’t started doing anything in an executor Jenkins will not know what hash it will actually checkout. In other words, ItemA #5 is running and occupying a single executor; ItemB #2 is triggered, in progress and waiting for next available executor; THE BASELINE GIT HASH used for ItemB to detect changes has not been yet updated. So ItemB’s polling to trigger #3 will end up detecting the same changes as before.

On the other hand, in Freestyle, ItemB #2 will be immediately to a pending status. Thus Item’s next polling triggers #3 but the job #3 will not be added in a queue. It just records to ItemB #2’s console output with “Started by an SCM change”.

This difference is annoying but I don’t know it is a bug.

Is it the same with any job? For example if you were to disable ItemB but keep ItemC, would it also trigger two builds with the identical poll logs?

Yes, it is. If ItemC is a pipeline and a single executor is still occupied by another job, ItemC works in a same way of ItemB. The second build will be triggered unless the first build occupies a single executor and updates a baseline git hash, I mean a build parameter.

If ItemB were kept and ItemC were triggered, ItemC could have two builds as below.

[Jenkins Dashboard]
Build Queue (4)
(ItemC)
(part of ItemC #2)
(ItemB)
(part of ItemB #2)

Build Executor Status
(1 of 1 executor busy)
-ItemA  #5

[ItemB Dashboard]
Builds
(...) #3 -> pending-Build #2 is already in progress
( ✓ ) #2 AM 09:15 -> the pop-up says 'In progress'

[ItemC Dashboard]
Builds
(...) #3 -> pending-Build #2 is already in progress
( ✓ ) #2 AM 09:30 -> the pop-up says 'In progress'