Still waiting to schedule task

rkmullapudi · April 18, 2024, 4:19am

Hi,

I have Jenkins controller with 2 agents configured. When I tried to deploy on any agent I see the following prompt and deployment job stays there for ever.

*[Pipeline] Start of Pipeline *
[Pipeline] node

*Still waiting to schedule task *
*‘FlutterNode’ is reserved for jobs with matching label expression *
‘PreProd’ is reserved for jobs with matching label expression

I’m not seeing any error in any logs related to this hang. All nodes systems resources are fine and don’t see any resource hogging issue. I’m wondering what could be this issue and our deployment is blocked now. Is there any limitation on number agents that can be configured on one node and what should be this issue?

Best regards,
rk

mawinter69 · April 18, 2024, 5:42am

There are no limitations
It seems the label expression you’re using in your pipeline is not matching anything what you have configured in your agents
Check what label you use in the node step or when using declarative what you have for agent. If that expression doesn’t match any labels on your agents then they job will wait forever.

rkmullapudi · April 18, 2024, 12:52pm

Hi,

What you mentioned is all looking good and I don’t see any discrepancies:

Excerpt from Jenkinsfile for each node

#!groovy
pipeline {
agent {label ‘FlutterNode’}

environment {

#!groovy
pipeline {
agent {label ‘PreProd’}

environment {

Interestingly I’m not seeing any log lines related to this hang and not even in system logs.

Could someone let me know what could be the issue?

Best regards,
RK

rkmullapudi · April 18, 2024, 12:57pm

I see these errors under security monitoring and not sure whether these are related?

MarkEWaite · April 18, 2024, 1:42pm

I can’t duplicate the problem that you’re describing. As @mawinter69 said:

I configured my Jenkins 2.452 agent named “linux-agent” with the setting “Only build jobs with label expressions matching this node”. I assigned an additional label “amd64-Ubuntu-22.04” and then ran a Pipeline job that used the label “linux-agent” and a Pipeline job that used the label " amd64-Ubuntu-22.04". Both jobs completed as expected.

On older Jenkins versions, sometimes I needed to configure the agent, change the agent labels, and save the configuration in order to assure that the label was visible. You might try that as a possible work around until you are able to upgrade to the most recent Jenkins LTS.

MarkEWaite · April 18, 2024, 1:43pm

I doubt that they are related, though running a Jenkins version with known security vulnerabilities is generally not a good practice.

rkmullapudi · April 18, 2024, 1:51pm

Hello Mark,

Thanks for your answers. I don’t know how to fix those security vulnerabilities but I will try to fix them. Coming back nodes configuration I am using these 3 as shown in this snap

When I tried to deploy on one node the other two nodes prompted in the console messages. That means when I tried to deploy on Built-in Node, I see following message:

[Pipeline] Start of Pipeline
[Pipeline] node
Still waiting to schedule task
‘FlutterNode’ is reserved for jobs with matching label expression
‘PreProd’ is reserved for jobs with matching label expression

I’m not sure what I’m missing. Let me fix the security errors and see if it helps.

Best regards,
RK

mawinter69 · April 18, 2024, 2:10pm

can you visit 2 both agents page and list all labels they have?
looks likes this:

This pipeline works fine with this setup:

pipeline {
    agent {
      label 'windows'
    }
    stages {
        stage("build") {
            steps {
    echo "works"
            }
        }
    }
}

also with scripted

node("windows") {
  echo "works"
}

MarkEWaite · April 18, 2024, 2:10pm

You need to assign the “DevNode” label to one or more of the existing agents. You need to assign the “PreProd” label to one or more of the existing agents.

Upgrading Jenkins won’t change the assignment of labels to agents. That’s something you need to do from the agent configuration page of each agent.

rkmullapudi · April 19, 2024, 1:51am

Hi Markus,

Please see these snapshots from all of agents page and I don’t see labels that you have in your snap, but I do see labels in each node configure screen:

Best regards,
RK

rkmullapudi · April 19, 2024, 2:11am

and configure screen for each node:

mawinter69 · April 19, 2024, 7:46am

The agent name is always also a label, so technically you don’t need to repeat the agent name as a label, though it shouldn’t hurt. But also when I do this, I have no problem executing pipeline jobs.

Please also check what you have configured as usage, from the logs you showed I assume it is Only build jobs with label expressions matching this node

I copied your setup

Using this pipeline works fine:

pipeline {
    agent {
      label 'PreProd'
    }
    stages {
        stage("build") {
            steps {
    echo "works"
            }
        }
    }
}

Output:

rkmullapudi · April 19, 2024, 1:13pm

Yes, I do have the same except for built-in Node.

mawinter69 · April 19, 2024, 1:27pm

I can only speculate that something with your pipeline is wrong.
Can you post the snippet again. Please put the snippet in triple single backticks so that it is properly formatted as code

rkmullapudi · April 19, 2024, 1:44pm

---------For the remote agent to deploy on FlutterNode-------

#!groovy
pipeline {
    agent {label 'FlutterNode'}
     }

    environment {
        VENV_PATH = 'x'
        APP_PATH = 'x'
        REQUIREMENTS_FILE = 'requirements.txt'   
    }

  stages {
     stage('Connect') {
        steps {
          sh 'ssh xxxx@x.x.x.x "uptime && ls -l"'
          }
     }

  stage('Checkout') {
       steps { 
                sh 'git status'
                sh 'git pull https://xxxxxx'
                checkout scm
            }
        }

 stage('Install Dependencies') {
   steps {
     script {
     sh "python3 -m venv ${VENV_PATH}"
         }
     }
   }

 stage('Database Migrations') {
    steps {
      script {
sh "xxx/manage.py makemigrations --settings=${DJANGO_SETTINGS_MODULE}"
sh "xxx/manage.py migrate --settings=${DJANGO_SETTINGS_MODULE}"
                }
            }
        }

stage('Collect Static Files') {
    steps {
       script {
sh "xxx}/manage.py collectstatic --settings=${DJANGO_SETTINGS_MODULE} --noinput"
              }
         }
     }
        
stage('Starting Django Application') {
   steps {
     script {
sh 'JENKINS_NODE_COOKIE=dontKillMe nohup xxx/manage.py runserver 0.0.0.0:xxxx &'
               }
         }
    }
   }

 post {
    success {
       echo 'Deployment successful'
    }
   failure {
       echo 'Deployment failed. Rolling back..'
          }
      }

--------------and for Built-In Node -----------------

#!groovy

node {

    try {
        stage 'Checkout'
            sh 'git status'
            sh 'git pull https://xxxxxxxxxxxx'
            sh 'git branch'
    
        stage 'Build'
            sh 'python3 -m venv rvksiteenv'
            sh '. xxxx/bin/activate'
            sh '/xxxx/pip install django gunicorn psycopg2-binary'
            sh '/xxxx/bin/pip freeze'
            sh '/xxxx/pip install -r requirements.txt'

        stage 'Test'
            sh '/xxxx/python manage.py makemigrations'
            sh '/xxxx/python manage.py migrate'
            sh '/xxxx/pytest'

  stage 'Deploy'

sh 'JENKINS_NODE_COOKIE=dontKillMe nohup /xxxx/python manage.py runserver 0.0.0.0:xxxx &'
sh 'echo "deployment is successfully completed."'

    }
catch (err) {
   sh 'echo "Build failed :face_with_head_bandage: \n`${env.JOB_NAME}#${env.BUILD_NUMBER}` <${env.BUILD_URL}|Open in Jenkins>"'

     throw err
     }
}

Note: Pardon me, I adjusted the indexing here to avoid horizontal scrolling when viewing the code and also ignored the quotes.

and deployment on either node hangs with the following console log lines

------------Console log-----------

Started by user [UserX]
Obtained Jenkinsfile from git https://xxxx.git 
[Pipeline] Start of Pipeline 
[Pipeline] node

Still waiting to schedule task 
‘FlutterNode’ is reserved for jobs with matching label expression 
‘PreProd’ is reserved for jobs with matching label expression

Best Regards,
RK

mawinter69 · April 19, 2024, 5:49pm

You can use markdown here to format text
The code you shared is unreadable and it is unclear if everything is correct.

rkmullapudi · April 20, 2024, 1:42am

I’ve deleted some of the unwanted lines to make it as generic code. Please see it if you can read it now. I am able to see it fine on my screen.

Best Regards,
RK

rkmullapudi · April 20, 2024, 4:47am

I upgraded the Jenkins version to

```  <version>2.440.3</version>

and the previous vulnerability errors are cleared, but the deployment result is still the same. I'm not sure what else I could try.

Best Regards,
RK

rkmullapudi · April 20, 2024, 5:46am

Thanks a lot to this post because the exact step recommended in this post helped me from the above blocker.

Topic		Replies	Views
Still waiting to schedule task Waiting for next available executor Using Jenkins question	4	2596	January 8, 2024
continuously get this warning "Still waiting to schedule task All nodes of label ‘slave’ are offline" while trying to use Jenkins master as an Agent. Using Jenkins	0	3645	July 17, 2023
Jenkins was stuck in 'There are no nodes with the label' for too long when creating the k8s agent node Using Jenkins kubernetes	4	6293	January 3, 2024
Jenkins-kubernetes Using Jenkins kubernetes	0	336	November 1, 2023
How does Jenkins monitor the resource scheduling status of the agents? Using Jenkins	1	327	November 7, 2023

Still waiting to schedule task

Related Topics