ERROR: Unable to determine cloud configuration using: Labels

Using Jenkins Ask a question
1

Jenkins setup:
Version 2.501

Deployment based on:

We updated from version 2.47 something earlier today and are struggling to get our inbound agents working again. After resolving a few issues we are now faced with this message:

ERROR: Unable to determine cloud configuration using: Labels: [simple-non-critical-jenkins-task-11-xtx88], inheritFrom: ‘fargate-cloud-jenkins-spot’, Cloud: ‘cloud-default’

fargate-cloud-jenkins-spot is basically a hello world instance built from jenkins/inbound-agent

We had this all working previously and I am at a loss regarding how to proceed.

2

You might try upgrading the EC2 plugin to the most recent release. JENKINS-75390 reported that 1872.v8757931fec80 has a bug that affects spot instances on AWS. That bug was fixed in 1873.vf0b_f234b_e2f3.

If that does not resolve the issue, then you’ll need to provide more information so that others can help. “How to report an issue” includes a list of items and scripts that help provide the necessary information

3

All of the plugins seem up to date, below is a copy of the system information.

Our installation is ECS based with a controller cluster and a spot cluster.

image
image2092×806 109 KB

system Info page:

executable-war

/usr/share/jenkins/jenkins.war
file.encoding

UTF-8
file.separator

/
hudson.lifecycle

hudson.lifecycle.ExitLifecycle
java.awt.headless

true
java.class.path

/usr/share/jenkins/jenkins.war
java.class.version

65.0
java.home

/opt/java/openjdk
java.io.tmpdir

/tmp
java.library.path

/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
java.runtime.name

OpenJDK Runtime Environment
java.runtime.version

21.0.6+7-LTS
java.specification.name

Java Platform API Specification
java.specification.vendor

Oracle Corporation
java.specification.version

21
java.vendor

Eclipse Adoptium
java.vendor.url

java.vendor.url.bug

adoptium/adoptium-support/issues
java.vendor.version

Temurin-21.0.6+7
java.version

21.0.6
java.version.date

2025-01-21
java.vm.compressedOopsMode

32-bit
java.vm.info

mixed mode
java.vm.name

OpenJDK 64-Bit Server VM
java.vm.specification.name

Java Virtual Machine Specification
java.vm.specification.vendor

Oracle Corporation
java.vm.specification.version

21
java.vm.vendor

Eclipse Adoptium
java.vm.version

21.0.6+7-LTS
jdk.debug

release
jenkins.install.runSetupWizard

false
jenkins.model.Jenkins.slaveAgentPort

50000
jetty.git.hash

c3f88bafb4e393f23204dc14dc57b042e84debc7
line.separator

mail.smtp.sendpartial

true
mail.smtps.sendpartial

true
native.encoding

UTF-8
os.arch

amd64
os.name

Linux
os.version

5.10.234-225.895.amzn2.x86_64
path.separator

:
stderr.encoding

UTF-8
stdout.encoding

UTF-8
sun.arch.data.model

64
sun.boot.library.path

/opt/java/openjdk/lib
sun.cpu.endian

little
sun.io.unicode.encoding

UnicodeLittle
sun.java.command

/usr/share/jenkins/jenkins.war
sun.java.launcher

SUN_STANDARD
sun.jnu.encoding

UTF-8
sun.management.compiler

HotSpot 64-Bit Tiered Compilers
user.dir

/
user.home

/var/jenkins_home
user.language

en
user.name

jenkins
user.timezone

Etc/UTC

4

a bit more detail from our configuration:

`jenkins:
authorizationStrategy:
globalMatrix:
entries:
- group:
name: “authenticated”
permissions:
- “Job/Read”
- “Overall/Administer”
- “Overall/Read”
- “View/Read”
clouds:

  • ecs:
    allowedOverrides: “inheritFrom,label,memory,cpu,image”
    cluster: “arn:aws:…:cluster/serverless-jenkins-spot”
    jenkinsUrl: “…:8080”
    name: “fargate-cloud-jenkins-spot”
    regionName: “us-east-1”
    retentionTimeout: 10
    templates:
    • assignPublicIp: false
      cpu: 512
      cpuArchitecture: “X86_64”
      defaultCapacityProvider: false
      enableExecuteCommand: false
      executionRole: “arn:aws:iam::…:role/serverless-jenkins-ecs-execution-role”
      image: “…dkr.ecr.us-east-1.amazonaws.com/jenkins-agent/jenkins:latest”
      label: “build-jenkins-spot”
      launchType: “FARGATE”
      memory: 0
      memoryReservation: 1024
      networkMode: “awsvpc”
      operatingSystemFamily: “LINUX”
      platformVersion: “LATEST”
      privileged: false
      remoteFSRoot: “/home/jenkins”
      securityGroups: “sg-01bd4216253794608”
      sharedMemorySize: 0
      subnets: “subnet-07d97943017e6d425,subnet-0ae8ba80d19f7d4b1”
      templateName: “build-jenkins”
      uniqueRemoteFSRoot: false
  • ecs:
    allowedOverrides: “inheritFrom,label,memory,cpu,image”
    cluster: “arn:aws:…:cluster/serverless-jenkins-spot”
    jenkinsUrl: “…:8080”
    name: “fargate-cloud-node-spot”
    regionName: “us-east-1”
    retentionTimeout: 10
    templates:
    • assignPublicIp: false
      cpu: 512
      cpuArchitecture: “X86_64”
      defaultCapacityProvider: false
      enableExecuteCommand: false
      executionRole: “arn:aws:iam::…:role/serverless-jenkins-ecs-execution-role”
      image: “…dkr.ecr.us-east-1.amazonaws.com/jenkins-agent/node:latest”
      label: “build-node-spot”
      launchType: “FARGATE”
      memory: 0
      memoryReservation: 1024
      networkMode: “awsvpc”
      operatingSystemFamily: “LINUX”
      privileged: false
      remoteFSRoot: “/home/jenkins”
      securityGroups: “sg-01bd4216253794608”
      sharedMemorySize: 0
      subnets: “subnet-07d97943017e6d425,subnet-0ae8ba80d19f7d4b1”
      templateName: “build-node”
      uniqueRemoteFSRoot: false
  • ecs:
    allowedOverrides: “inheritFrom,label,memory,cpu,image”
    cluster: “arn:aws:…:cluster/serverless-jenkins-spot”
    jenkinsUrl: “…:8080”
    name: “fargate-cloud-python-spot”
    regionName: “us-east-1”
    retentionTimeout: 10
    templates:
    • assignPublicIp: false
      cpu: 512
      cpuArchitecture: “X86_64”
      defaultCapacityProvider: false
      enableExecuteCommand: false
      executionRole: “arn:aws:iam::…:role/serverless-jenkins-ecs-execution-role”
      image: “…dkr.ecr.us-east-1.amazonaws.com/jenkins-agent/python:latest”
      label: “build-python-spot”
      launchType: “FARGATE”
      memory: 0
      memoryReservation: 1024
      networkMode: “awsvpc”
      operatingSystemFamily: “LINUX”
      privileged: false
      remoteFSRoot: “/home/jenkins”
      securityGroups: “sg-01bd4216253794608”
      sharedMemorySize: 0
      subnets: “subnet-07d97943017e6d425,subnet-0ae8ba80d19f7d4b1”
      templateName: “build-python”
      uniqueRemoteFSRoot: false
  • ecs:
    allowedOverrides: “inheritFrom,label,memory,cpu,image”
    cluster: “arn:aws:…:cluster/serverless-jenkins-main”
    jenkinsUrl: “…:8080”
    name: “fargate-cloud”
    regionName: “us-east-1”
    retentionTimeout: 10
    templates:
    • assignPublicIp: false
      cpu: 512
      cpuArchitecture: “X86_64”
      defaultCapacityProvider: false
      enableExecuteCommand: false
      executionRole: “arn:aws:iam::…:role/serverless-jenkins-ecs-execution-role”
      image: “jenkins/inbound-agent:latest-alpine-jdk21”
      label: “build-example”
      launchType: “FARGATE”
      memory: 0
      memoryReservation: 1024
      networkMode: “awsvpc”
      operatingSystemFamily: “LINUX”
      privileged: false
      remoteFSRoot: “/home/jenkins”
      securityGroups: “sg-01bd4216253794608”
      sharedMemorySize: 0
      subnets: “subnet-07d97943017e6d425,subnet-0ae8ba80d19f7d4b1”
      templateName: “build-example”
      uniqueRemoteFSRoot: false
      crumbIssuer:
      standard:
      excludeClientIPFromCrumb: false
      disableRememberMe: false
      disabledAdministrativeMonitors:
  • “jenkins.diagnostics.RootUrlNotSetMonitor”
  • “hudson.util.DoubleLaunchChecker”
    labelAtoms:
  • name: “build-example”
  • name: “build-jenkins-spot”
  • name: “build-node-spot”
  • name: “build-python-spot”
  • name: “built-in”
    markupFormatter: “plainText”
    mode: NORMAL
    myViewsTabBar: “standard”
    nodeMonitors:
  • “architecture”
  • “clock”
  • diskSpace:
    freeSpaceThreshold: “1GiB”
    freeSpaceWarningThreshold: “2GiB”
  • “swapSpace”
  • tmpSpace:
    freeSpaceThreshold: “1GiB”
    freeSpaceWarningThreshold: “2GiB”
  • “responseTime”
    numExecutors: 0
    primaryView:
    all:
    name: “all”
    projectNamingStrategy:
    roleBased:
    forceExistingJobs: false
    quietPeriod: 5
    remotingSecurity:
    enabled: true
    scmCheckoutRetryCount: 0
    securityRealm:
    local:
    allowsSignup: false
    enableCaptcha: false
    users:
    • id: “…”
      name: “…”
      properties:
      • “apiToken”
      • “consoleUrlProvider”
      • “myView”
      • preferredProvider:
        providerId: “default”
      • “timezone”
      • “experimentalFlags”
      • mailer:
        emailAddress: “…”
    • id: “…”
      name: “…”
      properties:
      • “apiToken”
      • “consoleUrlProvider”
      • “myView”
      • preferredProvider:
        providerId: “default”
      • “timezone”
      • “experimentalFlags”
      • mailer:
        emailAddress: “…”
    • id: “…”
      name: “…”
      properties:
      • “apiToken”
      • “consoleUrlProvider”
      • “mailer”
      • “myView”
      • preferredProvider:
        providerId: “default”
      • “timezone”
      • “experimentalFlags”
    • id: “…”
      name: “…”
      properties:
      • “apiToken”
      • “consoleUrlProvider”
      • “myView”
      • preferredProvider:
        providerId: “default”
      • “timezone”
      • “experimentalFlags”
      • mailer:
        emailAddress: “…”
    • id: “…”
      name: “…”
      properties:
      • “apiToken”
      • “consoleUrlProvider”
      • “myView”
      • preferredProvider:
        providerId: “default”
      • “timezone”
      • “experimentalFlags”
      • mailer:
        emailAddress: “…”
        slaveAgentPort: 50000
        systemMessage: “Build Server”
        updateCenter:
        sites:
    • id: “default”
      url: “https://updates.jenkins.io/update-center.json”
      views:
  • all:
    name: “all”
    viewsTabBar: “standard”
    globalCredentialsConfiguration:
    configuration:
    providerFilter: “none”
    typeFilter: “none”
    appearance:
    prism:
    theme: PRISM
    security:
    apiToken:
    creationOfLegacyTokenEnabled: false
    tokenGenerationOnCreationEnabled: false
    usageStatisticsEnabled: true
    cps:
    hideSandbox: false
    gitHooks:
    allowedOnAgents: false
    allowedOnController: false
    gitHostKeyVerificationConfiguration:
    sshHostKeyVerificationStrategy: “knownHostsFileVerificationStrategy”
    globalJobDslSecurityConfiguration:
    useScriptSecurity: true
    sSHD:
    port: -1
    scriptApproval:
    forceSandbox: false
    unclassified:
    buildDiscarders:
    configuredBuildDiscarders:
    • “jobBuildDiscarder”
      buildStepOperation:
      enabled: false
      fingerprints:
      fingerprintCleanupDisabled: false
      storage: “file”
      gitHubPluginConfig:
      hookUrl: “…/github-webhook/”
      globalTimeOutConfiguration:
      operations:
    • “abortOperation”
      overwriteable: false
      junitTestResultStorage:
      storage: “file”
      location:
      adminAddress: “address not configured yet nobody@nowhere”
      mailer:
      charset: “UTF-8”
      useSsl: false
      useTls: false
      mavenModuleSet:
      localRepository: “default”
      pollSCM:
      pollingThreadCount: 10
      scmGit:
      addGitTagAction: false
      allowSecondFetch: false
      createAccountBasedOnEmail: false
      disableGitToolChooser: false
      globalConfigName: “…”
      hideCredentials: false
      showEntireCommitSummaryInChanges: false
      useExistingAccountWithSameEmail: false
      timestamper:
      allPipelines: false
      elapsedTimeFormat: “'‘HH:mm:ss.S’ '”
      systemTimeFormat: “'‘HH:mm:ss’ '”
      tool:
      git:
      installations:
    • home: “git”
      name: “Default”
      mavenGlobalConfig:
      globalSettingsProvider: “standard”
      settingsProvider: “standard”`
5

I am suspecting this is related to Cloud: ‘cloud-default’ which I cannot find in the original repo I based our deployment on or any of the modifications we have made.

6

So the issue was complicated, the issue in my initial post was some confusion on my part between of what label to use in inheritFrom. In this case I should have been using label: “build-jenkins-spot” from the configuration of the ecs cloud. Additionally somehow terraform was setting the wrong security group, even on a clean deploy. I was able to uncover this by reviewing the logs when the builds were never connecting to spot instances.