Best practice to programmatically update Jenkins EC2 Cloud AMI IDs across many controllers (auth + API approach)

Using Jenkins
1

I’m looking for guidance on the best way to programmatically update the AMI ID in Jenkins Amazon EC2 plugin cloud configuration (Manage Jenkins → Clouds → Amazon EC2) across a large fleet of Jenkins controllers (hundreds of instances).

Goal

Automate updating the AMI ID used by EC2 worker nodes in the cloud configuration in a safe and idempotent way (only update if AMI differs), and avoid Selenium/UI automation.

Authentication challenge

I’m trying to start with a read-only call:

curl -u "user:password" https://<jenkins-host>/whoAmI/api/json

But I consistently get:

  • 401 Unauthorized
  • WWW-Authenticate: Basic realm="Jenkins"

Even though the same user/password works in the browser UI (SSO/LDAP based).

API tokens work on the Jenkins controller where the token was created, but fail on other controllers.

Questions

  1. In LDAP/SSO setups, is it expected that browser login works but password-based Basic auth for API calls returns 401?
  2. What is the recommended scalable authentication mechanism for automation across many Jenkins controllers?
  • service account + API token per controller?
  • OAuth/OIDC tokens?
  • reverse proxy / header-based auth?
  1. What is the recommended supported way to update Amazon EC2 plugin AMI configuration programmatically?
  • Groovy via Script Console (/scriptText)?
  • Jenkins Configuration as Code (JCasC)?
  • direct config.xml update + reload?
  • any plugin-supported REST API?

Any best practices, references, or examples would be appreciated.

Thanks!

2

You first need a crumb when you don’t use a token to access the API.

Using a token would be the preferred way. A token is instance specific so you can’t use the same token on all instances.

When your instances are already setup with JCasC that would be the preferred way.

3

Thanks, this is helpful.

A couple of clarifications so I can implement this cleanly at scale:

  1. Crumb vs token: If I use API token auth (user:token) for POST operations, do I still need to fetch and include a crumb (/crumbIssuer/api/json), or is the crumb only required when using username/password auth?
  2. Best supported update mechanism: For updating the Amazon EC2 plugin cloud config AMI ID, what is the recommended/supported approach?
  • Groovy via /scriptText (preferred?)
  • POSTing updated /config.xml
  • any plugin-specific REST API
  1. JCasC adoption: If we move to JCasC, can we manage only the cloud configuration section (AMI updates) without migrating the entire Jenkins config, and does it require a restart or support reload?
  2. Since tokens are instance-specific, is the recommended enterprise pattern typically service account + per-controller token provisioning (bootstrap/bake), or is there a better standard approach for large fleets?

Thanks again!

4

  1. You don’t need the crumb when you use an api token.

  2. The best approach also depends on the specific setup. Clouds are stored in the global config.xml. Afaik you can’t post the global config.xml to Jenkins. The only way would be to update the config.xml in the file system and then tell Jenkins to reload the configuration. This must ensure that you don’t delete/modify any unrelated things in the config.xml

  3. When using CasC it should be clear that you can’t configure configurable objects only partially. You can configure clouds only then it must be all clouds and any changes done by admins to the clouds via the UI that are not reflected in the casc yaml file are lost.
    You can’t use CasC to only change the AMI id of a single cloud . The CasC configuration must be complete for all clouds. That means when the admins of your instances apply manual changes to your clouds you would need to make sure that you first fetch the complete configuration of all clouds, then change the AMI id in affected clouds and then updload the yaml to apply it. So when you upload a yaml with only a single cloud and the instance had 2 clouds before, then you have only one cloud after applying.

  4. I just checked and you can create tokens with a plain value by doing a post call call to jenkins_root/user/<userid>/descriptorByName/jenkins.security.ApiTokenProperty/addFixedToken?newTokenName=<tokenName>&newTokenPlainValue=<value>. Once done with user / password and crumb initially, you can later use that and have the same token on all your instances. This method has following javadoc so use with care:

This method is dangerous and should not be used without caution.
The token passed here could have been tracked by different network system during its trip.
It is recommended to revoke this token after the generation of a new one.

So from that I guess that using the approach via /scriptText is the way to go
I’m not familiar with the EC2 cloud plugin so I don’t know if there are any rest apis that allow to change only the id.

I would suggest that you play around with a test instance and checkout the best approach for you before doing that on the complete landscape.