Attendees 
- @dduportal (Damien Duportal)
- @lemeurherve (Hervé Le Meur)
- @MarkEWaite (Mark Waite)
Announcements 
- Jenkins Weekly Releases
- 2 weeks ago: 2.585, completed with no issues
- Last Week: 2.586, completed with no issues
- Note: @en3hD3iMRx6_6IXLNY0Rag detected a bug but should fine - You're invited to talk on Matrix
- This Week: 2.587, completed with no issues
- Announcements:
- Last week (2026-10-13), team meeting cancelled in last minute (team members sick, Jay was alone to hold the fort)
- Jenkins Officer Elections: Jenkins Board and Officer Elections 2026 - Nominations Open
- Register!
- Nominate!
- Vote!
- Team capacity:
- 23 October: Damien off
- 22 October → 07 November: Mark unavailable
- 2nd November: Damien & Herve off
- Priorities:
- Infra Roadmap: Jenkins Roadmap
- Current EPICs (by priority): epics Milestone · jenkins-infra/helpdesk · GitHub
- New EPIC: Apache Central (aka “repo1”) new rate limit policies
- Topic worth mentioning (timeframe of the upcoming 2 months)
- Nginx ingress retirement - Inform of Ingress NGINX Retirement in March 2026 · Issue #4869 · jenkins-infra/helpdesk · GitHub
- Kubernetes 1.35 upgrade - Upgrade to Kubernetes 1.35 · Issue #5168 · jenkins-infra/helpdesk · GitHub
- Unblocked!
- Today is Java patch day. But Oracle has not released anything (yet). So let’s wait until they do (and that Adoptium downstream these patches)
- Sponsorings:
- DigitalOcean announced sunset of their OSS program
- NodeOPS
- Their plugin has been released, we can start full evaluation
- GitHub - jenkinsci/createos-sandbox-plugin · GitHub
- Damien to share access with the team to move to next steps
Upcoming Calendar 
- Next infra meeting: 2026-10-27
- Next Weekly: 2026-10-27 - 2.588
- Next LTS: 2026-10-28 - 2.580.2
- Release lead: Kris Stern
- Backports helpdesk: (in triage) Prepare release, packaging and docker backports for LTS 2.580.2 · Issue #5337 · jenkins-infra/helpdesk · GitHub
- Next Security Release as per jenkinsci-advisories: N.A.
- Upcoming credentials expirations (~3 weeks):
- 2026-10-31: [infra.ci] Netlify token expires. => issue to create @dduportal
- 2026-11-02: [infra.ci] contributors|stats|plugins FileShare service principal credentials expire => @7JoquqrvQFu6FDlR14JVhg
- 2026-11-09: [infra.ci] NPM token expires => issue to create (WHO?)
- Next major event: N.A.
Cloud Budgets
-
Azure (CDF paid)
- August: $2,179.52
- September: $2,080.67
- October: $1,423.81 (forecast at $2,310)
- publick8s is being removed: we might be able to reach 2.0k or even less
Max. threshold of $2.5k monthly will be met
-
Azure Sponsored subscription (Microsoft Credits): $63,277.09 credits on October 01 2026 (expires on Nov. 2027)
- August: $6,652.23
- Septemper: $5,843.97
- October: $4,584.23 (forecasted at $7,250)
- Increased cert.ci builds
- Added publick8s-sponsored
At current rate, we shall have about 9 months (at 6.4k/monthly), depleting credits on June 2027
-
DigitalOcean - Remaining $4,819.07 until January 02, 2027 (on October 6)
- August: $382.55
- September: $507.39
- October: $342.43 (forecast at ~$550)
-
AWS:
- Sponsored account, credits left: $19,675.63 until 01/31/2028 (on 13th October)
- August: $5,398.51 -8.3% compared to last month
- September: $4,749.17 -12% compared to last month, -19.4% compared to July
- October: $3,216.67 (forecasted at $4,900)
At current rate ($5.0k monthly), we shall deplete credits on January 2027- AWS OSS programm told us it is too early to request more credits but they are happy to renew. They asked us to retry in December.
- Sponsored account, credits left: $19,675.63 until 01/31/2028 (on 13th October)
-
JFrog Artifactory Usage
- Storage: 1.98 TB - steady
- Bandwidth:
- August: 36.53 TB
- September: 39.22 TB
- October: 24.83 TB (forecast: 40 TB)
- releases: 21.11 Tb (-18% since last month)
- atlassian-pkg-maven-public: 1.02 Tb (+112% since last month)
- incrementals: 0.9 Tb (+61% since last month)
Notes 
-
Done:
-
support
- issues.jenkins.io serving Kubernetes Ingress Controller Fake Certificate
- JIRA was down and we temporarily pointed its DNS to publick8s.
- issues.jenkins.io serving Kubernetes Ingress Controller Fake Certificate
-
keep-infra-sane-and-maintainable
- [archives.jenkins.io] Data disk passed the 80% space usage threshold
- Increased disk (no data to remove in this machine) following the steady rate of +50Tb once a year
- [infra.ci.jenkins.io] Terraform Cloudflare Job is failing since 4 September 2026
- The Terraform Cloudflare provider was buggy since almost 2 years but FINALLY fixed for our use case.
- Required a bit of code change and then performed an update wih no issues.
- Weird responses time from issues.jenkins.io
- JIRA was down. CDF took care of it. It’s back since 1 week at least.
- [stats.jenkins.io/infra-statistics] Move “data for the usage stats site” generation (from anonymized data) out from Andrew machine
- stats.jenkins.io is up to date with the data we have (from KK, up to mid-March 2026)
- We’ve opened issues for follow ups.
- [archives.jenkins.io] Data disk passed the 80% space usage threshold
-
keep-infra-up-to-date
-
-
Closed as not planned:
-
-
keep-infra-up-to-date
- Migrate
publick8sAKS cluster to the Azure 2026 Sponsored subscription- New cluster
publick8s-sponsoredis handling production
- Cleanup of publick8s in progress
- Need to test the NAT gatewayV2 (IPv6)
- Need to check get.jenkins.io mirrors (outbound IPs changed) and other minors things
- Need to do a team post mortem to see what/if we can improve things for when this cluster will be back to CDF subscription in 2027
- Document the inbound Public IP migration/evaluate green blue using a Global LB (since we stayed on the same Region in the end)
- Document how NOT to scale down LDAP during operation
- What did we learn?
- Green Blue seems easier to handle
- Next steps: cleaning up resources
- New cluster
- [get.jenkins.io,mirrors.updates.jenkins.io] Resume GeoIP database weekly update
- New storage is used but still aug. 2025 DB
- Next step: update the GeoIp DB in new storage and geoupdate mirrors
- Prepare update-center root CA rotation
- On hold: Damien to find a second person (checking with Wadeck/board)
- Migrate
-
support
-
BOM job on cert.ci.jenkins.io rate limited by repo1.maven.org
- EPIC to be created
- Jenkins Security team was blocked by this behavior
- Short term (custom Maven cache archive used from their pipelines) works.
- Next steps: create issues to generalize the client-side cachings (Maven, and Gradle for now) to all agents in Azure, on all controllers
-
[ci.jenkins.io] Dependency failures from Apache Central (
repo1)- EPIC to be created (same as cert.ci)
- Pinned issue, we can start working on it now that cert.ci and publick8s have been treated
- Next steps: check ACP errors and determine if updating the cache generation (from what we learned in cert.ci, moving cache to infra.ci as cronjobs with wider scope, and maybe generate daily instead of weekly) is sufficient
- Most probably we’ll have to set up a private artifactory/nexus to replace ACP in AWS
-
Jenkins Incrementals Publisher not available in PR
- Sounds like an edge case in the pipeline library
buildPlugin()when a windows agent is spot-reclaimed - Let’s open an issue in pipeline-library, and close this issue as the user problem is solved
- Sounds like an edge case in the pipeline library
-
- The Sonarcloud token does have needed permissions. We have to resume work on this.
-
-
keep-infra-sane-and-maintainable
- [packer-images] Migrate the self-contained install functions of ubuntu-provision.sh to Ansible roles
- WIP: [packer-images] Migrate the tier 1 self-contained install functions of ubuntu-provision.sh to Ansible roles
- Delivered examples of “direct binary” downloads and “apt packages”
- WiP: review of “archives” downloads to be delivered then review of “batch tiers 1”
- WIP: [packer-images] Migrate the tier 1 self-contained install functions of ubuntu-provision.sh to Ansible roles
- [packer-images] Migrate the self-contained install functions of ubuntu-provision.sh to Ansible roles
-
uncategorized
- [INFRA-249] Kohsuke might be the only one who can decrypt statistics data
- On hold: waiting for KK and/or Olivier for at least the private key
- [INFRA-249] Kohsuke might be the only one who can decrypt statistics data
-
-
Issues added to the next milestone:
-
Issues staying in backlog/triage:
- [ci.jenkins.io] DockerHub transparent mirror/proxy seems to be down, resulting in HTTP/429 from DockerHub
- Bump NodeJS from 22 to 24 (most recent LTS)
lintstage fails with JavaScript heap out of memory for docs.jenkins.io- [buildWebsite] Some failed stages don’t stop builds and
retrydirective doesn’t seems to be effective - [Azure] Decommission
privatek8scluster in favor of VMs - Sign and publish (CD)
winpbinaries from release.ci.jenkins.io - Use VM to publish Jenkins Core MSI installer from release.ci.jenkins.io
- Allow to use VM agents on release.ci.jenkins.io
- [(updates|get).jenkins.io] provide a mirror for Chinese users
- [All websites/pipeline library
buildWebsite] EnforceNODE_ENVtoproduction - Harmonise spot and nonspot agent template labels on ci.jenkins.io
- [updates.jenkins.io] Get rid of Cloudflare R2 buckets for UC mirrors in favor of plain old VMs
- Switch ci.jenkins.io git implementation from JGit to command line git
- [ci.jenkins.io] optimize Maven Cache loading during EC2 Agent startup
- [Maven Dependency] could not be resolve
org.eclipse.m2e:lifecycle-mapping:jar:1.0.0Maven plugin - [packer-images] Bump Ubuntu base image from 22.04 to 26.04
- Upgrade to Kubernetes 1.35
- Debian upgrade campaign:
bookworm(12.x) totrixie(13.x) - Tombstone
plugin-site-apiservice (in favor of a static report file) - See if we can get OIDC enabled on our artifactory
- main branch Jenkins build status is in progress but not build was triggered
- Sporadic HTTP 503 First byte timeout errors on jenkins.io
- Provide Maven 4 pre-release templates on ci.jenkins.io
- Move docker controller images publication job from trusted.ci.jenkins.io to release.ci.jenkins.io
- Enable Jenkins CLI
- [(status|contributors|stats|docs|stories|cn|jenkins-io-components).jenkins.io, Websites PR previews] Netlify billing plan changed: evaluate and act
- Prepare for post-Jira account management
- Inform of Ingress NGINX Retirement in March 2026
- [Azure Deprecation] Convert your OS disks to Standard SSD or Premium SSD before 8 September 2028
- [Azure] Merge webservices data storage accounts into a single one with NFS v4.1
- private docker image registry for staging core security releases
- [ci.jenkins.io] diagnose and eventually fix the EC2 Windows VM performances and build failures/retries
- [PHS/Datadog] improve monitoring using application logs
- Add a real-world job to weekly.ci.jenkins.io
- [ci.jenkins.io] Monitor and Garbage collect data volume of the DockerHub registry mirror and EC pull through cache
- Move collection of stats out from Kohsuke’s home
- Support [skip ci] on default branch
- Create build for jenkinsci/winp on release ci server
- [Update Center] HTTP/404 on
/current/updates/*.json*links - Add monitoring for CD secrets updates
- packer-images: git version not always available simultaneously on apt ppa for arm64 and amd
- Support CD workflow on Windows
- Add reminder about Jira notification email address
- [INFRA-3167] Move security settings to configuration-as-code for puppet managed instances
- [INFRA-3046] Monitor Jenkins mirrors Age
- [INFRA-2352] Add monitoring for age of crawler files