Infrastructure Team Meeting - October 06, 2026

[dduportal (Damien Duportal)]

2026 10 06 Jenkins Infra Meeting

Attendees

• @dduportal (Damien Duportal)
• @lemeurherve (Hervé Le Meur)
• @MarkEWaite (Mark Waite)

Announcements

1. Jenkins Weekly Releases
   • Last Week: 2.584, completed
   • This Week: 2.585, completed
2. Announcements:
   • New infra SIG meeting times!
     • Once every 2 weeks at 08:30AM UTC (like next week) to cover for Asia/EU => same event as before
     • Once every 2 weeks at 01:00pm UTC (like this week) to cover for USA/EU => new event, susbcribe or ask to be invited
   • Team capacity: N/A
   • Priorities:
     • Infra Roadmap: Jenkins Roadmap
     • Current EPICs (by priority): epics Milestone · jenkins-infra/helpdesk · GitHub
   • (Jenkins JIRA) issues.jenkins.io is down, LF ticket opened
   • Topic worth mentioning (timeframe of the upcoming 2 months)
     • Nginx ingress retirement - Inform of Ingress NGINX Retirement in March 2026 · Issue #4869 · jenkins-infra/helpdesk · GitHub
     • Kubernetes 1.35 upgrade - Upgrade to Kubernetes 1.35 · Issue #5168 · jenkins-infra/helpdesk · GitHub
       • Blocked by publick8s migration to Sponsored Subscription, in progress
   • Sponsoring: DigitalOcean announced sunset of their OSS program
     • See for details So long DigitalOcean and thanks for all the fish (and cloud credits)
   • Sponsoring: NodeOPS
     • Their plugin has been released, we can start full evaluation
     • GitHub - jenkinsci/createos-sandbox-plugin · GitHub
     • Need to share accesses with the team to move to next steps

Upcoming Calendar

• Next infra meeting: 2026-10-13 at 08:30am UTC (Asia/EU)
• Next Weekly: 2026-10-13 - 2.586
• Next LTS: 2026-10-28 - 2.580.2
  • Release lead: Kris Stern
  • Backports helpdesk: (pending issue creation)
• Next Security Release as per jenkinsci-advisories: N/A
• Upcoming credentials expirations (~3 weeks):
  • 2026-10-19: Expiration of the Digital Oceans PATs: [infra.ci.jenkins.io] DigitalOcean PAT used by terraform expires on the 2026-10-14 · Issue #5319 · jenkins-infra/helpdesk · GitHub
  • 2026-10-31: [infra.ci] Netlify token expires => @dduportal
• Next major event: N.A.

Cloud Budgets

• Azure (CDF paid)

  • May: ~$2,800
  • June: ~$2,108
  • July: $2,182.84
  • August: $2,179.52
  • September: $2,080.67
  • October: $363 (forecast at $2,136)
    • Max. threshold of $2.5k monthly met
    • Note: publick8s should move to sponsored subscription (~$990 monthly)

• Azure Sponsored subscription (Microsoft Credits): $60,447 credits on October 06 2026 (expires on Nov. 2027)

  • June: ~$5,542
  • July: $5,518.05
  • August: $6,652.23
  • Septemper: $5,843.97
  • October: $761 (forecasted at $4,369)
    • At current rate, we shall have about 11 months (at 5.5k/monthly), depleting credits on August 2027 (unless we have to move ci.jenkins.io here)

• DigitalOcean - Remaining $5,079.48 until January 02, 2027 (on October 6)

  • May: $1186.97
  • June: $740.32
  • July: $496.88
  • August: $382.55
  • September: $507.39
  • October: $84.72 (forecast at ~$438)
    • So long DigitalOcean and thanks for all the fish (and cloud credits)

• AWS:

  • Sponsored account, credits left: $24,424.79 until 01/31/2028 (on 22nd September)
    • May: $14.9k
    • June: ~$7,893
    • July: $5,891.74
    • August: $5,398.51 -8.3% compared to last month
    • September: $4,749.17 -12% compared to last month, -19.4% compared to July
    • October: $814.96 (-24% than same period last month) (forecasted at $4,346.44)
      • At current rate ($4.3k monthly), we shall deplete credits on January 2027
      • No more cost reduction effort: from now we focus on either getting more credits or moving to Azure (or somewhere else) - [Autumn/2026] Determine and set up the future Cloud location of `ci.jenkins.io` before EoY 2026 · Issue #5283 · jenkins-infra/helpdesk · GitHub

• JFrog Artifactory Usage

  • Storage: 1.76 TB (was 1.99 TB last week)
  • Bandwidth:
    • May: 29.98 TB
    • June: 23.24 TB
    • July: 31.3 TB
    • August: 36.53 TB
    • September: 39.22 TB
    • October: 6.37 TB (forecast: 33 TB)

Notes

• Done:

  • support

    • Requesting VPN for jenkins project
    • Requesting VPN for jenkins project
    • Request to delete Jenkins ticket
    • [get.jenkins.io] Ensure TUNA (China-based) mirror is reliable for Chinese users (original title: Support Mirror Selection in Update Center)
      • TUNA mirror has been removed
      • New issue created for creation of our own China mirror, links given by a contributor for other potential chinese university mirrors

  • keep-infra-up-to-date

    • Backport docker, packaging, and release changes for LTS 2.580.1
    • Upgrade controllers to Jenkins 2.580.1
    • [infra.ci.jenkins.io] Renew Algolia write API token used for plugins.jenkins.io before 2026-10-22
    • Renew ci.jenkins.io API token for incremental-publisher service before 2026-10-07
    • [infra.ci.jenkins.io] Azure Credential used by Packer Image builds expires on 2026-10-14

  • keep-infra-sane-and-maintainable

    • Use a common buildWebsite pipeline library function for websites
      • Issue to open about retry not working (?) => @hervelemeur
    • Optimise costs and maintainance by merging Windows 2022 and Windows 2025 templates
      • We can try out devdrive on those Windows 2025 agents

• Closed as not planned:

  • support
    • Windows build failure not propagated to overall job status

• Work in Progress:

  • support

    • Jenkins Incrementals Publisher deos not show in PR
    • Create sonarcloud-token
      • Pending contributor feedback

  • keep-infra-sane-and-maintainable

    • Weird responses time from issues.jenkins.io
    • [packer-images] Migrate the tier 1 self-contained install functions of ubuntu-provision.sh to Ansible roles
    • [packer-images] Migrate the self-contained install functions of ubuntu-provision.sh to Ansible roles
    • [stats.jenkins.io/infra-statistics] Move “data for the usage stats site” generation (from anonymized data) out from Andrew machine
      • Issue to be closed by @dduportal after the cleanup

  • keep-infra-up-to-date

    • Migrate publick8s AKS cluster to the Azure 2026 Sponsored subscription
    • Prepare update-center root CA rotation
    • [get.jenkins.io,mirrors.updates.jenkins.io] Resume GeoIP database weekly update

• New items added to the next milestone (infra-team-sync-2026-10-13 Milestone · jenkins-infra/helpdesk · GitHub):

  • [archives.jenkins.io] Data disk passed the 80% space usage threshold
  • [infra.ci.jenkins.io] Terraform Cloudflare Job is failing since 4 September 2026
  • [infra.ci.jenkins.io] DigitalOcean PAT used by terraform expires on the 2026-10-19
  • Maven 3.10.0 upgrade campaign

• Issues kept in backlog (after triage):

  • [Azure] Decommission privatek8s cluster in favor of VMs
  • Sign and publish (CD) winp binaries from release.ci.jenkins.io
  • Use VM to publish Jenkins Core MSI installer from release.ci.jenkins.io
  • Allow to use VM agents on release.ci.jenkins.io
  • Improve Jenkins release process
  • [(updates|get).jenkins.io] provide a mirror for Chinese users
  • [All websites/pipeline library buildWebsite] Enforce NODE_ENV to production
  • Harmonise spot and nonspot agent template labels on ci.jenkins.io
  • [updates.jenkins.io] Get rid of Cloudflare R2 buckets for UC mirrors in favor of plain old VMs
  • Switch ci.jenkins.io git implementation from JGit to command line git
  • [ci.jenkins.io] optimize Maven Cache loading during EC2 Agent startup
  • [Maven Dependency] could not be resolve org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 Maven plugin
  • [packer-images] Bump Ubuntu base image from 22.04 to 26.04
  • Upgrade to Kubernetes 1.35
  • Debian upgrade campaign: bookworm (12.x) to trixie (13.x)
  • Tombstone plugin-site-api service (in favor of a static report file)
  • See if we can get OIDC enabled on our artifactory
  • main branch Jenkins build status is in progress but not build was triggered
  • Sporadic HTTP 503 First byte timeout errors on jenkins.io
  • Provide Maven 4 pre-release templates on ci.jenkins.io
  • Move docker controller images publication job from trusted.ci.jenkins.io to release.ci.jenkins.io
  • Enable Jenkins CLI
  • [(status|contributors|stats|docs|stories|cn|jenkins-io-components).jenkins.io, Websites PR previews] Netlify billing plan changed: evaluate and act
  • Prepare for post-Jira account management
  • Inform of Ingress NGINX Retirement in March 2026
  • [Azure Deprecation] Convert your OS disks to Standard SSD or Premium SSD before 8 September 2028
  • [Azure] Merge webservices data storage accounts into a single one with NFS v4.1
  • private docker image registry for staging core security releases
  • [ci.jenkins.io] diagnose and eventually fix the EC2 Windows VM performances and build failures/retries
  • [PHS/Datadog] improve monitoring using application logs
  • Add a real-world job to weekly.ci.jenkins.io
  • [ci.jenkins.io] Monitor and Garbage collect data volume of the DockerHub registry mirror and EC pull through cache
  • Support [skip ci] on default branch
  • Create build for jenkinsci/winp on release ci server
  • [Update Center] HTTP/404 on /current/updates/*.json* links
  • Add monitoring for CD secrets updates
  • packer-images: git version not always available simultaneously on apt ppa for arm64 and amd
  • Support CD workflow on Windows
  • Matomo github/docker repos
  • Add reminder about Jira notification email address
  • [INFRA-3167] Move security settings to configuration-as-code for puppet managed instances
  • [INFRA-3046] Monitor Jenkins mirrors Age
  • [INFRA-2352] Add monitoring for age of crawler files
  • [INFRA-249] Kohsuke might be the only one who can decrypt statistics data
  • [ci.jenkins.io] DockerHub transparent mirror/proxy seems to be down, resulting in HTTP/429 from DockerHub