Infrastructure Team Meeting - January 28, 2025

Contributing Infrastructure
,
1

Attendees :busts_in_silhouette:

Announcements :loudspeaker:

  1. Jenkins Weekly Releases
    • Last week: 2.494 released with no issues
    • This Week: 2.495 started on time today
  2. Reminder: no weekly team meeting next week (Feburary 4, 2025). Next one is the 11.
  3. Azure Outages impacting Container Instances
    • The Infra/“homemade” ATH has not reported errors on the maven-8-windows since a few days
    • Proposal: revert label to use ACI and close the issue
  4. We had a security advisory (plugins only) last week
    • Minor issue with the new UC: the JenSec script was assuming all files have the same “Last-Modified” header which changed.
    • Our runbooks need updates (post mortem with Kevin):
      • Runbook “trusted.ci”:
        • Clarify how to choose why putting the SSH public key in a hieradata or the other
        • Clarify if the group “sudo” is needed or not
        • Clarify that an RSA 4096 or ECDSA key is needed
        • Describe the Windows + WSL “hosts” file to change (both Windows and WSL)
      • Runbook “updates.jenkins.io”
        • Remove SSH access (and any “legacy”) element
      • Runbook “pkg.origin.jenkins.io”
        • Create it!
        • Define the SSH access with “pkg.origin.jenkins.io” as entrypoint
        • Add a note that it “used” to host updates.jio
      • Runbook “Security”:
        • Make it point to the runbook “ci.jenkins.io” to avoid duplicating information
        • Make it point to the new pkg.origin (instead of updates.jio)
        • Use “ci.jenkins.io” for both SSH and HTTP endpoints (instead of private hostnames) to anticipater for AWS migration
  5. Artifactory Storage: Jfrog asked us to decrease disk space used
    • Goal: from ~9.8Tb to <5Tb
    • We have only 2 Artifactory admins. We’re going to temporarily grant admin permissions to Darin Pope if JenSec team is ok,so he can lead this process
      • Led by Mark, and executed by Darin
    • 2 phases:
      • Incrementals artifacts (e.g. builds from PR or branches used for early testing) are never deleted today. If we start deleting all of them older than 4 weeks, we should save more than 1Tb of storage.
      • Reducing the “mirror” cache size
        • Deletion of unused cached (e.g. not in “public”)
        • Some of the mirrors (mainly atlassian public mirror) should be shrinked to decrease the usage
    • Mark meets with JFrog tomorrow and will report during the Contributor summit

Upcoming Calendar :calendar:

Cloud Budgets

  • Azure CDF:
    • October: $4,0k (invoice)
    • November: $4,3k (invoice)
    • December: $4,4k
    • January: $3,816 (forecast at $4.3k)
  • Azure Sponsorship (Microsoft Credits) - Remaining: $20,869 until May 2025
    • October: $12,9k consumed
    • November: $13k
    • December: $9,5k
    • January: $11,2k (forecast at $11,8k) :warning:
      • Bom, bom, bom
  • DigitalOcean - Remaining ~$15k until January 02, 2026
    • October: $195.67 (invoice)
    • November: $146 (invoice)
    • December: $192 (invoice)
    • January: $200 (forecast at $216)
  • AWS:
    • CloudBees:
      • October: $6,4k
      • November: $3,9k
      • December: $540
      • January: $465 (forecast at $536)
    • Sponsored account (~$57k credits lefts)
      • October: $178
      • November: $482
      • December: $595
      • January: $1,077 (forecast at $1,212)
        • Bom bom bom

Notes :book: