Infrastructure Team Meeting - March 21, 2023

[dduportal (Damien Duportal)]

2023 03 21 Jenkins Infra Meeting

Attendees

• @dduportal (Damien Duportal)
• @MarkEWaite (Mark Waite)
• @smerle33 (Stéphane Merle)
• @poddingue (Bruno Verachten)
• @kmartens27 (Kevin Martens)
• @hlemeur (Hervé Le Meur)

Announcements

1. Weekly: in progress (final jenkins.io update delayed due to security advisory, should be good in next hours)
   • Upcoming check to be done (docker, changelog, etc.)
2. Security advisory today on plugins

Upcoming Calendar

• Next Weekly: 28 of March, 2.397
• Next LTS: 05 of April, 2.387.2 (RC2 soon, with Kris Stern as release lead)
• Next Security Release as per jenkinsci-advisories: today
• Next major event:
  • DevoXX France April 12 to 14
  • CDCon in May

Notes

• Done:

  • (March 2023) Update VPN CRL
  • (2023-02-15) Azure Credential for cert.ci.jenkins.io is expired
  • maven-17 label hanging on ci.jenkins.io
  • Create a new credential for Packer as code with terraform
  • pkg.origin.jenkins.io SSL certificate expires Mar 28, 2023
  • 401 when performing plugin release via CD workflow

• Closed as not planned:

  • jenkins/inbound-agent is deprecated in favor of jenkins/inbound-agent ?
  • Signup for Account Access blocked due possible spam requests
  • We can’t access our plugin account
  • I forgot Jenkins credentials. Please help on this matter.
  • forgot jenkins login password

• Work In Progress:

  • Could not create account
  • EC2s are not available
  • Maven central packages not found on build agent
  • agent instability
  • (2023-03-04) Azure Credential for ci.jenkins.io is expired
  • PGP keys for redhat and debian expire on March the 30th
    • URGENT: 9 days left
    • Does not depend on Digicert
    • Let’s check documentation/runbook/material. If no one: le’ts ask Olivier (but only after searching)
    • Do we need to extend the key validity? Or add a new key ? Impacts?
    • Last time: a blog post was written
  • Out of space on a ci.jenkins.io agent in bom build
    • Root problem fixed (no more disk full): 20 PRs treated and merged
    • BOM uses (pods) container with their workspace as emptyDir as expected
    • But the AWS Kubernetes nodes has a disk of only 20Gb which was filled => increased to 200 Gb (and better IOPS for free)
    • Study to eventualy decrease to a lower size (90 Gb) taking in account the 3 pods / node rule, to decrease the cost (storage based)
    • Eventually define /tmp and the Jenkins home as emptyDir
  • AKS: add cluster privatek8s
    • Last service to move here: release.ci.jenkins.io
    • Plan is ready: let’s define a datetime in the issue and communicate
    • Important steps to check:
      • VPN access to update (to private.vpn)
      • Agent allocation
      • Credential (Azure vault) endpoints
  • Apply to Docker Open Source for the jenkinsciinfra and jenkins4eval organizations
    • Damien to send email
    • If not possible, jenkinsciinfra could move to GHCR (GitHub registry)
      • Pros: better RBAC (managed per repository)
      • Risk: Rate Limit/ Costs in GHCR
    • jenkins4eval : same (GHCR ?). Let’s ask the contributors
  • Sunset the robobutler service
    • Let’s move to backlog
  • Add gatsby-plugin-jenkins-layout to ci.jenkins.io and infraci.jenkins.io
    • To be closed
    • Traede off in term of security and permissions
  • Grant limited access to release.ci to some security team folks
  • Update center job is failing
  • Document the code signing certificate renewal process
  • renew the signer certificate for jenkins
  • (Re) Introduce an artifact caching proxy for ci.jenkins.io
    • javadoc under the ACP (but only on ci.jenkins.io)
  • [INFRA-2754] Realign repo.jenkins-ci.org mission
    • LDAP-HA: WiP
  • Compuware and BMC plugins removal

• ToDo (next milestone) (infra-team-sync-2023-03-28 Milestone · GitHub)