Infrastructure Team Meeting - April 30, 2024

[dduportal (Damien Duportal)]

2024 04 30 Jenkins Infra Meeting

Attendees

• @dduportal (Damien Duportal)
• @hlemeur (Hervé Le Meur)
• @smerle33 (Stéphane Merle)
• @kmartens27 (Kevin Martens)

Announcements

1. Weekly:

• WAR and packages are out
• Changelog is out
• Docker image is out

2. May 1, 8 and 9 are off in most of EU: expect lower availability as usual
3. Microsoft credits (sponsored) => expiration of credits as been moved to end of August 2024

• Goal: consume this credits before expires
• Priorities shift: move cik8s to AKS instead of the new AWS account

Upcoming Calendar

• Next Weekly: 2.457, Tuesday 7 May
• Next LTS: 2.452.1 - (15 May 2024) - Alex Brandes is release lead
• Next Security Release as per jenkinsci-advisories: Thursday 2 May (https://groups.google.com/g/jenkinsci-advisories/c/7u4u0lHbDEc)

Notes

• Done:

  • Delate account and update the default assignee for the atlassian-jira-software-cloud-plugin

  • No email received after signing up

  • User mikehussy blocked for 3 advertising spam messages to issues.jenkins.io

  • not receive mail for reset password

  • Block nigelalvarado account due to same email address as a known spammer

  • Block jeffsnyder account due to issues.jenkins.io spam report

  • accountissue

  • [infra.ci.jenkins.io] Build of the jenkins/docker-inbound-agents are failing since 20 March 2024

  • Support for Visual Studio in windows container image

  • Release of jenkins/* container images on DockerHub errors with HTTP/429

  • renew Azure AD Password before expiration for sponsorship.ci.jenkins.io-tf-managed

  • https://updates.jenkins.io/download/plugins/ionicons-api/70.v2959a_b_74e3cf/ionicons-api.hpi

• Work In Progress:

  • [INFRA-3100] Migrate updates.jenkins.io to another Cloud

    • httpd is now serving the expected redirection. Note we discovered additionl “sub” htaccess for plugins, to get.jenkins.io :party:
    • WiP: separation of file storages for httpd and (mirrorbits + rsync) to ensure safety (no htaccess on mirrorbits)
      • Proposal: removing httpd from the mirrorbits-parent release for UC and use astandalone httpd release (no more value in sharing ingress and storage with httpd in this case)
    • Wip: PRs on update_center2 (publish.sh) to populate the 2 distinct file shares with 2 different contents (only htaccess for httpd and the other files on mirrorbits/rsync)

  • [AWS] Migrate ci.jenkins.io EKS clusters from CloudBees AWS account to Jenkins AWS (sponsored) account

    • Done: AWS admin setup in terraform-state
    • WiP: destination to be updated to AKS (see announcement)
      • Need to study S3 → Azure Blob
        • Back to ci.j’s hard drive (for archiving artifacts)
        • We use Azure blob storage plugin
    • Todo: create operational todolist
    • Todo (AWS): add Herve and Mark as AWS admin in terraform state so any of us can take over during the summer
      • Stephane do this so Damien is not the bus factor

  • migrate storage from premium to standard for jenkins-infra, jenkins-weekly and jenkins-release

    • Done: storage class are created
    • WiP: weekly.ci storage migration plan in details
      • Working on how to manage PV/PVC in Terraform (manage disk + PV + PVC / PVC only) => let’s go PVC so we are safe around disk deletion thanks to retain

  • JDK patch upgrade campaign (April 2024)

    • Wip: Still LTS controller and agents to be released with JDK updates
    • Issue to be closed once all agents are updated: controller are out of scope (for now?)

  • Discrepancy in Plugin Release Date

    • Algolia API keys updated and working
    • But no change despite rebuilding/de-caching
    • WiP: need to deep dive on Algolia to see if the index is good and if the proper API key is used either
    • Todo: what happened around the 28 of Feb. this year?
    • Let’s wait for Gavin

  • New Jenkins mirror in Romania by RCS&RDS

    • No news from them: let’s keep it “stale” in next milestone

  • New Jenkins mirror in Romania by Hostico

    • Back to Herve

  • Plugin health JSON sometimes missing

    • Analysis done: issue with variables in pipeline
    • Todo (short term) use scripted pipeline => Damien
    • Todo (proper fix): update publishReport library to add a parameter to fail when empty files => Herve

  • [Artifactory (repo.jenkins-ci.org)] perform a systematic audit of all non-plugin artifacts in our releases repository to identify and quarantine unused artifacts

    • Damien late. Email to send and remove from milestone

• ToDo (next milestone) (infra-team-sync-2024-05-07 Milestone · GitHub)

  • To host versioned jenkins.io docs on docs.jenkins.io
    • Herve write down task list in the issue (instead of full runbook)
    • Priority to unblock Kris on GSoc project
  • [infra.ci.jenkins.io / accountapp] Build stuck waiting for jdk17 label
    • Damien takes it
  • [infra.ci.jenkins.io] Builds of Kubernetes Management randomly failing
    • Damien takes it with Packer images failed to build due to `(429) Too Many Requests` errors from Chocolatey registry · Issue #4059 · jenkins-infra/helpdesk · GitHub (network outbound for infra.ci)
  • Add `.war.asc` to get.jenkins.io · Issue #4055 · jenkins-infra/helpdesk · GitHub
    • Mark takes it: work in the Core packagin process