Attendees ![:busts_in_silhouette: :busts_in_silhouette:](https://emoji.discourse-cdn.com/apple/busts_in_silhouette.png?v=12)
- @dduportal (Damien Duportal)
- @hlemeur (Hervé Le Meur)
- @smerle33 (Stéphane Merle)
- @poddingue (Bruno Verachten)
- @kmartens27 (Kevin Martens)
- Jay Reddy
Announcements ![:loudspeaker: :loudspeaker:](https://emoji.discourse-cdn.com/apple/loudspeaker.png?v=12)
- Weekly: 2.460
- Changelog: merged!
- WAR Released
- Package is OK (stuck at mirror sync, but done)
- Docker image: todo later today
- Weekly new meeting schedule
- Poll: Sondage - Jenkins Infra SIG weekly meeting time - Framadate
- Let’s try 2 hours earlier than doay for the upcoming weeks
- Don’t hesitate to raise your voice for other option such as alternating 2 different times to cover the whole globe
- (minor) security advisory last Friday: https://groups.google.com/g/jenkinsci-advisories/c/ZAwn9ooELD0
Upcoming Calendar ![:calendar: :calendar:](https://emoji.discourse-cdn.com/apple/calendar.png?v=12)
- Next Weekly: 2.461, Tuesday 4 June 2024
- Next LTS (2.452.2):
- RC this week
- Final June 12th, 2024
- Next Security Release as per jenkinsci-advisories: None
- Upcoming credentials expirations (~3 weeks):
- Azure credential for trusted.ci (8 June 2024)
- @smerle creates the issue and add it to new milestone
- Digital Ocean PATs (10 June 2024)
- @dduportal creates the issue and add it to new milestone
- Azure credential for trusted.ci (8 June 2024)
Cloud Budgets
- Azure (CDF paid)
- March: $4,398 (invoice)
- April:
$4,550 (invoice)
- May (current): $3771 consumed (Forecast at ~4.3k)
- Azure Sponsorship (Microsoft Credits)
- Global Status:
- Credits left: $26,349 until 31 August 2024
- March: $2,375
- April: $2,041
- May (current): $4,266 consumed (Forecast at ~4.7k)
- Global Status:
- DigitalOcean
- Global Status:
- Credits left: $16,522.71 until 2nd January 2025
- March: $938
- April: $840
- May (current): $645 consumed (Forecast at ~$714)
- Global Status:
- AWS:
- CloudBees:
- March: $9,567
- April: $9,782
- May (current): $7706 consumed (Forecast at ~$8.8k)
- Sponsored account
- Global Status:
- Credits left: $60,000 until 31 January 2025
- Untouched
- Global Status:
- CloudBees:
Notes ![:book: :book:](https://emoji.discourse-cdn.com/apple/book.png?v=12)
-
Done:
- ftp.halifax.rwth-aachen.de blocked
- Packer-Images: bug within updatecli to update goss since exclusion of windows 2019 for vsstudio
- YAML syntax error making
updatecli
fail (Go template inside the yaml, used by Goss) preventing us to create dependency bump PRs since 4 weeks - Updatecli fixed by changing the YAML parser (short term fix). Allowed us to release new image versions
- Goss syntax fixes to make the YAML valid: opportunity to add a “failfast” and correct error code: now if goss fail on Windows, then the pipeline fail \o/
- YAML syntax error making
- [ci.jenkins.io] Migrate ci.jenkins.io EKS clusters out from CloudBees AWS account
- Only cleanup was left: done!
- We see the decrease on AWS (CloudBees) and DO bills while consuming in Azure (sponsored account)
-
-
[INFRA-3100] Migrate updates.jenkins.io to another Cloud
- Handover from @lemeurherve to @smerle done (thanks!)
- Done:
- Designed the next steps around production architecture:
- Cloudflare R2: we already have a West-Europe bucket, we need a second one in US-East (same as PKG vm) to provide better latency for US users
- OSUOSL VMs: if Cloudflare fails (whatever reason) we need a fallback. We have 2 unused VMs, with enough resources (CPU/Memory/disk) to be used for this use case
- In case of terrible problem, we also can spin up a VM in DigitalOcean for fallback/alternative
- Credentials for Azure HTTPD (new) - bucket tested with success!
- Designed the next steps around production architecture:
- Wip:
- update-center2: we expect 4 PRs (1 already in draft)
- Abstract away the environment (bucket URLs/credentials) from the
publish.sh
script to allow real life testing without impacting production and manage multiple sync targets per "kind " (e.g. rsync/azcopy/s3/R2) => @dduportal - Adding the azcopy httpd sync support (Draft already there) => everyone
- New R2 bucket to be added => @smerle
- Later: OSUOSL VMs (rsync)
- Abstract away the environment (bucket URLs/credentials) from the
- update-center2: we expect 4 PRs (1 already in draft)
-
New Jenkins mirror in Romania by RCS&RDS
- They still need to add HTTPS on their server (and give us credentials for sync) => pinged them, still waiting
-
New Jenkins mirror in Romania by Hostico
- Done:
- Verified that we can specify user and password in the FTP or RSYNC inline URL (ftp://user:password@hostname/path for instance)
- WiP:
- Waiting for them to re-enable their FTP or rsync => @dduportal once they answer
- Done:
-
Packaging job on ci.jenkins.io never completes - retries fail
- Root cause: cluster change to Azure on ci.jenkins.io
- Wip:
- PR on the project to unblock the pod allocation
- Choice: merge pod template definition between admin-defined and developer defined for this particular one (packaging Docker image in,stead of all in one) => @dduportal with @smerle
-
ftp.halifax.rwth-aachen.de got blocked
- Closeable as C-Otto responded
-
packer-images: git version not always available simultaneously on apt ppa for arm64 and amd
- Proposal: we are experimenting compiling our own git instead of relying on packages (wether ubuntu or PPA) which is safer.
- WiP:
- Compilation work very well and is really fast
- Problem: on VMs, the meta-package
ubuntu-server
hasgit
as a direct dependency- Solution 1: let’s override the
git
package even if present. Need exhaustive file override and might lead the package tree broken. - Solution 2: build our own deb package as a post-step of compilation
- Solution 1: let’s override the
- Cleanup of unused build dependencies
- Updatecli manifest
-
Request Temporal “Admin Read” Access for https://ci.jenkins.io/
- Carlos want to run a “Jenkins Cloud-Friendly” talks in June. He wanted to show different CNCF/Cloud-Native projects linked to a real life Jenkins instance.
- Damien met Carlos, an we’re going to share links to the “public” setup we have for a few elements:
- Azure VM and Azure Container based agents
- Kubernetes agents
- Former EC2 setup
- Datadog setup
- If needed we’ll meet to record Damien’s screen to give him more material
- Note: with their Jenkins account, he can show builds logs with Pod template agent definition
-
Add
.war.asc
to get.jenkins.io- WiP by Mark and Basil
-
migrate storage from premium to standard for jenkins-infra, jenkins-weekly and jenkins-release
- We changed Kubernetes volume from “Dynamic Provisioning” to “Static Provisioning” to control the disk (in Azure API) and its attributes such as type
- We had to create a “dummy” storage class to ensure PV and PVC statically provisionned have the same “storage class” value (not empty and not default)
- Next steps: plan operation with the rsync migration (preparation involved) => @smerle
-
- Nothing done (on Damien to send email)
-
-
New issues:
- Maven 3.9.7 => need an issue @dduportal
- Start discussion about Jenkins core release with JDK17 and Maven 3.9.x => issue (@dduportal and @markewaite)
- NPM Jenkins account - MFA => Enable 2FA on jenkinsci npm account · Issue #4073 · jenkins-infra/helpdesk · GitHub => @lemeurherve
- Add a new private kubernetes cluster in the new sponsored azure subscription · Issue #3923 · jenkins-infra/helpdesk · GitHub => @smerle to see (least priority)
- Replace Blue Ocean in default display URL (or remove the Blue Ocean plugins) · Issue #2833 · jenkins-infra/helpdesk · GitHub => @dduportal with Jay as secondary
-
ToDo (next milestone) (infra-team-sync-2024-06-04 Milestone · GitHub)