Infrastructure Team Meeting - June 14, 2022

[dduportal (Damien Duportal)]

2022 06 14 Jenkins Infra Meeting

Attendees

• @dduportal (Damien Duportal)
• @hlemeur (Hervé Le Meur)
• @MarkEWaite (Mark Waite)
• @smerle33 (Stéphane Merle)
• @poddingue (Bruno Verachten)

Announcements

1. Weekly: going well, checklist to cover, no error this week
2. Outage: repo.jenkins-ci.org

Notes

• Done (infra-team-sync-2022-06-14 Milestone · GitHub)

  • Remove basil from plugins he is no longer maintaining github-permissions triage
  • [ci.jenkins.io] Try the kubernetes-plugin PR
  • Add an HTTP permanent redirect from https://jenkinsistheway.io to https://stories.jenkins.io/ azure jenkinsistheway kubernetes stories.jenkins.io triage
  • Weekly release 2.352/2.353/2.354 is failing release
  • eventually stop using img at all (removing code in the library and eventually cleanup docker-builder)
  • bootstrat a terraform project for oracle oracle terraform
  • Deprecation warning in packaging pipeline release.ci.jenkins.io
  • High memory runners fail to come online ci.jenkins.io
  • Activate signoff on web based commits github
  • GSoC: Plugin Health Scoring system gsoc gsoc-2022

• Work in Progress (infra-team-sync-2022-06-14 Milestone · GitHub)

  • [INFRA-3100] Migrate updates.jenkins.io to another Cloud
    • WiP. on terraform’s oracle resources
  • [ci.jenkins.io] Digital Ocean sponsorship runs out of credits: stopping the cluster
    • Meeting with DO later (Herve and Damien)
    • Different costs studied ([ci.jenkins.io] Digital Ocean sponsorship runs out of credits: stopping the cluster · Issue #2917 · jenkins-infra/helpdesk · GitHub)
  • DockerHub rate limiting
    • No feedbacks from Docker. Damien to send an email.
  • Upgrade to Kubernetes 1.22
    • kubectl upgraded
    • DigitalOcean WiP (almost done as per @smerle)
  • [ci.jenkins.io] Provides powershell on the Windows Docker images used on ACI
    • Tricky issue as both pwsh and powershell are valid
    • Issue to be raised to the pipeline plugin implementing the 2 steps pwsh and powershell: feature request to have auto-detection
    • As per @lemeurherve 's feedback: we have to update our pipeline-library to use pwsh (or bat as fallback like Jesse and James did) instead of powershell to avoid errors with our nanoserver agents
  • [JENKINS-49707] Evaluate retry to improve the stability of the builds
    • Feedback to do in https://github.com/jenkins-infra/pipeline-library/pull/405 to Jesse about “usage”
    • As per <[JENKINS-49707] Evaluate `retry` conditions to improve the stability of the builds · Issue #2984 · jenkins-infra/helpdesk · GitHub, we rolled it out on infra.ci earlier today. Le’ts wait and see how it behaves (focus: what happens when controller restart with kubernetes-management pod agents + check for failing VM agents when the agent process is done, like it would with reclaimed spot VMs)
    • Raised by @lemeurherve : comment from Jesse to check (and eventually answer) [JENKINS-49707] Experimental plugin updates for `node`-block retry by jglick · Pull Request #512 · jenkins-infra/docker-jenkins-weekly · GitHub
  • Auto notify people based on service routing rules
    • Removed from milestone (wasn’t able to work on it)

• New/Important infra-team-sync-next

  • [ Rollback JDK8 removal for the 2.346 LTS upcoming line]
    • @en3hD3iMRx6_6IXLNY0Rag to open an issue for this
    • @dduportal ready to work on it as he already did the same kind of rollback on 2.332 line for upcoming security releases
  • Maven 3.8.6 upgrade campaign ci.jenkins.io
    • Waiting for Daniel Beck’s go (ref. security release)
  • Replace s390x Ubuntu 18.04 agent with s390x Ubuntu 20.04 agent
    • Mark takes it
    • Low priority
    • As discussed end of meeting: outages for the s390x and powerpc agents does not justify updating status.jenkins.io: tt only bothers the infra team, not other users.
  • enable Development integration in JIRA
    • Legit request
    • Require a technical user in jenkinsci, dedicated for this as we need oauth

• ToDo (next milestone) (infra-team-sync-2022-06-21 Milestone · GitHub)