Infrastructure Team Meeting - Feb. 08 2022

[dduportal (Damien Duportal)]

2022 02 08 Jenkins Infra Meeting

Participants

Damien Duportal (@dduportal ), Hervé Le Meur (@hlemeur ), Stephane Merle (@smerle), Mark Waite (@MarkEWaite ), Tim Jacomb (@timja )

Official minutes on GitHub.

Notes

• Security releases

  • 2022-02-09:
    • Core(s) only (weekly 2.334 and LTS 2.319.3)
    • release.ci.jenkins.io and ci.jenkins.io must NOT be updated until the release is applied
    • ci.jenkins.io and release.ci (and trusted and infra) had been updated (plugins, kernels, packages)
    • Automatic weekly got triggered again. Daniel caught it and was kind enough to stop it.
      • Disable release.ci management for the sec. release of 2022-02 by dduportal · Pull Request #1979 · jenkins-infra/kubernetes-management · GitHub to remove release.ci from kubernetes management to avoid unexpected reload (at least MOST of the unexpected reloads)
    • Release (jenkins-infra/release) pipelines:
      • Weekly staged released
      • LTS (security) failed (missing jarsigner): I forgot to cherry pick from main (weekly) branch. Fixed for sec LTS with https://github.com/jenkins-infra/release/pull/220 and with current LTS with cherry-pick pipeline changes from weekly to current LTS by dduportal · Pull Request #221 · jenkins-infra/release · GitHub “just in case”
        • Mark and Stephane pair to cherry pick to stable-2.332
  • 2022-02-15:
    • Plugins only
    • trusted.ci.jenkins.io, release.ci and ci.jenkins.io must NOT be updated this day (as per Daniel’s comment there might be very minor weirdness with the mirrorsync script, but it probably doesn't matter … some plugins being available before they should be via direct download only) so let’s play it safe
      • Mark and Stephane update status.jenkins.io to predict downtime for tomorrow and next week

• Digital Ocean:

  • WiP by @lemeurherve , it’s going well (fighting with the Terraform process that need to have an improved documentation + updated, and it’s the first time that it is used outside jenkins-infra/aws so need some fixes)
  • Status: Works locally, CI job ready, CI builds to be fixed
    • Credentials for the backend configuration file not interpreted
      • Places the credential name instead of value into config file
      • Each configuration reload requires an interactive update
      • Required to set multiline content separately
        • Easier to use string variables rather than file content
        • File is needed for the DigitalOcean docs

• Updatecli:

  • WiP by @smerle on Docker image updates (hashicorp-tools, jenkins-infra/aws) and on kubernetes-management (AWS EC2 agent template updates directly from packer)

• Private AKS cluster:

  • WiP by @dduportal , back from holidays. Focus on the groovy pipeline library

• Service mirrors.jenkins.io

  • Plain HTTP only, legacy service, running on the VM pkg.origin.jenkins.io
  • Pager duty alert the 07th of Feb. about bad TLS certificats: but no HTTPS on it: we removed the alert as it make no sense
  • TODO: decommission this service

• rating.jenkins.io:

  • WiP by Jeremy Playout (@jplayout)
  • Status: PR opened with the helm chart (from @halkeye’s work )
  • Todo: validate helm chart, and then open PR for installation in kubernetes-management
  • Might need to create a Postgres DB in Azure + migrate data