Jenkins High Availablity

Jenkins setup:
3 node Dell server cluster.
Deployed jenkins using helm with directpv as provisioner of PV.

We want to deploy jenkins with high-availability. Is it possible to do so? Any solutions around modifying statefulsets or deployment config?
We also have minio in our cluster. Can we use minio for the distributed file system and form a solution around this?


Hi @raghu-d32 Jenkins does not support highly available controllers: don’t even try, you’ll end up corrupting the content of your JENKINS_HOME.

There are solutions to be taken to approach such a setup though:

  • Ensure that you have a prebuilt image with the plugins (to avoid downloading or upgrading plugins on restarts)
  • Ensure that the persistent volume hosting your JENKINS_HOME is fast not only with I/Os but also when mounting/unmounting to achieve a fast (< 1 min) restart of your controller
  • Test every plugin a nd core update offline (so any plugin error would not impact your production)
  • Add a “customized” reverse proxy to capture webhook incoming requests (to store them and redeliver them) such as GitHub - jenkins-infra/captain-hook (experimental)
  • Ensure that you offload the artifacts storage to an object storage (such as Artifact Manager on S3 for instance)
  • Ensure that you have a full Jenkins configuration as code system to ensure reproductibility including jobs (and to cover crash recovery)
  • Offload logs and metrics collection to another system such as datadog

Of course, this set of “good practises” might not cover all your cases. If you have a dire needs of high availability you can check with CloudBees commercial product: CloudBees High Availability Management<!-- --> | CloudBees CI plugins (spoiler: CloudBees is my employer but I only work with Open Source Jenkins as they pay me to work on the open source infrastructure, not the commercial product)

Hi @dduportal
Thanks for the reply. Will try the specified approaches.

Keeping high availability aside, is there a way to at least make the data persistent across 3 nodes? Using minio as the storage, will we be able to ensure data persistence in any way? Do let me know.


A given JENKINS_HOME must be only used by a single instance, otherwise the data will be corrupted.

You can absolutely use minIO to store it but:

If you want to set up a disaster recovery for your Jenkins instance, or a passive/active system, you need a way to take/restore snapshots of the underlying filesystem, or perform backup/restores of the directory to/from a location where there is NO I/O write concurency. More details on Jenkins backup/restore in Backing-up/Restoring Jenkins