# EKS Jenkins setup

**URL:** <https://community.jenkins.io/t/eks-jenkins-setup/366>\
**Category:** Ask a question\
**Created:** [August 29, 2021, 3:44pm UTC](https://community.jenkins.io/t/eks-jenkins-setup/366 "2021-08-29T15:44:45Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jeffreiher](https://avatars.discourse-cdn.com/v4/letter/j/eb8c5e/32.png) [@jeffreiher](https://community.jenkins.io/u/jeffreiher)\
**Post date:** [August 29, 2021, 3:44pm UTC](https://community.jenkins.io/t/eks-jenkins-setup/366/1 "2021-08-29T15:44:45Z")

</div>

Hi all, has anyone here set up Jenkins on AWS EKS? I have been looking at the docs and there are a ton of rabbit holes to go down. Which is the easiest most durable way with the least amount of headaches? Thanks all

---

<div class="post-metadata">

**Author:** ![halkeye](https://dub1.discourse-cdn.com/flex013/user_avatar/community.jenkins.io/halkeye/32/8_2.png) [@halkeye](https://community.jenkins.io/u/halkeye)\
**Post date:** [August 30, 2021, 1:01am UTC](https://community.jenkins.io/t/eks-jenkins-setup/366/2 "2021-08-30T01:01:31Z")

</div>

Let’s assume the answer is yes (I personally havnt, but have done k8s installs before).

What would be your actual question?

In general for any async environment it’s better to ask the question rather than to ask to ask the question? Just reduces round trip time.

---

<div class="post-metadata">

**Author:** ![jeffreiher](https://avatars.discourse-cdn.com/v4/letter/j/eb8c5e/32.png) [@jeffreiher](https://community.jenkins.io/u/jeffreiher)\
**Post date:** [August 30, 2021, 2:36pm UTC](https://community.jenkins.io/t/eks-jenkins-setup/366/3 "2021-08-30T14:36:23Z")

</div>

Hi, our team has been trying to install a durable version of Jenkins on EKS. As I see it there are 2 or 3 ways you can go about this. The way I am currently trying is using helm to install Jenkins and the Jenkins operator with virtual docker in docker agents using configuration as code. The problem with this setup is the Jenkins-operator doesn’t support persistent storage, and the backup only runs once a day. 2nd thing is that our org didn’t want to get a enterprise docker hub account so we have to move all the images to ECR because we where hitting our daily pull limit. (makes it harder to maintain - ie upgrades and such ) 3rd, we are using virtual docker agents which is using docker in docker so trying to figure out how to throttle the job queue and executers so the worker nodes don’t crash before the autoscaler has a chance to do its thing is also a problem. I was asking is someone has done an eks deployment of Jenkins that is in production without these problems and if so which way did you do it?.

---

<div class="post-metadata">

**Author:** ![olblak](https://dub1.discourse-cdn.com/flex013/user_avatar/community.jenkins.io/olblak/32/3_2.png) [@olblak](https://community.jenkins.io/u/olblak)\
**Post date:** [August 31, 2021, 1:17pm UTC](https://community.jenkins.io/t/eks-jenkins-setup/366/4 "2021-08-31T13:17:26Z")

</div>

> [@halkeye](#):
>
> In general for any async environment it’s better to ask the question rather than to ask to ask the question? Just reduces round trip time.

Hi @jeffreiher,  
We started switching our agents to EKS on [https://ci.jenkins.io](https://ci.jenkins.io) quite recently so maybe we can help if you have specific questions. Looking at your latest post, it seems to me that you are looking for guidance on the following topics

1. How to deploy Jenkins on Kubernetes
2. How to switch from DockerHub to ECR
3. Correctly handle time out when scaling nodes

**How to deploy Jenkins on Kubernetes**  
On the Jenkins infrastructure project, we decided to use the helm chart, so we have a “custom” helm chart which overrides the official one and add our custom resources [link](https://github.com/jenkins-infra/charts/tree/master/charts/jenkins) and you can find the jcasc configuration that we are using [here](https://github.com/jenkins-infra/charts/blob/f5692fa4ff366b4a8eaf1f01035371009fb37a11/config/default/jenkins-infra.yaml)  
I am not very familiar with the Jenkins operator so I can’t help with that one

**How to switch from DockerHub to ECR**  
It’s not clear to me what you are trying to solve but it appears to me that you want to configure your Kubernetes cluster to pull images from a pull-through docker registry cache to avoid DockerHub pull limit. I don’t think that ECR provides that feature out of the box but with a bit of creativity, I am sure you can figure it away.

**Correctly handle time out when scaling nodes**  
You’ll have to play with the configuration but I can share with you the configuration that we are using on [ci.jenkins.io](http://ci.jenkins.io). It’s defined on this public repository [jenkins-infra/jenkins-infra](https://github.com/jenkins-infra/jenkins-infra/blob/9a8a734d40ca958720381a3510d79b105a0f48cb/dist/profile/templates/azure.ci.jenkins.io/agents.yaml.erb#L378L409)  
Please note that it’s a ruby erb template as we are using Pupppet to manage [ci.jenkins.io](http://ci.jenkins.io)
