# Kubernetes plugins default workspace-volume to emptyDir (hostnode?)

**URL:** <https://community.jenkins.io/t/kubernetes-plugins-default-workspace-volume-to-emptydir-hostnode/1422>\
**Category:** Using Jenkins\
**Created:** [February 4, 2022, 6:49am UTC](https://community.jenkins.io/t/kubernetes-plugins-default-workspace-volume-to-emptydir-hostnode/1422 "2022-02-04T06:49:01Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![dev-samples](https://dub1.discourse-cdn.com/flex013/user_avatar/community.jenkins.io/dev-samples/32/292_2.png) [@dev-samples](https://community.jenkins.io/u/dev-samples)\
**Post date:** [February 4, 2022, 6:49am UTC](https://community.jenkins.io/t/kubernetes-plugins-default-workspace-volume-to-emptydir-hostnode/1422/1 "2022-02-04T06:49:02Z")

</div>

I am running the kubernetes plugin for jenkins i k8s in AKS. When I run a build I see that the agent defaults this workspace dir to:

```auto
name: "my-jenkins-agent"
tty: true
- mountPath: "/home/jenkins/agent"
  name: "workspace-volume"
  readOnly: false
env:
- name: "JENKINS_AGENT_WORKDIR"
  value: "/home/jenkins/agent"
volumes:
  - emptyDir:
      medium: ""
    name: "workspace-volume"

```

As I understand that means the files/folders used in the workspace during the build is stored in the emptyDir on the host node confirmed by ( **default** ):

> **[Kubernetes](https://plugins.jenkins.io/kubernetes/#plugin-content-pod-template)**
>
> This plugin integrates Jenkins with Kubernetes

- **workspaceVolume** The type of volume to use for the workspace.
  - `emptyDirWorkspaceVolume` ( **default** ): an empty dir allocated on the host machine
  - `dynamicPVC()` : a persistent volume claim managed dynamically. It is deleted at the same time as the pod.
  - `hostPathWorkspaceVolume()` : a host path volume
  - `nfsWorkspaceVolume()` : a nfs volume
  - `persistentVolumeClaimWorkspaceVolume()` : an existing persistent volume claim by name.

Before experimenting with using a PVC or e.g. hostPathWorkspaceVolume I guess using the default (emptyDir) should be just fine? Or are there some best practices when it comes to configuring the workspace dir?

The reason I am asking is that I am seeing increased pod (jenkins agent) evictions errors like:

```auto
Status: Failed
Reason: Evicted
Message: The node was low on resource: ephemeral-storage. Container jnlp was using 156Ki, which exceeds its request of 0. 
Container docker was using 56Ki, which exceeds its request of 0.

```

and I assume that might have to do with using emptyDir as workspace location for all my builds (I regularly purge unused docker tags/volumes from the host nodes).

I would like to make sure that when a build starts its using a clean workspace but I would also like to avoid exhausting the disk capacity on my worker nodes (eventually leading to the pod evictions).

And looking at the defaults:

```auto
volumes:
  - emptyDir:
      medium: ""
    name: "workspace-volume"

```

does the **medium** field correspond to `emptyDir.medium` described here (and “” string being the value meaning its NOT using RAM-backed filesystem):

> **[Volumes](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir)**
>
> On-disk files in a container are ephemeral, which presents some problems for non-trivial applications when running in containers. One problem occurs when a container crashes or is stopped. Container state is not saved so all of the files that were...

> Depending on your environment, `emptyDir` volumes are stored on whatever medium that backs the node such as disk or SSD, or network storage. However, if you set the `emptyDir.medium` field to `"Memory"` , Kubernetes mounts a tmpfs (RAM-backed filesystem) for you instead. While tmpfs is very fast, be aware that unlike disks, tmpfs is cleared on node reboot and any files you write count against your container’s memory limit.

?

Any input/recommendations?

---

<div class="post-metadata">

**Author:** ![halkeye](https://dub1.discourse-cdn.com/flex013/user_avatar/community.jenkins.io/halkeye/32/8_2.png) [@halkeye](https://community.jenkins.io/u/halkeye)\
**Post date:** [February 4, 2022, 5:35pm UTC](https://community.jenkins.io/t/kubernetes-plugins-default-workspace-volume-to-emptydir-hostnode/1422/2 "2022-02-04T17:35:38Z")

</div>

This is a very long post and I’m not really sure what your asking.

Jenkins controller writes everything to $JENKINS\_HOME (which is often default to /var/lib/jenkins)  
Jenkins agents write to whatever directory you specify, I guess $JENKINS\_AGENT though I don’t know if that’s a helm specific thing.

Nothing on agents need to be kept, it’ll be setup on demand as needed so you can use whatever filesystem you want and change it as you feel like.

The controller persists all the data. It is possible to have non persistent disk with configuration as code + jobdsl, but I’ve never found it works 100%, so I persist my controller’s directory.

---

<div class="post-metadata">

**Author:** ![dev-samples](https://dub1.discourse-cdn.com/flex013/user_avatar/community.jenkins.io/dev-samples/32/292_2.png) [@dev-samples](https://community.jenkins.io/u/dev-samples)\
**Post date:** [February 8, 2022, 2:28am UTC](https://community.jenkins.io/t/kubernetes-plugins-default-workspace-volume-to-emptydir-hostnode/1422/3 "2022-02-08T02:28:09Z")

</div>

Sorry for the long post/read.

So you use a single PVC for everything related to the jenkins master (controller?), like plugins, expanded war file, jobs?

And just host node storage (emptyDir) for diskspace needed during agent execution like workspace-dir, maven cache etc?

---

<div class="post-metadata">

**Author:** ![halkeye](https://dub1.discourse-cdn.com/flex013/user_avatar/community.jenkins.io/halkeye/32/8_2.png) [@halkeye](https://community.jenkins.io/u/halkeye)\
**Post date:** [February 8, 2022, 4:33am UTC](https://community.jenkins.io/t/kubernetes-plugins-default-workspace-volume-to-emptydir-hostnode/1422/4 "2022-02-08T04:33:21Z")

</div>

Yea essentially. Though I have plugins backed into my image so I can roll forward and back.

---

<div class="post-metadata">

**Author:** ![dev-samples](https://dub1.discourse-cdn.com/flex013/user_avatar/community.jenkins.io/dev-samples/32/292_2.png) [@dev-samples](https://community.jenkins.io/u/dev-samples)\
**Post date:** [February 8, 2022, 6:43am UTC](https://community.jenkins.io/t/kubernetes-plugins-default-workspace-volume-to-emptydir-hostnode/1422/5 "2022-02-08T06:43:21Z")

</div>

Ok so you have a docker build of the master image that downloads the necessary plugin in the Docker file so they are all ready when the master is started?

Right now I have this in the init script of the master:

` jenkins-plugin-cli --war "/usr/share/jenkins/jenkins.war" --plugin-file "/app-config/plugins.txt" --latest false --verbose`

using: plugin-installation-manager-tool

so when jenkins starts it will download the plugins if the are not already available in a PVC dedicated to the plugins.

Not sure if there are any benefits of having them as part of the docker image compared to the above?