Jenkins setup:
Jenkins 2.452.2
Azure pAKS
JENKINS_HOME on Azure file share
Web browser: not relevant since the problem occurs while bootstrapping
output from Script Console:
Jenkins: 2.452.2
OS: Linux - 5.15.0-1064-azure
Java: 17.0.11 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
---
ace-editor:1.1
ant:497.v94e7d9fffa_b_9
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
asm-api:9.6-3.v2e1fa_b_338cd7
authentication-tokens:1.53.v1c90fd9191a_b_
bootstrap5-api:5.3.2-4
bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
branch-api:2.1105.v472604208c55
build-timeout:1.32
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloudbees-folder:6.928.v7c780211d66e
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-109.vfe16c66636eb_
configuration-as-code:1810.v9b_c30a_249a_4c
credentials:1337.v60b_d7b_c7b_c9f
credentials-binding:677.vdc9d38cb_254d
dark-theme:439.vdef09f81f85e
display-url-api:2.204.vf6fddd8a_8b_e9
durable-task:550.v0930093c4b_a_6
echarts-api:5.4.0-7
email-ext:1814.v404722f34263
font-awesome-api:6.5.1-3
git:5.2.2
git-client:4.7.0
git-server:1.9
github:1.37.3.1
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1789.v5b_0c0cea_18c3
gradle:2.12
gson-api:2.10.1-15.v0d99f670e0a_7
instance-identity:185.v303dc7c645f9
ionicons-api:73.vb_f522f227457
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jjwt-api:0.11.5-77.v646c772fddb_0
joda-time-api:2.12.5-5.v5495a_235fedf
jquery3-api:3.7.1-1
jsch:0.1.55.2
json-api:20240303-41.v94e11e6de726
json-path-api:2.8.0-5.v07cb_a_1ca_738c
junit:1265.v65b_14fa_f12f0
kubernetes:4246.v5a_12b_1fe120e
kubernetes-client-api:6.10.0-240.v57880ce8b_0b_2
kubernetes-credentials:0.11
mailer:472.vf7c289a_4b_420
matrix-auth:3.2.2
matrix-project:822.824.v14451b_c0fd42
metrics:4.2.21-449.v6960d7c54c69
mina-sshd-api-common:2.12.0-90.v9f7fb_9fa_3d3b_
mina-sshd-api-core:2.12.0-90.v9f7fb_9fa_3d3b_
oic-auth:4.284.v0cc21de03d37
okhttp-api:4.11.0-157.v6852a_a_fa_ec11
pam-auth:1.11
pipeline-build-step:2.18
pipeline-github-lib:61.v629f2cc41d83
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-graph-view:287.v3ef017b_780d5
pipeline-groovy-lib:689.veec561a_dee13
pipeline-input-step:495.ve9c153f6067b_
pipeline-milestone-step:101.vd572fef9d926
pipeline-model-api:2.2198.v41dd8ef6dd56
pipeline-model-definition:2.2198.v41dd8ef6dd56
pipeline-model-extensions:2.2198.v41dd8ef6dd56
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2198.v41dd8ef6dd56
plain-credentials:179.vc5cb_98f6db_38
plugin-util-api:4.1.0
popper2-api:2.11.6-1
prism-api:1.29.0-15
resource-disposer:0.23
scm-api:690.vfc8b_54395023
script-security:1336.vf33a_a_9863911
snakeyaml-api:2.2-111.vc6598e30cc65
ssh-credentials:337.v395d2403ccd4
ssh-slaves:2.968.v6f8823c91de4
sshd:3.322.v159e91f6a_550
structs:337.v1b_04ea_4df7c8
theme-manager:215.vc1ff18d67920
timestamper:1.27
token-macro:400.v35420b_922dcb_
trilead-api:2.142.v748523a_76693
variant:60.v7290fc0eb_b_cd
workflow-aggregator:596.v8c21c963d92d
workflow-api:1291.v51fd2a_625da_7
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3894.3896.vca_2c931e7935
workflow-cps-global-lib:2.19
workflow-durable-task-step:1336.v768003e07199
workflow-job:1400.v7fd111b_ec82f
workflow-multibranch:756.v891d88f2cd46
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:657.v03b_e8115821b_
workflow-support:865.v43e78cc44e0d
ws-cleanup:0.46
We try to deploy Jenkins to Azure pAKS using the official helm chart.
This works as expected as long as Jenkins resides on some Azure managed disk. For failover reasons we want to run Jenkins off of some Azure file share (CIFS). But, when supplying Jenkins with a file share instead of a disk the pod goes into an infinite loop when unpacking and configuring plugins. We see an exception that points to not being able to change the modification timestamp of files like:
plugins/ace-editor/.timestamp2
The files can be created in the first place but later on it’s last modified timestamp can’t be changed.
The exception thrown is:
2024-07-16 10:38:30.865+0000 [id=41] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Inspecting plugin /var/jenkins_home/plugins/ant.jpi
java.io.IOException: Failed to set the timestamp of /var/jenkins_home/plugins/ant/.timestamp2 to 1721125544519
at hudson.FilePath$Touch.invoke(FilePath.java:1836)
at hudson.FilePath$Touch.invoke(FilePath.java:1821)
at hudson.FilePath.act(FilePath.java:1232)
at hudson.FilePath.act(FilePath.java:1215)
at hudson.FilePath.touch(FilePath.java:1818)
at hudson.ClassicPluginStrategy.explode(ClassicPluginStrategy.java:469)
at hudson.ClassicPluginStrategy.createPluginWrapper(ClassicPluginStrategy.java:163)
at hudson.PluginManager$1$3$1.run(PluginManager.java:442)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:177)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1175)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
The kubernetes storage class providing the Azure file shares is:
$ kubectl -n <my-namespace> get StorageClass azurefile-csi-standard-retain -o yaml
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
creationTimestamp: "2023-04-25T10:59:25Z"
name: azurefile-csi-standard-retain
resourceVersion: "137587763"
uid: <UID>
mountOptions:
- mfsymlinks
- nosharesock
- actimeo=30
- cache=strict
parameters:
resourceGroup: <my-resgroup>
server: <my-server>
storageAccount: <my-storage-account>
provisioner: file.csi.azure.com
reclaimPolicy: Retain
volumeBindingMode: Immediate
Moreover, if I re-use an Azure file share that has been created a day before - which still contains all the files of the previous bootstrapping attempts - the bootstrapping succeeds. Did anyone experience this problem as well and has a solution or workaround?
Regards, Ralf