Github Plugin configuration does not load after service restart

Hi! I recently posted this issue regarding my jenkins configuration and was referred to this forum instead. About 2 weeks ago, we upgraded jenkins to the latest LST version (2.401.2) plus upgraded all our plugins to their most recent versions. We use the GitHub plugin for integration with our repos. After the upgrade, whenever I restart the jenkins.service on my instance, the configuration for the github plugin fails to reload. the following exception is visible on the journal logs:

2023-07-27 11:44:32.039+0000 [id=35]        WARNING        hudson.model.Descriptor#load: Failed to load /mnt/data/github-plugin-configuration.xml
com.thoughtworks.xstream.mapper.CannotResolveClassException: github-plugin-configuration
        at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(DefaultMapper.java:81)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.DynamicProxyMapper.realClass(DynamicProxyMapper.java:55)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.PackageAliasingMapper.realClass(PackageAliasingMapper.java:88)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.ClassAliasingMapper.realClass(ClassAliasingMapper.java:79)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.ArrayMapper.realClass(ArrayMapper.java:74)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.SecurityMapper.realClass(SecurityMapper.java:71)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at hudson.util.XStream2$CompatibilityMapper.realClass(XStream2.java:452)
        at hudson.util.xstream.MapperDelegate.realClass(MapperDelegate.java:46)
        at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
        at com.thoughtworks.xstream.mapper.CachingMapper.realClass(CachingMapper.java:47)
        at com.thoughtworks.xstream.core.util.HierarchicalStreams.readClassType(HierarchicalStreams.java:29)
        at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:135)
        at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1464)
        at hudson.util.XStream2.unmarshal(XStream2.java:230)
        at hudson.util.XStream2.unmarshal(XStream2.java:201)
        at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1441)
        at hudson.XmlFile.unmarshal(XmlFile.java:196)
Caused: java.io.IOException: Unable to read /mnt/data/github-plugin-configuration.xml
        at hudson.XmlFile.unmarshal(XmlFile.java:199)
        at hudson.XmlFile.unmarshal(XmlFile.java:179)
        at hudson.model.Descriptor.load(Descriptor.java:933)
        at org.jenkinsci.plugins.github.config.GitHubPluginConfig.<init>(GitHubPluginConfig.java:86)
        at org.jenkinsci.plugins.github.config.GitHubPluginConfig$$FastClassByGuice$$374e4fd6.GUICE$TRAMPOLINE(<generated>)
        at org.jenkinsci.plugins.github.config.GitHubPluginConfig$$FastClassByGuice$$374e4fd6.apply(<generated>)
        at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
        at com.google.inject.internal.ConstructorInjector.access$000(ConstructorInjector.java:33)
        at com.google.inject.internal.ConstructorInjector$1.call(ConstructorInjector.java:98)
        at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:109)
        at hudson.ExtensionFinder$GuiceFinder$SezpozModule.onProvision(ExtensionFinder.java:610)
        at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:117)
        at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:66)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:93)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:300)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:169)
        at hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1.get(ExtensionFinder.java:445)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:45)
        at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1148)
        at hudson.ExtensionFinder$GuiceFinder._find(ExtensionFinder.java:403)
        at hudson.ExtensionFinder$GuiceFinder.find(ExtensionFinder.java:394)
        at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:359)
        at hudson.ExtensionList.load(ExtensionList.java:384)
        at hudson.ExtensionList.ensureLoaded(ExtensionList.java:320)
        at hudson.ExtensionList.getComponents(ExtensionList.java:184)
        at hudson.DescriptorExtensionList.load(DescriptorExtensionList.java:213)
        at hudson.ExtensionList.ensureLoaded(ExtensionList.java:320)
        at hudson.ExtensionList.iterator(ExtensionList.java:172)
        at hudson.ExtensionList.get(ExtensionList.java:149)
        at jenkins.plugins.shiningpanda.tools.PythonInstallation.list(PythonInstallation.java:128)
        at jenkins.plugins.shiningpanda.tools.PythonInstallation.isEmpty(PythonInstallation.java:119)
        at jenkins.plugins.shiningpanda.ShiningPanda.postInitialize(ShiningPanda.java:51)
        at hudson.PluginManager$2$1$2.run(PluginManager.java:583)
        at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:177)
        at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305)
        at jenkins.model.Jenkins$5.runTask(Jenkins.java:1164)
        at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:221)
        at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:120)
        at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

We are mounting an external EBS volume on AWS containing our jenkins data to /mnt/data. I’ve checked file permissions are correct and the jenkins user can most definitely read/write to this location. Because the configuration does not re-load, after each restart I need to go to Dashboard > Manage Jenkins > System and input my Github server configuration (which is super annoying):

gh-server

I’ve tried removing the file and re-configuring the plugin. After this /mnt/data/github-plugin-configuration.xml gets regenerated as expected. However when I restart it still fails to load.

I recall seeing this issue pop up with an older version of this plugin, but I cannot find the link. Any help solving this is appreciated!


Jenkins setup:
Jenkins version: 2.401.2
OS: Ubuntu 20.04.2 LTS
Kernel: 5.8.0-1038-aws
Java version: openjdk 11.0.20 2023-07-18
Github Plugin version: 1.37.2


List of installed plugins:

Amazon Web Services SDK :: All: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: CloudFormation: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: CodeBuild: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: EC2: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: ECR: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: ECS: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: EFS: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: Elastic Beanstalk: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: IAM: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: kinesis: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: Logs: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: Minimal: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: SNS: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: SQS: 1.12.481-392.v8b_291cfcda_09
Amazon Web Services SDK :: SSM: 1.12.481-392.v8b_291cfcda_09
AnsiColor: 1.0.2
Ant Plugin: 497.v94e7d9fffa_b_9
Any Build Step Plugin: 14.ve115ec1484f0
Apache HttpComponents Client 4.x API Plugin: 4.5.14-150.v7a_b_9d17134a_5
Audit Trail: 333.vb_e1b_b_0f1238c
Authentication Tokens API Plugin: 1.53.v1c90fd9191a_b_
AWS CodeDeploy Plugin for Jenkins: 1.23
AWS Secrets Manager Credentials Provider: 1.202.ve0ec0c17611c
Basic Branch Build Strategies Plugin: 81.v05e333931c7d
Bootstrap 4 API Plugin: 4.6.0-6
Bootstrap 5 API Plugin: 5.3.0-1
bouncycastle API Plugin: 2.29
Branch API Plugin: 2.1122.v09cb_8ea_8a_724
Build Discarder Plugin: 139.v05696a_7fe240
Build Name and Description Setter: 2.2.1
Build Timeout: 1.31
Caffeine API Plugin: 3.1.6-115.vb_8b_b_328e59d8
Checks API plugin: 2.0.0
CloudBees AWS Credentials Plugin: 191.vcb_f183ce58b_9
CloudBees Disk Usage Simple Plugin: 182.v62ca_0c992a_f3
CMake plugin: 4.1.1
Code Coverage API Plugin: 4.7.0
Command Agent Launcher Plugin: 100.v2f6722292ee8
Commons HttpClient 3.x API: 3.1-3
commons-lang3 v3.x Jenkins API Plugin: 3.12.0-36.vd97de6465d5b_
commons-text API Plugin: 1.10.0-36.vc008c8fcda_7b_
Conditional BuildStep: 1.4.2
Configuration as Code Plugin: 1670.v564dc8b_982d0
Copy Artifact Plugin: 705.v5295cffec284
Credentials Binding Plugin: 631.v861c06d062b_4
Credentials Plugin: 1271.v54b_1c2c6388a_
Custom Tools Plugin: 0.8
Dark Theme: 336.v02165cd8c2ee
Dashboard View: 2.487.vcf0ff9008a_c0
DataTables. net API Plugin: 1.13.5-1
Date Picker Plugin: 1.0
description setter plugin: 1.10
Design Language: 1.27.5
Disable GitHub Multibranch Status Plugin: 1.2
Display URL API: 2.3.7
Durable Task Plugin: 510.v324450f8dca_4
ECharts API Plugin: 5.4.0-5
Email Extension Plugin: 2.100
EnvInject API Plugin: 1.199.v3ce31253ed13
Environment Injector Plugin: 2.908.v66a_774b_31d93
Exclusive Execution Plugin: 0.8
Extended Choice Parameter Plugin: 375.v72e4b_a_d33d33
Extended Read Permission Plugin: 53.v6499940139e5
External Monitor Job Type Plugin: 207.v98a_a_37a_85525
Extra Columns Plugin: 1.26
Favorite: 2.4.2
File Parameter Plugin: 285.287.v4b_7b_29d3469d
Flexible Publish Plugin: 0.16.1
Folders Plugin: 6.815.v0dd5a_cb_40e0e
Font Awesome API Plugin: 6.4.0-2
Forensics API Plugin: 2.3.0
Generic Webhook Trigger Plugin: 1.86.4
Git client plugin: 4.4.0
Git Parameter Plug-In: 0.9.19
Git plugin: 5.2.0
Git server Plugin: 99.va_0826a_b_cdfa_d
GitHub API Plugin: 1.314-431.v78d72a_3fe4c3
GitHub Authentication plugin: 0.39
GitHub Branch Source Plugin: 1728.v859147241f49
GitHub Checks plugin: 545.v79a_a_68b_ca_682
GitHub Integration Plugin: 0.5.0
GitHub plugin: 1.37.2
Go Plugin: 1.4
Google Login Plugin: 1.7
Green Balls: 1.15.1
Groovy: 453.vcdb_a_c5c99890
Handy Uri Templates 2.x API Plugin: 2.1.8-22.v77d5b_75e6953
Heavy Job Plugin: 1.1
HTML Publisher plugin: 1.31
Instance Identity: 173.va_37c494ec4e5
Ionicons API: 56.v1b_1c8c49374e
Jackson 2 API Plugin: 2.15.2-350.v0c2f3f8fc595
Jakarta Activation API: 2.0.1-3
Jakarta Mail API: 2.0.1-3
Java JSON Web Token (JJWT) Plugin: 0.11.5-77.v646c772fddb_0
JavaBeans Activation Framework (JAF) API: 1.2.0-6
Javadoc Plugin: 233.vdc1a_ec702cff
JavaMail API: 1.6.2-9
JavaScript GUI Lib: ACE Editor bundle plugin: 1.1
JavaScript GUI Lib: Handlebars bundle plugin: 3.0.8
JavaScript GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin: 1.2.1
JavaScript GUI Lib: Moment.js bundle plugin: 1.1.1
JAXB plugin: 2.3.8-1
Job Configuration History Plugin: 1227.v7a_79fc4dc01f
Job DSL: 1.84
Job/Queue/Agents Monitoring Plugin: 37.vf50a_82a_0b_f32
jQuery plugin: 1.12.4-1
JQuery3 API Plugin: 3.7.0-1
JSch dependency plugin: 0.2.8-65.v052c39de79b_2
JUnit Plugin: 1217.v4297208a_a_b_ce
LDAP Plugin: 682.v7b_544c9d1512
List Git Branches Parameter PlugIn: 0.0.13
Lockable Resources plugin: 1172.v4b_8fc8eed570
Mailer Plugin: 457.v3f72cb_e015e5
Matrix Authorization Strategy Plugin: 3.1.10
Matrix Project Plugin: 789.v57a_725b_63c79
Maven Integration plugin: 3.22
Metrics Plugin: 4.2.18-439.v86a_20b_a_8318b_
Mina SSHD API :: Common: 2.10.0-69.v28e3e36d18eb_
Mina SSHD API :: Core: 2.10.0-69.v28e3e36d18eb_
Nexus Artifact Uploader: 2.14
OkHttp Plugin: 4.11.0-145.vcb_8de402ef81
Oracle Java SE Development Kit Installer Plugin: 66.vd8fa_64ee91b_d
OWASP Markup Formatter Plugin: 159.v25b_c67cd35fb_
PAM Authentication plugin: 1.10
Pipeline Graph Analysis Plugin: 202.va_d268e64deb_3
Pipeline Remote Loader Plugin: 1.6
Pipeline Utility Steps: 2.16.0
Pipeline: 596.v8c21c963d92d
Pipeline: API: 1241.v4edc8b_44933b_
Pipeline: Basic Steps: 1017.vb_45b_302f0cea_
Pipeline: Build Step: 496.v2449a_9a_221f2
Pipeline: Declarative Extension Points API: 2.2144.v077a_d1928a_40
Pipeline: Declarative: 2.2144.v077a_d1928a_40
Pipeline: Deprecated Groovy Libraries: 609.vd95673f149b_b
Pipeline: GitHub Groovy Libraries: 42.v0739460cda_c4
Pipeline: Groovy Libraries: 656.va_a_ceeb_6ffb_f7
Pipeline: Groovy: 3722.v85ce2a_c6240b_
Pipeline: Input Step: 468.va_5db_051498a_4
Pipeline: Job: 1316.vd2290d3341a_f
Pipeline: Milestone Step: 111.v449306f708b_7
Pipeline: Model API: 2.2144.v077a_d1928a_40
Pipeline: Multibranch: 756.v891d88f2cd46
Pipeline: Nodes and Processes: 1247.v7f9dfea_b_4fd0
Pipeline: REST API Plugin: 2.33
Pipeline: SCM Step: 415.v434365564324
Pipeline: Stage Step: 305.ve96d0205c1c6
Pipeline: Stage Tags Metadata: 2.2144.v077a_d1928a_40
Pipeline: Stage View Plugin: 2.33
Pipeline: Step API: 639.v6eca_cd8c04a_a_
Pipeline: Supporting APIs: 848.v5a_383b_d14921
Plain Credentials Plugin: 143.v1b_df8b_d3b_e48
Plugin Utilities API Plugin: 3.3.0
Popper.js 2 API Plugin: 2.11.6-2
Popper.js API Plugin: 1.16.1-3
Post build task: 1.9
Prism API Plugin: 1.29.0-7
Prometheus metrics plugin: 2.2.3
Pub-Sub “light” Bus: 1.17
Pyenv Pipeline Plugin: 2.1.2
Rebuilder: 320.v5a_0933a_e7d61
Resource Disposer Plugin: 0.22
Run Condition Plugin: 1.6
SCM API Plugin: 676.v886669a_199a_a_
Script Security Plugin: 1251.vfe552ed55f8d
Server Sent Events (SSE) Gateway Plugin: 1.26
ShiningPanda Plugin: 0.24
Slack Notification Plugin: 664.vc9a_90f8b_c24a_
SnakeYAML API Plugin: 1.33-95.va_b_a_e3e47b_fa_4
SSH Build Agents plugin: 2.877.v365f5eb_a_b_eec
SSH Credentials Plugin: 305.v8f4381501156
SSH Pipeline Steps: 2.0.65.vd26b_5b_9b_de4d
SSH server: 3.303.vefc7119b_ec23
Structs Plugin: 324.va_f5d6774f3a_d
Support Core Plugin: 1346.v976da_cf026e1
Theme Manager: 193.vcef22f6c5f2b_
ThinBackup: 1.17
Timestamper: 1.25
Token Macro Plugin: 359.vb_cde11682e0c
Trilead API Plugin: 2.84.v72119de229b_7
Unity3d plugin: 1.3
Variant Plugin: 59.vf075fe829ccb
WMI Windows Agents Plugin: 1.8.1
Workspace Cleanup Plugin: 0.45
Xvfb plugin: 1.2

I discovered there is a conflict with the ShiningPanda Plugin: 0.24. I was able to reproduce this on a fresh docker install (Jenkins version: 2.401.2) with recommended plugins only (among which is Github Plugin version: 1.37.2). I configured the GitHub plugin and restarted the docker - all is good. Then I installed ShiningPanda, restarted, and got the exception above - the GitHub plugin configuration is unable to load. No idea what might be triggering this problem, and the exception message is definitely not useful. :confused: