Upgrade from 2.263.4 to 2.277.1

Linux Based Jenkins, inherited machine, very old install and has been upgraded for years. I am attempting to get the software release current and rolling through the upgrades on a clone of the server and when I install and launch 2.277.1 it gives an error on every user xml file acting as if the user format was not rewritten. I’ve rolled back to 2.263-1 and then up to 2.277.1 and all the compatible upgrades for plugins are done. Is there a way to know if the a bad plugin is the problem or if something failed to convert the user format? I see the error An attempt to save the global configuration was made before it was loaded prior to the complaints about the user files. What should I do next? I can roll backwards again and try disabling all or most of the plugins, but I am not sure that a plugin is the issue.

The Jenkins 2.204.1 upgrade guide offers some background on that message and a possible workaround if you’re using the configuration as code plugin.

A stackoverflow article suggests that plugin upgrades might help, though performing the plugin upgrades reliably on that old a version is very difficult because the Jenkins update center does not provide compatibility data for releases from that long ago.

Another stackoverflow article suggests that matrix auth and role strategy might be colliding with each other in some way.

I recently did an upgrade from around the same starting point to somewhere in the high 300’s. I found that I had to choose short upgrade steps with plugin upgrades in between to avoid the worst of the upgrade issues.

2.263.x to 2.277.x doesn’t seem like it should be far enough apart for major issues, though.

The third suggestion was definitely the issue https://devops.stackexchange.com/questions/15750/how-to-solve-my-error-about-saving-the-global-configuration-before-it-is-loaded
. Matrix Auth and Role Strategy were the problem. Disabling them did not resolve the issue since it left remnant configurations for authorization in the config.xml file. I had to fully delete the data from the config file and delete the plugin itself as well as its data folder.