Back of Napkin upgrade guide, for the uninitiated

This is my “back of the napkin” “guide” that I used to fully research and upgrade my Jenkins Servers. Many of you are seasoned experts, I wasn’t. This represents my full journey and cited references. Hope it helps others.

~Yours in Jenkins~
Marc
“This is the way”

Prework:
Go thru the Change log, with your plugin list right next to you. Go thru EVERY change log from current version to target version, and each iteration between. Look for references to your Plugins and Java versions (client and primary) in the Change Log to build a validation check list (e.g. does SSH still work, Does plugin function properly). ADDED ROI: while you are looking at your Plug-ins, which ones are deprecated, do you still need them? Try deactivating them (before removing them) to see if anyone is complaining.

If you use LDAP/AD, make copies off line of Security groups and access, User Lists and Security groups( <jenkins_url>/manage/role-strategy/ && <jenkins_url>/manage/role-strategy/assign-roles. as well as configurations to access the AD (<jenkins_url>/configureSecurity/).

  1. know your backups for rollbacks. I played a lot with plug-ins… making a tar/zip archive of /JENKINS_HOME/plugins was super useful. Active Directory plugin threw me for a loop when I had to adapt to the “Require TLS” force for LDAPS. I work update that one after hours

  2. SNAPSHOTS beginning, and after each benchmark, if you can afford to.

  3. update Plugins to the highest Compatible level (I found “highest level” sometimes threw errors as a certain point). Archive that plugins/ folder :wink:

  4. Update the Core application. Use the same method that you used to install. We run on RHEL, and used yum… I initially tried downloading the war file initially; and it kept throwing different errors… Go back to the same install method. Depending on the versioning used, versions pre-2.400 used a different security key than post-2.400 (Jenkins 2.397 and 2.387.2: New Linux Repository Signing Keys); keep in mind.

  5. Update plugins again to highest “compatible” level (hopefully highest level)

  6. Plugin configuration. My CVS plug in required files that are not in the hpi install file… follow appropriate documentation. Establish connections to external database and services (AD/CVS/etc).
    I had had trouble with getting the “Require TLS” checkbox to work properly… because there was a conflict in the systemctl for Jenkins… confirm “-Dhudson.plugins.active_directory.ActiveDirectorySecurityRealm.forceLdaps=true” was removed before enabling that checkbox.

  7. Validation check with that list you made before… can you SSH our to client servers? Can you connect to your external services/databases?

Cleanup:
Depending on how big the upgrade was, re-examine Plug-ins. Some may be deprecated, now that you are on a higher Jenkins build and could be disabled, and removed at a later date. This will help close security hole =-O After each disable/remove and Application recycle.

my references:
Core app updated

Change log

Plug-in references

Supplementary reading:

Thank you so much, Marc! You’re amazing. :metal:

I believe this would make excellent material for a blog post on jenkins.io. If you ever consider doing it, I’d be happy to assist. Just let me know.