Managing Plugins

Using Jenkins
1

Hi everyone,

Our team manages plugins through infrastructure as code through plugins.txt files for our Jenkins. However, when managing plugins this way, it is hard to track dependencies through automation. Here is a documented proposal of things I learned for best practices to manage plugins. Does anyone have a solidifed way that they manage Jenkins plugins using automation that handles dependency issues and making sure you’re consistently tracking unused plugins or monitoring for plugins that fall short of health scores. Automated mechanisms would be greatly appreciated. This is my research:

Problem

Plugins load as soon as Jenkins starts, so if there are any issues with any plugins that we manage, the controllers we manage will not start up. This is problematic during weekly Jenkins updates as this not only affects devops, but can affect other developer teams if their controller is down. There are a few issues with managing Jenkins plugins that can be addressed.

1. Dependencies

When updating plugins through casc using plugins.txt, we make a call to the jenkins api with our file and it returns us the updated plugins.txt file with the updated plugins. However, if an update to one plugin requires a new dependent plugin, this is not captured in the process and will cause failures upon jenkins startup.

The same goes for installing plugins. It is a user’s responsibility when adding a new plugin to the plugins.txt file to ensure that all dependencies are also being added, but if this is missed, it will cause the controller to fail during startup period.

When deleting a plugin, it might have had a dependent plugin that was installed that can also be removed. By deleting a plugin and failing to delete its dependencies, we leave plugins in the system that may never be used

Overall, coupling dependency management during all jenkins plugin processes increases the risk of breaking our controllers and/or keeping unused plugins in our system

2. Plugin File Validation

Right now, there is no mechanism for linting our plugins.txt file. People can misspell plugins or make a mistake with the versioning, which will cause controller failure upon startup. Although we validate our plugin changes within our dev controllers, it is better altogether to mitigate all potential risks before merging into a develop environment as our dev controllers can still break.

3. Unused Plugins

Right now, we don’t have a standardized automated mechanism to determine when plugins are no longer used. Continuously upgrading and managing unused plugins is unnecessary and increases risks of plugins breaking controllers. Also managing plugins that aren’t used and contain security risks can threaten controller safety

What I Learned

I tried to learn about the ways that users of Jenkins propose or manage plugins to see if there are any mechanisms that we can adopt into our plugins processes to help make managing plugins 1) easier 2) effective 3)secure

  1. Minimize plugin usage: “ The fewer plugins you have, the fewer opportunities there are for them to cause problems. “ The Plugin Usage Tool is a great way to see which plugins are being used

  2. Choose high quality plugins. Plugins with high health scores, widely used, and are reliable. Right now are team has a standard for adding new plugins and are working on evaluating low quality plugins

  3. Use caution when updating plugins. When updating a plugin, check the plugin’s release notes for breaking changes and new dependencies. Also check compatibility between Jenkins version and updated plugin

  4. Test changes in a development environment.

    1. Last PI, I implemented a pipeline so that we can grab plugin updates and directly update the plugins.txt file instead of having to manually download plugins within dev controllers. I also created a develop branch in our jenkins repository so that when we push plugin changes to develop, the new release management process will restart all the dev controllers with the new changes.

  5. Keep security at the top of your mind. Regular security audits of your plugin ecosystem should be part of your maintenance routine. Remove unnecessary plugins to reduce potential security risks

  6. Start with a default deny rule. If the functionality you’re aiming to achieve can be done without a plugin, stick to that. The fewer the plugins the fewer dependencies to manage, fewer security risks

Suggestions on Improving Plugin Management

  1. Periodically audit for unused plugins. Proposed solution is to include this in maintenance planning to record unused plugins so these plugins can be scheduled for removal

  2. Separate common plugins into a plugins.txt file, and controller specific plugins should have their dedicated plugin files

    1. Keeps plugins organized and increases effectiveness by managing most plugins from one file rather then repeating same exact plugins in each controller’s plugins.txt

  3. Separate plugins into core plugins and let cli handle dependency installation

    1. By using the plugin installation manager to install plugins, itll automatically retrieve dependencies for core plugins and versions

  4. Lint plugins.txt files

    1. Jenkins plugin manager doesn’t have an official linter, but you can operate a dry run by asking it to run the --list command on your file. If there are any plugin naming issues, the command will fail

      1. This command will also list the dependency tree of your plugins

  5. Make sure when updating plugins none of the core plugins fail devops standards for plugins (adoption, security, etc) → we currently do this in our pipeline that retrieves updated plugins files when we’re planning plugin updates. PI planning will include evaluating plugins with low health scores and if we are able to remove any of them, schedule the removal.

  6. Find a way to evaluate breaking changes. Struggled to find a way to automate this. Is it better for this to be checked by developers during the planning process?