I have some automated tests for our Jenkins plugin using the Jenkins test infrastructure. In the past, we encountered bugs when the Jenkins controller was running on Windows and the agent on Linux. I’m looking for a way to automate testing of such cross-platform scenarios.
One option I considered was setting up a Jenkins controller on a Windows machine with a Linux agent and running the tests via GitHub Actions, but that approach raises maintainability concerns.
Is there a way to simulate or test these cross-platform setups using the Jenkins test infrastructure, or any recommended approach for automating such scenarios?
I think the Jenkins test harness doesn’t support this scenario out of the box. While you can create and start agents with it, they all run on the same machine under the same user. This makes it impossible to find errors where you use eg File instead of FilePath to access a file or you use ProcessBuilder directly instead of using a Launcher or wrapping it with ´FilePath#act`.
But you could of course programmatically create an agent that e.g. connects via ssh. You would need to provide though somehow hostname and user/pwd or ssh key file (via environment variables) as you don’t want to hard code those things. And if those env variables are not there the test is skipped.
Leaves the problem how to validate the results. Probably the job that you run in your tests has to archive the files from the agent so they end up on the controller and you can check things.
Set this up on a Jenkins and provide host name and credentials to your mvn run.
Should be clear that this will only work in your setup but not when the plugin is hosted in the Jenkins org and the build runs on ci.jenkins.io