Hi all,
I’m using Jenkins to implement CI\CD.
When I change script / pipeline file / job configuration or other global configuration, it affect the jobs behaver.
I’m looking for strategy to test my modifications.
I can’t run all the jobs every time because 1. it take lot of time 2. it affect production (or other stable environment) .
any idea?
how do you test yourself?
Thanks.
This is a very very brought question. I’ve tried answering a few times.
For shared libraries, there are some pipeline unit testing frameworks like GitHub - jenkinsci/JenkinsPipelineUnit: Framework for unit testing Jenkins pipelines
For Jenkinsfile, try to have as much logic as you can inside of scripts that can be tested outside of jenkins, preferably in isolation, then the number of changes you need to make to jenkins file are minimal, and jenkinsfile becomes flow control instead of build logic.
Otherwise, communication. Always communication. Don’t make changes in isolation.
Thank you for your answer.
I didn’t know about pipeline Junit. I will try it.
In additional to the pipeline Junit, I had an idea to use proxy to control an testing modifications.
I have 2 instances of jenkins + slaves.
One is production and one is for test. They both have the same configuration and scripts.
All the scripts stored in Git.
I’m doing modification in test env, and after it is stable, I commit to git repository, and pull them on production instance.
I think to add proxy to my production instance, and before commit the modifications, redirect requests for specific job/s to the test env .
This way I can test my modification with real requests.
Is it a good idea? what do you think?
I have open issue in this solution - The history of the builds can be unclear - Some builds will appear in other instance. If this idea will become applicable I will have to solve this problem somehow.
This can be somewhat challenging. Mainly because the whole point of most pipelines is to change something in the real world. For example you might be releasing a build artifact to npm or pypi. You could also be creating infrastructure with Terraform. Interacting with other services always increases the complexity of testing pipelines.
I think @halkeye gave some great advice but if your build and release process is really complex, than you probably are not going to want to orchestrate it through the sh step.
I tend to stick everything re-usable into shared libraries and than stick the pipeline directly into the applications repo.
When we want to change a library function, we create a PR on that repo and it has it’s own test suite that runs in a jenkins multibranch job. It’s mainly testing things like ansible and terraform against sample repos.
When an application repo gets a PR opened against it. We test different things in multiple environments. For example, we will deploy the change to dev environment but instead of using the code from the PR, we use the code from the master branch. This allows us to test the Jenkinsfile changes against known working application code. A failure in dev means the failure happened because the Jenkinsfile is incorrect or some third party dependency is failing. After a sucessful dev deployment we deploy to test environment, this will test the actual changes to the application from the feature branch.
While this process will work for most jobs, it will not work for all of them. You will still run into certain pipelines that make changes that are hard to test. For jobs like this I try and take the “side effect” code and put it behind a “dry-run” flag. That way when I want to test it, I call it with the dry-run flag and skip the side effects.
The process I outlined above is more like high level feature/smoke tests. It’s not the same as running unit tests or anything like that. I do find it quick and effective for what we do. If you are writing pipeline templates that will be shared by many teams, than you might find the unit test approach a better option. That also goes for more strict industries like finance or security etc.