Being an Auditor when we ask the Deployement Team to produce the logs for each deployement event , however they didnt retained the logs for more than 30 builds which is a default setting in General Tab with Log Rotation Strategy, Refer Image. When asked why only 30 events were retained the DevOps Team say that they have limitation of storage / size.
Is there a possibility that Logs for all CICD deployements could be retained and for greater duration without impacting the size / storage. As we need the logs for
“what version was deployed”
“Who deployed it”
“when was it deployed”
Keeping the logs requires some storage. You can individually define for each job how much of history should be kept, so reduce for unimportant ones and extend for the important ones.
You can use e.g. Logstash to forward logs to an external system.
Keeping the logs will eat up disk space on the controller and unfortunately I have seen jenkins controllers go down because of this.
This get compounded by folk running everything in debug mode and last year someone managed to log 12GB in a single run.
There is a plugin to limit the size of the logs, and abort the pipeline, but the plugin does not work and does nothing in practice unfortunately.
There are several plugins that will forward the console output to other logging systems such as cloudwatch, but then these systems are even more expensive than local storage.
One thing I recommend, and that we do, is to move most of the console output to separate log files, then at the end of the stage compress the log files and archive them as artifacts. This will massively reduce the size used by the logs. If you combine this with the S3 artifacts plugin, then these files get stored on S3 which scales much better and at a much better cost.
If you use datadog or jira they are capable of tracking versions deployments. If you use slack you could send a notification to a deployment channel with the audit details you are looking for. So look at your toolchain and see if you can track these things outside of Jenkins itself.
