Changelog files have a huge size (more than 100 Mb), how to clean them?

Using Jenkins
1

We have a Jenkins job that backs up our Jenkins server files into a GH repository.
Recently, we detected that some changelog files in some builds exceed 50 Mb, and one exceeds 100 Mb, so GH is raising errors.
I searched for a way to reduce/clean those files and avoid them can growing so much, but I found nothing.

Any help will be welcome.
Regards

2

If you’re looking to handle large changelog files in your Jenkins backups more efficiently before they get to your GitHub repository, here’s a guide on setting up a script to clean them up or truncate them. This should help keep your backups sleek and manageable.

Step-by-step approach:

  1. Identify Large Files: Start by finding changelog files that are bigger than your preferred size limit.
  2. Truncate or Clean Files: Next, either trim these files down to your set limit or clean out unnecessary content.
  3. Commit Changes: Once they’re tidied up, commit the cleaned-up files back to your GitHub repository.

Here’s a Bash script example you could use in your Jenkins job to automatically handle the large changelog files:

#!/bin/bash

# Directory containing the changelog files
CHANGELOG_DIR="/path/to/changelog/files"

# Maximum allowed file size in bytes (e.g., 50 MB)
MAX_SIZE=$((50 * 1024 * 1024))

# Loop through all changelog files in the directory
for file in "$CHANGELOG_DIR"/*; do
  if [ -f "$file" ]; then
    # Get the file size
    FILE_SIZE=$(stat -c%s "$file")
    
    # Check if the file size exceeds the maximum allowed size
    if [ "$FILE_SIZE" -gt "$MAX_SIZE" ]; then
      echo "Truncating $file (size: $FILE_SIZE bytes)"
      
      # Truncate the file to the maximum allowed size
      head -c "$MAX_SIZE" "$file" > "$file.tmp"
      mv "$file.tmp" "$file"
    fi
  fi
done

# Add and commit the changes to the GitHub repository
cd /path/to/your/repo
git add .
git commit -m "Cleaned large changelog files"
git push origin main

What does the script do?

  • Identify Large Files: It checks the size of each changelog file.
  • Truncate Files: Oversized files are cut down to the specified maximum size.
  • Commit Changes: It adds the changes to your GitHub repository.

Just make sure to replace /path/to/changelog/files and /path/to/your/repo with the correct paths for your setup, and adjust MAX_SIZE as needed. This should streamline your changelog management before hitting GitHub. Also, consider adding the log files to .gitignore or setting up your repo to use Git Large File Storage (LFS) for an even smoother process.