I am running into issues when I try to run a python build script from a Jenkins Pipeline, which I don’t get when I run the same script on the same machine outside of Jenkins. I am wondering if my approach has some obvious flaws or if someone has some insights into the issue I am running into.
My Jenkins setup looks like this:
- I have a windows jenkins controller
- The controller uses the EC2 plugin to allocate and spin up build agents also running on windows
- The controller connects to the agent using Winrm
- The Build agents are running on a custom windows AMI i have put together that contains everything needed to run UnrealEngine builds and I log in using a user that has all of the required permissions to run the builds
- When a build is triggered, the controller spins up an agent on a ec2 machine, which pulls down the project, runs the build and then uploads the completed builds to an S3 bucket
I have a Python build script that runs the various steps of my UE4 builds that I can start through powershell. If I run this script on the EC2 machine manually, the builds work just fine without issues. When I use the same powershell command in my Jenkins pipeline script to build the project, I run into issues that I never get outside of jenkins.
The error looks like this:
c1xx: error C3859: Failed to create virtual memory for PCH c1xx: note: the system returned code 1455: The paging file is too small for this operation to complete. c1xx: note: please visit [https://aka.ms/pch-help](https://aka.ms/pch-help) for more details c1xx: fatal error C1076: compiler limit: internal heap limit reached
The errors seem to change depending on the run, but always seem to concern memory limits, which the instance shouldn’t have. I spin up instances with 64gb of physical memory and I’ve assigned a 64gb of virtual memory (paging file size) as well. Again, the thing that confuses me is the lack of this issue when running the build manually on the agent machine.
I’ve looked into this for the last couple of days and I haven’t gotten closer to a solution. Based on what I’ve found, you can adjust the JVM memory limit using the -Xmx and -Xms options.
I’ve set those to these values: -Xmx16G -Xms8G which should be plenty to run the builds but i still run into the same issue.
I feel like I am missing something obvious. I am relatively new to using Jenkins so any insights into what could be happening here would be greatly appreciated! I would love to learn more about the differences of running a build on the machine vs what happens with a jenkins build. I am more than happy to provide any additional info that could be useful.