Running python script in declarative pipeline doesn’t work

Using Jenkins Ask a question
1

Hello,

I am using declarative pipeline to run a python program that uses a subprocess.run(…) to generate and run another python script.

The pipeline is very simple:

stage(‘Produce’){

steps {
sh ‘’’
python produceData.py
‘’’
}
}

The part of produceData.py that is related to the problem is:
————

command = “source framework_env.sh; script.py -option1 -option2 …”
sourceCmd = [‘/bin/bash’, ‘-c’, command]
sourceProc = subprocess.run(sourceCmd, stdout=logfile, stderr=logfile, check=True, text=True)

——————————
All the environment is installed with the “source framework_env.sh” before script.py …

The produceData.py works fine when I run it manually on Jenkins server. When it is used in the pipeline it crashes. The error message says that some libraries that should have been loaded are missing.

I do not understand why the program works in the first case and doesn’t work with Jenkins.

Is the part of the pipeline:
sh ‘’’
python produceData.py
‘’’
the same as doing
bash$ python produceData.py on the Jenkins terminal (connected as Jenkins user) ?

Thank you for your help!! :slight_smile:

2

Really depends on your setup, but short of it, likely either the libs are not installed on your agent or your environment is not setup correctly and it is not finding it. I would advise against assuming you know what is preinstalled on your agent and just use a venv and install what you need.

3

Hi,

This problem sounds like a case of differences between environments. Often caused by a different user, or even a non desktop environment running your agent vs your login session.

I would recommend running a tool to export your environmental variables (set in windows, export or printenv in linux) in both your job and your “command prompt” to see whats different.

Often its something like HOME is set different. Or PATH has changed since its managed by jenkins agent settings. Or even a license key is missing.

Its usually a good first step.

4

Hi,

Thank you mlasevich and halkeye for the suggestions!!

I investigated the setup using printenv and the PATH but I did not find any differences.
The code I am using is very complex. The “missing” libraries are created dynamically and included in the setup with a general command that is difficult to track.

Is there anything else I can try in order to check if Jenkins makes some changes?

Thank you for your help!!