Scripting log extraction from parallel stages

Using Jenkins Ask a question
,
1

My team has some jobs that run multiple parallel stages and I want to extract the output from each stage in a logical way. I know I could download all of the output from the job in one log file but the stages are all interspersed and very hard to make sense of. What I have been doing interactively is:

  1. Open Blue Ocean view for the job
  2. Focus on each individual stage and use the Download the log file widget to download just the output for that stage

This is very tedious and error-prone and I would like to write a script to accomplish the same steps.

I’ve been using the JSON REST API for some things but don’t think there’s anything quite the same for the Ocean Blue view. I see the URLs associated with the View and Download icons (such as https://SERVER/jenkins/sto/blue/rest/organizations/jenkins/pipelines/FOLDER/branches/master/runs/665/nodes/629/log/?start=0) but the node number is confusing and I don’t know how to derive a stage’s node number in my script. 665 is my job number and that’s not a problem but where does 629 come from? I would be happy with even starting at a sensible starting number (like 0 or 1?) and just iterate over a range of numbers until Jenkins doesn’t return anything for the current number but the numbers in the URLs don’t make a lot of sense. I would love to find an API that enumerated the node numbers. Does this make sense? Can anyone suggest a solution?

2

As an alternative you could take a look at the
graph analysis folder from the workflow-api-plugin.

Other plugins have good examples of how to use these APIs, such as: pipeline-graph-view-plugin/PipelineNodeGraphVisitor.java at 0f4275571632e34d6ef35fa3424732c2f1b14ad6 · jenkinsci/pipeline-graph-view-plugin · GitHub.

3

Thanks but I’m not sure about the context in which that would run. Ideally, I just want to use REST APIs from a command line in a Python script.

I poked around and think I found what I needed at https://SERVER/jenkins/sto/blue/rest/organizations/jenkins/FOLDER/branches/master/runs/JOB_NUMBER/nodes. I got more nodes than I need but I knew what to expect to see in the output I needed so I was able to select the output I needed. I’m pulling the output great! This will be a great timesaver for me!

4

Just pipe the output to a file (use tee on linux if you want the log to be visible online as well) and archive the file as artifact instead of trying to(ab-)use the gui?