Nunit marking build as unstable, but not the stage

Jenkins setup:

Jenkins: 2.414.1
OS: Windows Server 2019 - 10.0
Java: 11.0.17 - Microsoft (OpenJDK 64-Bit Server VM)

ant:497.v94e7d9fffa_b_9
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-150.v7a_b_9d17134a_5
atlassian-bitbucket-server-integration:3.6.0
authentication-tokens:1.53.v1c90fd9191a_b_
basic-branch-build-strategies:81.v05e333931c7d
blueocean:1.27.6
blueocean-autofavorite:1.2.5
blueocean-bitbucket-pipeline:1.27.6
blueocean-commons:1.27.6
blueocean-config:1.27.6
blueocean-core-js:1.27.6
blueocean-dashboard:1.27.6
blueocean-display-url:2.4.2
blueocean-events:1.27.6
blueocean-git-pipeline:1.27.6
blueocean-github-pipeline:1.27.6
blueocean-i18n:1.27.6
blueocean-jwt:1.27.6
blueocean-personalization:1.27.6
blueocean-pipeline-api-impl:1.27.6
blueocean-pipeline-editor:1.27.6
blueocean-pipeline-scm-api:1.27.6
blueocean-rest:1.27.6
blueocean-rest-impl:1.27.6
blueocean-web:1.27.6
bootstrap5-api:5.3.0-1
bouncycastle-api:2.29
branch-api:2.1122.v09cb_8ea_8a_724
build-discarder:139.v05696a_7fe240
build-timeout:1.31
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.0
cloudbees-bitbucket-branch-source:825.va_6a_dc46a_f97d
cloudbees-folder:6.848.ve3b_fd7839a_81
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.10.0-68.v0d0b_c439292b_
copyartifact:714.v28a_34f8c563f
credentials:1271.v54b_1c2c6388a_
credentials-binding:631.v861c06d062b_4
data-tables-api:1.13.5-1
display-url-api:2.3.9
durable-task:523.va_a_22cf15d5e0
echarts-api:5.4.0-5
email-ext:2.100
envinject:2.908.v66a_774b_31d93
envinject-api:1.199.v3ce31253ed13
favorite:2.4.3
font-awesome-api:6.4.0-2
forensics-api:2.3.0
generic-webhook-trigger:1.87.0
git:5.2.0
git-client:4.4.0
git-forensics:2.0.0
git-server:99.va_0826a_b_cdfa_d
github:1.37.3
github-api:1.314-431.v78d72a_3fe4c3
github-branch-source:1732.v3f1889a_c475b_
github-pullrequest:0.5.0
gradle:2.8.2
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
htmlpublisher:1.32
instance-identity:173.va_37c494ec4e5
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.15.2-350.v0c2f3f8fc595
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javadoc:243.vb_b_503b_b_45537
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.8-1
jdk-tool:73.vddf737284550
jenkins-design-language:1.27.6
jjwt-api:0.11.5-77.v646c772fddb_0
jquery3-api:3.7.0-1
jsch:0.2.8-65.v052c39de79b_2
junit:1217.v4297208a_a_b_ce
ldap:694.vc02a_69c9787f
lockable-resources:1185.v0c528656ce04
mailer:463.vedf8358e006b_
matrix-auth:3.2
matrix-project:808.v5a_b_5f56d6966
maven-plugin:3.23
mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_
mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_
nunit:394.v65c09a_8408b_a_
okhttp-api:4.11.0-157.v6852a_a_fa_ec11
pam-auth:1.10
pipeline-build-step:505.v5f0844d8d126
pipeline-github:2.8-147.3206e8179b1c
pipeline-github-lib:42.v0739460cda_c4
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:671.v07c339c842e8
pipeline-input-step:477.v339683a_8d55e
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2144.v077a_d1928a_40
pipeline-model-definition:2.2144.v077a_d1928a_40
pipeline-model-extensions:2.2144.v077a_d1928a_40
pipeline-rest-api:2.33
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2144.v077a_d1928a_40
pipeline-stage-view:2.33
pipeline-utility-steps:2.16.0
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.3.0
publish-over:0.22
publish-over-ssh:1.25
pubsub-light:1.17
resource-disposer:0.23
scm-api:676.v886669a_199a_a_
script-security:1271.vdede89739a_81
snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4
sse-gateway:1.26
ssh-agent:333.v878b_53c89511
ssh-credentials:308.ve4497b_ccd8f4
ssh-slaves:2.916.vd17b_43357ce4
sshd:3.312.v1c601b_c83b_0e
structs:325.vcb_307d2a_2782
timestamper:1.26
token-macro:384.vf35b_f26814ec
trilead-api:2.84.v72119de229b_7
variant:59.vf075fe829ccb
workflow-aggregator:596.v8c21c963d92d
workflow-api:1259.vb_47f14fffc8a_
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3769.v8b_e595e4d40d
workflow-durable-task-step:1284.v4fcd365b_75b_e
workflow-job:1341.vd9fa_65f771dd
workflow-multibranch:756.v891d88f2cd46
workflow-scm-step:415.v434365564324
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:848.v5a_383b_d14921
ws-cleanup:0.45

I have an nuint step to publish my nunit results. Everything here seems to work(test results are captured as expected) except when a test fails it doesnt mark the stage as unstable, only the build. So if I have multiple stages with nunit steps, I don’t know which one had the failure at a quick glance without diving in.

nunit testResultsPattern: 'TestFile.xml', debug: false, keepJUnitReports: true, skipJUnitArchiver: false, failIfNoResults: true

I have some junit steps that the are marking the build and stage as unstable as expected.
junit 'TEST-junit-jupiter.xml'

Is this a known issue with nunit publisher or am I missing something/doing something wrong?

Thanks

Hello @michig54 and welcome to this community. :wave:

The junit plugin and the nunit plugin handle test results differently, which can result in variations in how they affect the build and stage status.

Here are a few things to consider:

  1. NUnit Plugin Behavior: The nunit plugin for Jenkins primarily focuses on publishing NUnit test results for reporting purposes. It doesn’t inherently set the build or stage status based on test results. The failIfNoResults option in the nunit step only affects whether the build should be marked as failed if there are no test results found. It doesn’t control the build or stage status based on the test results.
  2. JUnit Plugin Behavior: The junit plugin for Jenkins, on the other hand, is designed to work specifically with JUnit-style test reports. It has built-in logic to mark the build and stages as unstable or failed based on the test results. This behavior is specific to the junit plugin.

Interesting! Thanks for your reply!

Is there a recommended way to use the nunit results to mark the stages as unstable?

Thanks