SessionNotCreatedException

Jenkins setup:
Jenkins: 2.401.1.3
OS: Linux - 5.10.184-175.731.amzn2.x86_64
Java: 11.0.19 - Red Hat, Inc. (OpenJDK 64-Bit Server VM)

ace-editor:1.1
antisamy-markup-formatter:159.v25b_c67cd35fb_
apache-httpcomponents-client-4-api:4.5.14-150.v7a_b_9d17134a_5
authentication-tokens:1.53.v1c90fd9191a_b_
aws-credentials:191.vcb_f183ce58b_9
aws-java-sdk-ec2:1.12.447-382.vda_68e2007233
aws-java-sdk-elasticbeanstalk:1.12.447-382.vda_68e2007233
aws-java-sdk-minimal:1.12.447-382.vda_68e2007233
blueocean-commons:1.27.3
blueocean-rest:1.27.3
bluesteel-cjoc:1.2.52
bouncycastle-api:2.27
caffeine-api:3.1.6-115.vb_8b_b_328e59d8
cloudbees-administrative-monitors:1.0.11
cloudbees-analytics:1.48
cloudbees-assurance:2.276.0.28
cloudbees-blueocean-default-theme:0.8
cloudbees-casc-server:2.3
cloudbees-folder:6.815.v0dd5a_cb_40e0e
cloudbees-folders-plus:3.29
cloudbees-jenkins-advisor:336.v4d00382fe22c
cloudbees-license:9.73
cloudbees-monitoring:2.16
cloudbees-platform-common:1.19
cloudbees-platform-data:1.33
cloudbees-plugin-usage:2.17
cloudbees-prometheus:1.2
cloudbees-ssh-slaves:2.20
cloudbees-support:3.31
cloudbees-uc-data-api:4.56
cloudbees-unified-ui:1.24
cloudbees-update-center-plugin:4.78
command-launcher:100.v2f6722292ee8
commons-lang3-api:3.12.0-36.vd97de6465d5b_
commons-text-api:1.10.0-36.vc008c8fcda_7b_
credentials:1254.vb_96f366e7b_a_d
credentials-binding:604.vb_64480b_c56ca_
display-url-api:2.3.7
docker-commons:419.v8e3cd84ef49c
durable-task:506.v1b_3e14b_6f5da_
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
infradna-backup:3.38.64
instance-identity:142.v04572ca_5b_265
ionicons-api:45.vf54fca_5d2154
jackson2-api:2.14.2-319.v37853346a_229
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.8-1
jdk-tool:66.vd8fa_64ee91b_d
jjwt-api:0.11.5-77.v646c772fddb_0
jquery3-api:3.6.4-1
kube-agent-management:1.1.59
kubernetes:3923.v294a_d4250b_91
kubernetes-client-api:6.4.1-215.v2ed17097a_8e9
kubernetes-credentials:0.10.0
ldap:676.vfa_64cf6b_b_002
mailer:448.v5b_97805e3767
mapdb-api:1.0.9-28.vf251ce40855d
controller-provisioning-core:2.6.41
controller-provisioning-kubernetes:2.6.41
metrics:4.2.13-420.vea_2f17932dd6
mina-sshd-api-common:2.9.2-62.v199162f0a_2f8
mina-sshd-api-core:2.9.2-62.v199162f0a_2f8
mina-sshd-api-scp:2.9.2-62.v199162f0a_2f8
mina-sshd-api-sftp:2.9.2-62.v199162f0a_2f8
nectar-license:8.41
nectar-rbac:5.80
node-iterator-api:49.v58a_8b_35f8363
oauth-credentials:0.645.ve666a_c332668
okhttp-api:4.10.0-132.v7a_7b_91cef39c
operations-center-agent:2.387.0.2
operations-center-clusterops:2.387.0.1
operations-center-context:2.387.0.5
operations-center-elasticsearch-provider:2.387.0.1
operations-center-jnlp-controller:2.387.0.5
operations-center-kubernetes-cloud:2.387.0.2
operations-center-license:2.387.0.1
operations-center-monitoring:2.387.0.1
operations-center-rbac:2.387.0.2
operations-center-server:2.387.0.6
operations-center-sso:2.387.0.2
operations-center-updatecenter:2.387.0.1
plain-credentials:143.v1b_df8b_d3b_e48
pubsub-light:1.17
scm-api:667.v8b_6e07cdc7f2
script-security:1244.ve463715a_f89c
snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4
sse-gateway:1.26
ssh-credentials:305.v8f4381501156
sshd:3.275.v9e17c10f2571
structs:324.va_f5d6774f3a_d
support-core:1283.v9ddb_0284a_00c
token-macro:359.vb_cde11682e0c
trilead-api:2.84.v72119de229b_7
unique-id:2.101.v21a_b_6390a_b_04
user-activity-monitoring:1.9
variant:59.vf075fe829ccb
workflow-api:1208.v0cc7c6e0da_9e
workflow-durable-task-step:1246.v5524618ea_097
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:839.v35e2736cfd5c

I am running ui tests via selenium grid(4.15) using pytest in jenkins. When I run 100+ tests with chrome/firefox webdriver(latest) either with single worker or with 4 workers, 1st test always returns SessionNotCreatedException but other tests run fine. What could be the reason for this? Locally I dont get this issue.

Below is the error log:

E   selenium.common.exceptions.SessionNotCreatedException: Message: Could not start a new session. Could not start a new session. Unable to obtain: Capabilities {browserName: chrome, browserVersion: 119.0, goog:chromeOptions: {args: [], extensions: [], prefs: {download.default_directory: /home/ec2-user/jenkins/work..., download.directory_upgrade: true, download.prompt_for_download: false, helperApps.neverAsk.saveToDisk: /home/ec2-user/jenkins/work..., profile.default_content_settings.popups: 0, safebrowsing.enabled: false, safebrowsing_for_trusted_sources_enabled: false}}, pageLoadStrategy: normal, platformName: linux, se:noVncPort: 7900, se:vncEnabled: true}, error Failed to parse json output, executed: [--browser, chrome, --output, json, --browser-version, 119.0]
E   
E   Build info: version: '4.15.0', revision: '1d14b5521b'
E   System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.1.38-59.109.amzn2023.x86_64', java.version: '11.0.20.1'
E   Driver info: driver.version: unknown 
E   Host info: host: 'a3cd4230924a', ip: '192.168.48.11'
E   Build info: version: '4.15.0', revision: '1d14b5521b'
E   System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.1.38-59.109.amzn2023.x86_64', java.version: '11.0.20.1'
E   Driver info: driver.version: unknown
E   Build info: version: '4.15.0', revision: '1d14b5521b'
E   System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.1.38-59.109.amzn2023.x86_64', java.version: '11.0.20.1'
E   Driver info: driver.version: unknown
E   Stacktrace:
E       at org.openqa.selenium.grid.node.remote.RemoteNode.newSession (RemoteNode.java:151)
E       at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession (LocalDistributor.java:645)
E       at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession (LocalDistributor.java:564)
E       at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest (LocalDistributor.java:824)
E       at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1 (LocalDistributor.java:784)
E       at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
E       at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
E       at java.lang.Thread.run (Thread.java:829)

Hello and welcome to this community, @eventsnotif. :wave:

Do you have the very same versions of Selenium and WebDriver locally and on your Jenkins agent?

@poddingue Yes, I have the same version of chrome & chromedriver.
chrome & chromedriver version (local) - 121.0.6167.184.
chrome & chromedriver version (jenkins) - 119.0.6045.105