Question about using Xvfb in multi-branch pipeline

Hello:

I’m an IDE(based on Eclipse Theia) developer and using a multi-branch pipeline to run the CI test.

Now I’m trying to add E2E test to eclipsesource/theia according to the pull request

I add a Jenkinsfile for it:

pipeline {
    agent { label "linux-x64" }
    stages {
        stage('The Matrix') {
            matrix {
                agent { label "${AGENT}" }
                axes {
                    axis {
                        name 'AGENT'
                        values 'linux-x64', 'win64'
                    }
                }
                stages {
                    stage('Test') {
                        steps {
                            nodejs(nodeJSInstallationName: 'Node_v16') {
                                sh 'yarn all'
                                sh 'yarn electron build'
                                sh '''
                                    if [ "x${AGENT}" = "xlinux-x64" ]; then export DISPLAY=:99; echo $DISPLAY; Xvfb :99 -ac & ps -ef | grep Xvfb; fi
                                    yarn --cwd examples/playwright ui-tests -g "electron"
                                '''
                            }
                        }
                    }
                }
            }
        }
    }
}

As you can see, I use Xvfb as the virtual window but the job failed:

+ [ xlinux-x64 = xlinux-x64 ]
+ export DISPLAY=:99
+ echo :99
:99
+ Xvfb :99 -ac
+ ps -ef
+ grep Xvfb
jenkins  1185628 1185626  0 11:37 ?        00:00:00 Xvfb :99 -ac
jenkins  1185631 1185626  0 11:37 ?        00:00:00 grep Xvfb
+ yarn --cwd examples/playwright ui-tests -g electron
yarn run v1.22.5
$ yarn && playwright test --config=./configs/playwright.config.ts -g electron
[1/5] Validating package.json...
[2/5] Resolving packages...
success Already up-to-date.
$ yarn clean && yarn build
$ rimraf lib *.tsbuildinfo
$ tsc --incremental && npx playwright install chromium

Running 7 tests using 1 worker

  ✘  1 ../../src/tests/theia-electron-app.test.ts:34:9 › Theia Electron Application › should load and show main content panel (10ms)
  -  2 ../../src/tests/theia-electron-app.test.ts:38:9 › Theia Electron Application › open about dialog using menu
  -  3 ../../src/tests/theia-electron-app.test.ts:47:9 › Theia Electron Application › toggle explorer view using menu
  -  4 ../../src/tests/theia-electron-app.test.ts:55:9 › Theia Electron Application › open quick command palette
  -  5 ../../src/tests/theia-electron-app.test.ts:62:9 › Theia Electron Application › open about dialog using command
  -  6 ../../src/tests/theia-electron-app.test.ts:73:9 › Theia Electron Application › select all using command
  -  7 ../../src/tests/theia-electron-app.test.ts:80:9 › Theia Electron Application › toggle explorer view using command

  1) ../../src/tests/theia-electron-app.test.ts:34:9 › Theia Electron Application › should load and show main content panel 
    electronApplication.firstWindow: Timeout 30000ms exceeded while waiting for event "window"
       at ../../src/theia-app-loader.ts:84
      82 |         const playwrightOptions = this.toPlaywrightOptions(launchOptions, initialWorkspace);
      83 |         const electronApp = await electron.launch(playwrightOptions);
    > 84 |         const page = await electronApp.firstWindow();
         |                                        ^
      85 |         const appFactory = theiaAppFactory<T>(factory);
      86 |         const app = new appFactory(page, workspace);
      87 |         await app.waitForShellAndInitialized();

        at Function.load (/jenkins/workspace/theia-electron-playwright_master@2/examples/playwright/src/theia-app-loader.ts:84:40)
        at /jenkins/workspace/theia-electron-playwright_master@2/examples/playwright/src/tests/theia-electron-app.test.ts:31:15

  1 failed
    ../../src/tests/theia-electron-app.test.ts:34:9 › Theia Electron Application › should load and show main content panel 
  6 skipped
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
script returned exit code 1

I tried the Xvfb plugin for the Jenkins. But it only supports freestyleJob since I added the Xvfb installation in Jenkins global tool configuration. But configuration for multi-branch pipeline does not have the Build Environment settings:

Note: 多分支流水线 means Multi-branch pipeline

So I don’t know how to set the Xvfb in my pipeline.

Any suggestions?

Really appreciate!

1 Like

Before I get started to review this all, I would say you have way too much going on to actually ask questions. I think you should (either now or in the future) try to make the smallest test case possible that shows the issue, it would help us help you because we don’t need to spend 20 minutes reading everything and trying to figure out what your question is, and would help you narrow down the specific problem. Looks like you were assuming it was a different line than the problem was on.

these should be .trim() btw, otherwise they will likely have newlines on the end of the variable.

export only lasts the length of that sh() statement, so this won’t do anything.
You can just set it using script { env.DISPLAY=":99" }
you could also script { env.XVFB_PID = sh(returnStdout: true, script: 'Xvfb :99 -ac 1>/dev/null &; echo $$').trim(); } or something so you know what pid to kill later.

your screenshot seems to say its running this step, which is calling tsc, which is then calling tets. So you don’t even have xvfb started then

1 Like

Really appreciate for your reply. I simplified the original post.

My question is:

I can’t use Xvfb directly in the “sh” of the step even though the e2e test works in terminal (all test passed, you can ignore the following output):

tptuser@linux-ide:~/Work/eclipsesource_theia$ sh
$ Xvfb :99 -ac &
$ export DISPLAY=:99
$ yarn --cwd examples/playwright ui-tests -g electron
yarn run v1.22.19
$ yarn && playwright test --config=./configs/playwright.config.ts -g electron
[1/5] Validating package.json...
[2/5] Resolving packages...
success Already up-to-date.
$ yarn clean && yarn build
$ rimraf lib *.tsbuildinfo
$ tsc --incremental && npx playwright install chromium

Running 7 tests using 1 worker

  ✓  1 ../../src/tests/theia-electron-app.test.ts:34:9 › Theia Electron Application › should load and show main content panel (7ms)
  ✓  2 ../../src/tests/theia-electron-app.test.ts:38:9 › Theia Electron Application › open about dialog using menu (134ms)
  ✓  3 ../../src/tests/theia-electron-app.test.ts:47:9 › Theia Electron Application › toggle explorer view using menu (229ms)
  ✓  4 ../../src/tests/theia-electron-app.test.ts:55:9 › Theia Electron Application › open quick command palette (5.1s)
  ✓  5 ../../src/tests/theia-electron-app.test.ts:62:9 › Theia Electron Application › open about dialog using command (5.6s)
  ✓  6 ../../src/tests/theia-electron-app.test.ts:73:9 › Theia Electron Application › select all using command (11.1s)
  ✓  7 ../../src/tests/theia-electron-app.test.ts:80:9 › Theia Electron Application › toggle explorer view using command (6.5s)

  Slow test file: ../../src/tests/theia-electron-app.test.ts (28.6s)
  Consider splitting slow test files to speed up parallel execution

  7 passed (32.5s)
Done in 35.98s.
$ exit

In addition, the Xvfb plugin appears to support only free jobs, not multi-branch pipelines, because the multi-branch configuration does not have the “build environment” Settings required by the Xvfb plugin

I’ll try it! :smile: Thanks a lot

A lot of the plugins are no longer needed with pipeline. They were designed for the limitations of the kinda ironically named freestyle jobs. Even the nodejs plugin you can replace with tool()

can’t use Xvfb directly in the “sh” of the step even th

Why can’t you?

The reason I use the NodeJS plugin is that several of my repositories require different versions of NodeJS. But when I installed the nvm(NodeJS version manager) in the built-in Jenkins node and tried to change the version of NodeJS by runing sh 'nvm use 12.22.0' or sh 'nvm use 16.18.0' in the pipeline steps, the sh cannot get the nvm executable. The $PATH of sh does not contain the nvm folder which confused me, too.

For example, the $PATH in the terminal is (The host jenkins is the built-in node):

tptuser@jenkins:~$ su jenkins
Password: 
jenkins@jenkins:/home/tptuser$ echo $PATH
/jenkins/.nvm/versions/node/v16.14.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
jenkins@jenkins:/home/tptuser$ sh
$ echo $PATH
/jenkins/.nvm/versions/node/v16.14.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
$ node -v
v16.14.0

But when I tried to echo $PATH in the sh of pipeline, it shows:

When I tried use sh to launch the Xvfb and run the test like the original post:

sh '''
    if [ "x${AGENT}" = "xlinux-x64" ]; then export DISPLAY=:99; echo $DISPLAY; Xvfb :99 -ac & ps -ef | grep Xvfb; fi
    yarn --cwd examples/playwright ui-tests -g "electron"
'''

The yarn xxx ui-test failed to connect the Xvfb. I’m wondering if the reason for the failure is similar to the failure of the nvm.

re nvm:
nvm needs shell hooks to function. Shell hooks are hard in jenkins because each sh() is a different session. Its fixable, but if what you have works and you are happy with it, then no real concern.

are you 100% certain that its that yarn statement that is failing? From the original screenshot, it was yarn build failing, not yarn ui-tests

Just confirm your assumptions and add an “echo before-ui-tests” and “echo after-ui-tests” around it, and same with other yarn commands, and confirm which one is failing. I’m pretty sure you are starting xfvb after you need it, not before.

I modified the pipeline as you said:

stage('Install & Build') {
    steps {
        nodejs(nodeJSInstallationName: 'Node_v16') {
            sh 'yarn all'
            sh '''
                echo "before building..."
                yarn electron build
                echo "after building..."
            '''
        }
    }
}

stage('Test') {
    steps {
        nodejs(nodeJSInstallationName: 'Node_v16') {
            sh '''
                if [ "x${AGENT}" = "xlinux-x64" ]; then export DISPLAY=:99; echo $DISPLAY; Xvfb :99 -ac & ps -ef | grep Xvfb; fi
                echo "before ui-test..."
                yarn --cwd examples/playwright ui-tests -g "electron"
                echo "after ui-test...
            '''
        }
    }
}

I put the launch of Xvfb and the yarn test into a same sh so that they could use a same session.
And use the ps -ef | grep Xvfb to check if the Xvfb is running:

sh '''
    if [ "x${AGENT}" = "xlinux-x64" ]; then export DISPLAY=:99; echo $DISPLAY; Xvfb :99 -ac & ps -ef | grep Xvfb; fi
    echo "before ui-test..."
    yarn --cwd examples/playwright ui-tests -g "electron"
    echo "after ui-test...
'''

The “yarn build” was successfully executed since Xvfb is required for yarn ui-test rather than yarn build. The output of ps -ef | grep Xvfb shows the Xvfb is running correctly, but the yarn ui-test failed:

The error message Timeout 30000ms exceeded while waiting for event "window" of yarn ui-test shows it doesn’t connect to Xvfb successfully. I can reproduce this error manually in the terminal without launching Xvfb.

Maybe its not ready yet?

Finally, I found a correct pipeline syntax for the Xvfb plugin:

stage('Test') {
    steps {
        nodejs(nodeJSInstallationName: 'Node_v16') {
            sh 'xvfb-run -s "-screen 0 1024x768x24" yarn --cwd examples/playwright ui-tests -g "electron"'
        }
    }
}

I add the xvfb-run prefix to let the Xvfb plugins to start the virtual screen to run the yarn command and it works.