Cypress Run - Do not relaunch browser in between each spec file

[renelux]

In our current project we have around 40 spec files with tests for different components or pages. The Cypress pipeline on our CI takes around 12-13 min. Instead of just throwing more server resources at it and paralyze the runs we looked into how we could improve the runtime by doing some optimizations.

We noticed that for every spec file it reloads the Cypress runner, this takes up some time which might not be necessary.

Our current solution is to create a index_spec.js that dynamically imports all other spec files in that folder. This cuts the runtime down with around 4-5 minutes, but this doesn't work well with plugins like cypress-plugin-snapshots as they use the spec files path to generate the path to store the snapshots.

Would it be possible to have the run command support a way to bundle specs?

[jennifer-shehane]

We noticed that for every spec file it reloads the Cypress runner, this takes up some time which might not be necessary.

Reloading the Cypress runner for each spec is intended behavior - and a recent change made in Cypress 3.0 that you can read about in more detail here: #681

[Lakitna]

Similar to #2943 I'd say

[lephuongbg]

@jennifer-shehane Has the team evaluated the option that instead of killing the browser completely, we could add a new tab to it then close the previous tab? I figure that at least we could save the time and resources that the browser needs when it initialize the first time.

[dipasqualew]

@jennifer-shehane Has the team evaluated the option that instead of killing the browser completely, we could add a new tab to it then close the previous tab? I figure that at least we could save the time and resources that the browser needs when it initialize the first time.

Seconding this - Any chance to execute the new spec into another tab?

Cheers

[a7madgamal]

or can someone at least suggest a way to avoid having 10000 tests in one file? can we split them between files without killing the whole browser in between?

[sagar-ag05]

what if we want to maintain the state of the application, instead of launching new tab which may cause to lose the state? For example, I login to the app. (My app is huge). Now after I login, i have many e2e tests in different spec files. And once all of my tests are done, I logout from the app. But i dont want to relaunch the browser/tab and lose the state. Have you thought of this use case ?

[jennifer-shehane]

@sagar-ag05 We're working on a session API that should help with this use case. #8301

[sagar-ag05]

@jennifer-shehane - What is the timeline of session API availability?
And any workaround to use in the interim to unblock us? I want the ability to record screenshots/videos and for that I need to use 'cypress run' command. But the moment I use this command, the browser relaunches after each spec file, causing tests to fail.

[sagar-ag05]

We noticed that for every spec file it reloads the Cypress runner, this takes up some time which might not be necessary.

Our current solution is to create a index_spec.js that dynamically imports all other spec files in that folder. This cuts the runtime down with around 4-5 minutes,

@renelux - How are you combining/importing diff spec file into index_spec.js file ?

[mcintyret]

+100 to this feature. I have 100s of tests split up logically into spec files, which is useful to a) avoid a single, massive spec file and b) to allow me to run a single spec at a time if needed. But when running all the tests (definitely in CI, but also locally sometimes) I don't need the browser to restart after each spec, and it adds a ton of time.

My tests are relatively simple - parallelism and user sessions would be overkill, or not applicable. I just need a way to run multiple spec files without killing the browser between each one!

[saiudayakella]

@mcintyret - Just curious to know if you have grouped just single test under each spec file?

[colin0117]

I'm with @mcintyret on this one - I've got about a dozen spec files (expected to grow rapidly), each testing a specific page in our app, and if I execute cypress run the majority of the test run is spent waiting for the browser to tear down or spin up. With our Jasmine/Karma tests, the browser remains open between the specs, meaning those are lightning fast to execute. This isn't about state like some of the other comments, each spec goes to a different page, it's just simply the time taken to relaunch browser.

[pmk1c]

It seems Cypress 10 brings the same behaviour to component tests, which is even more problematic.
Using the browser cache in between component specs can lead to dramatically better performance, which is now not possible anymore.

[lmiller1990]

@pmk1c Component Testing in 10.0 now uses the same code path as E2E. This did indeed slow things down, but improve spec isolation... I'm looking at some perf improvements, follow #22353 for more info.

As for the original feature request - as of Cypress 10, both E2E and CT now use 1 browser, a new tab per spec, so I'd say this feature (as in, the original request) is ✔️, which should solve some performance issues. This isn't the end - we are always looking to making things faster - just the end of the "single browser per run" issue.

The experimental session API also landed. See the docs: https://docs.cypress.io/api/commands/session

I'm going to close this since both requests (single tab, sessions) are implemented.

[bijojoseph99]

In our current project we have around 40 spec files with tests for different components or pages. The Cypress pipeline on our CI takes around 12-13 min. Instead of just throwing more server resources at it and paralyze the runs we looked into how we could improve the runtime by doing some optimizations.

We noticed that for every spec file it reloads the Cypress runner, this takes up some time which might not be necessary.

Our current solution is to create a index_spec.js that dynamically imports all other spec files in that folder. This cuts the runtime down with around 4-5 minutes, but this doesn't work well with plugins like cypress-plugin-snapshots as they use the spec files path to generate the path to store the snapshots.

Would it be possible to have the run command support a way to bundle specs?

Hi René Lux,
@renelux
Could you please guide me on the way you did the work to reduce the time taken from 1-12 mins to 4-5 Mins?
I'm in such a situation where the tests take 5-6 mins but the whole stage takes around 20 mins.
Thanks

[lmiller1990]

He is just doing something like

/** cypress.config.js */
export default defineConfig({
  e2e: {
    specPattern: 'cypress/e2e/**/*.spec.js'
  }
})

/** cypress/e2e/all_specs.spec.js */
import './spec-1.js'
import './spec-2.js'
import './spec-3.js'

Making 1 spec file that imports the others, so Cypress only launches once (for the 1 spec file).

Not sure this is ideal, but it answers the question you are asking (what did the user do). Most likely you'll want a different specPattern for development (so you don't need to run every single spec in open mode, which would make iterating on specs slow).