GUI and CLI Treat Top-Level beforeEach Differently

[CodingItWrong]

Current behavior:

• I have a test suite that has one test file with a top-level beforeEach block
• When I "Run all specs" in the GUI, the beforeEach block is run before every test file. As a result, the other test file was written to rely on the beforeEach block, and so it passes when run as part of the whole suite in the GUI:

• But when running this suite in the CLI, the beforeEach block is only run for the test file it's in. As a result, the other test file fails, even though it succeeded in the GUI:

Desired behavior:

• Running full test suite from the GUI or CLI should treat beforeEach blocks in the same way
• Note that I don't desire the beforeEach block to behave one way or the other: whether it fails or succeeds consistently across the GUI and CLI is fine. A top-level beforeEach block is not recommended by Mocha. The desires is for consistency on this behavior across GUI and CLI.

Test code to reproduce

https://github.com/CodingItWrong/cypress-persistent-stub - instructions in readme

Versions

• Cypress 4.5.0 (also seen in 4.2.0 in production project)
• macOS 10.15.4
• Node 12.16.2

[jennifer-shehane]

Yes, this is a known limitation of 'Runs all specs'. In the future we will change it to behave the same as cypress run, so recommend that you update your test code accordingly.

Duplicate of #1586

[bahmutov]

Just a note: I have wrote a blog post about it https://glebbahmutov.com/blog/run-all-specs/ and will add a note to our documentation