screenshots saved in different folders when invoked from app vs. cli

[phryneas]

Current behavior:

When using .screenshot() in a test from the command line with yarn cypress run -s cypress/integration/visual-regression/test-urls.js, screenshots are saved to cypress/screenshots/visual-regression/test-urls.js/.
When using the same test from the app (cypress open), the screenshots are saved to cypress/screenshots/integration/visual-regression/test-urls.js/

Desired behavior:

Screenshots are saved to the same folders, no matter how the test is invoked

Versions

3.0.2, I guess this is introduced with #1826 / #1858

[brian-mann]

This for sure is a bug. @chrisbreiding can you take a peek?

[chrisbreiding]

@brian-mann Which version is correct? With or without integration?

[brian-mann]

Without integration

[cangoektas]

I'm on 3.0.3 and I'm still experiencing this issue.

When running cypress run this is what my screenshots folder looks like:

cypress/screenshots
├── home.spec.js
│   └── home.png
└── login.spec.js
    └── login.png

When running cypress open and clicking on "Run all specs", an "All specs" folder is created:

cypress/screenshots
├── All\ Specs
│   ├── home.png
│   └── login.png
├── home.spec.js
│   └── home.png
└── login.spec.js
    └── login.png

Is this expected?

[chrisbreiding]

No, that's definitely not expected. Looks like we didn't take running all specs into account when implementing the new screenshot path rules. It's a different issue than the original here, so I'm going to create a new GitHub issue for this.

[edelgado]

Also, I wish the directories weren't named with .js in them. It's confusing. Instead of this:

cypress/screenshots
├── home.spec.js
│   └── home.png
└── login.spec.js
    └── login.png

We should have:

cypress/screenshots
├── home
│   └── home.png
└── login
    └── login.png