Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Provide more detailed debug when parallelization fails #24540

Closed
cgraham-rs opened this issue Nov 4, 2022 · 1 comment
Closed

Provide more detailed debug when parallelization fails #24540

cgraham-rs opened this issue Nov 4, 2022 · 1 comment
Labels
Cypress Cloud Feature request or issue in Cypress Cloud, not App E2E Issue related to end-to-end testing type: feature New feature that does not currently exist

Comments

@cgraham-rs
Copy link

What would you like?

As a user of Github workflow for CI, and of Cypress Dashboard for parallelization, when parallelization fails I want more detailed debug presented.

See additional comments in this ticket: #24514 (comment)

Why is this needed?

When parallelization fails there is not enough detail to figure out the root cause. Compare the output from these two runners in the same workflow. It does not output the same exact parameters to compare apples-to-apples to figure out what may be different.

Runner that failed to execute Cypress parallelized:

You passed the --parallel flag, but we do not parallelize tests across different environments.

This machine is sending different environment parameters than the first machine that started this parallel run.

The existing run is: https://dashboard.cypress.io/projects/dhybrn/runs

In order to run in parallel mode each machine must send identical environment parameters such as:

 - specs
 - osName
 - osVersion
 - browserName
 - browserVersion (major)

This machine sent the following parameters:

{
  "osName": "linux",
  "osVersion": "Ubuntu - 20.04",
  "browserName": "Chrome",
  "browserVersion": "106.0.5249.119",
  "specs": [
    "cypress/e2e/auth.spec.js",
    "cypress/e2e/dashboard.spec.js",
    etc. etc.
  ]
}

Runner that could successfully execute Cypress parallelized.

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        10.10.0                                                                        │
  │ Browser:        Chrome 107 (headless)                                                          │
  │ Node Version:   v16.18.0 (/opt/hostedtoolcache/node/16.18.0/x64/bin/node)                      │
  │ Specs:          22 found (auth.spec.js, dashboard.spec.js, home.spec.js, search.spec.js, sylla │
  │                 bus.spec.js, termsAndConditions.spec.js, assessment/assessment_logic_flow.spec │
  │                 .js, assessment/general_assessment.spec.js, assessment/group_report_panel.spec │
  │                 .js, exercise_bank/data...)                                                    │
  │ Searched:       cypress/e2e/**/*.{js,jsx,ts,tsx}                                               │
  │ Params:         Tag: false, Group: chrome, Parallel: true                                      │
  │ Run URL:        https://dashboard.cypress.io/projects/dhybrn/runs/1584                         │
  │ Experiments:    experimentalFetchPolyfill=true,experimentalSessionAndOrigin=true               │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

Other

No response

@mschile mschile added type: feature New feature that does not currently exist E2E-core labels Nov 4, 2022
@nagash77 nagash77 added E2E Issue related to end-to-end testing and removed E2E-core labels Nov 8, 2022
@emilyrohrbough emilyrohrbough added external: cloud Cypress Cloud Feature request or issue in Cypress Cloud, not App labels Dec 21, 2022
@emilyrohrbough
Copy link
Member

@cgraham-rs In Cypress 12.1.0 we updated this error message to provide additional details to simplify the process of identifying machine mismatches when a parallelized run fails. The run differences are now displayed in the error. Please see #24734.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Cypress Cloud Feature request or issue in Cypress Cloud, not App E2E Issue related to end-to-end testing type: feature New feature that does not currently exist
Projects
None yet
Development

No branches or pull requests

5 participants