Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Output inconsistency (ios vs android) #1388

Closed
pawelpasterz opened this issue Dec 9, 2020 · 1 comment · Fixed by #1395
Closed

Output inconsistency (ios vs android) #1388

pawelpasterz opened this issue Dec 9, 2020 · 1 comment · Fixed by #1395
Assignees

Comments

@pawelpasterz
Copy link
Contributor

Describe the bug

A clear and concise description of what the bug is.

There are differences in flank output for iOS and Android when all test are filtered
Below is the output for https://github.com/Flank/flank/blob/master/integration_tests/src/test/kotlin/integration/AllTestFilteredIT.kt

To Reproduce

Steps to reproduce the behavior:

  1. run ios test with config
    gcloud:
      test: [path to zip]
      xctestrun-file: [path to xctestrun]
    flank:
      test-targets:
        - nonExisting/Class
    
  2. run android test with config
    gcloud:
      app: [path to app apk]
      test: [path to test apk]
      test-targets:
        - class non.existing.Class
    flank:
      disable-sharding: true
    

Expected behavior

A clear and concise description of what you expected to happen.

There is a decision to be made if flank should (or not) print args.
Based on the answer:

  • fix should be implemented
  • IT must be updated

Details (please complete the following information):

Have you tested on the latest Flank snapshot?

Post the output of flank --version.

version: local_snapshot
revision: 2d2982474e9d38ee39639ed7282b84fb5e33ce01

Additional context

Add any other context about the problem here.

Introduced in #1219

@adamfilipow92
Copy link
Contributor

Plan

  1. Print args like on android
  2. When no tests, throw the same exception type in android and ios

mergify bot pushed a commit that referenced this issue Dec 11, 2020
Fixes #1315 

## Test Plan
> How do we know the code works?

1. run ```./gradlew integrationTests```
1. integration tests should pass

## Description

Flank should also verify XML results for android and ios integration tests.

This ticket should be merged after #1316 when we update all tests. 

```filter all tests - ios``` is disabled and should be updated in #1388

## Checklist
- [X] Add verification for basic android test
- [X] Add verification for tests from #1316
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants