[IOS] Failing to execute tests in Bitrise for debug variant

[GuiCavi]

Description

I'm using Bitrise to run Detox tests. The build command runs ok and is a success. But the test command always fails.

Link to StackOverflow question

---

Running locally:

• Command detox test -c ios.sim.debug without starting the packager: 'No bundle URL present' error

• Command detox test -c ios.sim.debug with the packager: Opens the packager and everything runs OK.

• Command detox test -c ios.sim.release without the packager: works fine.

• Command detox test -c ios.sim.release with the packager: works fine.

---

Running in Bitrise:

• Command detox test -c ios.sim.debug --cleanup without starting the packager: 'No bundle URL present' error
• Command detox test -c ios.sim.debug --cleanup with the packager:

Verify tests are up
    "before all" hook:
    Error: Timeout of 120000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/vagrant/git/e2e/tests.spec.js)

• Command detox test -c ios.sim.release --cleanup without the packager: works fine.
• Command detox test -c ios.sim.release --cleanup with the packager: works fine.

Disclaimer
With/Without the packager means the variable RCT_NO_LAUNCH_PACKAGER equals false/true, respectively.

---

The package.json file

"ios.sim.debug": {
    "binaryPath": "ios/build/Build/Products/Debug-iphonesimulator/PROJECT_NAME.app",
    "build": "export RCT_NO_LAUNCH_PACKAGER=true && xcodebuild -workspace ios/PROJECT_NAME.xcworkspace -scheme INTEGRATION_SCHEME -configuration Debug -sdk iphonesimulator -derivedDataPath ios/build",
    "type": "ios.simulator",
    "name": "iPhone 7"
},
"ios.sim.release": {
    "binaryPath": "ios/build/Build/Products/Release-iphonesimulator/PROJECT_NAME.app",
    "build": "export RCT_NO_LAUNCH_PACKAGER=true && xcodebuild -workspace ios/PROJECT_NAME.xcworkspace -scheme PRODUCTION_SCHEME -configuration Release -sdk iphonesimulator -derivedDataPath ios/build",
    "type": "ios.simulator",
    "name": "iPhone 7 Plus"
},

The bitrise.yml file

_init_install:
    steps:
    - activate-ssh-key@3.1.1:
        run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
    - git-clone@4.0.11: {}
    - script@1.1.5:
        inputs:
        - content: |-
            #!/usr/bin/env bash
            # fail if any commands fails
            set -e
            # debug log
            set -x

            # write your script here
            brew uninstall --ignore-dependencies node
            brew install node@8
            brew link --overwrite --force node@8
    - yarn@0.0.7: {}

_detox_tests:
    steps:
    - npm@0.9.0:
        title: Install Detox CLI
        inputs:
        - command: install -g detox-cli
    - npm@0.9.0:
        title: Install React-Native CLI
        inputs:
        - command: install -g react-native-cli
    - script@1.1.5:
        title: Install Detox Utils
        inputs:
        - content: |-
            #!/bin/bash
            brew tap facebook/fb
            export CODE_SIGNING_REQUIRED=NO
            brew install fbsimctl
            brew tap wix/brew
            brew install applesimutils --HEAD
    - script@1.1.5:
        title: Detox - Build Debug App
        inputs:
        - content: |-
            #!/usr/bin/env bash
            detox build -c ios.sim.debug
    - script@1.1.5:
        title: Detox - Run E2E Tests
        inputs:
        - content: |-
            #!/usr/bin/env bash
            detox test -c ios.sim.debug --cleanup
    - deploy-to-bitrise-io@1.3.10: {}

test_ios:
    before_run:
    - _init_install
    after_run:
    - _detox_tests
    steps:
    - cocoapods-install@1.7.2:
        inputs:
        - podfile_path: ios/Podfile
    - resource-archive@2.0.1:
        inputs:
        - extract_to_path: "$HOME/Documents/FacebookSDK/"
        - archive_url: https://origincache.facebook.com/developers/resources/?id=facebook-ios-sdk-current.zip
    - certificate-and-profile-installer@1.9.3: {}

---

Detox, Node, Device, Xcode and macOS Versions

• Detox: v7.4.3
• React Native: v0.51.0
• Node: v9.4.0
• Device: iPhone 7 (simulator)
• Xcode: Version 9.3 (9E145)
• macOS: 10.13.3 (17D102)

---

Device and verbose Detox logs

1. Error detox test -c ios.sim.debug --cleanup with the packager

Verify tests are up
detox info 12:14:57: server listening on localhost:49300...
detox info 2: Searching for device matching iPhone 7... 
detox info 5: Booting device 923035AA-8378-47FE-A4FE-CE5205AF323C 
detox info 7: Uninstalling com.PROJECT_NAME.PROJECT_NAME... 
detox info 7: com.PROJECT_NAME.PROJECT_NAME uninstalled 
detox info 8: Installing /Users/vagrant/git/ios/build/Build/Products/Debug-iphonesimulator/PROJECT_NAME.app... 
detox info 8: /Users/vagrant/git/ios/build/Build/Products/Debug-iphonesimulator/PROJECT_NAME.app installed 
detox info 9: Terminating com.PROJECT_NAME.PROJECT_NAME... 
detox info 9: com.PROJECT_NAME.PROJECT_NAME terminated 
detox info 10: Trying to set permissions... 
detox info 10: Permissions are set 
detox info 11: Launching com.PROJECT_NAME.PROJECT_NAME... 
detox info 11: com.PROJECT_NAME.PROJECT_NAME launched. The stdout and stderr logs were recreated, you can watch them with:
        tail -F $HOME/Library/Developer/CoreSimulator/Devices/923035AA-8378-47FE-A4FE-CE5205AF323C/data/tmp/detox.last_launch_app_log.{out,err} 
    1) "before all" hook
  0 passing (2m)
  1 failing
  1) Verify tests are up
       "before all" hook:
     Error: Timeout of 120000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/vagrant/git/e2e/tests.spec.js)
  
detox info 12: Shutting down 923035AA-8378-47FE-A4FE-CE5205AF323C... 
detox info 12: 923035AA-8378-47FE-A4FE-CE5205AF323C shut down 
child_process.js:644
    throw err;
    ^
Error: Command failed: node_modules/.bin/mocha e2e --opts e2e/mocha.opts --configuration ios.sim.debug  --cleanup   --grep :android: --invert  
    at checkExecSyncError (child_process.js:601:13)
    at Object.execSync (child_process.js:641:13)
    at runMocha (/Users/vagrant/git/node_modules/detox/local-cli/detox-test.js:114:6)
    at run (/Users/vagrant/git/node_modules/detox/local-cli/detox-test.js:75:7)
    at Object.<anonymous> (/Users/vagrant/git/node_modules/detox/local-cli/detox-test.js:181:1)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)

2. File detox.last_launch_app_log.err prints:

Failed to load bundle(http://localhost:8081/index.bundle?platform=ios&dev=true&minify=false) with error:(Could not connect to development server.
Ensure the following:
- Node server is running and available on the same network - run 'npm start' from react-native root
- Node server URL is correctly set in AppDelegate
- WiFi is enabled and connected to the same network as the Node Server
URL: http://localhost:8081/index.bundle?platform=ios&dev=true&minify=false The operation couldn’t be completed. (NSURLErrorDomain error -999.))

---

TL;DR

I get an error when running Detox with Bitrise. The command detox test -c ios.sim.debug --cleanup should open a packager but looks like it's not doing it.

The version that is needed to test is the Debug one, no other options. Maybe I missed something about the installation, but all the files were checked with official docs and other examples and are the identical.

Related issues: #742, #749, #740

[pkantsedalov]

@GuiCavi Hi. I have the similar issue.
How can it possible at all to run detox test without a packager? I mean is it possible to at first run detox build and then immediately run detox test?

[GuiCavi]

@pkantsedalov Hi, the only possible way to do that is by running a release version. This version doesn't need the packager. I'm not sure how Bitrise works internally, but I believe it:

• Runs the build command
• The build triggers the packager to open
• It closes by the end of the command
• Then it runs the test

The point is that the packager closes right before the tests starts, while running on a local machine it stays opened in another terminal window. Anyway, a workaround is to run the tests in Release version, pointing to a Test database (if needed).

What do you think of that?

[pkantsedalov]

@GuiCavi I arranged the next flow:

1. detox build -c {configurationFromPackageJson}
2. node_modules/react-native/local-cli/cli.js bundle --dev false --entry-file index.js --platform ios --reset-cache --bundle-output Build/Products/Debug-iphonesimulator/{myApp}.app/main.jsbundle --assets-dest Build/Products/Debug-iphonesimulator/{appName}.app
3. detox test -c {configurationFromPackageJson}

Works fine.

[GuiCavi]

That makes sense, I'll give it a try. All in the same step? Like:

- script@1.1.5:
        inputs:
        - content: |-
            #!/usr/bin/env bash
            # fail if any commands fails
            set -e
            # debug log
            set -x

            detox build -c {configurationFromPackageJson}
            node_modules/react-native/local-cli/cli.js bundle --dev false --entry-file index.js --platform ios --reset-cache --bundle-output Build/Products/Debug-iphonesimulator/{myApp}.app/main.jsbundle --assets-dest Build/Products/Debug-iphonesimulator/{appName}.app
            detox test -c {configurationFromPackageJson}

[pkantsedalov]

@GuiCavi sorry we don't have any CI solution for that yet. For now we have such script for local tests run.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.
Thank you for your contributions.

[stale]

The issue has been closed for inactivity.

[noomorph]

From what I see, it could be good if we were running Detox e2e test builds in debug mode as well, preferably including Bitrise. I think it needs to be filled as a separate issue though.