Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Author integration tests using Ginkgo #27

Closed
6 tasks done
cam-schultz opened this issue Sep 19, 2023 · 0 comments · Fixed by #168
Closed
6 tasks done

Author integration tests using Ginkgo #27

cam-schultz opened this issue Sep 19, 2023 · 0 comments · Fixed by #168
Assignees
Labels
enhancement New feature or request tests

Comments

@cam-schultz
Copy link
Contributor

cam-schultz commented Sep 19, 2023

Action items
The Ginkgo test framework has already been implemented in other tickets, for example #94. Much of the discussion in the following sections have already been resolved and implemented, but I'll leave the text there for context.

The main remaining action item to close this ticket out is to port the existing bash integration tests to our Ginkgo framework. Below are the test cases we need to port.

Context and scope
The current integration test setup consists of a set of docker containers that:

  1. Spin up a local avalanche network and deploys subnets and Teleporter contracts
  2. Runs an awm-relayer instance
  3. Executes integration tests

These steps are performed within the containers using a set of bash scripts. For example, https://github.com/ava-labs/teleporter/blob/main/docker/run_setup.sh starts the networks, and https://github.com/ava-labs/teleporter/blob/main/scripts/local/test.sh runs the integration tests.

Discussion and alternatives
Ginkgo is a test framework written in Go that performs very similar steps to our current setup, but in a much more user friendly and less error prone way. Ginkgo:

  • handles container setup and teardown
  • is easier to avoid bugs when authoring tests due to Go's static typing
  • produces much more readable tests with clearly defined expected behavior
  • is under active development

To convert our bash-based integration tests to Ginkgo, the following development strategy is proposed:

  1. Port awm-relayer E2E tests written in Ginkgo to teleporter. The high-level changes here are:
    a. Implement basic send and receive integration test.
    b. Modify message relaying to either build and run awm-relayer directly, or manually aggregate signatures and deliver the message from within the E2E test. subnet-evm Warp E2E tests take the latter approach.
  2. Consolidate awm-relayer and teleporter E2E tests to reduce code duplication.
  3. Iteratively expand teleporter Ginkgo E2E test suite to achieve parity with the existing bash integration tests.
  4. Remove bash integration test CI jobs, and optionally remove all docker and bash code.

Open questions

  • One nice feature of the docker setup is that we can start up the system, but not execute tests, (see run.sh) allowing the user to open a shell inside one of the containers and interact with the system in real time. Is this possible with Ginkgo?
@cam-schultz cam-schultz added the enhancement New feature or request label Sep 19, 2023
@cam-schultz cam-schultz moved this from 📋 Backlog to 🔖 Ready in Avalanche Warp Messaging Nov 9, 2023
@michaelkaplan13 michaelkaplan13 moved this from Backlog 🗄️ to In Progress 🏗 in Platform Engineering Group Nov 14, 2023
@michaelkaplan13 michaelkaplan13 added this to the production launch milestone Nov 14, 2023
@feuGeneA feuGeneA self-assigned this Nov 21, 2023
@cam-schultz cam-schultz moved this from In Progress 🏗 to In Review 👀 in Platform Engineering Group Nov 30, 2023
@github-project-automation github-project-automation bot moved this from In Review 👀 to Done ✅ in Platform Engineering Group Dec 5, 2023
minghinmatthewlam pushed a commit that referenced this issue Oct 1, 2024
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
enhancement New feature or request tests
Projects
Archived in project
Status: 🔖 Ready
Development

Successfully merging a pull request may close this issue.

5 participants