Refactored end-to-end tests fully orchestrated by pytest

[pyalex]

What this PR does / why we need it:
Reworked e2e tests consist of:

• Registering definitions in Core
• Retrieve historical features
• Retrieve online features (ingested by batch/stream)

Runners regrouped by environment:

• standalone (run spark locally, all services (kafka, redis, postgres, incl feast core & serving) spawned by pytest as subprocesses)
• gcp (dataproc, redis-cluster & kafka in kubernetes)
• (potentially) aws

Auth (enabled/disabled) is now just a parameterised fixture.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

[feast-ci-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pyalex

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [pyalex]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[woop]

The self-hosted runners don't clean up. If you aren't using docker then all the state will remain. I dont think this test is safe, it's going to fall over for our current runners because they are already so unstable (long running stateful pods)

[pyalex]

I don't use docker. It's regular test, which github runner was designed for, hence there should be way to easily clean up. Right now for example python w/ environment (and installed packages) being cleaned up by python action

[pyalex]

Also, we're on preemptible nodes right now. So we have additional clean up

[woop]

I don't use docker. It's regular test, which github runner was designed for, hence there should be way to easily clean up. Right now for example python w/ environment (and installed packages) being cleaned up by python action

ok, but self-hosted runners don't clean up right? state just hangs around as far as I know.

[feast-ci-bot]

@pyalex: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
test-end-to-end	5f706cf	link	/test test-end-to-end
test-end-to-end-auth	5f706cf	link	/test test-end-to-end-auth
test-end-to-end-redis-cluster	5f706cf	link	/test test-end-to-end-redis-cluster

Full PR test history

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[feast-ci-bot]

@pyalex: Updated the config configmap in namespace default at cluster default using the following files:

• key config.yaml using file .prow/config.yaml

In response to this:

What this PR does / why we need it:
Reworked e2e tests consist of:

• Registering definitions in Core
• Retrieve historical features
• Retrieve online features (ingested by batch/stream)

Runners regrouped by environment:

• standalone (run spark locally, all services (kafka, redis, postgres, incl feast core & serving) spawned by pytest as subprocesses)
• gcp (dataproc, redis-cluster & kafka in kubernetes)
• (potentially) aws

Auth (enabled/disabled) is now just a parameterised fixture.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[woop]

Do we have to use double templating (envsubst, helm)? What if we just used --set CLI command for Kafka IP and redis IP?