Skip to content

Files

Latest commit

 

History

History
97 lines (70 loc) · 2.32 KB

README.md

File metadata and controls

97 lines (70 loc) · 2.32 KB

E2E tests

Our automated test suite. Should match the official tests in the rigor. No compromise here at all.

Prerequisites

We need a lot of infrastructure setup for E2E tests.
This will be slowly automated as we make progress (Looking at you Temporal).

Minio Local?

brew install minio/stable/minio
mkdir -p ~/airbyte-replication-operator-data
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=miniostorage
minio server --console-address :9090 ~/airbyte-replication-operator-data
http://localhost:9090

Datasets

cargo run --bin nyc_taxi_data

Kind

kind create cluster --image kindest/node:v1.24.7 --config infra/kind/cluster.yaml

KubeVela

./infra/vela/setup.sh
./bin/vela install -y
./bin/vela addon enable velaux --version=v1.7.4
./bin/vela addon enable infra/vela/addons/minio
./bin/vela addon enable infra/vela/addons/airbyte-replication-operator

File-to-CSV, simple things first

cd hello-airbyte-file-to-csv/docker
docker compose up
docker compose down && cd ..
kubectl apply -f k8s --recursive
vela up -f vela/application.yaml --namespace default
vela status hello-airbyte-file-to-csv --namespace default
vela logs hello-airbyte-file-to-csv --namespace default
vela delete hello-airbyte-file-to-csv --namespace default --force
kubectl delete -f k8s --recursive

S3-to-S3, a good benchmarking exercise.

cd ../hello-airbyte-s3-to-s3/docker
docker compose up
docker compose down && cd ..
kubectl apply -f k8s --recursive
vela up -f vela/application.yaml --namespace default
vela status hello-airbyte-s3-to-s3 --namespace default
vela logs hello-airbyte-s3-to-s3 --namespace default
vela delete hello-airbyte-s3-to-s3 --namespace default --force
kubectl delete -f k8s --recursive

Postgres-to-S3, started to getting interesting.

cd ../hello-airbyte-postgres-to-s3/docker
docker compose up
docker compose down && cd ..
kubectl apply -f k8s --recursive
vela up -f vela/application.yaml --namespace default
vela status hello-airbyte-postgres-to-s3 --namespace default
vela logs hello-airbyte-postgres-to-s3 --namespace default
vela delete hello-airbyte-postgres-to-s3 --namespace default --force
kubectl delete -f k8s --recursive

Cleanup

kind delete cluster --name airbyte-replication-operator-platform