Skip to content

e-espootin/p0371_stream_argoCD_NYC_taxiTrips_AWS_S3

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Streaming Generated TLC Trip Data with Python, Kafka, Kubernetes, ArgoCD into AWS S3

Project Summary

This project simulates TLC trip data using the Faker library within a containerized environment. It leverages Docker images for both data producers and consumers, which are managed by ArgoCD and deployed on Kubernetes. The Helm configuration file dictates the number of producer and consumer containers. Generated data is continuously streamed and stored in AWS S3. The Kafka server, along with Minikube and ArgoCD, is set up on a local Ubuntu machine to facilitate this process.

project structure

Project Architecture

ArogCD & kubernetes & kafka

kafka producer

kafka consumer

ArgoCD

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published