[x] Project 1. Chicago-transit-authority
- Tech stack: Kafka, ksql, faust, stream processing web app.
- In this project, you will construct a streaming event pipeline around Apache Kafka and its ecosystem. Using public data from the Chicago Transit Authority we will construct an event pipeline around Kafka that allows us to simulate and display the status of train lines in real time.
- Chicago-transit-authority report
[x] Project 2. SF Crime Statistics with Spark Streaming
- Tech stack: Spark, Kafka, Kaggle.
- In this project, you will be provided with a real-world dataset, extracted from Kaggle, on San Francisco crime incidents, and you will provide statistical analyses of the data using Apache Spark Structured Streaming. You will draw on the skills and knowledge you've learned in this course to create a Kafka server to produce data, and ingest data through Spark Structured Streaming.
- SF Crime Statistics with Spark Streaming report
[x] Project 3. STEDI Ecosystem
- Tech stack: Spark, Kafka, Redis.
- You work for the data science team at STEDI, a small startup focused on assessing balance for seniors. STEDI has an application that collects data from seniors during a small exercise. The user logs in, and then selects the customer they are working with. Then the user starts a timer, and clicks a button with each step the senior takes. When the senior has reached 30 steps, their test is finished. The data transmitted enables the application to monitor seniors’ balance risk.
- Evaluate Human Balance with Spark Streaming