Skip to content

Final project for the Makers Academy Data Engineering Bootcamp! In this amazing, complex group project we had to analyse a massive dataset and extract insightful data that could be used to improve education world-wide!

Notifications You must be signed in to change notification settings

igorlangoni/final_project_data_eng_makers

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PISA 2018 Education Insights - Data Pipeline

Project overview

Global Education Insights, a mock company that we work for, had recently acquired the PISA 2018 dataset, a comprehensive survey of student performance in various countries.

This dataset contained a wealth of information that could help GEI understand the factors that contribute to student success and identify areas where education systems can be improved.

However, the dataset was large and complex, making it difficult for GEI's team to extract meaningful insights from it.

That's where we came in.

Our task was to leverage our data engineering skills to analyze the PISA 2018 dataset and develop a functioning dashboard that GEI could use to easily visualise and interpret the data.

This dashboard would be a critical tool for GEI, helping them to make data-driven decisions and recommendations.

The project was carried out in a distributed environment in the Cloud, which will allow for efficient data processing and collaboration among team members.

The tools stack for this project: Python, SQL, Apache Airflow, AWS EC2, AWS RDS, Flask, Git.

Architecture

pipeline_architecture_logos

DAG

dag

Dashboard

dashboard

Collaborators

About

Final project for the Makers Academy Data Engineering Bootcamp! In this amazing, complex group project we had to analyse a massive dataset and extract insightful data that could be used to improve education world-wide!

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%