A set of Airflow Hooks, Operators and Sensors for Debussy Concert,
the core component of Debussy Framework
This package provides operators, sensors, and hooks that enables running Debussy Concert on Apache Airflow. However, it's not a traditional Airflow provider package in the sense Debussy Concert is not a tool, but rather a framework for data pipeline development and code generation. Therefore, our hooks, operators and sensors acts as interfaces, extending other Airflow providers, and abstracting their usage.
You can install this package via
pip install debussy-airflow
An environment running apache-airflow 2.2.0+ with the following requirements (already managed as dependencies):
- apache-airflow-providers-google == 8.0.0
- facebook-business == 13.0.0
- mysql-connector-python == 8.0.24
- paramiko == 2.8.1- basic_operator
- db_to_storage_operator
- facebook_ads_to_storage_operator
- http_operator
- rest_api_to_storage_operator
- storage_to_rdbms_operator
- storage_to_storage_operator
See the examples directory for an example DAG.
Please submit issues and pull requests in our official repo: https://github.com/DotzInc/debussy_airflow
We are happy to hear from you. Please email any feedback to the authors at dataengineer@dotz.com.
This project is built with active contributions from:
This project is formatted via black:
pip install black
black .