ETL Discovery with Apache Airflow

Arthur Kotkowiak and Tom Torres' project

The goal of this project is to create and to orchestrate ETL (Extract, Load, Transform) pipelines between a data warehouse tool and a CRM (Customer Relationship Management) tool for the University of Rhode Island Foundation. The tools and services involved in this project are the following:

• Salesforce, the CRM tool
• Apache Airflow, more specifically a cloud-based service offered by AWS (Amazon Web Services) called Amazon MWAA (Amazon Managed Workflow for Apache Airflow), to create and orchestrate the pipelines
• The data warehouse tool has not been decided yet, but it will probably be Snowflake

This documentation details the different steps of the project.