Skip to main content

Proof of Concept

The first step of this project was to establish a PoC (Proof of Concept) to show that it is possible to create a basic ETL pipeline between a data warehouse tool and Salesforce using Apache Airflow. As the data warehouse tool had not been chosen at the time, it was decided to use a PostgreSQL database instead. For Salesforce, a sandbox populated with mock data is used rather than the instance running in production, which is more appropriate for experimentation and testing.

This documentation details the different steps for creating this proof of concept. It can be used as a reference for the remaining of the project, especially to get instructions on how to set up an Airflow environment and how to connect Airflow to different external services.

As mentioned in the first paragraph, the goal was to create a pipeline between a data warehouse tool and Salesforce. Before being able to write the code for this pipeline, the tools and services involved had to be set up and connected together. The different steps that were required are listed below, in order:

  • Set up and configure Apache Airflow
  • Set up and configure the PostgreSQL database
  • Connect Airflow to the PostgreSQL database
  • Connect Airflow to Salesforce

The explanations and technical details for each of these steps can be found in their respective page, listed below.

Back to top