Skip to content

tomekmatuszewski/Airflow-playground

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

To run: docker-compose up --build

Playing with Airflow -> orchestrate simply ETL job

- pull data from Spotify API (EXTRACT)
- transform data (TRANSFORM)
- load data to sqlite3 database (LOAD)
- create in parallel csv and json copy of data
- finish task

to run job creation of .env file/ exporting env variables/ adding them to docker-compose path (--env-file) is needed -> variables URL and TOKEN (credentials to Spotify API)

About

Playing with Airflow -> creation of ETL job

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published