Pinned Loading

1. ETL_spotify ETL_spotify Public

   here is a data pipeline (Extract, Transform,Load) from my spotify account managed by Apache Airflow

   Python 1

2. Amazon_S3_databricks Amazon_S3_databricks Public

   Setting up AWS S3 and Getting Data into Databricks notebook. Then doing engineering and analysis with Python, Spark and SQL

   Jupyter Notebook 2

3. PySpark PySpark Public

   i wrote a small tutorial on Databricks and on premise about how to utilize Spark Resilient Distributed Datasets and Spark Data Frames to explore a datasets, analyze them and save as parquet file.

   Jupyter Notebook 1 1

4. Azure_Databricks Azure_Databricks Public

   this repo contains the code from Microsoft learning path: Data engineering with Azure Databricks

   Jupyter Notebook 1

5. most_useful_spark most_useful_spark Public

   I wrote this code to illustrate Spark (Python) basics for newcomers at Sberbank

   Jupyter Notebook 1

6. Databricks_data_engineering Databricks_data_engineering Public

   Jupyter Notebook 1