🎨 UI for the Free Data Engineering Zoomcamp Course provided by DataTalksClub
-
Updated
Aug 2, 2024 - Python
#
mageai
Here are 24 public repositories matching this topic...
An ETL data pipeline that extracts data from source and loads it to destination, automated using mage.ai
-
Updated
Nov 21, 2023 - Python
Using machine learning, feature engineering, and web scraping, I created an end-to-end laptop price prediction website by scraping data from a popular Iranian source. Empowering users with accurate pricing estimates and model comparisons
-
Updated
Dec 26, 2023 - Jupyter Notebook
pokéFastApi: A backend API with Docker Compose integration for retrieving and searching Pokémon data using FastAPI and DuckDB.
-
Updated
Jan 11, 2024 - Python
Data Engineering Zoomcamp Final Project
python
bigquery
csv
spark
docker-compose
terraform
jupyter-notebook
gcp
cloud-storage
spreadsheets
looker-studio
mageai
-
Updated
Jul 14, 2024 - Jupyter Notebook
Datatalks Club Free Data Engineering Zoomcamp Project
python
linux
bigquery
kafka
spark
docker-compose
terraform
postgresql
jupyter-notebook
gcp
pandas
dbt
prefect
github-actions
redpanda
gcp-cloud-storage
duckdb
risingwave
looker-studio
mageai
-
Updated
Jul 14, 2024 - Jupyter Notebook
Using Mage.ai orchestration tool for ETL process
-
Updated
Mar 28, 2024 - Python
python
aws
data-science
sql-server
sentiment-analysis
kafka-streams
powerbi
webscraping
nlp-machine-learning
forecasting-models
hosting-aws
mageai
-
Updated
Mar 3, 2024 - Jupyter Notebook
Data Engineering Swiss Air Quality
-
Updated
Mar 31, 2024 - Python
An ETL Pipeline built over GCP and orchestrated by Mage, which involves Extracting Data from GCS Bucket, building Dimensional Model, loading the Data into BigQuery and a Looker Dashboard for further analysis.
-
Updated
Nov 18, 2023 - Jupyter Notebook
DataTalks Club - Machine Learning Operations Zoomcamp (Cohort 2024)
git
testing
dockerfile
aws
monitoring
docker-compose
model
terraform
scikit-learn
jupyter-notebook
grafana-dashboard
mlops
mlflow
cicd-pipeline
evidently
mageai
-
Updated
Jul 13, 2024 - Jupyter Notebook
-
Updated
Jun 20, 2024 - Python
DE Zoomcamp 2024 Final Project 🧙
python
bigquery
google-cloud
data-warehouse
data-engineering
data-lake
dbt
etl-pipeline
dbt-cloud
looker-studio
mageai
-
Updated
Jul 24, 2024 - Python
This project is a Data Engineering solution implemented to validate the data types of columns in PostgreSQL tables in a DataMart. It aims to validate whether the data stored in tables conforms to the expected data types, improving data integrity and reliability.
-
Updated
Jul 28, 2024 - Python
Automated script developed in Mage.ai to monitor and validate the status of AWS DMS CDC tasks, ensuring data integrity and synchronization. Sends notifications for any detected validation issues.
notifications
python
validation
data-engineering
monitoring-scripts
aws-dms
aws-dms-task-status
cdc-data
mageai
-
Updated
Aug 5, 2024 - Python
This script automates cleaning up a specific folder in an S3 bucket, deleting all objects within it. It uses secure AWS credentials and is built on the Mage.ai platform. Additionally, the roadmap includes approach handling and logging for greater robustness and monitoring.
python
s3-bucket
data-engineering
boto3
cleanup-script
automation-script
s3-aws
monitoring-automation
mageai
cloud-storage-management
s3-cleanup
-
Updated
Aug 15, 2024 - Python
The AWS DMS Task Restart and Status Checker is a Python script designed to restart various AWS Database Migration Service (DMS) tasks and check their status. This script leverages AWS SDK (Boto3) and Mage.ai for DMS task integration and management, ensuring efficient and reliable task handling.
python
data-engineering
boto3
data-migration
task-management
script-automation
aws-dms
aws-dms-task-status
mageai
dms-tasks-restart
-
Updated
Aug 10, 2024 - Python
Improve this page
Add a description, image, and links to the mageai topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the mageai topic, visit your repo's landing page and select "manage topics."