Skip to content

MobileTeleSystems/syncmaster

Repository files navigation

Data.SyncMaster

Repo Status PyPI PyPI License PyPI Python Version Docker image Documentation Build Status Coverage pre-commit.ci

What is Data.SyncMaster?

Data.SyncMaster is as low-code ETL tool for transfering data between databases and file systems. List of currently supported connections:

  • Apache Hive
  • Clickhouse
  • Postgres
  • Oracle
  • MSSQL
  • MySQL
  • HDFS
  • S3

Current Data.SyncMaster implementation provides following components:

  • REST API
  • Celery Worker

Note: service is under active development, and is not ready to use.

Goals

  • Make transfering data between databases and file systems as simple as possible
  • Provide a lot of builtin connectors to transfer data in heterogeneous environment
  • RBAC and multitenancy support

Non-goals

  • This is not a backup system
  • Only batch, no streaming

Documentation

See https://syncmaster.readthedocs.io