Skip to content

Latest commit

 

History

History
94 lines (66 loc) · 2.9 KB

README.md

File metadata and controls

94 lines (66 loc) · 2.9 KB
Raw

Binance ETL

License: MIT Python

Extract, Transform and Load real-time trading events from Binance exchange.

Getting started • Data • Export • Kaggle

Getting started

Install dependencies

pip install -r requirements.txt

Configure

Configure the ETL by editing config.json.

Here you can define the symbols you want to track, the types of events to record and the desired storage destination.

Start

python3 -m binance_etl.main

Data

The program supports the collection and storage of the following trading events.

Event type Spot Futures
Depth updates ✅ ❌
Trades ✅ ❌

Depth updates

Depth updates events are order book updates streamed from binance websockets depth stream.

We store depth updates in the following format:

Column Description
timestamp Exchange timestamp in milliseconds since epoch
local_timestamp Message arrival timestamp in milliseconds since epoch
side Side of the order: bid or ask
price Price of the order
quantity Quantity of the order
is_snapshot true if the update is part of the initial snapshot, else false

Trades

Trade events are market trades streamed from binance websockets trade stream.

We store depth updates in the following format:

Column Description
timestamp Exchange timestamp in milliseconds since epoch
local_timestamp Message arrival timestamp in milliseconds since epoch
id Unique identifier of the trade
side Can be buy or sell, and reflects who is the liquidity taker
price Execution price of the trade
quantity Quantity of the trade

Export

The program supports the following export destinations:

Export type Supported
Local CSV files ✅
Google BigQuery ❌ (coming soon)

Kaggle

If you're looking for some ready to use datasets, I've exported some sample data to this kaggle dataset, and you can check some started code in this kaggle notebook.