Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync / Dashboard development -> main #171

Merged
merged 54 commits into from
Aug 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
2edc6be
update docker buildx workflow
david-hummingbot Jul 8, 2024
2f2230c
(feat) update credentials.yml
cardosofede Jul 8, 2024
a42b845
(feat) improve main.py to use the latest version of stauth
cardosofede Jul 8, 2024
a12f74a
(feat) remove pre-auth
cardosofede Jul 8, 2024
0a57d3b
(feat) simplify main.py
cardosofede Jul 8, 2024
51d24f5
(feat) clean up imports
cardosofede Jul 8, 2024
56fa94b
(feat) add permission to manage public and private pages
cardosofede Jul 8, 2024
6e3b025
(feat) add auth function that can be reused
cardosofede Jul 8, 2024
41d55f8
(feat) revert pre-authorized
cardosofede Jul 8, 2024
3c3ee37
(feat) format CONFIG and make auth env
cardosofede Jul 9, 2024
eb64ca6
(feat) update main.py with new video
cardosofede Jul 9, 2024
edcad77
(feat) update container text
cardosofede Jul 9, 2024
bcac036
(feat) remove container
cardosofede Jul 9, 2024
9beb1d6
(feat) add space to improve formatting
cardosofede Jul 9, 2024
1fbe59e
Merge pull request #156 from hummingbot/feat/docker-buildx-workflow
cardosofede Jul 9, 2024
256e193
(feat) remove unnecessary space
cardosofede Jul 9, 2024
61f4c57
(feat) remove unnecessary imports
cardosofede Jul 11, 2024
7934522
Merge pull request #157 from hummingbot/feat/auth_system
cardosofede Jul 11, 2024
f836bd4
(feat) remove ms
cardosofede Jul 12, 2024
5e871a4
(feat) remove ms from get candles
cardosofede Jul 12, 2024
d61b817
(feat) remove ms from download candles
cardosofede Jul 12, 2024
536ba1c
Merge pull request #159 from hummingbot/feat/adapt_ms_to_s
cardosofede Jul 12, 2024
8981974
(feat) add pre-commit file
cardosofede Jul 16, 2024
be148eb
(feat) update styling dependencies
cardosofede Jul 16, 2024
dec22c7
(feat) remove quants lab
cardosofede Jul 16, 2024
ff35581
(feat) update pyproject.toml
cardosofede Jul 16, 2024
e8071a1
(feat) add more config to pyproject
cardosofede Jul 16, 2024
f4fcf54
(feat) update pyproject
cardosofede Jul 16, 2024
75908d6
(feat) improve pyproject
cardosofede Jul 16, 2024
1f686e9
(feat) update precommit
cardosofede Jul 16, 2024
a87e45e
(feat) update precommit
cardosofede Jul 16, 2024
37f63e1
(feat) improve formatting
cardosofede Jul 16, 2024
033853d
(feat) final version of pyproject
cardosofede Jul 16, 2024
ae8142c
(feat) remove unused components
cardosofede Jul 16, 2024
f163c58
(feat) update base components
cardosofede Jul 16, 2024
785eab1
(feat) update controllers configs components
cardosofede Jul 16, 2024
7a87588
(feat) update distribution inputs
cardosofede Jul 16, 2024
e7eba7e
(feat) formatting controllers
cardosofede Jul 16, 2024
f123570
(feat) format visualizations
cardosofede Jul 16, 2024
81b47f4
(feat) update st utils
cardosofede Jul 16, 2024
2637949
(feat) update config pages
cardosofede Jul 16, 2024
ffd1799
(feat) update main page
cardosofede Jul 16, 2024
cd8c1bb
(feat) update backend style
cardosofede Jul 16, 2024
759c286
(feat) update glom import
cardosofede Jul 16, 2024
9967322
(feat) general precommit run
cardosofede Jul 16, 2024
7f0df00
Merge pull request #160 from hummingbot/feat/clean_up_repo
nikspz Jul 18, 2024
de2ac2f
update Makefile
david-hummingbot Jul 22, 2024
2814b08
add headless flag
david-hummingbot Jul 23, 2024
c750f11
(feat) set non expiring cookies
cardosofede Jul 23, 2024
ef24915
(feat) improve auth initialization
cardosofede Jul 23, 2024
907132e
Merge pull request #167 from hummingbot/feat/improve_auth
cardosofede Jul 23, 2024
961f520
update CONFIG.py
david-hummingbot Jul 26, 2024
b8c3a75
Merge pull request #164 from hummingbot/feat/update-makefile
rapcmia Jul 29, 2024
22b3116
Merge pull request #168 from hummingbot/fix/deploy-issue
rapcmia Jul 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 54 additions & 21 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,47 +1,80 @@
name: Build and Push Docker image
name: Dashboard Docker Buildx Workflow

on:
pull_request:
types: [closed]
branches:
- main
- development
release:
types: [published, edited]

jobs:
build:
build_pr:
if: github.event_name == 'pull_request' && github.event.pull_request.merged == true
runs-on: ubuntu-latest
if: github.event.pull_request.merged == true
steps:
- name: Checkout code
uses: actions/checkout@v3.5.3
uses: actions/checkout@v4.1.1

- name: Set up QEMU
uses: docker/setup-qemu-action@v2.2.0
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2.9.1
with:
version: latest
uses: docker/setup-buildx-action@v3.1.0

- name: Login to DockerHub
uses: docker/login-action@v2.2.0
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build and push
id: docker_build
uses: docker/build-push-action@v4.1.1
- name: Build and push Development Image
if: github.base_ref == 'development'
uses: docker/build-push-action@v5
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: hummingbot/dashboard:development

- name: Build and push Latest Image
if: github.base_ref == 'main'
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: hummingbot/dashboard:latest

build_release:
if: github.event_name == 'release'
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4.1.1

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3.1.0

- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Extract tag name
id: get_tag
run: echo ::set-output name=VERSION::${GITHUB_REF#refs/tags/}

- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
platforms: linux/amd64,linux/arm64
build-args: |
BRANCH=${{ github.ref }}
COMMIT=${{ github.sha }}
BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
cache-from: type=registry,ref=hummingbot/dashboard:latest
cache-to: type=inline

- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
push: true
tags: hummingbot/dashboard:${{ steps.get_tag.outputs.VERSION }}
33 changes: 33 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: detect-private-key

- repo: https://github.com/pre-commit/mirrors-eslint
rev: v8.10.0
hooks:
- id: eslint
files: \.[jt]sx?$ # *.js, *.jsx, *.ts and *.tsx
types: [file]

- repo: https://github.com/CoinAlpha/git-hooks
rev: 78f0683233a09c68a072fd52740d32c0376d4f0f
hooks:
- id: detect-wallet-private-key
types: [file]
exclude: .json

- repo: https://github.com/pycqa/isort
rev: 5.12.0
hooks:
- id: isort
files: "\\.(py)$"
args: [--settings-path=pyproject.toml]

- repo: https://github.com/pycqa/flake8
rev: 3.9.2
hooks:
- id: flake8
additional_dependencies: ['flake8']
args: [--max-line-length=130]
7 changes: 3 additions & 4 deletions CONFIG.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@

from dotenv import load_dotenv


load_dotenv()

MINER_COINS = ["Algorand", "Avalanche", "DAO Maker", "Faith Tribe", "Fear", "Frontier",
"Harmony", "Hot Cross", "HUMAN Protocol", "Oddz", "Shera", "Firo",
"Vesper Finance", "Youclout", "Nimiq"]
"Harmony", "Hot Cross", "HUMAN Protocol", "Oddz", "Shera", "Firo",
"Vesper Finance", "Youclout", "Nimiq"]
MINER_EXCHANGES = ["Binance", "FTX", "Coinbase Exchange", "Huobi Global", "OKX", "KuCoin",
"Kraken", "Bybit (Spot)", "FTX.US", "Crypto.com Exchange", "Binance US",
"MEXC Global", "Gate.io", "BitMart", "Bitfinex", "AscendEX (BitMax)",
Expand All @@ -18,7 +17,7 @@
CERTIFIED_EXCHANGES = ["ascendex", "binance", "bybit", "gate.io", "hitbtc", "huobi", "kucoin", "okx", "gateway"]
CERTIFIED_STRATEGIES = ["xemm", "cross exchange market making", "pmm", "pure market making"]

AUTH_SYSTEM_ENABLED = False
AUTH_SYSTEM_ENABLED = os.getenv("AUTH_SYSTEM_ENABLED", "False").lower() in ("true", "1", "t")

BACKEND_API_HOST = os.getenv("BACKEND_API_HOST", "127.0.0.1")
BACKEND_API_PORT = os.getenv("BACKEND_API_PORT", 8000)
36 changes: 33 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,19 +1,49 @@
.ONESHELL:
.SHELLFLAGS := -c

.PHONY: run
.PHONY: uninstall
.PHONY: install
.PHONY: install-pre-commit
.PHONY: docker_build
.PHONY: docker_run


detect_conda_bin := $(shell bash -c 'if [ "${CONDA_EXE} " == " " ]; then \
CONDA_EXE=$$((find /opt/conda/bin/conda || find ~/anaconda3/bin/conda || \
find /usr/local/anaconda3/bin/conda || find ~/miniconda3/bin/conda || \
find /root/miniconda/bin/conda || find ~/Anaconda3/Scripts/conda || \
find $$CONDA/bin/conda) 2>/dev/null); fi; \
if [ "${CONDA_EXE}_" == "_" ]; then \
echo "Please install Anaconda w/ Python 3.10+ first"; \
echo "See: https://www.anaconda.com/distribution/"; \
exit 1; fi; \
echo $$(dirname $${CONDA_EXE})')

CONDA_BIN := $(detect_conda_bin)

run:
streamlit run main.py
streamlit run main.py --server.headless true

uninstall:
conda env remove -n dashboard

install:
conda env create -f environment_conda.yml
if conda env list | grep -q '^dashboard '; then \
echo "Environment already exists."; \
else \
conda env create -f environment_conda.yml; \
fi
$(MAKE) install-pre-commit

install-pre-commit:
/bin/bash -c 'source "${CONDA_BIN}/activate" dashboard && \
if ! conda list pre-commit | grep pre-commit &> /dev/null; then \
pip install pre-commit; \
fi && pre-commit install'

docker_build:
docker build -t hummingbot/dashboard:latest .

docker_run:
docker run -p 8501:8501 dashboard:latest
docker run -p 8501:8501 hummingbot/dashboard:latest
5 changes: 3 additions & 2 deletions backend/services/backend_api_client.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from typing import Optional, Dict
from typing import Dict, Optional

import pandas as pd
import requests
Expand Down Expand Up @@ -115,7 +115,8 @@ def start_bot(self, start_bot_config: dict):
def stop_bot(self, bot_name: str, skip_order_cancellation: bool = False, async_backend: bool = True):
"""Stop a Hummingbot bot."""
endpoint = "stop-bot"
return self.post(endpoint, payload={"bot_name": bot_name, "skip_order_cancellation": skip_order_cancellation, "async_backend": async_backend})
return self.post(endpoint, payload={"bot_name": bot_name, "skip_order_cancellation": skip_order_cancellation,
"async_backend": async_backend})

def import_strategy(self, strategy_config: dict):
"""Import a trading strategy to a bot."""
Expand Down
4 changes: 2 additions & 2 deletions backend/services/coingecko_client.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import re
import time

from pycoingecko import CoinGeckoAPI
import pandas as pd
import re
from pycoingecko import CoinGeckoAPI


class CoinGeckoClient:
Expand Down
2 changes: 1 addition & 1 deletion backend/services/miner_client.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import pandas as pd
import requests
from glom import *
from glom import glom


class MinerClient:
Expand Down
14 changes: 1 addition & 13 deletions backend/utils/optuna_database_manager.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import os
import json
import os
from typing import Optional

import pandas as pd
Expand Down Expand Up @@ -102,18 +102,6 @@ def _get_trial_system_attributes_table(self):
except Exception as e:
return f"Error: {str(e)}"

@property
def trial_system_attributes(self):
return self._get_trial_system_attributes_table()

def _get_trial_system_attributes_table(self):
try:
with self.session_maker() as session:
df = pd.read_sql_query(text("SELECT * FROM trial_system_attributes"), session.connection())
return df
except Exception as e:
return f"Error: {str(e)}"

@property
def version_info(self):
return self._get_version_info_table()
Expand Down
13 changes: 9 additions & 4 deletions backend/utils/os_utils.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
import glob
import subprocess
import importlib.util
import inspect
import os
import subprocess

import pandas as pd

import yaml
from hummingbot.strategy_v2.controllers.directional_trading_controller_base import DirectionalTradingControllerBase, DirectionalTradingControllerConfigBase
from hummingbot.strategy_v2.controllers.market_making_controller_base import MarketMakingControllerBase, MarketMakingControllerConfigBase
from hummingbot.strategy_v2.controllers.directional_trading_controller_base import (
DirectionalTradingControllerBase,
DirectionalTradingControllerConfigBase,
)
from hummingbot.strategy_v2.controllers.market_making_controller_base import (
MarketMakingControllerBase,
MarketMakingControllerConfigBase,
)


def remove_files_from_directory(directory: str):
Expand Down
13 changes: 7 additions & 6 deletions credentials.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
cookie:
expiry_days: 30
key: random_signature_key
name: random_cookie_name
credentials:
usernames:
admin:
email: admin@gmail.com
name: Admin User
password: abc # To be replaced with hashed password: hashed_passwords = stauth.Hasher(['abc', 'def']).generate()
preauthorized: # the preferred way to add users since there is no need for manual hashing of passwords
logged_in: False
password: abc
cookie:
expiry_days: 0
key: some_signature_key # Must be string
name: some_cookie_name
pre-authorized:
emails:
- admin@admin.com
3 changes: 3 additions & 0 deletions environment_conda.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,6 @@ dependencies:
- streamlit-elements==0.1.*
- streamlit-authenticator
- pydantic==1.10.4
- flake8
- isort
- pre-commit
11 changes: 6 additions & 5 deletions frontend/components/backtesting.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import streamlit as st
from datetime import datetime, timedelta

import streamlit as st


def backtesting_section(inputs, backend_api_client):
st.write("### Backtesting")
Expand All @@ -13,7 +14,8 @@ def backtesting_section(inputs, backend_api_client):
end_date = st.date_input("End Date", default_end_time,
help="End date is inclusive, make sure that you are not including the current date.")
with c3:
backtesting_resolution = st.selectbox("Backtesting Resolution", options=["1m", "3m", "5m", "15m", "30m", "1h", "1s"], index=0)
backtesting_resolution = st.selectbox("Backtesting Resolution",
options=["1m", "3m", "5m", "15m", "30m", "1h", "1s"], index=0)
with c4:
trade_cost = st.number_input("Trade Cost (%)", min_value=0.0, value=0.06, step=0.01, format="%.2f")
with c5:
Expand All @@ -24,16 +26,15 @@ def backtesting_section(inputs, backend_api_client):
end_datetime = datetime.combine(end_date, datetime.max.time())
try:
backtesting_results = backend_api_client.run_backtesting(
start_time=int(start_datetime.timestamp()) * 1000,
end_time=int(end_datetime.timestamp()) * 1000,
start_time=int(start_datetime.timestamp()),
end_time=int(end_datetime.timestamp()),
backtesting_resolution=backtesting_resolution,
trade_cost=trade_cost / 100,
config=inputs,
)
except Exception as e:
st.error(e)
return None

if len(backtesting_results["processed_data"]) == 0:
st.error("No trades were executed during the backtesting period.")
return None
Expand Down
Loading