From 68b020a503e63d11a8c7a8478302082f389e7497 Mon Sep 17 00:00:00 2001 From: trentmc Date: Fri, 20 Oct 2023 18:49:34 +0200 Subject: [PATCH] For #255: EthMiami: first cut of competition README. Has end-to-end flow --- README.md | 1 + READMEs/ethmiami.md | 164 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 165 insertions(+) create mode 100644 READMEs/ethmiami.md diff --git a/README.md b/README.md index ca453c59c..3869cf55b 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,7 @@ Less common: - [Release process](READMEs/release-process.md) - **[Run dfbuyer bot](READMEs/dfbuyer.md)** - runs Predictoor DF rewards - **[Run publisher](READMEs/publisher.md)** - publish new feeds +- [ETHMiami competition](READMEs/ethmiami.md) - Oct 27-29, 2023 ## Repo structure diff --git a/READMEs/ethmiami.md b/READMEs/ethmiami.md new file mode 100644 index 000000000..0ba800c20 --- /dev/null +++ b/READMEs/ethmiami.md @@ -0,0 +1,164 @@ + + +# ETHMiami: Accurate ETH Predictoor + +## Introduction + +This README provides instructions for the ETHMiami data science competition run by Ocean Protocol, Oct 27-29, 2023. + +The goal is to accurately predict whether ETH with rise or fall, every five minutes, over an 8-hour interval. + +Through this README, you're supplied with: +- A baseline approach to gather data and build an AI/ML prediction model +- A "simulation flow" to quickly test performance, enabling rapid looping to improve the model +- A "predictoor bot flow" that uses the model to submit predictions to the chain every 5 minutes + +The competition is run on [Oasis Sapphire testnet](https://docs.oasis.io/dapp/sapphire/). + +Bonus! You can use the outcome from this competition to run [your own predictoor bot](predictoor.md) bot on Oasis Sapphire _mainnet_, to earn real $ in the Ocean Predictoor ecosystem. + +### Key Parameters + +Criteria / prizes: +- Accuracy = % correct predictions in the interval +- 1st place: (FIXME) OCEAN -> highest accuracy +- 2nd place: (FIXME) OCEAN -> 2nd highest accuracy +- 3rd place: (FIXME) OCEAN -> 3rd highest accuracy + +Timing: +- Your predictoor must make submissions **every 5 minutes** in the interval from **4:00am to 12:00pm EDT on Sun Oct 29**. +- It's ok if it predicts before or after too! But those won't be counted +- For each 5-min slot that it neglects to submit a prediction, that miss is treated as an incorrect prediction (vs being ignored) + +### 0.4 Support + +If you encounter issues, feel free to reach out :raised_hand: + +- Ask someone from the Ocean core team who will be physically present at the hackathon +- Ask in [Ocean #predictoor Discord](https://discord.com/channels/612953348487905282/1151066755796574389). + +### Outline of this README + +This README shows how to line up an AI/ML model-testing flow, then run a predictoor bot on testnet. + +Here are the steps: + +1. **[Install pdr-backend repo](#install-pdr-backend-repo)** +1. **[Try out simulation flow (modeling & trading)](#try-out-simulation-flow)** +1. **[Try out bot flow on testnet](#try-out-bot-flow)** +1. **[Improve model: looping in simulation flow](#improve-model)** +1. **[Submit to competion: Run bot in time for deadline](#run-bot-for-real)** + +You'll want to go through steps (1)-(3) quickly in case of snags. You'll spend most time in (4) flexing your data science muscles. Finally, put your work to the test in (5). + +After the interval completes, the judges will run a script to compute the accuracy of each team's predictoor bot. And you'll hear how you did. + +## Install pdr-backend repo + +In a new console: + +```console +# clone the repo and enter into it +git clone https://github.com/oceanprotocol/pdr-backend +cd pdr-backend + +# Create & activate virtualenv +python -m venv venv +source venv/bin/activate + +# Install modules in the environment +pip install -r requirements.txt +``` + +If you're running MacOS, then in console: +```console +codesign --force --deep --sign - venv/sapphirepy_bin/sapphirewrapper-arm64.dylib +``` + +## Try out simulation flow + +Simulation allows us to quickly build intuition, and assess the performance of the data / model / trading strategy (backtest). + +Let's simulate! In console: +```console +python pdr_backend/predictoor/approach3/runtrade.py +``` + +What `runtrade.py` does: +1. Set simulation parameters. +1. Grab historical price data from exchanges and stores in `csvs/` dir. It re-uses any previously saved data. +1. Run through many 5min epochs. At each epoch: + - Build a model + - Predict up/down + - Trade. + - (It logs this all to screen, and to `out*.txt`.) +1. Plot total profit versus time. + +(The "improve model" section below describes how to change the data, modeling approach, etc.) + +## Try out bot flow + +In this step, you'll try out the "predictoor bot flow", so that when it comes time to submit using the bot flow, you'll be ready. Every five minutes, the bot builds a new model, predicts, then submits the prediction to chain. + +First, tokens! You need (fake) ROSE to pay for gas, and (fake) OCEAN to stake and earn. [Get them here](testnet-faucet.md). + +Then, copy & paste your private key as an envvar. In console: +```console +export PRIVATE_KEY= +``` + +Now, set other envvars. In console: +```console +#other envvars for testnet and mainnet +export ADDRESS_FILE="${HOME}/.ocean/ocean-contracts/artifacts/address.json" +export PAIR_FILTER=BTC/USDT +export TIMEFRAME_FILTER=5m +export SOURCE_FILTER=binanceus + +#testnet-specific envvars +export RPC_URL=https://testnet.sapphire.oasis.dev +export SUBGRAPH_URL=https://v4.subgraph.sapphire-testnet.oceanprotocol.com/subgraphs/name/oceanprotocol/ocean-subgraph +export STAKE_TOKEN=0x973e69303259B0c2543a38665122b773D28405fB # (fake) OCEAN token address +export OWNER_ADDRS=0xe02a421dfc549336d47efee85699bd0a3da7d6ff # OPF deployer address +``` + +([envvars.md](envvars.md) has details.) + +Then, run your predictoor bot. In console: +```console +python pdr_backend/predictoor/main.py 3 +``` + +Your bot is running, congrats! Sit back and watch it in action. It will loop continuously. +- It has behavior to maximize accuracy without missing submission deadlines, as follows. 60 seconds before predictions are due, it will build a model then submit a prediction. It will repeat submissions every few seconds until the deadline. +- It does this for every 5-minute epoch. + +(You can track at finer resolution by writing more logs to the [code](../pdr_backend/predictoor/approach3/predictoor_agent3.py), or [querying Predictoor subgraph](subgraph.md).) + +## Improve model + +This is the fun part! The game is to improve model accuracy with the help of the "simulation flow" presented above. + +The baseline model is linear, inputting prices of the previous 10 epochs as inputs. (There's a reference trading of "buy if predict up; sell 5min later", but note that your focus is accuracy.) + +What you'll be doing: +- Change values in [runtrade.py](../pdr_backend/predictoor/approach3/runtrade.py) and elsewhere to choose: what input data to use, how to process the input data, what modeling approach to use, and more. +- Run the simulation, and see how well it does +- Repeat! + +You'll want to understand how to work with the simulation & modeling codebase. Please go **[here](dynamic-model-codebase.md)** to learn more. Like any code, it will take some effort for you to understand. Don't be scared of it! The better you know it, the better your accuracy:) + +Have fun! + +And don't forget to do the final step: running your bot for real. + +## Run bot for real + +Let's put all your hard work to the test: follow the "bot flow" provided above. But now it should be running _your_ setup from the previous step, having with more accurate predictions. + +⚠️ The simulation flow and bot flow have different places to set some parameters: simulation uses `runtrade.py`, and bot uses `predictoor_config3.py`. [Here are details](dynamic-model-codebase.md). So, make sure that your bot flow has the parameters you want. + +That's it! Take some time off while the time interval passes and your bot does its thing.