Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare bot for public use #1

Merged
merged 30 commits into from
Jun 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
3d2c75a
Initial trimming and refactoring
aquarat Jun 17, 2024
c0d2728
Remove typechain
aquarat Jun 17, 2024
fc2cbaf
Further trimming and removal
aquarat Jun 17, 2024
e4e2f8b
Lint fix
aquarat Jun 17, 2024
afd3db8
More triming
aquarat Jun 18, 2024
75e0e33
Fixes, CI fixes, deployment bytecode
aquarat Jun 18, 2024
fd943ec
Add persistence
aquarat Jun 18, 2024
b566482
Refactor, comment docs and README
aquarat Jun 19, 2024
2d9a98f
Add more docs, fix docker build, add quote stripping for mnemonic
aquarat Jun 19, 2024
dd2944f
Remove pointless docs link check
aquarat Jun 19, 2024
78d2d69
Remove HH files from ignores and replace with symlinks
aquarat Jun 19, 2024
0eebfd9
Remove release scripts
aquarat Jun 20, 2024
9a6f550
Refactor orbit-bot directory
aquarat Jun 20, 2024
cb58f67
Minor polishing
aquarat Jun 20, 2024
204635d
More comments and docs
aquarat Jun 20, 2024
0b23a3e
Additional JSDoc
aquarat Jun 20, 2024
7919f97
Add constants for loop sleep
aquarat Jun 21, 2024
e426c55
Add lots of links into the OEV docs
aquarat Jun 24, 2024
ce519f7
Remove Renovate and docker amd64 build command
aquarat Jun 24, 2024
39e620c
Remove JSDoc types, add funding and compilation for liquidator contract
aquarat Jun 24, 2024
1586cca
Refactor fixes
aquarat Jun 24, 2024
cbf8b3f
Add back contracts and HH components
aquarat Jun 24, 2024
d219a33
PR comments: simplify
aquarat Jun 25, 2024
4409cef
Remove unused deps, swap logger for console, remove immer
aquarat Jun 25, 2024
70b9c16
Split account watch function, comment them to explain what they're doing
aquarat Jun 26, 2024
f0d09a8
Add back fat contracts build (fat for consistency)
aquarat Jun 26, 2024
35fc3b3
Swap out contracts with interface-based contract
aquarat Jun 26, 2024
2cfc81e
Integrate upstream changes into accounts to watch
aquarat Jun 26, 2024
c218fd6
Fix linter, add hardhat config
aquarat Jun 26, 2024
b5fb311
Integrate upstream changes, expand interfaces, swap out Orbit functions
aquarat Jun 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# NOTE: Keep in sync with .gitignore.
#
# From: https://shisho.dev/blog/posts/how-to-use-dockerignore/
#
# In .gitignore, the file or directory name is ignored in any hierarchy below the .gitignore file, but in .dockerignore,
# all paths must be relative to the way where .dockerignore is located. However, in .dockerignore, all paths must be
# listed relative to the path.
**/.build
**/.env
**/.idea
**/.log
**/.tsbuildinfo
**/.vscode
**/build
**/dist
**/node_modules
**/coverage
**/.DS_Store
**/secrets.env
**/.eslintcache

# Custom ignore files
**/*.ignore
10 changes: 10 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
LOGGER_ENABLED=true
LOG_COLORIZE=true
LOG_LEVEL=debug
LOG_FORMAT=pretty

# Bot variables
MNEMONIC="test test test test test test test test test test test junk"

# Optional: PERSIST_ACCOUNTS_TO_WATCH=true
# Optional: ETHER_LIQUIDATOR_ADDRESS=0xAb6702A6Fd7f0F2596f70c273376036B44a10709
1 change: 1 addition & 0 deletions .eslintignore
36 changes: 36 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
module.exports = {
root: true, // https://github.com/eslint/eslint/issues/13385#issuecomment-641252879
env: {
es6: true,
node: true,
},
parser: '@typescript-eslint/parser',
parserOptions: {
project: ['./tsconfig.json'],
ecmaVersion: 11,
sourceType: 'module',
},
globals: {
Atomics: 'readonly',
SharedArrayBuffer: 'readonly',
},
plugins: ['@typescript-eslint', 'import'],
rules: {
'unicorn/no-process-exit': 'off',
'unicorn/prefer-top-level-await': 'off',

// Typescript
'@typescript-eslint/consistent-return': 'off', // Does not play with no useless undefined when function return type is "T | undefined" and does not have a fixer.
'@typescript-eslint/max-params': 'off',
'@typescript-eslint/no-dynamic-delete': 'off',
'@typescript-eslint/no-var-requires': 'off',
'@typescript-eslint/use-unknown-in-catch-callback-variable': 'off',

// Lodash
'lodash/prefer-immutable-method': 'off',
'lodash/prop-shorthand': 'off',

// Removal of typechain
'@typescript-eslint/no-unsafe-call': 'off',
},
};
40 changes: 40 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Continuous Build

on:
push:
branches:
- main
pull_request:

jobs:
lint-build:
runs-on: ubuntu-latest
name: Build, lint and test
steps:
- name: Clone repo
uses: actions/checkout@v4
- name: Install pnpm
uses: pnpm/action-setup@v3
with:
version: 9.x
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: '20.x'
cache: 'pnpm'
- name: Install Dependencies
run: pnpm install --frozen-lockfile
- name: Build
run: pnpm run build
- name: Lint
run: pnpm run prettier:check && pnpm run eslint:check
- name: Lint Typescript
run: pnpm run tsc

# Leaving this as we may want to re-add tests and docs later
aquarat marked this conversation as resolved.
Show resolved Hide resolved
required-checks-passed:
name: All required checks passed
runs-on: ubuntu-latest
needs: [lint-build]
steps:
- run: exit 0
18 changes: 18 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
.build
.env
.idea
.log
.tsbuildinfo
.vscode
artifacts
cache
build
dist
node_modules
coverage
.DS_Store
secrets.env
.eslintcache

# Custom ignore files
*.ignore
4 changes: 4 additions & 0 deletions .husky/pre-push
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

pnpm run prettier:check && pnpm run eslint:check
1 change: 1 addition & 0 deletions .prettierignore
17 changes: 17 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"bracketSpacing": true,
"printWidth": 120,
"singleQuote": true,
"tabWidth": 2,
"trailingComma": "es5",
"useTabs": false,
"overrides": [
{
"files": "*.md",
"options": {
"parser": "markdown",
"proseWrap": "always"
}
}
]
}
36 changes: 36 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# =============================================================================================
# Base image
# =============================================================================================
FROM node:20-slim AS base
WORKDIR /app

ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable

# Fetch all packages into a virtual store (specifically intended for Docker images)
# See: https://pnpm.io/cli/fetch
COPY pnpm-lock.yaml /app
RUN pnpm fetch

# Install and build dependencies
COPY . /app
RUN pnpm install --prefer-offline --recursive
RUN pnpm run build

# =============================================================================================
# OEV Bot image
# =============================================================================================
# Prepare the app image
FROM base AS oev-bot
ENV name="oev-bot"
LABEL application="oev-bot" description="OEV Bot container"
WORKDIR /app

# "node" Docker images come with a built-in, least-privileged user called "node"
USER node
COPY --chown=node:node --from=base /app/ .

ENV NODE_ENV=production

ENTRYPOINT ["node", "dist/src/index.js"]
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2024 API3

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
168 changes: 167 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,167 @@
# oev-seeker
# OEV Orbit Bot Example

This repository contains an example OEV Searcher bot implementation targeting [Orbit Lending](https://orbitlending.io/).
To understand how OEV works, visit
[the OEV documentation](https://oev-docs--pr12-new-oev-docs-0y2wddya.web.app/reference/oev-network/overview/oev-network.html).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to remember to update this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will in the following PR. The docs are WIP as of now.


Before running this application, be sure to read and understand the code.

## Process Overview

The OEV Bot follows this flow to extract OEV from Orbit Lending:

- Initialisation
- Get log events from the target chain and build a list of accounts to watch for possible liquidation opportunities
- Get log events from the OEV Network to determine awarded/live/lost bids
- Main Loop
- Continuously watch log events from Orbit to maintain a list of accounts to watch
- Attempt liquidations when opportunities are detected

## Opportunity Detection and Value Extraction - In Depth

Given a list of accounts to watch, the app does the following: (Refer to `findOevLiquidation()`)

### Search for an OEV Liquidation Opportunity

- Simulate liquidation potential by [transmuting](#transmutation) the oracle's value for a feed
- Refer to the [Transmutation section of this README](#transmutation) for more information.
- Find Orbit's Price Oracle: `orbitSpaceStation.oracle`
- Read the current value of the oracle for the target feed: `priceOracle.getUnderlyingPrice(oEtherV2)`
- Apply a transmutation value to the read price: `getPercentageValue(currentEthUsdPrice, 100.2)`
- Create a set of calls that can be used to transmute the value of the oracle temporarily
- Set the dAPI Name to a beacon we control:
`api3ServerV1Interface.encodeFunctionData('setDapiName', [dapiName, beaconId])`
- Set the value of the target beacon to our transmuted value: `updateBeaconWithSignedData`
- In a single call, apply the transmutation call data and retrieve account liquidity for all accounts
- refer to
[ExternalMulticallSimulator.sol](https://github.com/api3dao/oev-searcher/blob/main/contracts/api3-contracts/utils/ExternalMulticallSimulator.sol)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this repo is private so the link won't work. I'm not sure what the plans to make it public are though (if there are any)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be linked from the main branch of api3/contracts once this is done

- For all accounts assessed using a transmuted oracle value sort by the biggest shortfall.
- For all shortfall accounts, re-simulate the transmutation call, simulate a liquidation and determine the profit.
- Find the most profitable liquidation (using ETH and USD components).
- At this point, the OEV bidding process begins. To understand the OEV bidding lifecycle refer to
[these OEV docs](https://oev-docs--pr12-new-oev-docs-0y2wddya.web.app/reference/oev-network/overview/auction-cycle.html)

- [Bid on an update for the feed](https://oev-docs--pr12-new-oev-docs-0y2wddya.web.app/reference/oev-network/searchers/submit-bids.html#with-an-expiration-timestamp)

- ```typescript
const bidDetails: BidDetails = {
oevProxyAddress: contractAddresses.api3OevEthUsdProxy,
conditionType: BID_CONDITION.GTE,
conditionValue: transmutationValue,
updateSenderAddress: contractAddresses.multicall3,
nonce,
};
```

- Store the active bid's parameters

### Attempt to Exploit the OEV Liquidation Opportunity

Refer to `attemptLiquidation()`

- [Listen for the award, expiry or loss of the active bid](https://oev-docs--pr12-new-oev-docs-0y2wddya.web.app/reference/oev-network/searchers/submit-bids.html#checking-bid-status-and-listening-for-awarded-bids)
- If the bid is awarded, encode a multicall call set containing
- Call #1: Call to the API3 Server with the awarded bid details as call data
- Call #2: Call the Orbit Ether Liquidator contract with the liquidation parameters
- Simulate the liquidation multicall and determine the profitability - bail if the profit is below the minimum
- [Execute the liquidation transaction](https://oev-docs--pr12-new-oev-docs-0y2wddya.web.app/reference/oev-network/searchers/submit-bids.html#performing-the-oracle-update-using-the-awarded-bid)
- Report the fulfilment on the OEV Network #TODO there's no page for this in the OEV docs
https://github.com/api3dao/oev-docs/pull/12#issuecomment-2186092191

### Transmutation

In order to bid on an OEV update an application will need to determine
[the bid's parameters](https://oev-docs--pr12-new-oev-docs-0y2wddya.web.app/reference/oev-network/searchers/submit-bids.html#arguments-for-placebid),
and in particular:

- The value of the bid (what will be paid for the bid in the target chain's native token)
- The conditions under which the bid will be considered (less-than or greater-than a specific dAPI value)

Determining these values would generally require re-implementing the mathematical logic of the dApp being targeted,
something which is often very onerous. To make integrating into a target dApp easier, API3 has built a contract that
facilitates the "transmutation" of a dAPI (from the concept of transmuting silver to gold).

The contract's relevant function is quoted below:

```solidity
/// @notice eth_call'ed while impersonating address-zero with zero gas
/// price to simulate an external call
/// @param target Target address of the external call
/// @param data Calldata of the external call
/// @return Returndata of the external call
function functionCall(
address target,
bytes memory data
) external override returns (bytes memory) {
require(msg.sender == address(0), "Sender address not zero");
require(tx.gasprice == 0, "Tx gas price not zero");
return Address.functionCall(target, data);
}
```

[//]: # 'TODO add a link to the actual contract'

The function can only be called with a signer address of zero, and such a signer is only valid for non-write operations,
like a simulated RPC contract call. This can be executed via the
[eth_call](https://www.quicknode.com/docs/ethereum/eth_call) RPC method. The deployed contract instance this function
belongs to has been granted the
[DAPI_NAME_SETTER_ROLE on the Api3ServerV1](https://github.com/api3dao/contracts/blob/d3c7dc6683445df14bf5f43b07e6ad9cc2813cc5/contracts/api3-server-v1/DapiServer.sol#L66).
This allows this contract to change the datafeed a dApi name points to - but there is no risk to anyone as this can only
be called inside a non-writing and/or simulated transaction.

Therefore, within a simulated contract call, the app can do the following (via an intermediate contract):

- [Create and sign a new datafeed data point](https://github.com/api3dao/contracts/blob/d3c7dc6683445df14bf5f43b07e6ad9cc2813cc5/test/api3-server-v1/Api3ServerV1.sol.ts#L22)
(value and timestamp)
- Within a multicall transaction
- Use the data feed update created earlier to initialise a datafeed our app controls, with a value we have specified
- As an example, this could be the current target data feed's value + 1%
- Set the target datafeed of the dApp's dApi to the newly-initialised datafeed
- Read the necessary functions on the target dApp to determine OEV opportunities and profitability of a liquidation

For the implementation in this project, refer to the `getDapiTransmutationCalls` function for the transmutation
component. Also refer to `simulateTransmutationMulticall` for the actual transmutation simulation.

## Run the OEV Bot Locally

- Copy `.env.example` to `.env` and populate it
- `cp .env.example .env`
- If this app is being run for the first time you'll need to deploy and fund the OrbitLiquidator contract:
- Build everything, including the contract: `pnpm build`
- Deploy the contract: Run `pnpm orbit-bot:cli-utils deploy`
- Populate the `ETHER_LIQUIDATOR_ADDRESS` in .env with the address of the contract deployed above
- Fund the contract: `pnpm orbit-bot:cli-utils deposit 1` (for 1 ETH)
- Note that you can withdraw ETH and tokens with:
- `pnpm orbit-bot:cli-utils withdraw-all-eth`
- `pnpm orbit-bot:cli-utils withdraw-all-token`
- Ensure that the account on Blast, associated with the `MNEMONIC` you provided has some funds on the OEV Network and
Blast.

Finally, run the app: `pnpm orbit-bot`

## Running the OEV Bot in Docker

### Configuration

Ensure that the .env file has been populated, as described above. This is necessary for running the app, but not for
building the Docker image.

### Build Docker image

Build the docker images locally using any of the following commands (as per your requirements):

```bash
# Builds all three bots using the host machine's CPU architecture
pnpm docker:build

# Builds all three bots using the x86_64 (aka amd64) CPU architecture
pnpm docker:build:amd64
aquarat marked this conversation as resolved.
Show resolved Hide resolved

# Run the bot
pnpm docker:run
```

## Other notes

- To withdraw all Eth funds from the liquidator contract, run: `pnpm orbit-bot:cli-utils withdraw-all-eth`
- To withdraw all tokens from the liquidator contract, run: `pnpm orbit-bot:cli-utils withdraw-all-token`
Loading