Firehose on StarkNet

Firehose implementation for StarkNet, bootstrapped from streamingfast/firehose-acme. This project is maintained by the zkLend team.

Docker images

Docker images are available on Docker Hub. Three types of tags are provided:

starknet/firestark:${FIRESTARK_VERSION}

These images are pure firestark images with no StarkNet client bundled. These are suitable for using as non-reader services (e.g. relayer) that are not client-dependent.

starknet/firestark:${FIRESTARK_VERSION}-pathfinder-${PATHFINDER_VERSION}

These images are firestark images bundled with instrumented pathfinder builds. They work as an out-of-the-box solution that you can use to quickly spin up a working cluster.

starknet/firestark:${FIRESTARK_VERSION}-juno-${PATHFINDER_VERSION}

These images are firestark images bundled with instrumented juno builds. They work as an out-of-the-box solution that you can use to quickly spin up a working cluster.

starknet/firestark:${FIRESTARK_VERSION}-jsonrpc-${JSONRPC_TO_FIRESTARK_VERSION}

Similar to the pathfinder variant but these images come with jsonrpc-to-firestark bundled instead. jsonrpc-to-firestark is useful when you have an already-syned Starknet node, as it's much faster than syncing an instrumented node from scratch.

Running Firehose

Quickstart with docker-compose

An example docker-compose.yml file is available for getting the whole system to run in no time. Make sure to modify the .env file with the L1 Ethereum RPC URL, and then:

$ cd ./docker
$ sudo docker-compose up -d

Once everything is up and running, you should be able to subscribe to the block stream with grpcurl:

$ grpcurl -plaintext -d '{"start_block_num": 0}' localhost:10015 sf.firehose.v2.Stream.Blocks

As noted in the docker-compose.yml file, it only serves as an example of running different services separately. For production workload you might want to deploy a setup with high-availability, taking the file as a starting point.

Running without Docker

Note that these instructions are only suitable for development purposes. For production workload, you'd probably want to run a high-availability setup.

To run a Firehose-enabled pathfinder node, you need to install the instrumented fork version we also maintain:

$ cargo install --locked --git https://github.com/starknet-graph/pathfinder --branch patch pathfinder

Alternatively, you can also build the binary without installing. Just make sure to modify standard.yaml to point to the resulting binary.

Then, replace ETHEREUM_URL in standard.yaml with a valid Ethereum JSON-RPC URL for the L1 network you're going to sync.

After which you can just run:

$ cd ./devel/standard
$ mkdir -p ./pathfinder-data
$ ./start.sh

The instrumented pathfinder node should now start syncing. You can test the setup by subscribing to the block stream with grpcurl:

$ grpcurl -plaintext -d '{"start_block_num": 0}' localhost:10015 sf.firehose.v2.Stream.Blocks

Generating protobuf types

A script is available for generating Go types from protobuf files. Unless you have a specific reason to use another version, make sure you have the exact versions of the following tools installed:

$ protoc --version
libprotoc 3.19.4
$ protoc-gen-go --version
protoc-gen-go v1.27.1
$ protoc-gen-go-grpc --version
protoc-gen-go-grpc 1.2.0

You will also need to have streamingfast/proto cloned in the same folder where the current repository lives.

Then, invoke the script to generate the types:

$ ./types/pb/generate.sh

If any diff is found, make sure to also commit the last_generate.txt file so that others know what exact code versions were used to generate the types. If you're making changes to the proto files, make sure to commit those proto changes first before running the script, otherwise checking out to the revision specified in last_generate.txt would yield the proto files before the changes.

License

Apache 2.0