Skip to content
/ cole Public

Open source code of FAST' 24 paper, COLE: A Column-based Learned Storage for Blockchain Systems

License

Notifications You must be signed in to change notification settings

hkbudb/cole

Repository files navigation

COLE: A Column-based Learned Storage for Blockchain Systems

Components

  • cole-index and cole-star are COLE and COLE with asynchronous merge
  • patricia-trie is the implementation of the MPT
  • lipp is the the updatable learned index with node persistence
  • non-learn-cmi is the column-based Merkle index (CMI) that uses non-learned index
  • exp is the evaluation backend of all systems including the throughput and the provenance queries

Install Dependencies

sudo apt install -y curl
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
  • Run sudo apt update and sudo apt -y install git make clang pkg-config libssl-dev libsqlite3-dev llvm m4 build-essential

Download Repository to $HOME directory

cd ~
git clone https://github.com/hkbudb/cole

Build

  • Build the latency testing binary and provenance testing binary
cd ~/cole
cargo build --release --bin latency
cargo build --release --bin prov
  • If the program is built successfully, you should find two executable programs latency and prov in the directory ~/cole/target/release/

Prepare YCSB Dataset

  • Download the latest release of YCSB to the $HOME directory:
cd ~
curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.17.0/ycsb-0.17.0.tar.gz
tar xfvz ycsb-0.17.0.tar.gz
  • Install Java
sudo apt -y install default-jdk
sudo apt -y install default-jre
  • Use script build_ycsb.sh to generate readonly, writeonly, and readwriteeven datasets
cd ~/cole/exp
./build_ycsb.sh
  • After the build process finishes, three txt files will be generate:

    • cole/exp/readonly/readonly-data.txt
    • cole/exp/writeonly/writeonly-data.txt
    • cole/exp/readwriteeven/readwriteeven-data.txt
  • Next, prepare the dataset for provenance queries:

cd ~/cole/exp/
./build_prov_ycsb.sh
  • After the build process finishes, a file named cole/exp/prov/prov-data.txt will be generated.

Run Script

cd ~/cole/exp/
python3 run.py
  • Use functions like test_overall_kvstore(), test_overall_smallbank(), and test_prov() in cole/exp/run.py to evaluate the workload of KVStore, SmallBank, and provenance query performance.
  • You may select different scales scale = [1000, 10000, 100000, 1000000, 10000000] or different indexes indexes = ["mpt", "cole", "cole_star", "non_learn_cmi"]

Check the Result

The result json files can be found in each workload directory (e.g., smallbank, writeonly, prov)

  • *-storage.json stores the storage information
  • *-ts.json stores the block timestamp information including start timestamp, end timestamp, and block latency, which can be used to compute the system throughput and latency

About

Open source code of FAST' 24 paper, COLE: A Column-based Learned Storage for Blockchain Systems

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages