Dockerfile doesn't have compatible librocksdb-dev; how to package dynamic lib into whl like on pypi

[jrcavani]

Hello,

I was able to install the x86 python whl from pypi: https://pypi.org/project/faust-streaming-rocksdb/#files

Each whl contains a few precompiled dynamic libs:

# LD_LIBRARY_PATH=/usr/local/lib/python3.8/site-packages/faust_streaming_rocksdb.libs ldd /usr/local/lib/python3.8/site-packages/faust_streaming_rocksdb.libs/librocksdb-e568f8f5.so.6.14.6
        linux-vdso.so.1 (0x00007ffc148af000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8ae8b51000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8ae8b4c000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8ae8b47000)
        libsnappy-ddb1bad0.so.1.1.4 => /usr/local/lib/python3.8/site-packages/faust_streaming_rocksdb.libs/libsnappy-ddb1bad0.so.1.1.4 (0x00007f8ae8800000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f8ae8b2b000)
        libbz2-a273e504.so.1.0.6 => /usr/local/lib/python3.8/site-packages/faust_streaming_rocksdb.libs/libbz2-a273e504.so.1.0.6 (0x00007f8ae8400000)
        liblz4-af1653fb.so.1.8.3 => /usr/local/lib/python3.8/site-packages/faust_streaming_rocksdb.libs/liblz4-af1653fb.so.1.8.3 (0x00007f8ae8000000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8ae7dd6000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8ae8a42000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8ae8a22000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8ae7bae000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f8ae9228000)
# ldd /usr/local/lib/python3.8/site-packages/faust_streaming_rocksdb.libs/lib
libbz2-a273e504.so.1.0.6       liblz4-af1653fb.so.1.8.3       librocksdb-e568f8f5.so.6.14.6  libsnappy-ddb1bad0.so.1.1.4

I was wondering which script compiles these libraries and how they can be packaged into the whl. The Dockerfile installs librocksdb-dev from Ubuntu 20.04 package, which is RockDB 5, rather than 6.14.

[wbarnha]

Getting a compatible version of librocksdb-dev has been an issue for me as well, hence why RocksDB libraries are packaged into the wheel.

We test with multiple versions of RocksDB as indicated in:

python-rocksdb/.github/workflows/build.yml

Line 22 in 91e60f2

rocksdb_ver: ['v6.29.3', 'v6.25.3', 'v6.11.4']

However, we have a specific version of RocksDB hardcoded into our release workflow:

python-rocksdb/.github/workflows/dist.yml

Line 20 in 91e60f2

rocksdb_ver: ['v6.14.6']

Before we had wheels for faust-streaming-rocksdb, I built this library inside an Ubuntu:22.04 container. As you can see on https://packages.ubuntu.com/search?suite=default&section=all&arch=any&keywords=rocksdb&searchon=names, you'll have different versions available dependent on your version of Ubuntu:

[bionic (18.04LTS)](https://packages.ubuntu.com/bionic/librocksdb-dev) (libdevel): persistent Key-Value Store for Flash and RAM Storage (development) [universe]
5.8.8-1: amd64 arm64 ppc64el
[focal (20.04LTS)](https://packages.ubuntu.com/focal/librocksdb-dev) (libdevel): persistent Key-Value Store for Flash and RAM Storage (development) [universe]
5.17.2-3: amd64 arm64 ppc64el s390x
[jammy (22.04LTS)](https://packages.ubuntu.com/jammy/librocksdb-dev) (libdevel): persistent Key-Value Store for Flash and RAM Storage (development) [universe]
6.11.4-3: amd64 arm64 ppc64el s390x
[kinetic (22.10)](https://packages.ubuntu.com/kinetic/librocksdb-dev) (libdevel): persistent Key-Value Store for Flash and RAM Storage (development) [universe]
7.3.1-2: amd64 arm64 armhf ppc64el s390x
[lunar](https://packages.ubuntu.com/lunar/librocksdb-dev) (libdevel): persistent Key-Value Store for Flash and RAM Storage (development) [universe]
7.3.1-2: amd64 arm64 armhf ppc64el s390x

Upgrade your Ubuntu Docker image and you should be fine.