Skip to content

Your go-to for easy access to a plethora of compression algorithms, all neatly bundled in one simple installation.

License

Notifications You must be signed in to change notification settings

milesgranger/cramjam

Repository files navigation

cramjam

Code Style CI PyPI Anaconda-Server Badge Downloads

API Documentation

Install

pip install --upgrade cramjam  # Requires no Python or system dependencies!

CLI

A CLI interface is available as cramjam-cli

libcramjam

A Rust crate and C friendly library available at libcramjam


Extremely thin and easy-to-install Python bindings to de/compression algorithms in Rust. Allows for using algorithms such as Snappy, without any system or other python dependencies.


Benchmarks

Some basic benchmarks are available in the benchmarks directory


Available algorithms:

  • Snappy      cramjam.snappy
  • Brotli          cramjam.brotli
  • Bzip2          cramjam.bzip2
  • Lz4              cramjam.lz4
  • Gzip            cramjam.gzip
  • Zlib              cramjam.zlib
  • Deflate       cramjam.deflate
  • ZSTD           cramjam.zstd
  • XZ / LZMA  cramjam.xz
  • Blosc2         cramjam.experimental.blosc2
  • ISA-L backend (only on 64-bit targets)
    • igzip          cramjam.experimental.igzip
    • ideflate     cramjam.experimental.ideflate
    • izlib           cramjam.experimental.izlib

All available for use as:

>>> import cramjam
>>> import numpy as np
>>> compressed = cramjam.snappy.compress(b"bytes here")
>>> decompressed = cramjam.snappy.decompress(compressed)
>>> decompressed
cramjam.Buffer(len=10)  # an object which implements the buffer protocol
>>> bytes(decompressed)
b"bytes here"
>>> np.frombuffer(decompressed, dtype=np.uint8)
array([ 98, 121, 116, 101, 115,  32, 104, 101, 114, 101], dtype=uint8)

Where the API is cramjam.<compression-variant>.compress/decompress and accepts bytes/bytearray/numpy.array/cramjam.File/cramjam.Buffer / memoryview objects.

de/compress_into Additionally, all variants support decompress_into and compress_into. Ex.

>>> import numpy as np
>>> from cramjam import snappy, Buffer
>>>
>>> data = np.frombuffer(b'some bytes here', dtype=np.uint8)
>>> data
array([115, 111, 109, 101,  32,  98, 121, 116, 101, 115,  32, 104, 101,
       114, 101], dtype=uint8)
>>>
>>> compressed = Buffer()
>>> snappy.compress_into(data, compressed)
33  # 33 bytes written to compressed buffer
>>>
>>> compressed.tell()  # Where is the buffer position?
33  # goodie!
>>>
>>> compressed.seek(0)  # Go back to the start of the buffer so we can prepare to decompress
>>> decompressed = b'0' * len(data)  # let's write to `bytes` as output
>>> decompressed
b'000000000000000'
>>>
>>> snappy.decompress_into(compressed, decompressed)
15  # 15 bytes written to decompressed
>>> decompressed
b'some bytes here'