Skip to content

Latest commit

 

History

History
251 lines (193 loc) · 6.9 KB

README.md

File metadata and controls

251 lines (193 loc) · 6.9 KB

SeqProto

Nodejs npm package minimized gzipped size (select exports)

This library provides a simple way to serialize and deserialize objects in binary format.

Why another serialization library?

While I have been writing this library, I have in mind the following main goals:

  • Runtime independent - I want to have a library that runs in every javascript runtime, from Node.JS, through browsers, to CloudFlare.
  • Performance - I want to have a library that is fast and easy to use.
  • Small size - I want to have a library that is small and easy to use.
  • Customizable - Due to the JavaScript nature, the data structures are limited.
  • TypeScript support - I want to have a library that is easy to use in TypeScript.

Installation

Seqproto works in any JavaScript environment. You can install it via npm:

npm install seqproto

Or via CDN:

import { createSer, createDes } from 'https://unpkg.com/seqproto@latest/dist/esm/index.js'

Usage

Examples

For more examples, see the examples directory.

import { createSer, createDes } from 'seqproto'

// Create a serializer
const ser = createSer()

// Serialize some data
ser.serializeBoolean(true)
ser.serializeUInt32(42)
ser.serializeFloat32(-0.5)
ser.serializeString('hello world')
ser.serializeArray([1, 2, 3], (ser, n) => ser.serializeUInt32(n))

// Get ArrayBuffer with serialized data
const buffer = ser.getBuffer()

// Create a deserializer
const des = createDes(buffer)

// Deserialize data
const b = des.deserializeBoolean()
const i = des.deserializeUInt32()
const f = des.deserializeFloat32()
const s = des.deserializeString()
const a = des.deserializeArray((des) => des.deserializeUInt32())

console.log({ b, i, f, s, a })

Object

import type { Ser, Des } from 'seqproto'
import { createSer, createDes } from 'seqproto'

interface Todo {
  id: number
  userId: number
  title: string
  completed: boolean
}

function serializeTodo (ser: Ser, todo: Todo) {
  ser.serializeUInt32(todo.id)
  ser.serializeUInt32(todo.userId)
  ser.serializeString(todo.title)
  ser.serializeBoolean(todo.completed)
}

function deserializeTodo (des: Des): Todo {
  const id = des.deserializeUInt32()
  const userId = des.deserializeUInt32()
  const title = des.deserializeString()
  const completed = des.deserializeBoolean()
  return { id, userId, title, completed }
}

const ser: Ser = createSer()

serializeTodo(ser, {
  id: 1,
  userId: 1,
  title: 'hello',
  completed: false,
})

const buffer = ser.getBuffer()

const des: Des = createDes(buffer)
const todo = deserializeTodo(des)

console.log(JSON.stringify(todo, null, 2))

Array of object

import type { Ser, Des } from 'seqproto'
import { createSer, createDes } from 'seqproto'

let buffer

// Serialize
const todos = [
  { userId: 1, id: 1, completed: false, title: "delectus aut autem" },
  { userId: 1, id: 2, completed: true, title: "quis ut nam facilis et officia qui" }
]

const ser: Ser = createSer()

ser.serializeArray(todos, (ser, todo) => {
  ser.serializeUInt32(todo.id)
  ser.serializeUInt32(todo.userId)
  ser.serializeString(todo.title)
  ser.serializeBoolean(todo.completed)
})

buffer = ser.getBuffer()

// Deserialize
const des: Des = createDes(buffer)

const deserializedTodos = des.deserializeArray((des) => {
  const id = des.deserializeUInt32()
  const userId = des.deserializeUInt32()
  const title = des.deserializeString()
  const completed = des.deserializeBoolean()
  return { id, userId, title, completed }
})

console.log(deserializedTodos)

API

This library exports the following functions:

  • createSer()/createSer({ bufferSize: number }): creates a new serializer.
  • createDes(buffer): creates a new deserializer.
  • ser.reset(): reset the serializer.
  • ser.serializeBoolean(b): serializes a boolean value.
  • des.deserializeBoolean(): deserializes a boolean value.
  • ser.serializeUInt32(uint32): serializes a 32-bit unsigned integer.
  • des.deserializeUInt32(uint32): deserializes a 32-bit unsigned integer.
  • ser.serializeNumber(n): serializes a 32-bit unsigned integer or signed integer or float.
  • des.deserializeNumber(n): deserializes a 32-bit unsigned integer or signed integer or float.
  • ser.serializeString(string): serializes a string.
  • des.deserializeString(): deserializes a string.
  • ser.serializeArray(array, (ser, item) => { ... }): serializes an array.
  • des.deserializeArray((des) => { ... }): deserializes an array.
  • ser.serializeIterable(iterable, (ser, item) => { ... }): serializes an iterable.
  • des.deserializeIterable((des) => { ... }): deserializes an iterable.
  • ser.serializeFloat32(float32): serializes float 32bit.
  • des.deserializeFloat32(): deserializes float 32bit.
  • ser.getBuffer(): returns the serialized buffer.

Benchmarks

We created 3 different benchmarks to compare the performance of SeqProto with JSON and Avro.

Isolated benchmark

You can run the benchmarks with the following command:

npm run benchmark:serdes

Serialization / Deserialization:

name ops margin percentSlower
seqproto 29764 0.72 0
protobuf 13698 0.19 53.98
avro 24204 0.14 18.68
cbor 803 0.22 97.3
cborx 9707 0.32 67.39
msgpack 6857 0.06 76.96
msgpackr 10449 0.27 64.89
JSON 14434 0.07 51.51

Http benchmark

You can run the benchmarks using 2 shells.

cd bechmarks/e2e
pnpm install
pnpm start
cd bechmarks/e2e
pnpm run autocannon:json
pnpm run autocannon:seqproto
pnpm run autocannon:avro
type req (in 10s) Avg req/sec Avg Bytes/Sec Avg Latency (ms)
JSON 164k 14892 275 0.11
SeqProto 269k 26865.6 321 0.01
Avro 197k 17926.55 169 0.04

e2e benchmark

You can run the benchmarks with the following command:

cd bechmarks/e2e
pnpm install
pnpm start

And go to http://localhost:3000/public/index.html.

iteration parallelism type ms
10 1 JSON 30.69999998807907
10 1 SeqProto 25.600000023841858
10 1 Avro 30.399999976158142
100 1 JSON 108.80000001192093
100 1 SeqProto 96.80000001192093
100 1 Avro 96
100 3 JSON 162.10000002384186
100 3 SeqProto 152.4000000357628
100 3 Avro 167.5
100 6 JSON 277.19999998807907
100 6 SeqProto 263.30000001192093
100 6 Avro 308.19999998807907

Contributing

Contributions are welcome! Please open an issue if you have any ideas for improvement or found a bug.

License

Apache-2.0