Skip to content
/ serde Public

Serialization and Deserialization library for motoko

License

Notifications You must be signed in to change notification settings

NatLabs/serde

Repository files navigation

serde for Motoko

An efficient serialization and deserialization library for Motoko.

The library contains four modules:

  • Candid

    • fromText() - Converts Candid text to its serialized form.

    • toText() - Converts serialized candid to its textual representation.

    • encode() - Converts the Candid variant to a blob.

    • decode() - Converts a blob to the Candid variant.

    • encoding and decoding functions also support conversion between the ICRC3 value type and candid. Checkout the example in the usage guide

      • fromICRC3Value() - Converts an ICRC3 value to a candid variant.
      • toICRC3Value() - Converts a candid variant to an ICRC3 value.
  • CBOR

    • encode() - Converts serialized candid to CBOR.
    • decode() - Converts CBOR to a serialized candid.
  • JSON

    • fromText() - Converts JSON text to serialized candid.
    • toText() - Converts serialized candid to JSON text.
  • URL-Encoded Pairs

    • fromText() - Converts URL-encoded text to serialized candid.
    • toText() - Converts serialized candid to URL-encoded text.

Getting Started

Installation

mops

  1. Install mops.
  2. Inside your project directory, run:
mops install serde

Usage

To start, import the necessary modules:

import { JSON; Candid; CBOR; UrlEncoded } from "mo:serde";

JSON

The following code can be used for converting data between the other modules (Candid and URL-Encoded Pairs).

Example: JSON to Motoko

  1. Defining Data Type: This critical step informs the conversion functions (from_candid and to_candid) about how to handle the data.

    Consider the following JSON data:

    [
      {
        "name": "John",
        "id": 123
      },
      {
        "name": "Jane",
        "id": 456,
        "email": "jane@gmail.com"
      }
    ]

    The optional email field translates to:

    type User = {
        name: Text;
        id: Nat;
        email: ?Text;
    };
  2. Conversion: a. Parse JSON text into a candid blob using JSON.fromText. b. Convert the blob to a Motoko data type with from_candid.

    let jsonText = "[{\"name\": \"John\", \"id\": 123}, {\"name\": \"Jane\", \"id\": 456, \"email\": \"jane@gmail.com\"}]";
    
    let #ok(blob) = JSON.fromText(jsonText, null); // you probably want to handle the error case here :)
    let users : ?[User] = from_candid(blob);
    
    assert users == ?[
        {
            name = "John";
            id = 123;
            email = null;
        },
        {
            name = "Jane";
            id = 456;
            email = ?"jane@gmail.com";
        },
    ];

Example: Motoko to JSON

  1. Record Keys: Collect all unique record keys from your data type into an array. This helps the module convert the record keys correctly instead of returning its hash.

    let UserKeys = ["name", "id", "email"];
  2. Conversion:

    let users: [User] = [
        {
            name = "John";
            id = 123;
            email = null;
        },
        {
            name = "Jane";
            id = 456;
            email = ?"jane@gmail.com";
        },
    ];
    
    let blob = to_candid(users);
    let json_result = JSON.toText(blob, UserKeys, null);
    
    assert json_result == #ok(
         "[{\"name\": \"John\",\"id\": 123},{\"name\": \"Jane\",\"id\":456,\"email\":\"jane@gmail.com\"}]"
     );

Example: Renaming Fields

  • Useful way to rename fields with reserved keywords in Motoko.
import Serde from "mo:serde";

    // type JsonSchemaWithReservedKeys = {
    //     type: Text; // reserved
    //     label: Text;  // reserved
    //     id: Nat;
    // };

type Item = {
    item_type: Text;
    item_label: Text;
    id: Nat
};

let jsonText = "{\"type\": \"bar\", \"label\": \"foo\", \"id\": 112}";
let options: Serde.Options = {
    renameKeys = [("type", "item_type"), ("label", "item_label")]
};

let #ok(blob) = Serde.JSON.fromText(jsonText, ?options);
let renamedKeys: ?Item = from_candid(blob);

assert renamedKeys == ?{ item_type = "bar"; item_label = "foo"; id = 112 };

Checkout the usage guide for additional examples:

Limitations

  • Users must provide a list of record keys and variant names during conversions from Motoko to other data formats due to constraints in the candid format.
  • Lack of specific syntax for conversion between Blob, Principal, and bounded Nat/Int types.
  • Cannot deserialize Tuples as they are not candid types. They are just shorthands for records with unnamed fields. See https://forum.dfinity.org/t/candid-and-tuples/17800/7
  • Floats are only recognised if they have a decimal point, e.g., 1.0 is a Float, but 1 is an Int / Nat.
  • Only supports candid data types (i.e primitive and constructed types). Service and function reference types are not supported.

Running Tests

  1. Install dependencies:

  2. Inside the project directory, run:

mops test

Happy coding with serde! 🚀