diff --git a/http-gateways/IPLD_GATEWAY.md b/http-gateways/IPLD_GATEWAY.md new file mode 100644 index 000000000..9e041b25f --- /dev/null +++ b/http-gateways/IPLD_GATEWAY.md @@ -0,0 +1,202 @@ +# IPLD Gateway Specification + +![wip](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square) + +**Authors**: + +- Mauve Signweaver ([@RangerMauve](https://github.com/RangerMauve)) + +---- + +**Abstract** + +IPLD Gateway is an extension of [PATH_GATEWAY.md](./PATH_GATEWAY.md) that enables lower level interaction with IPLD data structures under a specific path. + +This document describes the delta between [PATH_GATEWAY.md](./PATH_GATEWAY.md) and this gateway type. + +Summary: + +- Adds a new `/ipld/{cid}[/{segments}][?{params}]` subpath to the gateway +- Defines a specification for parsing out extra parameters for individual path segments. +- Describes how to map to and from `ipld://` URLs + +## HTTP API + +### `GET /ipld/{cid}[/{segments}][?{params}]` + +Resolve IPLD paths to some data. + +The path segments will be traversed with any parameters used to transform data along the way. + +The `format` query string parameter, or the `Accept` request header can be used to control the format which will be used to return the data. + +By default, data will be returned as [DAG-JSON](https://ipld.io/specs/codecs/dag-json/). Implementations MUST also support [DAG-CBOR](https://ipld.io/specs/codecs/dag-cbor/) as an opt-in (requested by the client). + + + +### `HEAD /ipld/{cid}[/{segments}][?{params}]` + +Resolves IPLD paths, and yields the same status code and headers as `GET`. + +### `POST /ipld/` and `POST /ipld/localhost/` + +Upload raw data to IPLD. +The `body` of the request shall be parsed according to the `Content-Type` as IPLD data via standard encodings. +`/localhost/` is used to support `POST ipld://localhost/` for uploading IPLD data to local nodes in web browsers that support it. + +The response will contain an `ipld://{cid}/` URL pointing at your data. + + + +### `PATCH /ipld/{cid}[/{segments}][?{params}]` + +This endpoint enables you to apply an [IPLD Patch](https://ipld.io/specs/patch/) to existing IPLD data. + + + +The response will be an `ipld://` URL with your updated data. + +Note that the CID in the response URL will contain the same `segments` as in the request URL. +e.g. if you patch data at `ipld://{cid1}/some/path/`, you will get back a URL that looks like `ipld://{updated cid}/some/path/` +This enables you to make complex changes to a subtree in a dataset and get back a new root CID to use in your application. + +## HTTP Request + +### Request Headers + +#### `Accept` (request header) + +For `/ipld/{cid}/*` paths, the `Accept` header is used to indicate the encoding that should be used to return the data. +This means that data initially encoded as `dag-json` will be transcoded to `dag-cbor` if the `application/vnd.ipld.dag-cbor` Accept header is used. + +- `application/json`: Interpret in the same way as `application/vnd.ipld.dag-json`. +- `application/vnd.ipld.dag-json`: Return the block specified by the path encoded in `dag-json`. +- `application/vnd.ipld.dag-cbor`: Return the block specified by the path ecnoded in `dag-cbor`. + +If no `Accept` header is present in the request, it will be assumed to be `application/vnd.ipld.dag-json`. + +#### `Content-Type` (request header) + +This header applies to `PUT/POST/PATCH` requests for `/ipld/*` paths on IPLD gateways which also support the [writable gateways spec](./WRITABLE_GATEWAY.md). +Including it will hint to the writable IPLD gateway which encoding to use to parse the request body into the IPLD Data Model. + +- `application/json`: Interpret in the same way as `application/vnd.ipld.dag-json`. +- `application/vnd.ipld.dag-json`: Return the block specified by the path encoded in `dag-json`. +- `application/vnd.ipld.dag-cbor`: Return the block specified by the path ecnoded in `dag-cbor`. + +### Request Query Parameters + +#### `format` (request query parameter) + +Optional, `format=` can be used to request specific encodings. + +This is a URL-friendly alternative to sending `Accept: application/vnd.ipld.` header, see [Accept](#accept-request-header) for more details. + +### Path Segments and Path Segment Parameters + +Path segments are used for IPLD data model [traversal](https://ipld.io/docs/data-model/traversal/). +Each segment is separated by a `/` and contains a utf8 `name` followed by an optional set of parameters using the [Matrix URI format](https://www.w3.org/DesignIssues/MatrixURIs.html). + +Example: + +``` +/ipld/bafywhatever/foo/bar;extra=thing;whatever=here/ +``` + +Path segments can have additional parameters added to them by separating them using semicolons (`;`) and having key-value pairs separated by an `=`. +This format is based on the [Matrix URI proposal from the W3C](https://www.w3.org/DesignIssues/MatrixURIs.html). +Note that these parameters are stripped from the segment name when passed to any underlying traversal code. + +This spec only perscribes two reserved parameter names: `adl` for specifying [Advanced Data Layouts](https://ipld.io/docs/advanced-data-layouts/) to process the data with, and `schema` to specify an [IPLD schema](https://ipld.io/docs/schemas/intro/) to use to interpret the data. +Other names may be specified in future specs that build upon this one. + +#### ADL (segment parameter) + +Segments may contain an `adl` key which points to a name of an [Advanced Data Layout](https://ipld.io/docs/advanced-data-layouts/intro/) to process the node with. + +If no `adl` key is specified, then no ADL will be applied to this point in the traversal. + +The supported ADL names will vary based on gateway. + +An example value would be `;adl=hamt` to specify the [HAMT](https://ipld.io/specs/advanced-data-layouts/hamt/) ADL that's used to represent large maps. + +For example `/ipld/bafyreic672jz6huur4c2yekd3uycswe2xfqhjlmtmm5dorb6yoytgflova;adl=hamt/yes` (taken from the HAMT examples), should resolve to the following: + +```json +[ + { + "line": 9, + "column": 501 + } +] +``` + +Note that in the path, the CID refers to a root of the HAMT which is what the HAMT ADL is being applied to. There is then a subpath, `yes` which contains the instances of the word `yes`. + +#### Schema (segment parameter) + +Segments may contain a `schema` key which points to the `CID` of an [IPLD Schema](https://ipld.io/docs/schemas/) in its [DMT](https://ipld.io/specs/schemas/#dsl-vs-dmt) form. +If a `schema` key is provided, there must also be a `type` parameter which references one of the named Types within the IPLD Schema. + +The node at that point in the traversal will then be transformed by the schema, and any typed nodes it links to will also be transformed by their respective schemas. + +For example, given the following schema (note it is written in DSL form, but must be converted to the DMT in order to be refernced): + +```ipldschema +type Example struct { + Hello String + Goodbye &NestedExample +} representation tuple + +type NestedExample struct { + region String +} representation tuple +``` + +The CID for the DMT of this schema is `bafyreibvheoym4avfsjfw63yhsymovm7o54ftcnxwxovqf5xxcbjddanze` + +A raw node of type `NestedExample`, whose CID is `bafyreia5mssvef4owvyols2bduwxl6csvlb35oigyj4gc7wm6wzg44udtq`: + +```json +['Cyberspace'] +``` + +A raw node of type `Example` which references the first node and whose CID is `bafyreifuyjaq3u3izc7qaf4shh76lk6565e72njgjxtava7q4s7bxheyxa`: + +```json +['Hello', {'/': 'bafyreia5mssvef4owvyols2bduwxl6csvlb35oigyj4gc7wm6wzg44udtq'}] +``` + +We can construct the path `/ipld/bafyreifuyjaq3u3izc7qaf4shh76lk6565e72njgjxtava7q4s7bxheyxa/Goodbye?schema=bafyreibvheoym4avfsjfw63yhsymovm7o54ftcnxwxovqf5xxcbjddanze&type=Example`, or more succinctly `ipld://${cid2}/Goodbye?schema=${schemaCID}&type=Example`. + +The resolved node should look like: + +```JSON +{ + "region": "Cyberspace" +} +``` + +#### Escaping / Encoding + +IPLD path segments and path segment keys/values may use [escape sequences that follow RFC1738](https://www.rfc-editor.org/rfc/rfc1738) to represent raw values like `/` which would otherwise be interpreted by URL parsers as being structurally significant. + +Specifically, any values in path segments that are part of the "reserved" list of characters `";" | "/" | "?" | ":" | "@" | "&" | "="`, or are non-ascii characters, must be escaped when encoding to the path. + +For example, path segment name `escape;this` should be escaped to `escape%3Bthis` so that `this` doesn't get accidentally parsed as a parameter. + +Similarly, the path segment name `😁` should be escaped to `%F0%9F%98%81`. + +## Interaction with URLs + +`/ipld/` HTTP paths map directly to `urn:ipld:` URNs, `ipld:` URIs or `ipld://` pseudo-URLs. +Similarly, `ipld://` URLs can be mapped back to `/ipld/` paths on the gateway. +This gives us an easy way to convert between URLs within applications and paths on gateways running either locally or remotely.