Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DIP-263 Use User Data for Public Keys #276

Merged
merged 2 commits into from
Jun 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .spellcheckerdict.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ DSNP
DSNP-compatible
DSNP-referenced
ECIES
EdDSA
[Ee]num(s)?
Ethereum
Extrinsics
Expand Down Expand Up @@ -67,7 +68,7 @@ Polkadot
Poly1305
pre-configured
PRId([ABs])?
publicKey
[Pp]ublicKey
repo
RFC[1-9][0-9]*
Ristretto
Expand All @@ -78,6 +79,7 @@ schemaless
Schnorrkel
SDK
SHA-256
SHA-512
Stateful
stringified
subkey
Expand Down
6 changes: 4 additions & 2 deletions pages/DSNP/Announcements.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,11 @@ Each Announcement has an enumerated type for use when separating out a stream of
| 4 | [Reaction](Types/Reaction.md) | a public visual reply to a Broadcast | no | no |
| 5 | [Profile](Types/Profile.md) | a profile | YES | no |
| 6 | [Update](Types/Update.md) | an update to content| YES | no |
| 7 | [Public Key](Types/PublicKey.md) | a public key for secure communication | no | no |
| 7 | ~~[Public Key](Types/PublicKey.md)~~<sup>b</sup> | ~~a public key for secure communication~~ | ~~no~~ | ~~no~~ |
wilwade marked this conversation as resolved.
Show resolved Hide resolved

<sup>a</sup> Since DSNP version 1.2, social graph changes use [User Data](UserData.md) operations as described in the [Graph](Graph.md) section.
<sup>a</sup> Since DSNP version 1.2, social graph changes use [User Data](UserData.md) operations as described in the [Graph](Graph.md) section.

<sup>b</sup> Since DSNP version 1.3, public keys use [User Data](UserData.md) operations.

## Announcement Validation

Expand Down
2 changes: 1 addition & 1 deletion pages/DSNP/Operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Compliant implementations may respond to error conditions either synchronously,
| <a id="remove-control-key">Remove Control Key</a> | YES | User | Key | [Control Key Removal Record](Records.md#control-key-removal) |
| <a id="publish-announcement">Publish Announcement</a> | no* | User OR Delegate | [Announcement](Announcements.md) | [Announcement Published Record](Records.md#announcement-published) |
| <a id="publish-batch">Publish Batch</a> | no* | User OR Delegate | [Announcement Type](Announcements.md#announcement-types), [Batch Publication](BatchPublications.md) URL, Batch Publication Content Hash | [Batch Published Record](Records.md#batch-published) |
| <a id="get-user-data">Get User Data</a> | no | Any | User's Identifier, Set of Requested [User Data Types](UserData.md#user-data-types) | Map of [User Data Types](UserData.md#user-data-types) to [Data Chunks](UserData.md#data-chunks) with optional [Key Identifiers](Types/PublicKey.md#keyid) |
| <a id="get-user-data">Get User Data</a> | no | Any | User's Identifier, Set of Requested [User Data Types](UserData.md#user-data-types) | Map of [User Data Types](UserData.md#user-data-types) to [Data Chunks](UserData.md#data-chunks) with optional key identifiers of encryption keys for each chunk |
| <a id="replace-user-data">Replace User Data</a> | no | User OR Delegate | User's Identifier, [Key Identifier](Types/PublicKey.md#keyid), Map of [User Data Types](UserData.md#user-data-types) to [Data Chunks](UserData.md#data-chunks) | [User Data Replaced Record](Records.md#user-data-replaced) |

\* For each Announcement Type, an implementation may support one or both of these operations.
Expand Down
2 changes: 2 additions & 0 deletions pages/DSNP/Types/PublicKey.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Public Key Announcement

<mark>_Since DSNP version 1.3, public keys use [User Data](../UserData.md) operations._</mark>

A Public Key Announcement is a way to note a new cryptographic key that can be used in DSNP to secure and verify the authenticity of communications.

The most recently published key (if one exists) for a given key type should be treated as the active key of that key type.
Expand Down
37 changes: 37 additions & 0 deletions pages/DSNP/Types/PublicKeyUserData.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Public Key

Represents an encoding of a public key, one half of a cryptographic key pair.

## Serialization

PublicKey object serialization MUST conform to the following [Avro](https://avro.apache.org) schema:

```
{
"namespace": "org.dsnp",
"name": "PublicKey",
"type": "record",
"fields": [
{
"name": "publicKey",
"type": "bytes",
"doc": "Multicodec public key"
}
]
}
```

## Generation

### publicKey

- MUST be a public key of an allowed key type for the associated User Data type, encoded in `multicodec` format

The byte encoding consists of a [multicodec](https://github.com/multiformats/multicodec/blob/master/table.csv) key identifier (as a varint) followed by the public key's binary data in the codec's described format.

#### Allowed Key Types

| User Data Type | Allowed Algorithms ([multicodec](https://github.com/multiformats/multicodec/blob/master/table.csv)) | Purpose |
| --- | --- | --- |
| `keyAgreementPublicKeys` | `x25519-pub` | A Curve25519 public key that can be used in key exchange protocols to generate a shared secret |
| `assertionMethodPublicKeys` | `ed25519-pub` | A public key for the EdDSA signature scheme using SHA-512 and Curve25519 that can be used to verify cryptographic signatures |
19 changes: 11 additions & 8 deletions pages/DSNP/UserData.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,11 @@ DSNP implementations MUST support the following User Data Types:
| <a name="private-follows">`privateFollows`</a> | 1.2 | `curve25519xsalsa20poly1305` | [`DEFLATE`](https://en.wikipedia.org/wiki/Deflate) | [GraphEdge](Types/GraphEdge.md) |
| <a name="private-connections">`privateConnections`</a> | 1.2 | `curve25519xsalsa20poly1305` | [`DEFLATE`](https://en.wikipedia.org/wiki/Deflate) | [GraphEdge](Types/GraphEdge.md) |
| <a name="private-connection-prids">`privateConnectionPRIds`</a> | 1.2 | NONE | NONE | [PRId](Types/PRId.md) |
| <a name="key-agreement-public-keys">`keyAgreementPublicKeys`</a> | 1.3 | NONE | NONE | [PublicKey](Types/PublicKeyUserData.md) |
| <a name="assertion-method-public-keys">`assertionMethodPublicKeys`</a> | 1.3 | NONE | NONE | [PublicKey](Types/PublicKeyUserData.md) |

Data for each data type is initially formatted as a stream of Avro objects that should conform to the schema specified.
A DSNP system MAY limit the number of objects allowed for a given user data type; if so, this MUST be documented.
Avro file- and block-level information (including in-stream schema) is omitted.
The Avro stream is then compressed and/or encrypted as specified.

Expand All @@ -29,7 +32,7 @@ In the specification of cryptographic operations below, relevant methods from th

## Data Chunks

Because blockchain systems often have specific limits to the amount of data that can be included in a given transaction, operations on user data deal with the data in discrete chunks.
Because consensus systems often have specific limits to the amount of data that can be included in a given transaction, operations on user data deal with the data in discrete chunks.
As implementation strategies may vary, implementations MUST define their own maximum chunk size in bytes to be used in the operations described below.

## Entity Tags
Expand All @@ -46,8 +49,8 @@ The Replace User Data Operation takes the following parameters:

* A DSNP User Id
* Implementations MUST ensure that the principal invoking this Operation is this user, or a transparent chain of delegation from the user to the principal exists.
* A [Key Identifier](Types/PublicKey.md#keyid) for the `keyAgreement` key pair used to encrypt any private data in the operation.
(If only unencrypted user data types are included, the key identifier is optional.)
* The index of the `keyAgreementPublicKeys` key pair used to encrypt any private data in the operation.
(If only unencrypted user data types are included, the key index may be omitted.)
* A map containing the set of data types to update as the keys, and tuples consisting of (1) the schema version used to encode the data type, and (2) a list where each element includes a data chunk and its associated entity tag, as the values.

If the Operation is successful, any previous data associated with the user for each data type included in the input MUST be removed and replaced by the new data.
Expand All @@ -61,9 +64,9 @@ Data chunks should be generated for each included data type using the following
3. For each chunk generated, the application should then:
1. If the data type requires compression, apply the compression codec noted.
1. If the data type requires encryption,
1. Retrieve the user's active (most recently announced) `keyAgreement` public key, U<sub>public</sub>.
The `keyId` in the announcement should match the key identifier provided for this Operation.
If no key exists, one should be created and published as an Announcement before invoking the Operation.
1. Retrieve the user's active `keyAgreementPublicKey` key, U<sub>public</sub>, and note its index.
If no key exists, one should be created and published as User Data before invoking the Operation.
By convention, the key with the highest index (the last object in the Avro stream) is the active key.
1. Create a sealed box (a payload encrypted with a symmetric key derived from an ephemeral key pair, and accompanied by the ephemeral public key), as in the [libsodium](https://doc.libsodium.org/public-key_cryptography/sealed_boxes) function `crypto_box_seal`, using U<sub>public</sub>.
1. Include the previous `etag` value for the chunk. If the chunk is new, `etag` should be set to `null`.
If any chunks are to be deleted, they should be included in the input identified with the existing `etag` and a `null` value for the data.
Expand Down Expand Up @@ -151,13 +154,13 @@ The Get User Data Operation takes the following parameters:
* Note: While _writing_ user data is reserved for the user and any delegates, anyone on the network can read any user's data (though it may be encrypted).
* The User Data Types (by system name) that should be retrieved.

The operation returns a mapping of User Data Type to data chunks, with each data chunk annotated with an entity tag and (optionally) a key identifier. (Note that this is the same general structure as the input data for [Replace User Data](#replace-user-data-operation), for each requested data type.
The operation returns a mapping of User Data Type to data chunks, with each data chunk annotated with an entity tag and (optionally) a key index. (Note that this is the same general structure as the input data for [Replace User Data](#replace-user-data-operation), for each requested data type.
If no chunks for a requested data type exist, an implementation MAY omit that data type from the response.

To transform the data from the output to Avro binary records, a consumer should apply the following algorithm to each data type included:
1. Determine the relevant encryption algorithm, compression codec, and object schema from the User Data Type and version noted.
1. For each chunk,
1. If encryption is indicated, decrypt the chunk data using the user's secret key (identified using the key identifier) as in the [libsodium](https://doc.libsodium.org/public-key_cryptography/sealed_boxes) function `crypto_box_seal_open`.
1. If encryption is indicated, decrypt the chunk data using the user's secret key (identified using the key index) as in the [libsodium](https://doc.libsodium.org/public-key_cryptography/sealed_boxes) function `crypto_box_seal_open`.
1. If compression is required, uncompress the chunk data using the specified codec.
1. Deserialize the uncompressed data to logical records according to the Avro object schema.
1. Retain the chunk's `etag` value if needed for any updates.
Expand Down
27 changes: 14 additions & 13 deletions pages/Frequency/Overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,19 @@ Official schemas may be found in [GitHub](https://github.com/LibertyDSNP/schemas

<!-- These ids are duplicated here for quick reference. -->

| Name | Schema Id Mainnet | Schema Id Testnet (Paseo) | Schema Id Testnet (Rococo) |
| --- | --- | --- | --- |
| [Tombstone](./Publishing.md) | 1 | 1 | 1 |
| [Broadcast](./Publishing.md) | 2 | 2 | 2 |
| [Reply](./Publishing.md) | 3 | 3 | 3 |
| [Reaction](./Publishing.md)| 4 | 4 | 4 |
| [Profile](./Publishing.md) | 6 | 6 | 5 |
| [Update](./Publishing.md)| 5 | 5 | 6 |
| [Public Key](./Publishing.md)| 7 | 7 | 18 |
| [Public Follows](./UserData.md)| 8 | 8 | 13 |
| [Private Follows](./UserData.md) | 9 | 9 | 14 |
| [Private Connections](./UserData.md) | 10 | 10 | 15 |
| Name | Schema Name | Schema Id Mainnet | Schema Id Testnet (Paseo) | Schema Id Testnet (Rococo) |
| --- | --- | --- | --- | --- |
| [Tombstone](./Publishing.md) | `dsnp.tombstone` | 1 | 1 | 1 |
| [Broadcast](./Publishing.md) | `dsnp.broadcast` | 2 | 2 | 2 |
| [Reply](./Publishing.md) | `dsnp.reply` | 3 | 3 | 3 |
| [Reaction](./Publishing.md)| `dsnp.reaction` | 4 | 4 | 4 |
| [Profile](./Publishing.md) | `dsnp.profile` | 6 | 6 | 5 |
| [Update](./Publishing.md)| `dsnp.update` | 5 | 5 | 6 |
| [Key Agreement Public Keys](./UserData.md)| `dsnp.public-key-key-agreement` | 7 | 7 | 18 |
| [Public Follows](./UserData.md)| `dsnp.public-follows` | 8 | 8 | 13 |
| [Private Follows](./UserData.md) | `dsnp.private-follows` | 9 | 9 | 14 |
| [Private Connections](./UserData.md) | `dsnp.private-connections` | 10 | 10 | 15 |
| [Assertion Method Public Keys](./UserData.md)| `dsnp.public-key-assertion-method` | TBD | 11 | 100 |

<!--
### Obsolete
Expand All @@ -50,4 +51,4 @@ Official schemas may be found in [GitHub](https://github.com/LibertyDSNP/schemas

| Last Update Date | Frequency Release | DSNP Version |
| --- | --- | --- |
| 2023-07-17 | 1.5.2+ | 1.2.0 |
| 2024-04-12 | 1.10.0+ | pre-1.3.0 |
4 changes: 1 addition & 3 deletions pages/Frequency/Publishing.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# Announcement Publishing

On Frequency, [Announcements](../DSNP/Announcements.md) are mapped to Schemas which in turn publish Frequency Messages or Stateful Storage changes.
On Frequency, [Announcements](../DSNP/Announcements.md) are mapped to Schemas which in turn publish Frequency Messages.
Frequency Messages are either individual Announcements from a particular user, or a Batch Publication with a multitude of possible users.
Frequency Stateful Storage is either direct Announcements from a particular user or [User Data](./UserData.md) changes.

<!-- Links to https://frequency-chain.github.io/frequency should be updated with links to docs.frequency.xyz when able to be -->

Expand All @@ -16,7 +15,6 @@ Frequency Stateful Storage is either direct Announcements from a particular user
| 4 | [Reaction](../DSNP/Types/Reaction.md) | Batched | Mainnet: 4<br />Testnet (Paseo): 4<br />Testnet (Rococo): 4 | [`Parquet`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
| 5 | [Profile](../DSNP/Types/Profile.md) | Batched | Mainnet: 6<br />Testnet (Paseo): 6<br />Testnet (Rococo): 5 | [`Parquet`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
| 6 | [Update](../DSNP/Types/Update.md) | Batched | Mainnet: 5<br />Testnet (Paseo): 5<br />Testnet (Rococo): 6 | [`Parquet`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
| 7 | [Public Key](../DSNP/Types/PublicKey.md) | [Stateful](./UserData.md#announcements) | Mainnet: 7<br />Testnet (Paseo): 7<br />Testnet (Rococo): 18 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Itemized`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Itemized) |

Source code for each schema is located in the [LibertyDSNP/schemas](https://github.com/LibertyDSNP/schemas) repository.

Expand Down
12 changes: 4 additions & 8 deletions pages/Frequency/UserData.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,23 @@
# Frequency User Data

On Frequency, User Data and select Announcements are mapped to Schemas which use [Stateful Storage](https://frequency-chain.github.io/frequency/pallet_stateful_storage/index.html) for storage and retrieval of the data.
On Frequency, User Data is mapped to Schemas which use [Stateful Storage](https://frequency-chain.github.io/frequency/pallet_stateful_storage/index.html) for storage and retrieval of the data.

## User Data Sets

<!-- Update ./Overview.md if a Schema Id is updated -->

| User Data Set | Deployed Schema Ids | Frequency Model Type | Frequency Payload Location | Settings |
| --- | --- | --- | --- | --- |
| [Public Key (Key Agreement)](../DSNP/Types/PublicKeyUserData.md) | Mainnet: 7<br />Testnet (Paseo): 7<br />Testnet (Rococo): 7 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Itemized`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Itemized) | [Append Only](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.AppendOnly), [Signature Required](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.SignatureRequired) |
| [Public Follows](../DSNP/Graph.md#public-follows) | Mainnet: 8<br />Testnet (Paseo): 8<br />Testnet (Rococo): 13 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Paginated`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Paginated) | None |
| [Private Follows](../DSNP/Graph.md#private-follows) | Mainnet: 9<br />Testnet (Paseo): 9<br />Testnet (Rococo): 14 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Paginated`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Paginated) | None |
| [Private Connections](../DSNP/Graph.md#private-connections) | Mainnet: 10<br />Testnet (Paseo): 10<br />Testnet (Rococo): 15 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Paginated`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Paginated) | None |
| [Public Key (Assertion Method)](../DSNP/Types/PublicKeyUserData.md) | Mainnet: TBD <br />Testnet (Paseo): 11<br />Testnet (Rococo): 100 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Itemized`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Itemized) | [Append Only](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.AppendOnly), [Signature Required](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.SignatureRequired) |

[Pseudonymous Relationship Identifiers](./../DSNP/Graph.md#pseudonymous-relationship-identifiers) (PRIds) are stored along side Private Connections in the same Stateful Storage page.
[Pseudonymous Relationship Identifiers](./../DSNP/Graph.md#pseudonymous-relationship-identifiers) (PRIds) are stored alongside Private Connections in the same Stateful Storage page.

Source code for each schema is located in the [LibertyDSNP/schemas](https://github.com/LibertyDSNP/schemas) repository.

## Announcements

| Announcement | Deployed Schema Ids | Frequency Model Type | Frequency Payload Location | Settings |
| --- | --- | --- | --- | --- |
| [Public Key](../DSNP/Types/PublicKey.md) | Mainnet: 7<br />Testnet (Paseo): 7<br />Testnet (Rococo): 7 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Itemized`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Itemized) | [Append Only](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.AppendOnly), [Signature Required](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.SignatureRequired) |

## Read Operation Mapping

Stateful data is retrieved via state queries (`pallet.stateQuery`) or RPC calls (`pallet.rpcCall()`).
Expand Down
Loading