Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conceptual topic on the DAL #270

Merged
merged 31 commits into from
Feb 13, 2024
Merged

Conceptual topic on the DAL #270

merged 31 commits into from
Feb 13, 2024

Conversation

timothymcmackin
Copy link
Collaborator

@timothymcmackin timothymcmackin commented Jan 19, 2024

Overview of the DAL based on what I learned from the DAL tutorial. This does not go into as much detail as https://tezos.gitlab.io/shell/dal_overview.html; I've tried to include only what I think developers will need to know about sharing data with Smart Rollups via the DAL.

Preview: https://docs-staging-git-dal-overview-trili-tech.vercel.app/architecture/data-availability-layer

This is in the architecture section but still has some user tasks (sending data to the DAL and retrieving it in a Smart Rollup). Not sure if this is organized well or if the concept should be separated from the task.

@timothymcmackin timothymcmackin self-assigned this Jan 19, 2024
Copy link

vercel bot commented Jan 19, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
docs-staging ✅ Ready (Inspect) Visit Preview Feb 13, 2024 3:51pm

Copy link
Contributor

@germanD germanD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made some Minor pass with language tweaks.

docs/architecture/data-availability-layer.mdx Outdated Show resolved Hide resolved
docs/architecture/data-availability-layer.mdx Outdated Show resolved Hide resolved
The number of shards needed depends on how the data is spread across shards, which is controlled by a parameter called the _redundancy factor_.
- The _proof_ certifies the length of the data to prevent malicious users from overloading the layer with data.

1. Users post the certificate to layer 1 via the Octez client.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Users post the certificate to layer 1 via the Octez client.
1. Users post the certificate to Tezos' Layer 1 via the Octez client.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be "Tezos layer 1." I'll make that correction.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make it possessive, it would be "Tezos's layer 1" because Tezos is singular, but I don't think we need the "'s" here.

docs/architecture/data-availability-layer.mdx Outdated Show resolved Hide resolved
with proof "${proof}"
```

For an example of sending larger amounts of data, see [Implement a file archive with the DAL and a Smart Rollup](../tutorials/build-files-archive-with-dal).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For an example of sending larger amounts of data, see [Implement a file archive with the DAL and a Smart Rollup](../tutorials/build-files-archive-with-dal).
For an example involving larger amounts of data, see [Implement a file archive with the DAL and a Smart Rollup](../tutorials/build-files-archive-with-dal).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For an example of sending larger amounts of data, see [Implement a file archive with the DAL and a Smart Rollup](../tutorials/build-files-archive-with-dal).
For an example sending larger amounts of data, see [Implement a file archive with the DAL and a Smart Rollup](../tutorials/build-files-archive-with-dal).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Involving" is more general than "sending," so I prefer "sending" here. It could be "For an example that sends larger amounts of data..." or "For an application that sends larger amounts of data, see..."


## Getting data from the DAL

Smart Rollups can use data from the DAL only after it has been attested by the bakers.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Smart Rollups can use data from the DAL only after it has been attested by the bakers.
Smart Rollups can use data from the DAL **only** after it has been attested by the bakers.

emphasize

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To put the highlighting on the most important part, I'm going to change this to "only after it has been attested by the bakers."

- The _proof_ certifies the length of the data to prevent malicious users from overloading the layer with data.

1. Users post the certificate to layer 1 via the Octez client.
1. When the certificate is confirmed in a block, layer 1 splits the data into shards and assigns those shards to bakers, who verify that the data is available.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The layer 1 doesn't split the data into shards, the DAL node does.

The layer 1 "only" assign shards ids to bakers

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed this to "When the certificate is confirmed in a block, the DAL splits the data into shards and layer 1 assigns those shards to bakers, who verify that the data is available."


1. Users post the certificate to layer 1 via the Octez client.
1. When the certificate is confirmed in a block, layer 1 splits the data into shards and assigns those shards to bakers, who verify that the data is available.
1. Bakers verify that the data is available and attest that the data is available in their usual block attestations to layer 1.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Baker verify that they are able to download the portion of shards they are assigned, and attest ...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to "Bakers verify that they are able to download the shards that they are assigned to and attest..."

docs/architecture/data-availability-layer.mdx Outdated Show resolved Hide resolved
1. When the certificate is confirmed in a block, layer 1 splits the data into shards and assigns those shards to bakers, who verify that the data is available.
1. Bakers verify that the data is available and attest that the data is available in their usual block attestations to layer 1.
They have a certain number of blocks to do so, known as the _attestation lag_, and if enough shards are not attested by the end of this period, the certificate is considered bogus and the related data is dropped.
1. Other DAL nodes get the data from the initial DAL node through the peer-to-peer network.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily from the "initial" DAL node. Any node that receives a data can forward it to other nodes which are interested in the same data.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to "DAL nodes share the data through the peer-to-peer network."

docs/architecture/data-availability-layer.mdx Outdated Show resolved Hide resolved
docs/architecture/data-availability-layer.mdx Outdated Show resolved Hide resolved
Clients can get information about the current DAL parameters from the RPC endpoint `GET /chains/main/blocks/head/context/constants` or the Smart Rollup kernel SDK function `reveal_dal_parameters`.
These parameters include:

- `number_of_slots`: The number of slots in each block
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an upper bound of the number of slots that could be published per block, but it's not clear from the text.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to "The maximum number of slots in each block."

- `number_of_slots`: The number of slots in each block
- `slot_size`: The size of each slot in bytes
- `page_size`: The size of each page in bytes
- `attestation_lag`: The number of subsequent blocks in which bakers can attest that the data is available; if enough attestations are available by the time this number of blocks have been created, the data becomes available to Smart Rollups
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, we attest at level published_level + attestation_lag currently.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to "The number of blocks after a certificate is published that bakers attest that the data is available; if enough attestations are available in this block, the data becomes available to Smart Rollups"

Comment on lines 93 to 94
"commitment": "sh1u3tr3YKPDY",
"commitment_proof": "8229c63b8e858d9a9"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe paste the full lines?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I abbreviated them because they were long and would mess up the layout. Maybe I could make it:

   {
     "commitment": "sh1u3tr3YKPDY...",
     "commitment_proof": "8229c63b8e858d9a9..."
   }

Comment on lines 110 to 119
## Getting data from the DAL

Smart Rollups can use data from the DAL only after it has been attested by the bakers.
Therefore, they cannot access DAL data in the current level, because not enough blocks have elapsed to allow bakers to attest the data.

The latest level that Smart Rollups can access is the current level minus the attestation lag.
They can access the data in that level with the Smart Rollup kernel SDK function `reveal_dal_page`, which accepts the level, slot, and page to receive, as in this example:

```rust
let param = host.reveal_dal_parameters();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could probably have a simpler piece of code (just an RPC) that retrieves the slot on a DAL node playing the role of a slots consumer

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is that the RPC endpoints GET /slot/pages/<commitment> and GET /shards/<commitment>? I can give those a try when I have time to run though the setup again.

@timothymcmackin timothymcmackin merged commit 7924bb2 into staging Feb 13, 2024
4 checks passed
@timothymcmackin timothymcmackin deleted the dal-overview branch February 13, 2024 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants