Skip to content

Commit

Permalink
Clarify DAL file tutorial (#312)
Browse files Browse the repository at this point in the history
* link to Smart Rollup page

* Clarify how to use the script

* spell out the deployment steps

* Use the same case as the constant

* troubleshooting section

* result of publishing

* Clarify

* Clarify that you don't have to run nodes

* Hard-code diagram

* hard-code file and move MDX file to MD

* Split environment setup to a separate page

* remove info about the reveal data channel

* Move to MD file and hard-code diagram

* clarify slots, commitments, and pages

* correct path to setup page

* these can be MD files

* clarify what you do and don't set up in the tutorial

* forgot to mention the node

* tutorial diagram
  • Loading branch information
timothymcmackin authored Feb 13, 2024
1 parent 8af3a02 commit 588eb9c
Show file tree
Hide file tree
Showing 11 changed files with 323 additions and 227 deletions.
104 changes: 104 additions & 0 deletions docs/tutorials/build-files-archive-with-dal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
---
title: Implement a file archive with the DAL and a Smart Rollup
authors: 'Tezos Core Developers'
last_update:
date: 9 February 2024
---

:::note Experimental
The Data Availability Layer is an experimental feature that is not yet available on Tezos Mainnet.
The way the DAL works may change significantly before it is generally available.
:::

The Data Availability Layer (DAL) is a companion peer-to-peer network for the Tezos blockchain, designed to provide additional data bandwidth to Smart Rollups.
It allows users to share large amounts of data in a way that is decentralized and permissionless, because anyone can join the network and post and read data on it.

In this tutorial, you will set up a file archive that stores and retrieves files with the DAL.
You will learn:

- How data is organized and shared with the DAL and the reveal data channel
- How to read data from the DAL in a Smart Rollup
- How to host a DAL node
- How to publish data and files with the DAL

Because the DAL is not yet available on Tezos Mainnet, this tutorial uses the [Weeklynet test network](https://teztnets.com/weeklynet-about), which runs on a newer version of the protocol that includes the DAL.
Weeklynet runs just like other Tezos networks like Mainnet and Ghostnet, with its own nodes, bakers, and accusers, so you don't need to run your own nodes and bakers.

See these links for more information about the DAL:

- For technical information about how the DAL works, see [Data Availability Layer](https://tezos.gitlab.io/shell/dal.html) in the Octez documentation.
- For more information about the approach for the DAL, see [The Rollup Booster: A Data-Availability Layer for Tezos](https://research-development.nomadic-labs.com/data-availability-layer-tezos.html).

## Tutorial applications

In this tutorial, you set up these components:

- The Octez client, which you use to manage a local wallet, deploy a Smart Rollup, and send data to the DAL
- A Data Availability Layer node (not to be confused with a layer 1 node), which stores data temporarily and distributes it to Smart Rollups
- A Smart Rollup that listens for data published to the DAL, retrieves it from the DAL node, and stores it locally
- A Smart Rollup node that runs your Smart Rollup

For simplicity, you do not set up a layer 1 node or a baker, which are responsible for verifying that the data is available before Smart Rollups can access it.
Instead, you use the existing nodes and bakers that are running on Weeklynet.

## Tutorial diagram

Here is a diagram that shows the components that you set up in this tutorial in a light blue background:

![A diagram of the DAL file tutorial, highlighting the Octez client, DAL node, and Smart Rollup that you create with a light blue background to distinguish them from the existing DAL nodes, layer 1 nodes, and bakers](/img/tutorials/dal-file-tutorial-setup.png)

<!-- https://lucid.app/lucidchart/58f5577e-91b5-4237-89c4-a8cdf81c71ad/edit -->

## Prerequisites

This article assumes some familiarity with Smart Rollups.
If you are new to Smart Rollups, see the tutorial [Deploy a Smart Rollup](./smart-rollup).

## Why the DAL?

The DAL has earned the nickname "Rollup Booster" from its ability to address
the last bottleneck Smart Rollups developers could not overcome without
sacrificing decentralization: block space. Smart Rollups offload
*computation* from layer 1, but the transactions that they process still need to
originate from somewhere.

By default, that "somewhere" is the layer 1 blocks, yet the size of a Tezos
block is limited to around 500KBytes. In this model, while Smart Rollups do not
compete for layer 1 gas anymore, they still compete for block space.

The DAL allows third parties to publish data and have bakers attest that the data is available.
When enough bakers have attested that the data is available, Smart Rollups can retrieve the data without the need for additional trusted third-parties.

## How the DAL works

In this tutorial, you create a file archive application that allows clients to upload data to the DAL.
You also create a Smart Rollup that listens to the DAL and responds to that data.

The DAL works like this:

1. Users post data to a DAL node.
1. The DAL node returns a certificate.
This certificate includes a commitment that the data is available and a proof of the data.
1. Users post the certificate to layer 1 via the Octez client, which is much cheaper than posting the complete data.
1. When the certificate is confirmed in a block, layer 1 splits the data into shards and assigns those shards to bakers, who verify that the data is available.
1. Bakers verify that the data is available and attest that the data is available in their usual block attestations to layer 1.
They have a certain number of blocks to do so, known as the _attestation lag_, and if they don't by the end of this period, the certificate is considered bogus and the related data is dropped.
1. Other DAL nodes get the data from the initial DAL node through the peer-to-peer network.
1. The Smart Rollup node monitors the blocks and when it sees attested DAL data, it connects to a DAL node to request the data.
1. The Smart Rollup node stores the data in its durable storage, addressed by its hash.
Smart Rollups must store the data because it is available on the DAL for only a short time.
1. Users who know the hash of the data can download it from the Smart Rollup node.

The overall workflow is summarized in the following figure:

![Overall diagram of the workflow of the Data Availability Layer](/img/architecture/dal-workflow.png)
<!-- https://lucid.app/lucidchart/cc422278-7319-4a2f-858a-a7b72e1ea3a6/edit -->

There are many steps in the DAL process, but the most complicated parts (storing and sharing data) are handled automatically by the various daemons in the Octez suite.

:::note
When you install a Smart Rollup, you provide only the installer kernel on layer 1 and the full kernel via the reveal data channel.
Currently, you cannot send the full kernel data over the Data Availability Layer, so this tutorial relies on the reveal data channel to install the kernel as usual.
:::

When you are ready, get started by going to [Part 1: Setting up an environment](./build-files-archive-with-dal/set-up-environment).
207 changes: 0 additions & 207 deletions docs/tutorials/build-files-archive-with-dal.mdx

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
---
title: "Part 1: Getting the DAL parameters"
title: "Part 2: Getting the DAL parameters"
authors: 'Tezos Core Developers'
last_update:
date: 17 January 2024
date: 9 February 2024
---

import LucidDiagram from '@site/src/components/LucidDiagram';

The Data Availability Layer stores information about the available data in layer 1 blocks.
Each block has several byte-vectors called _slots_, each with a maximum size.
DAL users can add information about the available data as _pages_ in these slots, as shown in this figure:
DAL users can add information about the available data as a _commitment_ in a slot.
These commitments refer to the data that is stored on the DAL, which stores the data in _pages_ as shown in this diagram:

<LucidDiagram width="640px" height="240px" src="https://lucid.app/documents/embedded/46fa8412-8443-4491-82f6-305aafaf85f2" id="Hxs62lrO0C4d" />
![Two example blocks with different DAL slots in use in each](/img/architecture/dal-slots-in-blocks.png)
<!-- https://lucid.app/lucidchart/46fa8412-8443-4491-82f6-305aafaf85f2/edit -->

The data in a slot is broken into pages to ensure that each piece of data can fit in a single Tezos operation.
The data is broken into pages to ensure that each piece of data can fit in a single Tezos operation.
This data must fit in a single operation to allow the Smart Rollup refutation game to work, in which every execution step of the Smart Rollup must be provable to layer 1.
{/* TODO link to Smart Rollup topic for more info on the refutation game */}
For more information about Smart Rollups, see [Smart Rollups](../../architecture/smart-rollups).

When clients add data, they must specify which slot to add it to.
Note that because the DAL is permissionless, clients may try to add data to the same slot in the same block.
Expand All @@ -31,7 +31,7 @@ In these steps, you set up a simple Smart Rollup to get the current DAL paramete

## Prerequisites

Before you begin, make sure that you have installed the prerequisites and set up an environment and an account as described in [Implement a file archive with the DAL and a Smart Rollup](../build-files-archive-with-dal).
Before you begin, make sure that you have installed the prerequisites and set up an environment and an account as described in [Part 1: Setting up an environment](./set-up-environment).

## Fetching the DAL parameters in a kernel

Expand Down Expand Up @@ -143,8 +143,7 @@ These parameters are:
## Setting up a deployment script

In later parts of this tutorial, you will update and redeploy the Smart Rollup multiple times.
To simplify the process, you can use this script.
To use it, pass the alias of your account in the Octez client:
To simplify the process, you can use this script:

```bash
#!/usr/bin/bash
Expand All @@ -171,6 +170,13 @@ octez-smart-rollup-node --endpoint ${ENDPOINT} \
--dal-node http://localhost:10732 --log-kernel-debug
```

To use it, save it in a file with an `sh` extension, such as `deploy_smart_rollup.sh` and give it executable permission.
Then you can run it any tme you update the `lib.rs` or `Cargo.toml` files to deploy a new Smart Rollup by passing your account alias, as in this example:

```bash
./deploy_smart_rollup.sh $MY_ACCOUNT
```

If you run this script and see an error that says that the file was not found, update the first line of the script (the shebang) to the path to your shell interpreter.
For example, if you are using the Tezos Docker image, the path is `/bin/sh`.

Expand Down
Loading

0 comments on commit 588eb9c

Please sign in to comment.