Support new Blob TX type and deployment in chunks

[arboleya]

Important

• Tasks are prioritized in order.
• It is possible to address each one in isolation with a different PR.
• Only the third and last one can close this issue; the rest should only mention it.

Tasks:

• 1) Change TX error message from invalid to unsupported (source)
• 2) Throw error prohibiting deployment of >100kb contracts (source)
• 3) Implement support for >100kb contracts
  • Support blob TX type (requirement for contract chunk deployments)
  • Deploy contract validation for contracts that exceed the max contract size (taken from consensus parameters)
  • Chunk and deploy large contracts via blob tx and store contract IDs
  • Generate loader contract
  • Call contract functions via the loader contract

---

Note

This is the rationale for the above.

Options analysis:

1. Keep everything as is for <=100k contracts; throws for bigger ones
   • ✅ Keeps browser compatibility
   • ✅ Doesn't block mainnet
   • ⚠️ >100kb contracts must be deployed via forc deploy
2. Implement new [chunk] deployment mechanism
   • ✅ Keeps browser compatibility
   • ⛔ Blocks mainnet
3. Use forc deploy under the hood
   • ⛔ Breaks browser compatibility
   • ⛔ Blocks mainnet

Conclusions:

1. Clear, immediate winner.
2. May still be considered.
3. Probably a no-go.

---

Related PRs:

• Blob tx and instructions fuel-vm#780
• Blob tx support and fuel-vm 0.56.0 fuel-core#1988
• feat: Implement ldc based automatic contract chunking with auto split and loader generation sway#6250

---

Specs:

• Blob tx and loading fuel-specs#589

---

[danielbate]

Inspiration can be taken from the forc implementation however the main problem for us is the generation of the loader contract. Right now forc is doing this via the compiler which we can't do in browser. This element is currently WIP but work is almost there for blob TX support and contract chunking/deployment.

[arboleya]

Can we use a pre-compiled .sw contract (the loader) to deploy using an array of ContractIDs via configurables?

I'm thinking of a single loader .sw contract that could be reused by TS/RS.

cc @segfault-magnet

[segfault-magnet]

If we compile with the compiler we'd have to provide two function selectors -- one to get to the method of the proxy contract, another to be consumed once we jump into the code loaded from blobs.

I think this needs to be handcrafted.

Even so we're not missing out on much -- even if we wrote it in sway it's going to be almost totally in asm statements anyway.

As to how to write the proxy contract we have two choices:
a) write it to accept any number of blob ids
b) hardcode the blob ids

a)
pros:

• compression sometime in the future (there was talk about having reusable scripts for contract calls so that they may be replaced with a flag of some sorts)
• we can write it once and store it somewhere to be shared
  cons:
• it's always going to have the same id so you'd need to change the deployment to include a mandatory salt. But that might not be expected by the user. If you don't make the salt derived from the blob ids (and their order) then every time the user deploys there will be a different contract id, which is not expected behavior normally.

b)
pros:
identical behavior as current contract deployments -- if you change the original contract, you'll change the blob ids and that will change the proxy contract and change its contract id.
cons:
not easily template-able since we're customizing it per each deployment

I'll try out b first and see how it goes.

[segfault-magnet]

Ok so approach b tested and it works.

Loader is 26 instructions:

pub fn loader_contract(blob_ids: &[[u8; 32]]) -> Vec<u8> {
    const BLOB_ID_SIZE: u16 = 32;
    let get_instructions = |num_of_instructions, num_of_blobs| {
        [
            // 0x12 is going to hold the total size of the contract
            op::move_(0x12, RegId::ZERO),
            // find the start of the hardcoded blob ids, which are located after the code ends
            op::move_(0x10, RegId::IS),
            // 0x10 to hold the address of the current blob id
            op::addi(0x10, 0x10, num_of_instructions * Instruction::SIZE as u16),
            // loop counter
            op::addi(0x13, RegId::ZERO, num_of_blobs),
            // LOOP starts here
            // 0x11 to hold the size of the current blob
            op::bsiz(0x11, 0x10),
            // update the total size of the contract
            op::add(0x12, 0x12, 0x11),
            // move on to the next blob
            op::addi(0x10, 0x10, BLOB_ID_SIZE),
            // decrement the loop counter
            op::subi(0x13, 0x13, 1),
            // Jump backwards 3 instructions if the counter has not reached 0
            op::jneb(0x13, RegId::ZERO, RegId::ZERO, 3),
            // move the stack pointer by the contract size since we need to write the contract on the stack
            op::cfe(0x12),
            // find the start of the hardcoded blob ids, which are located after the code ends
            op::move_(0x10, RegId::IS),
            // 0x10 to hold the address of the current blob id
            op::addi(0x10, 0x10, num_of_instructions * Instruction::SIZE as u16),
            // 0x12 is going to hold the total bytes loaded of the contract
            op::move_(0x12, RegId::ZERO),
            // loop counter
            op::addi(0x13, RegId::ZERO, num_of_blobs),
            // LOOP starts here
            // 0x11 to hold the size of the current blob
            op::bsiz(0x11, 0x10),
            // the location where to load the current blob (start of stack)
            op::move_(0x14, RegId::SSP),
            // move to where this blob should be loaded by adding the total bytes loaded
            op::add(0x14, 0x14, 0x12),
            // load the current blob
            op::bldd(0x14, 0x10, RegId::ZERO, 0x11),
            // update the total bytes loaded
            op::add(0x12, 0x12, 0x11),
            // move on to the next blob
            op::addi(0x10, 0x10, BLOB_ID_SIZE),
            // decrement the loop counter
            op::subi(0x13, 0x13, 1),
            // Jump backwards 6 instructions if the counter has not reached 0
            op::jneb(0x13, RegId::ZERO, RegId::ZERO, 6),
            // what follows is called _jmp_mem by the sway compiler
            op::move_(0x16, RegId::SSP),
            op::sub(0x16, 0x16, RegId::IS),
            op::divi(0x16, 0x16, 4),
            op::jmp(0x16),
        ]
    };

    let real_num_of_instructions = get_instructions(0, blob_ids.len() as u16).len() as u16;

    let instruction_bytes: Vec<u8> =
        get_instructions(real_num_of_instructions, blob_ids.len() as u16)
            .into_iter()
            .collect();

    let blob_bytes: Vec<u8> = blob_ids.iter().flatten().copied().collect();

    [instruction_bytes, blob_bytes].concat()
}

Basically copied what the Sway compiler does with configurables, they place them after the code.

We then figure out how big the whole contract is, expand the stack and fill it with the contract code and then jump to the first instruction.

I've deployed and successfully executed a 1.8MB contract 🥳

Gonna write back if there are any issues with this loader after I adjust all the e2e tests to deploy in chunks (temporarily).

e2e tests passing, revealed some issues with predicates but the core team fixed them quickly.

[danielbate]

Great work for the above @segfault-magnet!

It's worth noting that for us we can't use the BSIZ and BLDD opcodes until we've had a release of @fuels-vm/asm, which will be once this is merged.

[arboleya]

Just a heads up, this has been released in v0.56.0:

• Blob tx and instructions fuel-vm#780

[segfault-magnet]

I've changed the loader a bit to adapt to the changes made in the vm:

    fn loader_contract(blob_ids: &[BlobId]) -> Result<Vec<u8>> {
        const BLOB_ID_SIZE: u16 = 32;
        let get_instructions = |num_of_instructions, num_of_blobs| {
            // There are 2 main steps:
            // 1. Load the blob contents into memory
            // 2. Jump to the beginning of the memory where the blobs were loaded
            // After that the execution continues normally with the loaded contract reading our
            // prepared fn selector and jumps to the selected contract method.
            [
                // 1. load the blob contents into memory
                // find the start of the hardcoded blob ids, which are located after the code ends,
                op::move_(0x10, RegId::IS),
                // 0x10 to hold the address of the current blob id
                op::addi(0x10, 0x10, num_of_instructions * Instruction::SIZE as u16),
                // The contract is going to be loaded from the current value of SP onwards, save
                // the location into 0x16 so we can jump into it later on
                op::move_(0x16, RegId::SP),
                // loop counter
                op::movi(0x13, num_of_blobs),
                // LOOP starts here
                // 0x11 to hold the size of the current blob
                op::bsiz(0x11, 0x10),
                // push the blob contents onto the stack
                op::ldc(0x10, 0, 0x11, 1),
                // move on to the next blob
                op::addi(0x10, 0x10, BLOB_ID_SIZE),
                // decrement the loop counter
                op::subi(0x13, 0x13, 1),
                // Jump backwards (3+1) instructions if the counter has not reached 0
                op::jnzb(0x13, RegId::ZERO, 3),
                // 2. Jump into the memory where the contract is loaded
                // what follows is called _jmp_mem by the sway compiler
                // subtract the address contained in IS because jmp will add it back
                op::sub(0x16, 0x16, RegId::IS),
                // jmp will multiply by 4 so we need to divide to cancel that out
                op::divi(0x16, 0x16, 4),
                // jump to the start of the contract we loaded
                op::jmp(0x16),
            ]
        };

        let num_of_instructions = u16::try_from(get_instructions(0, 0).len())
            .expect("to never have more than u16::MAX instructions");

        let num_of_blobs = u32::try_from(blob_ids.len()).map_err(|_| {
            error!(
                Other,
                "the number of blobs ({}) exceeds the maximum number of blobs supported: {}",
                blob_ids.len(),
                u32::MAX
            )
        })?;

        let instruction_bytes = get_instructions(num_of_instructions, num_of_blobs)
            .into_iter()
            .flat_map(|instruction| instruction.to_bytes());

        let blob_bytes = blob_ids.iter().flatten().copied();

        Ok(instruction_bytes.chain(blob_bytes).collect())
    }