Skip to content

Commit

Permalink
Update split block docs
Browse files Browse the repository at this point in the history
  • Loading branch information
calpt committed Oct 9, 2023
1 parent 6322035 commit 7770f43
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions docs/adapter_composition.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,10 +160,10 @@ In the example, `attention_scores` holds a dictionary of the following form:
Splitting the input between two adapters using the 'Split' block.
```

The `Split` block can be used to split an input sequence between two adapters.
This is done by specifying a split index, at which the sequences should be divided.
The `Split` block can be used to split an input sequence between multiple adapters.
This is done by specifying split indices at which the sequences should be divided.
In the following example, we split each input sequence between adapters `g` and `h`.
For each sequence, all tokens from 0 up to 63 are forwarded through `g` while all tokens beginning at index 64 are forwarded through `h`:
For each sequence, all tokens from 0 up to 63 are forwarded through `g` while the next 64 tokens are forwarded through `h`:

```python
import adapters.composition as ac
Expand All @@ -173,7 +173,7 @@ import adapters.composition as ac
model.add_adapter("g")
model.add_adapter("h")

model.active_adapters = ac.Split("g", "h", split_index=64)
model.active_adapters = ac.Split("g", "h", splits=[64, 64])
```

## `BatchSplit`
Expand Down Expand Up @@ -286,7 +286,7 @@ E.g., we can nest a `Split` block within a `Stack` of adapters:
```python
import adapters.composition as ac

model.active_adapters = ac.Stack("a", ac.Split("b", "c", split_index=60))
model.active_adapters = ac.Stack("a", ac.Split("b", "c", splits=60))
```

However, combinations of adapter composition blocks cannot be arbitrarily deep. All currently supported possibilities are visualized in the table below.
Expand Down

0 comments on commit 7770f43

Please sign in to comment.