Examples WMT: What is need to shift decoder inputs to the right?

[svarunid]

I've been going through the flax transformer example noticed the the inputs to the decoder are shifted to right while also applying a causal mask. Is there a reason for that?

Code from transformer example

class Decoder:
    config: TransformerConfig

    def __call__(...):
        ...
        if not config.decode:
            y = shift_right(y)
        ...

class Transformer:
    config: TransformerConfig

    def decode(...):
        ...
        decoder_mask = nn.combine_masks(
            nn.make_attention_mask(targets > 0, targets > 0, dtype=config.dtype),
            nn.make_causal_mask(targets, dtype=config.dtype),
        )
        ...