DOC.md

Whisper

安装修改过的transformers代码

cd transformers
pip install -e .

Whipser的运行过程

1. 只调用一次WhisperEncoder
2. 使用greedy策略多次调用WhisperDecoder

• WhisperForConditionalGeneration.generate
  • GenerationMixin.generate
    • GenerationMixin._prepare_encoder_decoder_kwargs_for_generation
      • WhisperEncoder.forward
        • WhisperEncoderLayer.forward
          • WhisperAttention不带cache的一个很典型的qkv self attention
    • GenerationMixin.greedy_search
      • WhisperForConditionalGeneration.forward
        • WhisperModel.forward
          • WhisperDecoder.forward
            • WhisperDecoderLayer.forward
              • WhisperAttention self/cross attn + w/wo cache