Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Parallel_Attention_Blocks (3 of 3) #457

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Add Parallel_Attention_Blocks (3 of 3) #457

wants to merge 8 commits into from

Commits on Aug 27, 2023

  1. add parallel_attention.py

    @lessw2020
    lessw2020 committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    ec9d728 View commit details
    Browse the repository at this point in the history

  2. add first unit tests for parallel_attention_blocks

    @lessw2020
    lessw2020 committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    615225d View commit details
    Browse the repository at this point in the history

  3. confirm num Q heads matches num_heads

    @lessw2020
    lessw2020 committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    290ce85 View commit details
    Browse the repository at this point in the history

  4. extend unit tests to check output numerics and attn_output shape

    @lessw2020
    lessw2020 committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    7972d4d View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2023

  1. fix typing for missing function returns

    @lessw2020
    lessw2020 committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    5d83d48 View commit details
    Browse the repository at this point in the history

  2. add typing for inline function kv_expansion params

    @lessw2020
    lessw2020 committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    2f8e6ff View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2023

  1. remove .shape from item assert attribute

    @lessw2020
    lessw2020 committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    8b0608b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into parallel_attn_blocks_merge

    @lessw2020
    lessw2020 authored Aug 29, 2023
    Configuration menu
    Copy the full SHA
    2156f2f View commit details
    Browse the repository at this point in the history