Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[wip] Entmax loss #3

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open

[wip] Entmax loss #3

wants to merge 26 commits into from

Commits on Jan 9, 2024

  1. add entmax loss training for gpt-like models

    @bpopeters
    bpopeters committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    8543c0e View commit details
    Browse the repository at this point in the history

  2. add entmax gpt example for debugging

    @bpopeters
    bpopeters committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    d3be645 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2024

  1. fix loss indentation problem that breaks entmax training

    @bpopeters
    bpopeters committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    f946b8f View commit details
    Browse the repository at this point in the history

  2. remove nonfunctional, unused, and unnecessary shell script

    @bpopeters
    bpopeters committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    1c76bf8 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. compute support during training (although do not log it yet)

    @bpopeters
    bpopeters committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    563f968 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. fix model training bug, accumulate support across gpus

    @bpopeters
    bpopeters committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    7228a8b View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. slightly more informative support logging

    @bpopeters
    bpopeters committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    202caed View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. update return_support to return_support_size in all cases

    @bpopeters
    bpopeters committed May 20, 2024
    Configuration menu
    Copy the full SHA
    8a83cf0 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. [wip] compute more sparsity stats

    @bpopeters
    bpopeters committed May 22, 2024
    Configuration menu
    Copy the full SHA
    48a1781 View commit details
    Browse the repository at this point in the history

  2. add more support logging statistics

    @bpopeters
    bpopeters committed May 22, 2024
    Configuration menu
    Copy the full SHA
    19b478f View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. add multiple loss functions (with some print statements to make sure … 

    …my assumptions of dimensions are correct)
    @bpopeters
    bpopeters committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    c4cf60c View commit details
    Browse the repository at this point in the history

  2. add force_decoded_accuracy as a zero-shot evaluation metric

    @bpopeters
    bpopeters committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    a02cf72 View commit details
    Browse the repository at this point in the history

  3. update logging with teacher forced accuracy

    @bpopeters
    bpopeters committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    a8733c4 View commit details
    Browse the repository at this point in the history

  4. update xavier_uniform init (which will probably be unused

    @bpopeters
    bpopeters committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    e51656d View commit details
    Browse the repository at this point in the history

  5. avoid unpacking error when labels is None

    @bpopeters
    bpopeters committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    055f869 View commit details
    Browse the repository at this point in the history

  6. remove print statements for eval computation

    @bpopeters
    bpopeters committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    e24ebc6 View commit details
    Browse the repository at this point in the history

  7. call item() because json cannot handle tensors

    @bpopeters
    bpopeters committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    f77cf0c View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. add accuracy at k

    @bpopeters
    bpopeters committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    68a4a1b View commit details
    Browse the repository at this point in the history

  2. fix guardrail for new task

    @bpopeters
    bpopeters committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    24a5ed3 View commit details
    Browse the repository at this point in the history

  3. small change to try to get LAMBADA to run

    @bpopeters
    bpopeters committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    77fe108 View commit details
    Browse the repository at this point in the history

  4. don't load optimizer state for lambada

    @bpopeters
    bpopeters committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    c41f8b0 View commit details
    Browse the repository at this point in the history

  5. write lambada to results file

    @bpopeters
    bpopeters committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    4cfc133 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. fix entmax_bisect_loss return type

    @bpopeters
    bpopeters committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    0182532 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. refactor zeroshot gpt evaluation for sparsemax score

    @bpopeters
    bpopeters committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    9e7f53d View commit details
    Browse the repository at this point in the history

  2. add missing parentheses

    @bpopeters
    bpopeters committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    f3c66c4 View commit details
    Browse the repository at this point in the history

  3. get item from sparsemax score tensor

    @bpopeters
    bpopeters committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    4f2c4fc View commit details
    Browse the repository at this point in the history