Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fineweb10B/gpt2 recipe, and supporting changes #32

Merged
merged 8 commits into from
Oct 18, 2024
Merged

fineweb10B/gpt2 recipe, and supporting changes #32

merged 8 commits into from
Oct 18, 2024

Commits on Jun 14, 2024

  1. support adamw, cosine decay, weight decay config, new param_init_meth… 

    …od supporting normal
    @francoishernandez
    francoishernandez committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    33f3098 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:eole-nlp/eole into fineweb_stuff

    @francoishernandez
    francoishernandez committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    a45b1f4 View commit details
    Browse the repository at this point in the history

  3. add fineweb10B recipe, recipe config check test

    @francoishernandez
    francoishernandez committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    6d358f2 View commit details
    Browse the repository at this point in the history

  4. add parse_fineweb_10B.py, fix flake

    @francoishernandez
    francoishernandez committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    3ed8af6 View commit details
    Browse the repository at this point in the history

  5. warning if xavier_uniform and param_init != 0

    @francoishernandez
    francoishernandez committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    7b2d1ae View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. Merge branch 'main' into fineweb10B_recipe

    @francoishernandez
    francoishernandez authored Oct 18, 2024
    Configuration menu
    Copy the full SHA
    19482e8 View commit details
    Browse the repository at this point in the history

  2. Create README.md

    @francoishernandez
    francoishernandez authored Oct 18, 2024
    Configuration menu
    Copy the full SHA
    133778c View commit details
    Browse the repository at this point in the history

  3. fix recipes

    @francoishernandez
    francoishernandez committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    b34895f View commit details
    Browse the repository at this point in the history