Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/move multiturn data #84

Merged
merged 17 commits into from
Nov 6, 2023
Merged

Feature/move multiturn data #84

merged 17 commits into from
Nov 6, 2023

Conversation

ruiyiw
Copy link
Collaborator

@ruiyiw ruiyiw commented Nov 6, 2023

Closes #

📑 Description

✅ Checks

  • My pull request adheres to the code style of this project
  • My code requires changes to the documentation
  • I have updated the documentation as required
  • All the tests have passed
  • Branch name follows type/descript (e.g. feature/add-llm-agents)
  • Ready for code review

ℹ Additional Information

lwaekfjlk and others added 17 commits October 11, 2023 19:23
@lwaekfjlk
support qlora mistral training
638064c
@Jasonqi146
added deep speed to requirements
dd939f3
@Jasonqi146
temporary save for switching disk region
8937459
@Jasonqi146
added shuffle and access token
a208d44
@Jasonqi146
finished training pipeline; need to fix inference
7938f13
@Jasonqi146
finished training pipeline; need to fix inference
2e41e86
@Jasonqi146
fixed inference pipeline
042ad7d
@Jasonqi146
commiting to test deepspeed
7f7481f
@Jasonqi146
added featurere to remove seq longer than 2048
cd5fdeb
@Jasonqi146
try to merge
4adcdf8
@Jasonqi146
added data preprocessing
daa6c59
@lwaekfjlk
Merge branch 'main' into feature/llama2-13b-train
1a07153
@zqi2cmu
minor changes
929e2ab
@zqi2cmu
minor changes
5db80be
@Jasonqi146
Merge pull request #64 from sotopia-lab/feature/llama2-13b-train
703ff73 
support llama2 13b train and inference pipeline in fastchat
@ruiyiw
Move together data
5248438
@ruiyiw
rename data process files and add together multiturn data preprocess
5b5720c
@ruiyiw ruiyiw merged commit d2d9215 into main Nov 6, 2023
3 checks passed
lwaekfjlk added a commit that referenced this pull request Nov 16, 2023
@ruiyiw @lwaekfjlk
@Jasonqi146 @zqi2cmu
Feature/move multiturn data (#84)
84c8afa 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
lwaekfjlk pushed a commit that referenced this pull request Nov 17, 2023
@Michaelvll
Minor fix (#84)
9ca3023 
* fix train yaml

* update
lwaekfjlk added a commit that referenced this pull request Nov 17, 2023
@ruiyiw @lwaekfjlk
@Jasonqi146 @zqi2cmu
Feature/move multiturn data (#84)
616e8cf 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
@Jasonqi146 @zqi2cmu
Feature/move multiturn data (#84)
ad45be0 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
@Jasonqi146 @zqi2cmu
Feature/move multiturn data (#84)
f44a4c9 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/move multiturn data (#84)
eacdeb7 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/move multiturn data (#84)
827f1d4 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
@Jasonqi146 @zqi2cmu
Feature/move multiturn data (#84)
896854b 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
Signed-off-by: Haofei Yu <1125027232@qq.com>
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/move multiturn data (#84)
99b71d3 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/move multiturn data (#84)
8056fc1 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/move multiturn data (#84)
92cf1b5 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/move multiturn data (#84)
491c1b0 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/move multiturn data (#84)
c018e01 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
@Jasonqi146 @zqi2cmu
Feature/move multiturn data (#84)
fe6efc4 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
@Jasonqi146 @zqi2cmu
Feature/move multiturn data (#84)
37216f2 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
@Jasonqi146 @zqi2cmu
Feature/move multiturn data (#84)
6a72eb3 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
Signed-off-by: Haofei Yu <1125027232@qq.com>
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
Feature/move multiturn data (#84)
a5ff492 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
(cherry picked from commit 6f285ba)
lwaekfjlk pushed a commit that referenced this pull request Mar 14, 2024
@Michaelvll
Minor fix (#84)
33dcfdb 
* fix train yaml

* update
lwaekfjlk added a commit that referenced this pull request Mar 14, 2024
@ruiyiw @lwaekfjlk
@Jasonqi146 @zqi2cmu
Feature/move multiturn data (#84)
7e9e4fb 
* support qlora mistral training

* added deep speed to requirements

* temporary save for switching disk region

* added shuffle and access token

* finished training pipeline; need to fix inference

* finished training pipeline; need to fix inference

* fixed inference pipeline

* commiting to test deepspeed

* added featurere to remove seq longer than 2048

* try to merge

* minor changes

* minor changes

* Move together data

* rename data process files and add together multiturn data preprocess

---------

Co-authored-by: lwaekfjlk <1125027232@qq.com>
Co-authored-by: Jasonqi146 <jasonqi146@gmail.com>
Co-authored-by: zqi2cmu <zqi2@andrew.cmu.edu>
Co-authored-by: Wonderplex <50866817+Jasonqi146@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ruiyiw @lwaekfjlk @Jasonqi146 @zqi2cmu