Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model][WIP] feat: Add internvl2 model for training #611

Closed
wants to merge 23 commits into from

Conversation

yinfan98
Copy link

@yinfan98 yinfan98 commented Jul 17, 2024

受散步 @sanbuphy 和 百度飞桨(厦门)人工智能产业赋能中心 邀请来给PaddleMIX套件支持InternVL2-8B。
基本搞完了但没对精度所以先来WIP下,肝到天亮有点昏昏欲睡😪。
顺便列下TODO:

  • 定义模型结构
  • 下载模型
  • 权重转换成paddle格式并贡献转换脚本
  • 修正Tokenizer bug
  • 修正预处理逻辑
  • 修正模型前向代码
  • 测试dataset,dataloader
  • 跑起SFT,LoRA训练任务
  • loss 精度对齐。

预计在8月底之前搞完zzz

Copy link

paddle-bot bot commented Jul 17, 2024

Thanks for your contribution!

@sanbuphy
Copy link
Contributor

牛! 需要支持请说

@nemonameless
Copy link
Collaborator

谢谢贡献,如需要支持请随时提出

@nemonameless nemonameless marked this pull request as draft July 22, 2024 12:56
@nemonameless nemonameless mentioned this pull request Aug 1, 2024
@luotao1 luotao1 added the HappyOpenSource 快乐开源活动issue与PR label Aug 21, 2024
@luotao1 luotao1 closed this Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contributor HappyOpenSource 快乐开源活动issue与PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants