Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

why pre-processed 4 bit model on huggingface is larger than normal 4 bit model ? and what about Qwen2.5,only saw Qwen2 #30

Open
chuangzhidan opened this issue Dec 30, 2024 · 0 comments

Comments

@chuangzhidan
Copy link

chuangzhidan commented Dec 30, 2024

couple of things i am wondering:
1.it is generally 4G larger in terms of disk usage for a 72b-sized model,even without considering pissa init folder size. nou sure why

2.i can just funtune this model directly on a pre-processed 4 bit model and saved chekpoint will also be a 4 bit model ,yes?

3.last thing ,do u have pre-processed Qwen2.5 series models ? only saw Qwen2 on huggingface,not sure how much GPU i need to process a large 72b sized model

thanks ,for u attention on this matter

@chuangzhidan chuangzhidan changed the title why pre-processed 4 bit model is larger than normal 4 bit model ? and what about Qwen2.5,only saw Qwen2 why pre-processed 4 bit model on huggingface is larger than normal 4 bit model ? and what about Qwen2.5,only saw Qwen2 Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant