LLM.md

Introduction

@Author: Yingding Wang

this document comprises information of load, compressing and utilizing LLM, SLM.

4-bit quantization and QLoRA

• Introducing bitsandbytes with HuggingFace: https://huggingface.co/blog/4bit-transformers-bitsandbytes
• Important Info about using LLaMA2 with HuggingFace: https://huggingface.co/docs/transformers/model_doc/llama2

Gemma Model Family

• Blog post about working with Gemma Model Family using Huggingface Transformers https://huggingface.co/blog/gemma

Try models on HuggingChat

• https://huggingface.co/chat