Merge pull request #507 from ymcui/33b

Add Chinese LLaMA and Alpaca 33B version
ymcui · Jun 8, 2023 · 5e7a50e · 5e7a50e
2 parents 7bc1f3d + a016d7e
commit 5e7a50e
Show file tree

Hide file tree

Showing 33 changed files with 1,178 additions and 98 deletions.
diff --git a/README.md b/README.md
diff --git a/README_EN.md b/README_EN.md
diff --git a/SHA256.md b/SHA256.md
@@ -4,7 +4,7 @@
 
 **To ensure the completeness of the model, please check the folllowing SHA256 before using them.**
 
-### Original LLaMA (by MetaAI)
+### Original LLaMA (by Meta AI)
 
 #### consolidated.*.pth format (original)
 
@@ -60,18 +60,20 @@ The followings are SHA256 values for `adapter_model.bin`  files.
 | Chinese-LLaMA-Plus-7B          | 8c928db86b2a0cf73f019832f921eb7e1e069ca21441b4bfa12c4381c6cc46be |
 | Chinese-LLaMA-13B              | 6a4ce789d219bde122f8d9a20371937f2aa2ee86a2311d9f5e303df2e774f9fc |
 | Chinese-LLaMA-Plus-13B         | 784fcff9c4bdf4e77d442a01158e121caf8fcce0f97ffb32396fe7a3617ee7e8 |
-| Chinese-LLaMA-33B              |                                                              |
+| Chinese-LLaMA-33B              | 93a449bafb71ff1bb74a4a21e64e102e5078e5c3898eb40d013790072a0fa3de |
 | Chinese-Alpaca-7B              | 0d9b6ed8e4a7d1ae590a16c89a452a488d66ff07e45487972f61c2b6e46e36de |
 | Chinese-Alpaca-Plus-7B         | 4ee0bf805c312a9a771624d481fbdb4485e1b0a70cd2a8da9f96137f177b795d |
 | Chinese-Alpaca-13B             | cb8dda3c005f3343a0740dcd7237fbb600cb14b6bff9b6f3d488c086a2f08ada |
 | Chinese-Alpaca-Plus-13B        | a1fcdcb6d7e1068f925fb36ec78632c76058ba12ba352bed4d44060b8e6f4706 |
-| Chinese-Alpaca-33B             |                                                              |
+| Chinese-Alpaca-33B             | 6b39da4c682e715a9de30b247b7e9b812d2d54f7d320ec9b452000a5cd4d178d |
 
 
 ### Merged files (consolidated.*.pth)
 
 下表展示了合并LoRA权重后的全量模型权重（PyTorch版）的SHA256。PyTorch版本不影响实际权重数据，但影响meta信息，所以SHA256也会不同。**建议合并模型时使用PyTorch >= 1.13.0版本，以确保以下SHA256有参考性。**
 
+⚠️ 请优先确保合并前的基模型和LoRA权重的SHA256是否与上述表中所述值一致。
+
 The followings are SHA256 values for merged files (`consolidated.*.pth`). Note that the version of PyTorch does not affect actual weights but meta informations are slightly different. **Please check SHA256 according to PyTorch version >= 1.13.0.**
 
 | Model                   | SHA256 (PyTorch >= 1.13.0)                                   |
@@ -80,12 +82,12 @@ The followings are SHA256 values for merged files (`consolidated.*.pth`). Note t
 | Chinese-LLaMA-Plus-7B   | f8d380d63f77a08b7f447f5ec63f0bb1cde9ddeae2207e9f86e6b5f0f95a7955 |
 | Chinese-LLaMA-13B       | aa7f4599487ea2b0d0aca2b522c39370897f9afd9839aac7d02155957f1f019f<br/>3954f3e7f7264994f23800a04423e6563cc1959ac699d9eaaa6801b4f9392ebd |
 | Chinese-LLaMA-Plus-13B  | 4de7d188003c778f216342de2dc5c9a9c74278c701c63a7b6bcd7957f5ebfdf5<br/>ff8046f9eb8b05dd86597c21edd07894aec00b31842a4c11996a4003091ea7c9 |
-| Chinese-LLaMA-33B       |                                                              |
+| Chinese-LLaMA-33B       | 054e9b7dffa3b92a053ca32acac6e22b27c184ed2b8563f8e44e6570ba416357<br/>a0fe86c45a0819f45a509776d82778b7de75fbff8d37afa97159b24de5448b7b<br/>13df5f74dc7bc1204076b1febef818fb3cec978de27bf8fc85c70e7d62282df9<br/>f4f28106c343c5804613faa9852f29fbc60764366bcb0d37ef2811a17be2d336 |
 | Chinese-Alpaca-7B       | fbfccc91183169842aac8d093379f0a449b5a26c5ee7a298baf0d556f1499b90 |
 | Chinese-Alpaca-Plus-7B  | 8b8f6551d0d83f93e378622b9f8dad0bec189da6c29d8a78de493e6aee9bd35f |
 | Chinese-Alpaca-13B      | 30cefb5be9091c3e17fbba5d91bf16266a2ddf86cde53370a9982b232ff8a2f4<br/>ce946742b0f122f472e192c3f77d506e0c26578b4b881d07d919553333affecd |
 | Chinese-Alpaca-Plus-13B | 1834558214c1dddc0d8b2826ece086908b9d2293241d0e12cecb48a035ec561b<br/>bf70001600ce166f6ca4ef59df5510f0582cdc119fb74e27d9cf3e4c7b142015 |
-| Chinese-Alpaca-33B      |                                                              |
+| Chinese-Alpaca-33B      | 72bfe67481c0df1b8c3b536acd15ac42c1163b0727b1beb6409ee31d14cb2490<br/>fd2151ea714a6e0706a60cca5ab7abf8558e665d4cb001481c6df616c0821c16<br/>4a7e3de6881769f9c2413f0867e67da20efdf4502602ab90483cb99c593e51ed<br/>99c81a7a310802dcc579fe96288fbc18d4486f92020eaf925e1c33db8311378a |
 
 
 ### How To Check SHA256
@@ -107,3 +109,4 @@ In Windows,
 ```
 > certutil -hashfile your-model-file sha256
 ```
+
diff --git a/data/README.md b/data/README.md
@@ -0,0 +1,13 @@
+# 数据 Data
+
+###  alpaca_data_zh_51k.json
+
+中文Alpaca数据，包含51k个从ChatGPT (gpt-3.5-turbo)爬取的指令数据。
+
+Chinese Alpaca dataset, containing 51k instruction data crawled from ChatGPT (gpt-3.5-turbo).
+
+###  pt_sample_data.txt
+
+CLM任务预训练样例数据
+
+Pre-training sample data
diff --git a/examples/README.md b/examples/README.md
@@ -1,4 +1,4 @@
-# 效果对比
+# 效果对比 Performance Comparison
 
 以下分数应视为paired score，也就是说分数是一个相对值，而不是绝对值，是多个系统相比较得到的结果。详细结果见相应目录。
 
@@ -18,4 +18,10 @@
 
 |               | 样例数 | Alpaca-13B | Alpaca-Plus-7B | Alpaca-Plus-13B |
 | ------------- | :----: | :--------: | :------------: | :-------------: |
-| **💯总平均分** |  200   |    74.3    |      78.2      |   **👍🏻80.8**    |
+| **💯总平均分** |  200   |    74.3    |      78.2      |   **👍🏻80.8**    |
+
+### f16_p7b-p13b-33b：对比了8-bit量化版的Alpaca-Plus-7B、Plus-13B和F16的33B
+
+|               | 样例数 | Alpaca-Plus-7B | Alpaca-Plus-13B | Alpaca-33B |
+| ------------- | :----: | :------------: | :-------------: | :--------: |
+| **💯总平均分** |  200   |      75.3      |      79.4       | **👍🏻82.0** |