bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-12 05:07:12 +08:00
commit 2619a8c550
24 changed files with 239 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

56
.gitattributes vendored Normal file
View File

@@ -0,0 +1,56 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q6_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2-f16.gguf filter=lfs diff=lfs merge=lfs -text
Chocolatine-3B-Instruct-DPO-v1.2.imatrix filter=lfs diff=lfs merge=lfs -text

3
Chocolatine-3B-Instruct-DPO-v1.2-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:706541812f842dcaa5b1b0112c0e595e8e344413899f52f9b51f737523f9dc5e
size 1316395392

3
Chocolatine-3B-Instruct-DPO-v1.2-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fb3d43ec54258c70aa81f9e347946af89bbaa7e6705c5f774ac8d0bd6ad43476
size 1855600512

3
Chocolatine-3B-Instruct-DPO-v1.2-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:77fb46ee5b246a702ec8d24caf6d59e301fb573ba50c8eac0b5969dc114a2e63
size 1625175936

3
Chocolatine-3B-Instruct-DPO-v1.2-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:601e9b898c864d66b8684d1d9da616d7d7f22c78f848a35022736b35e92177e6
size 2059853184

3
Chocolatine-3B-Instruct-DPO-v1.2-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:874afa8e07c0ab87da9ee123719dd6e6d107c5c3b29bf6381ea9df9c79f39d67
size 1416204672

3
Chocolatine-3B-Instruct-DPO-v1.2-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3e546b0f4acfdaa49d1c8bc1b7e084fd83a35da915316d155f72d1fd2229747c
size 1512396672

3
Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:819ccecd813a51fe78ec72904e773a712a4fbe6160d037b0e21a5429681f7bb5
size 2087597952

3
Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:51fa995d84dae4016c2c5b6653d8f4c6b9658840257dc52636233bba31b1c84d
size 1955477376

3
Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aafd28c04c96a0c6c0cd52a7ee8ff925230bf325b6a84b7e08da9ab9f5c63055
size 1681799040

3
Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1ddde00e87e533842ce86235523eb093f7c772f785395caaa66cfc3a90de4654
size 2173785984

3
Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:44a1ab0df5d80d79e9ba7110b6667e927303383832cdc980b92c84d1875ab160
size 2466338688

3
Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e99653b1b6ee529f22c8c03842a7f298503277fe0c019c3b4b8787f002c399cc
size 2393232768

3
Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8e654bb8355f818baf44c9397ce9f6b790850e6b3d3a38fdffa3613eb6776e29
size 2188760448

3
Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6d3157a1849d53b4d82a8050de1aaafd66daab629bb52ee10e8bc744c5207831
size 2876069760

3
Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:246700871c0bb9089af1f8b04737d6c23038d38f6017aa52ee7607f209ad5fdb
size 2815276416

3
Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c872b157d10166ec03df2d63038a752e4681c1152f5d47246b167f67a1ca0f2a
size 2641474944

3
Chocolatine-3B-Instruct-DPO-v1.2-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2f1f4545570c14ce5d0ba8816e5a04fb1852c5ab33049bbe7fab0e2835d9f92f
size 3135853440

3
Chocolatine-3B-Instruct-DPO-v1.2-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e2783c4dd47d5427e7accbde45fd5743fbbfedf36049fbdde81a88f2083e2d22
size 3183564672

3
Chocolatine-3B-Instruct-DPO-v1.2-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dadfa1f0ed27c0da82e095fbce7d310db993de77c471cb551c5f89abcce1ea12
size 4061222784

3
Chocolatine-3B-Instruct-DPO-v1.2-f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5026d426afb3132e6a46dfcd45a5c882b3e62e250a32a26978a08d9045c0815a
size 7643297344

3
Chocolatine-3B-Instruct-DPO-v1.2.imatrix Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b4bf4778916044b00e2d038fabb770f71a09d0a00c33daad9d618102b02d5d70
size 2232616

119
README.md Normal file
View File

@@ -0,0 +1,119 @@
---
base_model: jpacifico/Chocolatine-3B-Instruct-DPO-v1.2
datasets:
- jpacifico/french-orca-dpo-pairs-revised
language:
- fr
- en
library_name: transformers
license: mit
pipeline_tag: text-generation
tags:
- french
- chocolatine
quantized_by: bartowski
---
## Llamacpp imatrix Quantizations of Chocolatine-3B-Instruct-DPO-v1.2
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3615">b3615</a> for quantization.
Original model: https://huggingface.co/jpacifico/Chocolatine-3B-Instruct-DPO-v1.2
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
Run them in [LM Studio](https://lmstudio.ai/)
## Prompt format
```
<|system|> {system_prompt}<|end|><|user|> {prompt}<|end|><|assistant|>
```
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [Chocolatine-3B-Instruct-DPO-v1.2-f16.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-f16.gguf) | f16 | 7.64GB | false | Full F16 weights. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q8_0.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q8_0.gguf) | Q8_0 | 4.06GB | false | Extremely high quality, generally unneeded but max available quant. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q6_K_L.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q6_K_L.gguf) | Q6_K_L | 3.18GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q6_K.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q6_K.gguf) | Q6_K | 3.14GB | false | Very high quality, near perfect, *recommended*. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_L.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_L.gguf) | Q5_K_L | 2.88GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_M.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_M.gguf) | Q5_K_M | 2.82GB | false | High quality, *recommended*. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_S.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q5_K_S.gguf) | Q5_K_S | 2.64GB | false | High quality, *recommended*. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_L.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_L.gguf) | Q4_K_L | 2.47GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_M.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_M.gguf) | Q4_K_M | 2.39GB | false | Good quality, default size for must use cases, *recommended*. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_S.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_S.gguf) | Q4_K_S | 2.19GB | false | Slightly lower quality with more space savings, *recommended*. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_XL.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_XL.gguf) | Q3_K_XL | 2.17GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_L.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_L.gguf) | Q3_K_L | 2.09GB | false | Lower quality but usable, good for low RAM availability. |
| [Chocolatine-3B-Instruct-DPO-v1.2-IQ4_XS.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-IQ4_XS.gguf) | IQ4_XS | 2.06GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_M.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_M.gguf) | Q3_K_M | 1.96GB | false | Low quality. |
| [Chocolatine-3B-Instruct-DPO-v1.2-IQ3_M.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-IQ3_M.gguf) | IQ3_M | 1.86GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_S.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q3_K_S.gguf) | Q3_K_S | 1.68GB | false | Low quality, not recommended. |
| [Chocolatine-3B-Instruct-DPO-v1.2-IQ3_XS.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-IQ3_XS.gguf) | IQ3_XS | 1.63GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q2_K_L.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q2_K_L.gguf) | Q2_K_L | 1.51GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [Chocolatine-3B-Instruct-DPO-v1.2-Q2_K.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-Q2_K.gguf) | Q2_K | 1.42GB | false | Very low quality but surprisingly usable. |
| [Chocolatine-3B-Instruct-DPO-v1.2-IQ2_M.gguf](https://huggingface.co/bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF/blob/main/Chocolatine-3B-Instruct-DPO-v1.2-IQ2_M.gguf) | IQ2_M | 1.32GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
Some say that this improves the quality, others don't notice any difference. If you use these models PLEASE COMMENT with your findings. I would like feedback that these are actually used and useful so I don't keep uploading quants no one is using.
Thanks!
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset
Thank you ZeroWw for the inspiration to experiment with embed/output
## Downloading using huggingface-cli
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF --include "Chocolatine-3B-Instruct-DPO-v1.2-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/Chocolatine-3B-Instruct-DPO-v1.2-GGUF --include "Chocolatine-3B-Instruct-DPO-v1.2-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (Chocolatine-3B-Instruct-DPO-v1.2-Q8_0) or download them all in place (./)
## Which file should I choose?
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU and Apple Metal, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have an AMD card double check if you're using the rocBLAS build or the Vulcan build. At the time of writing this, LM Studio has a preview with ROCm support, and other inference engines have specific builds for ROCm.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}