bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-18 12:34:06 +08:00
commit 86f270bf37
29 changed files with 291 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
.gitattributes vendored Normal file
View File

@@ -0,0 +1,47 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.gguf* filter=lfs diff=lfs merge=lfs -text
*.ggml filter=lfs diff=lfs merge=lfs -text
*.llamafile* filter=lfs diff=lfs merge=lfs -text
*.pt2 filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

165
README.md Normal file
View File

@@ -0,0 +1,165 @@
---
quantized_by: bartowski
pipeline_tag: text-generation
base_model_relation: quantized
base_model: nvidia/NVIDIA-Nemotron-Nano-12B-v2
---
## Llamacpp imatrix Quantizations of NVIDIA-Nemotron-Nano-12B-v2 by nvidia
Using <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b6317">b6317</a> for quantization.
Original model: https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-12B-v2
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8) combined with a subset of combined_all_small.parquet from Ed Addario [here](https://huggingface.co/datasets/eaddario/imatrix-calibration/blob/main/combined_all_small.parquet)
Run them in [LM Studio](https://lmstudio.ai/)
Run them directly with [llama.cpp](https://github.com/ggml-org/llama.cpp), or any other llama.cpp based project
## Prompt format
No prompt format found, check original model page
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [NVIDIA-Nemotron-Nano-12B-v2-bf16.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-bf16.gguf) | bf16 | 24.63GB | false | Full BF16 weights. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q8_0.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q8_0.gguf) | Q8_0 | 13.09GB | false | Extremely high quality, generally unneeded but max available quant. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q6_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q6_K_L.gguf) | Q6_K_L | 10.44GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q6_K.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q6_K.gguf) | Q6_K | 10.11GB | false | Very high quality, near perfect, *recommended*. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q5_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q5_K_L.gguf) | Q5_K_L | 9.18GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q5_K_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q5_K_M.gguf) | Q5_K_M | 8.76GB | false | High quality, *recommended*. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q5_K_S.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q5_K_S.gguf) | Q5_K_S | 8.57GB | false | High quality, *recommended*. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q4_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_K_L.gguf) | Q4_K_L | 7.99GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q4_1.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_1.gguf) | Q4_1 | 7.84GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q4_K_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_K_M.gguf) | Q4_K_M | 7.49GB | false | Good quality, default size for most use cases, *recommended*. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q4_K_S.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_K_S.gguf) | Q4_K_S | 7.21GB | false | Slightly lower quality with more space savings, *recommended*. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q4_0.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_0.gguf) | Q4_0 | 7.16GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
| [NVIDIA-Nemotron-Nano-12B-v2-IQ4_NL.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ4_NL.gguf) | IQ4_NL | 7.11GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q3_K_XL.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q3_K_XL.gguf) | Q3_K_XL | 6.96GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [NVIDIA-Nemotron-Nano-12B-v2-IQ4_XS.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ4_XS.gguf) | IQ4_XS | 6.75GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q3_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q3_K_L.gguf) | Q3_K_L | 6.37GB | false | Lower quality but usable, good for low RAM availability. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q3_K_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q3_K_M.gguf) | Q3_K_M | 6.02GB | false | Low quality. |
| [NVIDIA-Nemotron-Nano-12B-v2-IQ3_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ3_M.gguf) | IQ3_M | 5.69GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q3_K_S.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q3_K_S.gguf) | Q3_K_S | 5.57GB | false | Low quality, not recommended. |
| [NVIDIA-Nemotron-Nano-12B-v2-IQ3_XS.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ3_XS.gguf) | IQ3_XS | 5.46GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q2_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q2_K_L.gguf) | Q2_K_L | 5.36GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [NVIDIA-Nemotron-Nano-12B-v2-IQ3_XXS.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ3_XXS.gguf) | IQ3_XXS | 4.96GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
| [NVIDIA-Nemotron-Nano-12B-v2-Q2_K.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q2_K.gguf) | Q2_K | 4.70GB | false | Very low quality but surprisingly usable. |
| [NVIDIA-Nemotron-Nano-12B-v2-IQ2_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ2_M.gguf) | IQ2_M | 4.38GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
| [NVIDIA-Nemotron-Nano-12B-v2-IQ2_S.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ2_S.gguf) | IQ2_S | 4.07GB | false | Low quality, uses SOTA techniques to be usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF --include "nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/nvidia_NVIDIA-Nemotron-Nano-12B-v2-GGUF --include "nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q8_0) or download them all in place (./)
</details>
## ARM/AVX information
Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.
Now, however, there is something called "online repacking" for weights. details in [this PR](https://github.com/ggml-org/llama.cpp/pull/9921). If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.
As of llama.cpp build [b4282](https://github.com/ggml-org/llama.cpp/releases/tag/b4282) you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.
Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to [this PR](https://github.com/ggml-org/llama.cpp/pull/10541) which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.
<details>
<summary>Click to view Q4_0_X_X information (deprecated</summary>
I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.
<details>
<summary>Click to view benchmarks on an AVX2 system (EPYC7702)</summary>
| model | size | params | backend | threads | test | t/s | % (vs Q4_0) |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------------: | -------------------: |-------------: |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp512 | 204.03 ± 1.03 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp1024 | 282.92 ± 0.19 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp2048 | 259.49 ± 0.44 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg128 | 39.12 ± 0.27 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg256 | 39.31 ± 0.69 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg512 | 40.52 ± 0.03 | 100% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp512 | 301.02 ± 1.74 | 147% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp1024 | 287.23 ± 0.20 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp2048 | 262.77 ± 1.81 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg128 | 18.80 ± 0.99 | 48% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg256 | 24.46 ± 3.04 | 83% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg512 | 36.32 ± 3.59 | 90% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp512 | 271.71 ± 3.53 | 133% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp1024 | 279.86 ± 45.63 | 100% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp2048 | 320.77 ± 5.00 | 124% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg128 | 43.51 ± 0.05 | 111% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg256 | 43.35 ± 0.09 | 110% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg512 | 42.60 ± 0.31 | 105% |
Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation
</details>
</details>
## Which file should I choose?
<details>
<summary>Click here for details</summary>
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggml-org/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
</details>
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
Thank you to LM Studio for sponsoring my work.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b3c3256ee6c07aa9c492a3cc91b0ff66e6e5ea03ec0c1f1ea92fc85e0e4e2c1f
size 4380288512

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:87f7f13367713f86ef351e032eb38ecbb68575fcd8dd3085567b3e2728cb1ee8
size 4066452992

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e793b56a0425cec439f7115aa2f774fe1f072f4e1b062792ff5ad2b945d96b2e
size 5690394112

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:85e4b32a6120e87de1bd9224c87fbe9189e3d5fc48f9521d1bb878978170f0a8
size 5455775232

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:39353a4a8148e33d110f8c838a92984c6a88a102238a67fb8a6947bf21b1bbef
size 4960282112

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fff62128106b0225f3980fad65a8b62314f314b27dd6eacaf30366033e69b729
size 7113324032

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:68ee170c811de128ebfe22c4119b9634d5548541b4dad6215f4342abe3d4cee2
size 6749681152

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c214b78f436ef303afb93a2f43ccf7f7cf9bd641ef573e48d84176cecddb8239
size 4703360512

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8497bf83362de8afcf859dbf38e27a99b0943c223427704fbe6be443be11812a
size 5358720512

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d6c4a00a2373430f3dafec9eafeda32ed5636739d9adebb4aba4cd9293fc1d30
size 6373443072

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ce3f66b7ca156d3e5916b5fe456455a3c45a4f14f692416eb85cfdec11362f7b
size 6023480832

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8eafbb4c4d1c3203d40a6ea417a2034d0d9a3c06dbd7c0a9aa5d4efa29a4b188
size 5567841792

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:13de977515fc71c46d477dc2baa2251f92b97dd17dedbc16f2e82f9628c80503
size 6960645632

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:81965f7ef4d1f7de3ae37ae6c3dbfdfeeabd207b9fda1c322b77de620249ba71
size 7159199232

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:12a8285d430533fa87048d484afbccb6b705338804a545d60068b683ab69b3cf
size 7840609792

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:035b07bb44f88f39e1226cfb60b109e4e294b5e9b13c6e5cbb11c7d19163e412
size 7992571392

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1ad1588e11778deb39a597523a4985289ce93626ae67bb3d762797b7c22dd28f
size 7494497792

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:83c449bc73bd815213ca1048f8ab78f289b4a41e70618b4502fb7ae5eb962080
size 7207695872

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:13fa8cc4d5d0fff3b686bcd4d0901d48957e0cd576670a65c4105afa115d24ec
size 9178445312

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:53e8eb38bee318b6aa06a968f6f8c03cb960f84d2064646b80ad0e32a04e92ae
size 8764257792

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f15dad05f68eff63cfaaca51f0d95c8a3b80611bd883644f03ab2a15001be32b
size 8567895552

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:659661190d9f650ae4bba252de3a88ed0bbcb60ebb4e833cc6d1db39610416dc
size 10113377792

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2d5305c7a4630b5762aeafb15f14179a2bae9d0982da16bf3103f93ed2d52e59
size 10438436352

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9d4b383c8add38649530e94978fd5b5b72a7b908d263ba6467049886279ac33a
size 13094139392

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-bf16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f3254914615ac3632af4c9b0d596ded75e440869b2c06eba232ead72ffbfa223
size 24632571104

3
nvidia_NVIDIA-Nemotron-Nano-12B-v2-imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d3906eeeef6388e62b5e605000f8a50d629fa63eb7929aa173a149f548e7a561
size 5100000