bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-11 22:55:06 +08:00
commit 105523fb68
29 changed files with 291 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
.gitattributes vendored Normal file
View File

@@ -0,0 +1,47 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.gguf* filter=lfs diff=lfs merge=lfs -text
*.ggml filter=lfs diff=lfs merge=lfs -text
*.llamafile* filter=lfs diff=lfs merge=lfs -text
*.pt2 filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

165
README.md Normal file
View File

@@ -0,0 +1,165 @@
---
quantized_by: bartowski
pipeline_tag: text-generation
base_model_relation: quantized
base_model: nvidia/NVIDIA-Nemotron-Nano-9B-v2
---
## Llamacpp imatrix Quantizations of NVIDIA-Nemotron-Nano-9B-v2 by nvidia
Using <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b6317">b6317</a> for quantization.
Original model: https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-9B-v2
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8) combined with a subset of combined_all_small.parquet from Ed Addario [here](https://huggingface.co/datasets/eaddario/imatrix-calibration/blob/main/combined_all_small.parquet)
Run them in [LM Studio](https://lmstudio.ai/)
Run them directly with [llama.cpp](https://github.com/ggml-org/llama.cpp), or any other llama.cpp based project
## Prompt format
No prompt format found, check original model page
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [NVIDIA-Nemotron-Nano-9B-v2-bf16.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-bf16.gguf) | bf16 | 17.79GB | false | Full BF16 weights. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q8_0.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q8_0.gguf) | Q8_0 | 9.46GB | false | Extremely high quality, generally unneeded but max available quant. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q6_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q6_K_L.gguf) | Q6_K_L | 9.14GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q6_K.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q6_K.gguf) | Q6_K | 9.14GB | false | Very high quality, near perfect, *recommended*. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q5_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q5_K_L.gguf) | Q5_K_L | 7.25GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q5_K_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q5_K_M.gguf) | Q5_K_M | 7.07GB | false | High quality, *recommended*. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q5_K_S.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q5_K_S.gguf) | Q5_K_S | 6.78GB | false | High quality, *recommended*. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q4_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_K_L.gguf) | Q4_K_L | 6.75GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q4_K_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_K_M.gguf) | Q4_K_M | 6.53GB | false | Good quality, default size for most use cases, *recommended*. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q4_K_S.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_K_S.gguf) | Q4_K_S | 6.21GB | false | Slightly lower quality with more space savings, *recommended*. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q4_1.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_1.gguf) | Q4_1 | 5.83GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q3_K_XL.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q3_K_XL.gguf) | Q3_K_XL | 5.78GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q3_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q3_K_L.gguf) | Q3_K_L | 5.49GB | false | Lower quality but usable, good for low RAM availability. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q3_K_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q3_K_M.gguf) | Q3_K_M | 5.38GB | false | Low quality. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q4_0.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_0.gguf) | Q4_0 | 5.34GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
| [NVIDIA-Nemotron-Nano-9B-v2-IQ4_NL.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ4_NL.gguf) | IQ4_NL | 5.31GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q2_K_L.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q2_K_L.gguf) | Q2_K_L | 5.30GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [NVIDIA-Nemotron-Nano-9B-v2-IQ4_XS.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ4_XS.gguf) | IQ4_XS | 5.27GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [NVIDIA-Nemotron-Nano-9B-v2-IQ3_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ3_M.gguf) | IQ3_M | 5.21GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q3_K_S.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q3_K_S.gguf) | Q3_K_S | 5.13GB | false | Low quality, not recommended. |
| [NVIDIA-Nemotron-Nano-9B-v2-IQ3_XS.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ3_XS.gguf) | IQ3_XS | 5.13GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [NVIDIA-Nemotron-Nano-9B-v2-IQ3_XXS.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ3_XXS.gguf) | IQ3_XXS | 5.07GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
| [NVIDIA-Nemotron-Nano-9B-v2-Q2_K.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q2_K.gguf) | Q2_K | 5.01GB | false | Very low quality but surprisingly usable. |
| [NVIDIA-Nemotron-Nano-9B-v2-IQ2_M.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ2_M.gguf) | IQ2_M | 5.00GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
| [NVIDIA-Nemotron-Nano-9B-v2-IQ2_S.gguf](https://huggingface.co/bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF/blob/main/nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ2_S.gguf) | IQ2_S | 4.96GB | false | Low quality, uses SOTA techniques to be usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF --include "nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/nvidia_NVIDIA-Nemotron-Nano-9B-v2-GGUF --include "nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q8_0) or download them all in place (./)
</details>
## ARM/AVX information
Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.
Now, however, there is something called "online repacking" for weights. details in [this PR](https://github.com/ggml-org/llama.cpp/pull/9921). If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.
As of llama.cpp build [b4282](https://github.com/ggml-org/llama.cpp/releases/tag/b4282) you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.
Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to [this PR](https://github.com/ggml-org/llama.cpp/pull/10541) which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.
<details>
<summary>Click to view Q4_0_X_X information (deprecated</summary>
I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.
<details>
<summary>Click to view benchmarks on an AVX2 system (EPYC7702)</summary>
| model | size | params | backend | threads | test | t/s | % (vs Q4_0) |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------------: | -------------------: |-------------: |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp512 | 204.03 ± 1.03 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp1024 | 282.92 ± 0.19 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp2048 | 259.49 ± 0.44 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg128 | 39.12 ± 0.27 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg256 | 39.31 ± 0.69 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg512 | 40.52 ± 0.03 | 100% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp512 | 301.02 ± 1.74 | 147% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp1024 | 287.23 ± 0.20 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp2048 | 262.77 ± 1.81 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg128 | 18.80 ± 0.99 | 48% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg256 | 24.46 ± 3.04 | 83% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg512 | 36.32 ± 3.59 | 90% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp512 | 271.71 ± 3.53 | 133% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp1024 | 279.86 ± 45.63 | 100% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp2048 | 320.77 ± 5.00 | 124% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg128 | 43.51 ± 0.05 | 111% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg256 | 43.35 ± 0.09 | 110% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg512 | 42.60 ± 0.31 | 105% |
Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation
</details>
</details>
## Which file should I choose?
<details>
<summary>Click here for details</summary>
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggml-org/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
</details>
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
Thank you to LM Studio for sponsoring my work.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bd48428408ed93efb7bf0bab8a26e610b4e06e5bc4aafe011d783987f00aa5ca
size 4996515680

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8a31bde795884c4a32bfa9208b8605e57acce38814b981f611d56ea201a910a2
size 4957808480

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6be313382440dcf3d89f3a491716b68093ce5ee1a2d7dc8cd5d29cd0aea23a91
size 5207935840

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e6c2652a5cfa135a0f50cc512864fdb3488b0e94e0d820afc9f7c9e0a8f23eb9
size 5131990880

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a60d3e176af3929b8781f979774fd5765cb2aae6200561b6c34da2afb5262460
size 5073930080

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3c408e4a79065b0dc93cdeb982583ed6632bf69d21badde9307c201a24812522
size 5308682080

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4b4e85cae0115a6a6aac311810accc27a605a761feae1ef0aa83be7c5015ba19
size 5267107680

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7bb683abcf19655bb0e447963f5aa3d29d602864603202144b78e4d641973f36
size 5006192480

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7040e11f2cd5d9986fbdcbac57ba473fbbccc679be0689ae97ec17d7db202b75
size 5299793760

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d289c6aa20e3da30eb01952659b7bb6c918afe36978daac52b581aabb42836b3
size 5488365920

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9174cfdea9161b0100f42ca4f684410c27c0d1a1329e3581286d6788d6fa6204
size 5379734880

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:778c4ad9b0a131671dce9db6b881a732b4f2e188e6149478b94159253894a421
size 5131990880

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c1c229b9abc4925000614a31928be5e4d00ca2822f411c97afae2f3dea24222c
size 5781967200

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:468ea3a6d895ccb5c5b4e4281c7a889e17b3e3ac263339a927e38e8312dc932c
size 5339414880

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:25dcc7d5252977909a9b32da18e3a1aadd1fbf51e82dea4be7a1ab97962e0cff
size 5827358560

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eaf3137fc5fc6132439175d8218cd6004c748828f041bfef9414084211120dc7
size 6745830240

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:be84528231457f766a35612386e2638d9af10beb10a9d8d23f0cc73c20818da7
size 6525629280

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2cfba6891ddd07219daa30e511226684216bb7b0215604f6d49e44808bae6a85
size 6211617120

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:359e5d955b8ded2fffde671b93dd68158c0c72b3186f31c1ec40efe75f5b6261
size 7253306720

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:09dae3403033e80a411f1f9b03c55712f9ea0c4a4302537027d293f3d9718b3d
size 7069805920

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:47b741b672dafef9dff9de8ca784063c65f786acd83ee0946e1bab55f7715f07
size 6781562720

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:95ee1b5339df8b4c735564cc2297572c972f181222cced619e726ba428adfc83
size 9135892320

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:95ee1b5339df8b4c735564cc2297572c972f181222cced619e726ba428adfc83
size 9135892320

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5c43a91e9d200cb2444a8ddc4f1b64c06adca98f86ae390e12c23605dfddef1c
size 9458093920

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-bf16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8ef005da11f47cf640cec564b3b761759a360b83fc42d33057e981343094d027
size 17788743232

3
nvidia_NVIDIA-Nemotron-Nano-9B-v2-imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:50c4f52d141f4fbb2a41aa81e8ef354e83831079f6c32f3295b9a7dff670eb7d
size 3921216