qwp4w3hyb/Llama-3-8B-Instruct-Coder-v2-iMat-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: qwp4w3hyb/Llama-3-8B-Instruct-Coder-v2-iMat-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-16 01:42:29 +08:00
commit 6a4e2f98de
22 changed files with 167 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

55
.gitattributes vendored Normal file
View File

@@ -0,0 +1,55 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imat-f16-gmerged.dat filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-f16.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
llama-3-8b-instruct-coder-v2-imat-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

52
README.md Normal file
View File

@@ -0,0 +1,52 @@
---
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
- gguf
- imatrix
- importance matrix
base_model: rombodawg/Llama-3-8B-Instruct-Coder-v2
---
# Quant Infos
- Quantized with recent bpe pre-tokenizer fixes https://github.com/ggerganov/llama.cpp/pull/6920
- quants done with an importance matrix for improved quantization loss
- 0, K & IQ quants in basically all variants from Q8 down to IQ1_S
- Quantized with [llama.cpp](https://github.com/ggerganov/llama.cpp) commit [04976db7a819fcf8bfefbfc09a3344210b79dd27](https://github.com/ggerganov/llama.cpp/commit/04976db7a819fcf8bfefbfc09a3344210b79dd27) (master from 2024-05-07)
- Imatrtix generated with [this](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384) dataset.
```
./imatrix -c 512 -m $model_name-f16.gguf -f $llama_cpp_path/groups_merged.txt -o $out_path/imat-f16-gmerged.dat
```
# Original Model Card
Llama-3-8B-Instruct-Coder-v2
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/0O4cIuv3wNbY68-FP7tak.jpeg)
_________________________________________________________________________
How is this model diffrent from rombodawg/Llama-3-8B-Instruct-Coder? Well the first model was trained on a dataset that had some major flaws that I originally had missed, with version 2 all of those flaws are fixed, and the model is fully retrained so it performs much better than the previous iteration.
_________________________________________________________________________
This model is llama-3-8b-instruct from Meta (uploaded by unsloth) trained on the full 150k Code Feedback Filtered Instruction dataset. You can find that dataset linked below. This AI model was trained with the new Qalore method developed by my good friend on Discord and fellow Replete-AI worker walmartbag.
The Qalore method uses Qlora training along with the methods from Galore for additional reductions in VRAM allowing for llama-3-8b to be loaded on 14.5 GB of VRAM. This allowed this training to be completed on an RTX A5000 24GB in 50 hours for less than $15.
Dataset used for training this model:
- https://huggingface.co/datasets/Replete-AI/CodeFeedback-Filtered-Instruction-Simplified-Pairs
Qalore notebook for training:
- https://colab.research.google.com/drive/1bX4BsjLcdNJnoAf7lGXmWOgaY8yekg8p?usp=sharing
Quantizations for easier inference:
- https://huggingface.co/bartowski/Llama-3-8B-Instruct-Coder-v2-GGUF
- https://huggingface.co/bartowski/Llama-3-8B-Instruct-Coder-v2-exl2

3
imat-f16-gmerged.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:380482e799aaeed74d48d70fe8e53796c92c6f78d5b9e2c9ee6fcfcd6003c023
size 4988185

3
llama-3-8b-instruct-coder-v2-f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:45667d91e0f038741ba8893e3469d83964cf90d83d66d1253fab87d723a7466a
size 16068890912

3
llama-3-8b-instruct-coder-v2-imat-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:73edcd2abbf18b4391f7fd870d3af36a3648cc114180049d89fc51eed69de6c0
size 2019627648

3
llama-3-8b-instruct-coder-v2-imat-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c5a1fa82b5be0c23b7764b0a721369825bd6db030d0c46d94f83d9cfc1c64104
size 2948280960

3
llama-3-8b-instruct-coder-v2-imat-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0930c3c20b34dc64fac02c88515fdd3a6f631bfa27e0f25dc676d858f63e430d
size 2758488704

3
llama-3-8b-instruct-coder-v2-imat-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5a245ce9dfad3b74298a50b58d39f8d12a7bb84927d9a884bb4a901b093564f8
size 2605781632

3
llama-3-8b-instruct-coder-v2-imat-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f24467deda5a6aebca5d31005b6e1ca228fd2d7ca10af7eb6fb48f1ecc29b819
size 2399212160

3
llama-3-8b-instruct-coder-v2-imat-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8c533453a52c0b59f5d7911e0f9ec2996f0b3103a8057e0d77ace64c322203e4
size 3784823424

3
llama-3-8b-instruct-coder-v2-imat-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5328b084f313aa505fa5aa3591c1cffcd3684d18b2c162ff3cf735823e862e99
size 3682325120

3
llama-3-8b-instruct-coder-v2-imat-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:16f75fc721478bfadc262b70c1dc3a81a9c6995422578419eae05d06f066d185
size 3518747264

3
llama-3-8b-instruct-coder-v2-imat-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fa69e05323159cccbc9e47f5114853566aece40165e81268220cef70084ab048
size 3274912384

3
llama-3-8b-instruct-coder-v2-imat-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cea1b6ae810e994b6bfc6f186c7bf42a4dce01108092ad28e4866ac1653ae90b
size 4677988992

3
llama-3-8b-instruct-coder-v2-imat-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:30cb11611c9be4f476237e2677d63bbadf5fc5615132773ac229f5133a4be25d
size 4447662720

3
llama-3-8b-instruct-coder-v2-imat-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c3df55e82c4a87ed44dfdff6a7125f99568d1a85094c720de91d375544f14e16
size 4675891840

3
llama-3-8b-instruct-coder-v2-imat-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b5c992be83ba7bae01cd8c992d0cd426638226ff32c8bf02d1a4f4f5b9ecddee
size 4920734336

3
llama-3-8b-instruct-coder-v2-imat-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:74a9d0a6b3a0923360eb8711e7da47c47e956809e5f29157b8b8d117427c7c1e
size 4692669056

3
llama-3-8b-instruct-coder-v2-imat-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5e49c74fa5162c03796c2c8ef0f739d989bf8e3e63d107450f6400b2f4dd9e65
size 5732987520

3
llama-3-8b-instruct-coder-v2-imat-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1d9f949737bd447d4dbccbf719bfc509d3e82d4f81c54665371b1530c482887c
size 5599294080

3
llama-3-8b-instruct-coder-v2-imat-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6af469b2ca85067a4e45932ace670f57bc577375c7af2e2557a89abbf3e4eeb7
size 6596006528

3
llama-3-8b-instruct-coder-v2-imat-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d0b1e5df82a568385c38708134fbd5e349612d7aed7260bfe10742f9e9a8eebb
size 8540770944