初始化项目,由ModelHub XC社区提供模型

Model: rombodawg/Llama-3-8B-Instruct-Coder
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-23 17:02:13 +08:00
commit 6e40cc8561
17 changed files with 413375 additions and 0 deletions

32
README.md Normal file
View File

@@ -0,0 +1,32 @@
---
language:
- en
license: llama3
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
---
llama-3-8B-Instruct-Coder
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/0O4cIuv3wNbY68-FP7tak.jpeg)
This model is llama-3-8b-instruct from Meta (uploaded by unsloth) trained on the full 65k Codefeedback dataset + the additional 150k Code Feedback Filtered Instruction dataset combined. You can find that dataset linked below. This AI model was trained with the new Qalore method developed by my good friend on Discord and fellow Replete-AI worker walmartbag.
The Qalore method uses Qlora training along with the methods from Galore for additional reductions in VRAM allowing for llama-3-8b to be loaded on 14.5 GB of VRAM. This allowed this training to be completed on an RTX A4000 16GB in 130 hours for less than $20.
Qalore notebook for training:
- https://colab.research.google.com/drive/1bX4BsjLcdNJnoAf7lGXmWOgaY8yekg8p?usp=sharing
__________________________________________________________________________________
## Join the Replete-Ai discord! We are a great and Loving community!
- https://discord.gg/ZZbnsmVnjD