language, license, library_name, base_model, tags, pipeline_tag, inference, model_creator

language	license	library_name	base_model	tags	pipeline_tag	inference	model_creator
en	apache-2.0	transformers	Qwen/Qwen3-8B	text-generation fine-tune coding gguf llama.cpp ollama lm-studio	text-generation	true	nachikethreddyy

Qwen3.5-8B Distilled - GGUF Format

Fine-tuned Qwen3.5-8B for software engineering & coding tasks. GGUF-optimized version for local inference.

📦 What's Included

Variant	Size	Format	Best For
Full Precision (BF16)	16.39 GB	Safetensors	Maximum quality, research
Q8 Quantized	8.8 GB	Safetensors	Balanced speed/quality
GGUF F16	15.3 GB	GGUF	Ollama, llama.cpp, LM Studio

🚀 Quick Start

Ollama

ollama run nachikethreddyy/qwen3.5-8b-distilled-GGUF:F16

llama.cpp

# Install
brew install llama.cpp

# Run
llama-cli -hf nachikethreddyy/qwen3.5-8b-distilled-GGUF:F16

LM Studio

1. Download LM Studio
2. Search: nachikethreddyy/qwen3.5-8b-distilled-GGUF
3. Download & run!

Transformers (Full/Q8)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "nachikethreddyy/qwen3.5-8b-distilled-GGUF",
    device_map="auto"
)

📊 Training Details

• Base: Qwen/Qwen3-8B
• Method: LoRA Fine-tuning (r=16, alpha=32)
• Data: 256 coding examples
• Framework: MLX
• Iterations: 1600

📄 License

Apache 2.0 (inherited from Qwen/Qwen3-8B)

---

For MLX/Apple Silicon: See qwen3.5-8b-distilled-MLX