83 lines
2.4 KiB
Markdown
83 lines
2.4 KiB
Markdown
|
---
|
||
|
|
title: Fox 1.5 Nova
|
||
|
|
base_model: deepseek-ai/DeepSeek-Coder-7B-Instruct
|
||
|
|
datasets:
|
||
|
|
- teolm30/fox-nova-training
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- code-generation
|
||
|
|
- lora
|
||
|
|
- deepseek
|
||
|
|
- foxos
|
||
|
|
license: apache-2.0
|
||
|
|
---
|
||
|
|
|
||
|
|
# 🦊 Fox 1.5 Nova
|
||
|
|
|
||
|
|
Fox 1.5 Nova is Teo's code generation model, fine-tuned on DeepSeek-Coder-7B-Instruct for competitive programming, systems design, and real-world code patterns across 50+ languages.
|
||
|
|
|
||
|
|
## 🏆 Comparison
|
||
|
|
|
||
|
|
| Metric | 🦊 Fox 1.5 Nova (7B) | Claude Mythos |
|
||
|
|
|--------|------------------------|---------------|
|
||
|
|
| Parameters | ~7B | ~200B+ |
|
||
|
|
| Speed | ~40+ tok/s (fp16) | N/A (API only) |
|
||
|
|
| Size | 6.6GB (4-bit) / 13GB (fp16) | ~80GB |
|
||
|
|
| RAM Required | ~16GB | ~256GB |
|
||
|
|
| VRAM Required | ~6GB | N/A |
|
||
|
|
| Cost | Free | $5-25 / 1M tokens |
|
||
|
|
| Runs on CPU | ✅ Yes | ❌ No |
|
||
|
|
| Internet Required | ❌ No | ✅ Yes |
|
||
|
|
|
||
|
|
## 📊 Specs
|
||
|
|
|
||
|
|
| Metric | Value |
|
||
|
|
|--------|-------|
|
||
|
|
| Base Model | DeepSeek-Coder-7B-Instruct |
|
||
|
|
| Fine-tune Method | QLoRA (4-bit NF4) |
|
||
|
|
| LoRA r | 16 |
|
||
|
|
| LoRA alpha | 64 |
|
||
|
|
| Max Length | 512 tokens |
|
||
|
|
| Trainable Params | ~40M |
|
||
|
|
| Training Steps | 220 |
|
||
|
|
| Epochs | 10 |
|
||
|
|
| Output Precision | fp16 merged |
|
||
|
|
|
||
|
|
## 💻 Hardware
|
||
|
|
|
||
|
|
- **Training**: NVIDIA RTX 3050 (6GB VRAM) via QLoRA + Unsloth
|
||
|
|
- **Inference**: ~6GB VRAM (4-bit) or fp16 with 8GB+ VRAM
|
||
|
|
|
||
|
|
## 🚀 Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
|
|
||
|
|
model_name = "teolm30/Fox-1.5-Nova"
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
|
||
|
|
|
||
|
|
prompt = "Write a Python LRU cache"
|
||
|
|
messages = [{"role": "user", "content": prompt}]
|
||
|
|
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||
|
|
inputs = tokenizer(text, return_tensors="pt").to("cuda")
|
||
|
|
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
## ⚠️ Limitations
|
||
|
|
|
||
|
|
- fp16 model is 13GB — requires more VRAM than 4-bit version
|
||
|
|
- For 4-bit version (~6.6GB), see teolm30/Fox-1.5-Nova-4bit
|
||
|
|
- No built-in tool-use (use OpenClaw agent framework)
|
||
|
|
|
||
|
|
## 🔗 Links
|
||
|
|
|
||
|
|
- **HuggingFace**: https://huggingface.co/teolm30/Fox-1.5-Nova
|
||
|
|
- **FoxOS**: https://github.com/teolm30/FoxOS
|
||
|
|
- **OpenClaw**: https://openclaw.ai
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
*🦊 Built by FoxModelClaw agent for Teo's FoxOS development.*
|