76 lines
2.9 KiB
Markdown
76 lines
2.9 KiB
Markdown
---
|
|
base_model:
|
|
- meta-llama/Llama-3.2-1B-Instruct
|
|
datasets:
|
|
- WithinUsAI/Llama_4_Maverick_Distilled_5k
|
|
- WithinUsAI/Meta_Muse_Spark_Distilled_5k
|
|
---
|
|
license: llama3.2
|
|
---
|
|
tags:
|
|
|
|
llama
|
|
llama3.2
|
|
distill
|
|
text-generation
|
|
conversational
|
|
WithinUsAI
|
|
11-47
|
|
Muse Spark
|
|
Llama 4 Maverick
|
|
1B
|
|
language:
|
|
en
|
|
---
|
|
Llama-3.2-Mavrick.Spark-1B
|
|
By 11-47 / WithinUsAI
|
|
Distilled from Muse Spark and Llama 4 Maverick into Llama 3.2 1B Instruct — fast 1.23B model with Spark-style persona + Maverick reasoning traces.
|
|
Built with Llama 3.2. This is a community research distillation. Not affiliated with Meta. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc.
|
|
What you used
|
|
Base: meta-llama/Llama-3.2-1B-Instruct (1.23B, 128k context)
|
|
Dataset 1: WithinUsAI/Meta_Muse_Spark_Distilled_5k - 5k Spark-style instructions, persona, and tool traces
|
|
Dataset 2: WithinUsAI/Llama_4_Maverick_Distilled_5k - 5k Maverick long-form CoT / reasoning traces
|
|
Total: 10k distilled conversations -> SFT into 1B
|
|
Why this build
|
|
Llama 3.2 1B is great for edge/CPU but lacks deep reasoning. Muse Spark (closed, API-only) and Maverick (frontier) have strong reasoning but can't run locally. This project compresses their trace style into a 2.49GB model that runs on a phone / laptop.
|
|
How to run
|
|
python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
model_id = "11-47/Llama-3.2-Mavrick.Spark-1B"
|
|
tok = AutoTokenizer.from_pretrained(model_id)
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
|
|
|
|
messages = [
|
|
{"role": "system", "content": "You are Mavrick.Spark, a concise, helpful assistant built by WithinUsAI."},
|
|
{"role": "user", "content": "Explain how distillation works in 3 bullets."}
|
|
]
|
|
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
|
out = model.generate(inputs, max_new_tokens=400, temperature=0.7, top_p=0.9, do_sample=True)
|
|
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
|
|
Your chat_template.jinja is already in the repo, so apply_chat_template will work and HF Inference will show the chat widget.
|
|
Training recipe (fill in your actual values)
|
|
Method: Supervised fine-tune on assistant tokens only
|
|
Data format: ShareGPT / ChatML with system + user + assistant
|
|
Base: Llama-3.2-1B-Instruct
|
|
Epochs: [e.g. 3]
|
|
LR: [e.g. 2e-5]
|
|
Batch: [e.g. 64]
|
|
Optimizer: AdamW
|
|
Leave as SFT - no RLHF needed for this scale.
|
|
Limitations
|
|
1B still hallucinates vs 400B Maverick / Spark
|
|
Distilled style may over-explain
|
|
Knowledge cutoff from base Llama 3.2 (Dec 2023) + synthetic traces
|
|
Not hardened for tool use / code exec
|
|
License / Attribution
|
|
Base weights: Llama 3.2 Community License
|
|
Requirement: Keep "Llama" prefix (you do: Llama-3.2-Mavrick.Spark-1B) and display "Built with Llama"
|
|
Datasets: Your own WithinUsAI sets
|
|
Push this README
|
|
bash
|
|
git pull
|
|
# replace README.md with this file
|
|
git add README.md
|
|
git commit -m "Add full model card - base + WithinUsAI datasets"
|
|
git push |