初始化项目,由ModelHub XC社区提供模型
Model: 11-47/Llama-3.2-Mavrick.Spark-1B Source: Original Platform
This commit is contained in:
76
README.md
Normal file
76
README.md
Normal file
@@ -0,0 +1,76 @@
|
||||
---
|
||||
base_model:
|
||||
- meta-llama/Llama-3.2-1B-Instruct
|
||||
datasets:
|
||||
- WithinUsAI/Llama_4_Maverick_Distilled_5k
|
||||
- WithinUsAI/Meta_Muse_Spark_Distilled_5k
|
||||
---
|
||||
license: llama3.2
|
||||
---
|
||||
tags:
|
||||
|
||||
llama
|
||||
llama3.2
|
||||
distill
|
||||
text-generation
|
||||
conversational
|
||||
WithinUsAI
|
||||
11-47
|
||||
Muse Spark
|
||||
Llama 4 Maverick
|
||||
1B
|
||||
language:
|
||||
en
|
||||
---
|
||||
Llama-3.2-Mavrick.Spark-1B
|
||||
By 11-47 / WithinUsAI
|
||||
Distilled from Muse Spark and Llama 4 Maverick into Llama 3.2 1B Instruct — fast 1.23B model with Spark-style persona + Maverick reasoning traces.
|
||||
Built with Llama 3.2. This is a community research distillation. Not affiliated with Meta. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc.
|
||||
What you used
|
||||
Base: meta-llama/Llama-3.2-1B-Instruct (1.23B, 128k context)
|
||||
Dataset 1: WithinUsAI/Meta_Muse_Spark_Distilled_5k - 5k Spark-style instructions, persona, and tool traces
|
||||
Dataset 2: WithinUsAI/Llama_4_Maverick_Distilled_5k - 5k Maverick long-form CoT / reasoning traces
|
||||
Total: 10k distilled conversations -> SFT into 1B
|
||||
Why this build
|
||||
Llama 3.2 1B is great for edge/CPU but lacks deep reasoning. Muse Spark (closed, API-only) and Maverick (frontier) have strong reasoning but can't run locally. This project compresses their trace style into a 2.49GB model that runs on a phone / laptop.
|
||||
How to run
|
||||
python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_id = "11-47/Llama-3.2-Mavrick.Spark-1B"
|
||||
tok = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "You are Mavrick.Spark, a concise, helpful assistant built by WithinUsAI."},
|
||||
{"role": "user", "content": "Explain how distillation works in 3 bullets."}
|
||||
]
|
||||
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
||||
out = model.generate(inputs, max_new_tokens=400, temperature=0.7, top_p=0.9, do_sample=True)
|
||||
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
|
||||
Your chat_template.jinja is already in the repo, so apply_chat_template will work and HF Inference will show the chat widget.
|
||||
Training recipe (fill in your actual values)
|
||||
Method: Supervised fine-tune on assistant tokens only
|
||||
Data format: ShareGPT / ChatML with system + user + assistant
|
||||
Base: Llama-3.2-1B-Instruct
|
||||
Epochs: [e.g. 3]
|
||||
LR: [e.g. 2e-5]
|
||||
Batch: [e.g. 64]
|
||||
Optimizer: AdamW
|
||||
Leave as SFT - no RLHF needed for this scale.
|
||||
Limitations
|
||||
1B still hallucinates vs 400B Maverick / Spark
|
||||
Distilled style may over-explain
|
||||
Knowledge cutoff from base Llama 3.2 (Dec 2023) + synthetic traces
|
||||
Not hardened for tool use / code exec
|
||||
License / Attribution
|
||||
Base weights: Llama 3.2 Community License
|
||||
Requirement: Keep "Llama" prefix (you do: Llama-3.2-Mavrick.Spark-1B) and display "Built with Llama"
|
||||
Datasets: Your own WithinUsAI sets
|
||||
Push this README
|
||||
bash
|
||||
git pull
|
||||
# replace README.md with this file
|
||||
git add README.md
|
||||
git commit -m "Add full model card - base + WithinUsAI datasets"
|
||||
git push
|
||||
Reference in New Issue
Block a user