98 lines
3.5 KiB
Markdown
98 lines
3.5 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
base_model:
|
||
|
|
- ibm-granite/granite-4.1-8b
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- unsloth
|
||
|
|
- lora
|
||
|
|
- trl
|
||
|
|
- sft
|
||
|
|
---
|
||
|
|
|
||
|
|
# Granite-4.1-8B-SFT-Fable5-Glint
|
||
|
|
|
||
|
|
A LoRA fine-tune of [`ibm-granite/granite-4.1-8b`](https://huggingface.co/ibm-granite/granite-4.1-8b), supervised fine-tuned on `ermiaazarkhalili/Fable-5-Glint-Clean` (private).
|
||
|
|
|
||
|
|
| | |
|
||
|
|
| --- | --- |
|
||
|
|
| **Base model** | [`ibm-granite/granite-4.1-8b`](https://huggingface.co/ibm-granite/granite-4.1-8b) |
|
||
|
|
| **Architecture** | `GraniteForCausalLM` |
|
||
|
|
| **Parameters** | 8.8B |
|
||
|
|
| **Training data** | `ermiaazarkhalili/Fable-5-Glint-Clean` (private) |
|
||
|
|
| **Method** | LoRA supervised fine-tuning via [Unsloth](https://github.com/unslothai/unsloth) + [TRL](https://github.com/huggingface/trl) |
|
||
|
|
| **License** | `apache-2.0` (inherited from the base model) |
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
model_id = "ermiaazarkhalili/Granite-4.1-8B-SFT-Fable5-Glint"
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, dtype='auto', device_map='auto')
|
||
|
|
|
||
|
|
messages = [{"role": "user", "content": "Explain gradient checkpointing in two sentences."}]
|
||
|
|
inputs = tokenizer.apply_chat_template(
|
||
|
|
messages, add_generation_prompt=True, return_tensors='pt'
|
||
|
|
).to(model.device)
|
||
|
|
|
||
|
|
outputs = model.generate(inputs, max_new_tokens=256)
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
## Training configuration
|
||
|
|
|
||
|
|
| Setting | Value |
|
||
|
|
| --- | --- |
|
||
|
|
| LoRA rank (r) | 16 |
|
||
|
|
| LoRA alpha | 16 |
|
||
|
|
| Learning rate | 0.0002 |
|
||
|
|
| Epochs | 3 |
|
||
|
|
| Effective batch size | 8 (2 x 4 grad accum) |
|
||
|
|
| Max sequence length | 4096 |
|
||
|
|
| Base precision | 4-bit (QLoRA) |
|
||
|
|
| Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
|
||
|
|
|
||
|
|
## Held-out evaluation
|
||
|
|
|
||
|
|
Next-token accuracy on a deterministic held-out split of `ermiaazarkhalili/Fable-5-Glint-Clean` (n = 199 samples), scored against the base model [`ibm-granite/granite-4.1-8b`](https://huggingface.co/ibm-granite/granite-4.1-8b). Assistant tokens only; both models are scored identically.
|
||
|
|
|
||
|
|
| Metric | Base | This model | Δ |
|
||
|
|
| --- | ---: | ---: | ---: |
|
||
|
|
| Top-1 accuracy | 0.5502 | 0.7258 | +0.1756 |
|
||
|
|
| Top-5 accuracy | 0.8136 | 0.9305 | +0.1169 |
|
||
|
|
|
||
|
|
A delta measures how far fine-tuning moved this model from its own starting point; it is not a ranking against other models, which start from different baselines.
|
||
|
|
|
||
|
|
## Observed training loss
|
||
|
|
|
||
|
|
Measured from our SLURM logs for this configuration. These are training-loss
|
||
|
|
observations only — see the held-out evaluation above for measured accuracy.
|
||
|
|
|
||
|
|
| SLURM job | Steps | First loss | Final loss |
|
||
|
|
| --- | --- | --- | --- |
|
||
|
|
| `46020984` | 1,557 | 1.2249 | 0.6531 |
|
||
|
|
| `45987985` | 900 | 1.3237 | 0.8099 |
|
||
|
|
|
||
|
|
## Limitations
|
||
|
|
|
||
|
|
- No benchmark evaluation has been run on this checkpoint. The only reported
|
||
|
|
numbers are training-loss observations.
|
||
|
|
- Inherits the biases, knowledge cutoff and failure modes of the base model.
|
||
|
|
- Fine-tuned on a single instruction-following dataset; behaviour outside that
|
||
|
|
distribution is untested.
|
||
|
|
- LoRA adapters were merged into the base weights, so the merged model cannot
|
||
|
|
be detached from this fine-tune.
|
||
|
|
|
||
|
|
## Reproducing
|
||
|
|
|
||
|
|
Trained by `notebooks/fable_distillation_granite41-8b_fable-glint_unsloth.ipynb`, executed non-interactively with
|
||
|
|
papermill on a SLURM H100 partition (Unsloth + TRL, LoRA).
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
*Card generated from the training run's own configuration and logs by*
|
||
|
|
*`scripts/generate_hub_model_card.py`.*
|