45 lines
1004 B
Markdown
45 lines
1004 B
Markdown
|
---
|
||
|
|
base_model: ibm-granite/granite-3.1-1b-a400m-instruct
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- granite
|
||
|
|
- reap
|
||
|
|
- pruning
|
||
|
|
- multilingual
|
||
|
|
license: apache-2.0
|
||
|
|
---
|
||
|
|
|
||
|
|
# Granite 3.1 1B A400M Instruct REAP Pruned Checkpoint
|
||
|
|
|
||
|
|
This repository contains a REAP layerwise-pruned checkpoint of
|
||
|
|
`ibm-granite/granite-3.1-1b-a400m-instruct`.
|
||
|
|
|
||
|
|
## Base Model
|
||
|
|
|
||
|
|
- Model: `ibm-granite/granite-3.1-1b-a400m-instruct`
|
||
|
|
- Architecture: `GraniteMoeForCausalLM`
|
||
|
|
- `num_local_experts=32`
|
||
|
|
- `num_experts_per_tok=8`
|
||
|
|
|
||
|
|
## Pruning Setup
|
||
|
|
|
||
|
|
- Dataset: `theblackcat102/evol-codealpaca-v1`
|
||
|
|
- `model_max_length=2048`
|
||
|
|
- `batches_per_category=128`
|
||
|
|
- `batch_size=1`
|
||
|
|
- `batch_group_size=8`
|
||
|
|
- `renormalize_router_weights=true`
|
||
|
|
- `truncate=false`
|
||
|
|
|
||
|
|
## Checkpoint
|
||
|
|
|
||
|
|
- Requested ratio: `0.10`
|
||
|
|
- Experts pruned: `3`
|
||
|
|
- Experts retained: `29`
|
||
|
|
- Actual ratio: `9.38%`
|
||
|
|
- Local output dir: `layerwise_reap-renorm_true-seed_42-0.09`
|
||
|
|
|
||
|
|
## Provenance
|
||
|
|
|
||
|
|
- Created with Codex using GPT-5.5.
|
||
|
|
- Generated for the REAP pruning experiment log and multilingual evaluation.
|