初始化项目,由ModelHub XC社区提供模型
Model: sasa2000/granite-3.1-1b-a400m-instruct-reap-pruned-010 Source: Original Platform
This commit is contained in:
45
README.md
Normal file
45
README.md
Normal file
@@ -0,0 +1,45 @@
|
||||
---
|
||||
base_model: ibm-granite/granite-3.1-1b-a400m-instruct
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- granite
|
||||
- reap
|
||||
- pruning
|
||||
- multilingual
|
||||
license: apache-2.0
|
||||
---
|
||||
|
||||
# Granite 3.1 1B A400M Instruct REAP Pruned Checkpoint
|
||||
|
||||
This repository contains a REAP layerwise-pruned checkpoint of
|
||||
`ibm-granite/granite-3.1-1b-a400m-instruct`.
|
||||
|
||||
## Base Model
|
||||
|
||||
- Model: `ibm-granite/granite-3.1-1b-a400m-instruct`
|
||||
- Architecture: `GraniteMoeForCausalLM`
|
||||
- `num_local_experts=32`
|
||||
- `num_experts_per_tok=8`
|
||||
|
||||
## Pruning Setup
|
||||
|
||||
- Dataset: `theblackcat102/evol-codealpaca-v1`
|
||||
- `model_max_length=2048`
|
||||
- `batches_per_category=128`
|
||||
- `batch_size=1`
|
||||
- `batch_group_size=8`
|
||||
- `renormalize_router_weights=true`
|
||||
- `truncate=false`
|
||||
|
||||
## Checkpoint
|
||||
|
||||
- Requested ratio: `0.10`
|
||||
- Experts pruned: `3`
|
||||
- Experts retained: `29`
|
||||
- Actual ratio: `9.38%`
|
||||
- Local output dir: `layerwise_reap-renorm_true-seed_42-0.09`
|
||||
|
||||
## Provenance
|
||||
|
||||
- Created with Codex using GPT-5.5.
|
||||
- Generated for the REAP pruning experiment log and multilingual evaluation.
|
||||
Reference in New Issue
Block a user