53 lines
1.4 KiB
Markdown
53 lines
1.4 KiB
Markdown
---
|
|
model-index:
|
|
- name: llama3-8b
|
|
results:
|
|
- task:
|
|
type: text-generation
|
|
dataset:
|
|
name: Wikitext
|
|
type: wikitext
|
|
metrics:
|
|
- type: perplexity (BASELINE)
|
|
value: 7.259655927392236
|
|
- type: perplexity (BASIC)
|
|
value: 109.31862113631051
|
|
---
|
|
This is a d-Matrix functional reference of the LLAMA3-8B model.
|
|
The reference provides the following functional *configurations*:
|
|
Configuration | Explanation
|
|
:-- | :--
|
|
**`BASELINE`** | a reference functionally equivalent to the original model
|
|
**`BASIC`** | all linear algebraic operands quantized to `MXINT8-64`
|
|
|
|
|
|
### Usage
|
|
|
|
Install d-Matrix [Dmx_Compressor](https://github.com/d-matrix-ai/dmx-compressor) first.
|
|
```sh
|
|
pip install dmx_compressor
|
|
```
|
|
|
|
The following is an example model and its evaluation.
|
|
|
|
```sh
|
|
git clone https://github.com/EleutherAI/lm-evaluation-harness
|
|
cd lm-evaluation-harness
|
|
pip install -e .
|
|
```
|
|
|
|
```python
|
|
from dmx.compressor.modeling import DmxModel
|
|
import lm_eval
|
|
from lm_eval.models.huggingface import HFLM
|
|
|
|
lm_eval.api.registry.register_model("hf", HFLM)
|
|
model_args = "pretrained=d-matrix/Llama3-8b,trust_remote_code=True"
|
|
|
|
lm = lm_eval.api.registry.get_model("hf").create_from_arg_string(model_args, {"batch_size": 1})
|
|
|
|
# Transform the model with DMX
|
|
lm._model = DmxModel.from_torch(lm._model)
|
|
|
|
eval_results = lm_eval.evaluate(lm, lm_eval.tasks.get_task_dict(["wikitext"])) # Assign desired task, i.e. "wikitext"
|
|
``` |