初始化项目,由ModelHub XC社区提供模型
Model: d-matrix/Llama3-8b Source: Original Platform
This commit is contained in:
53
README.md
Normal file
53
README.md
Normal file
@@ -0,0 +1,53 @@
|
||||
---
|
||||
model-index:
|
||||
- name: llama3-8b
|
||||
results:
|
||||
- task:
|
||||
type: text-generation
|
||||
dataset:
|
||||
name: Wikitext
|
||||
type: wikitext
|
||||
metrics:
|
||||
- type: perplexity (BASELINE)
|
||||
value: 7.259655927392236
|
||||
- type: perplexity (BASIC)
|
||||
value: 109.31862113631051
|
||||
---
|
||||
This is a d-Matrix functional reference of the LLAMA3-8B model.
|
||||
The reference provides the following functional *configurations*:
|
||||
Configuration | Explanation
|
||||
:-- | :--
|
||||
**`BASELINE`** | a reference functionally equivalent to the original model
|
||||
**`BASIC`** | all linear algebraic operands quantized to `MXINT8-64`
|
||||
|
||||
|
||||
### Usage
|
||||
|
||||
Install d-Matrix [Dmx_Compressor](https://github.com/d-matrix-ai/dmx-compressor) first.
|
||||
```sh
|
||||
pip install dmx_compressor
|
||||
```
|
||||
|
||||
The following is an example model and its evaluation.
|
||||
|
||||
```sh
|
||||
git clone https://github.com/EleutherAI/lm-evaluation-harness
|
||||
cd lm-evaluation-harness
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
```python
|
||||
from dmx.compressor.modeling import DmxModel
|
||||
import lm_eval
|
||||
from lm_eval.models.huggingface import HFLM
|
||||
|
||||
lm_eval.api.registry.register_model("hf", HFLM)
|
||||
model_args = "pretrained=d-matrix/Llama3-8b,trust_remote_code=True"
|
||||
|
||||
lm = lm_eval.api.registry.get_model("hf").create_from_arg_string(model_args, {"batch_size": 1})
|
||||
|
||||
# Transform the model with DMX
|
||||
lm._model = DmxModel.from_torch(lm._model)
|
||||
|
||||
eval_results = lm_eval.evaluate(lm, lm_eval.tasks.get_task_dict(["wikitext"])) # Assign desired task, i.e. "wikitext"
|
||||
```
|
||||
Reference in New Issue
Block a user