初始化项目,由ModelHub XC社区提供模型
Model: plawanrath/phi-3.5-mini-instruct-bf16-mlx-cba Source: Original Platform
This commit is contained in:
105
README.md
Normal file
105
README.md
Normal file
@@ -0,0 +1,105 @@
|
||||
---
|
||||
library_name: mlx
|
||||
license: mit
|
||||
license_link: https://huggingface.co/microsoft/Phi-3.5-mini-instruct/resolve/main/LICENSE
|
||||
language:
|
||||
- multilingual
|
||||
pipeline_tag: text-generation
|
||||
base_model: microsoft/Phi-3.5-mini-instruct
|
||||
tags:
|
||||
- nlp
|
||||
- code
|
||||
- mlx
|
||||
- quantization
|
||||
- bias-evaluation
|
||||
---
|
||||
|
||||
# phi-3.5-mini-instruct-bf16 (MLX, CBA artifact)
|
||||
|
||||
MLX-format BF16 (uncompressed baseline) variant of [`microsoft/Phi-3.5-mini-instruct`](https://huggingface.co/microsoft/Phi-3.5-mini-instruct).
|
||||
|
||||
This is one of the **15 model artifacts** from the paper:
|
||||
|
||||
> **Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels**
|
||||
> Plawan Kumar Rath, Rahul Maliakkal. *IEEE Cloud Summit 2026*.
|
||||
> Code: <https://github.com/plawanrath/compression-bias-amplification>
|
||||
> arXiv: <https://arxiv.org/abs/2605.15208>
|
||||
|
||||
## Quantization
|
||||
|
||||
This is the **BF16 baseline** used as the uncompressed reference in the paper. Weights have been re-serialized via `mlx_lm.convert` (no quantization) so this directory is loadable directly by MLX without an extra conversion step.
|
||||
|
||||
## How this artifact was produced
|
||||
|
||||
```bash
|
||||
python -m mlx_lm.convert \
|
||||
--hf-path microsoft/Phi-3.5-mini-instruct \
|
||||
--mlx-path ./phi-3.5-mini-instruct-bf16 \
|
||||
|
||||
```
|
||||
|
||||
This is the **exact** artifact used to produce the inference results in §4.3 of the paper (911,100 records over BBQ ambiguous, 5 seeds × 12,148 items × 15 configs).
|
||||
|
||||
## Usage (MLX)
|
||||
|
||||
```bash
|
||||
pip install mlx-lm
|
||||
```
|
||||
|
||||
```python
|
||||
from mlx_lm import load, generate
|
||||
|
||||
model, tokenizer = load("plawanrath/phi-3.5-mini-instruct-bf16-mlx-cba")
|
||||
prompt = tokenizer.apply_chat_template(
|
||||
[{"role": "user", "content": "Hello!"}],
|
||||
add_generation_prompt=True,
|
||||
tokenize=False,
|
||||
)
|
||||
print(generate(model, tokenizer, prompt=prompt, max_tokens=128))
|
||||
```
|
||||
|
||||
Or via CLI:
|
||||
|
||||
```bash
|
||||
mlx_lm.generate --model plawanrath/phi-3.5-mini-instruct-bf16-mlx-cba --prompt "Hello!"
|
||||
```
|
||||
|
||||
## Paper findings relevant to this variant
|
||||
|
||||
The paper documents a **dose-response** relationship between quantization aggressiveness and emergent stereotypical behavior on BBQ ambiguous questions:
|
||||
|
||||
| Variant | % of BF16-unbiased items that became biased |
|
||||
|---|---|
|
||||
| Q8 | 0.1–0.9% |
|
||||
| Q6 | 0.3–1.3% |
|
||||
| Q4 | 2.2–5.6% |
|
||||
| Q3 | 6.0–21.1% |
|
||||
|
||||
These changes are largely **invisible to perplexity** (<0.5% shift at Q8, <3% at Q4 across all three families). Treat any deployment of compressed instruction-tuned models on fairness-sensitive tasks accordingly.
|
||||
|
||||
## Model details
|
||||
|
||||
- **Base model:** [`microsoft/Phi-3.5-mini-instruct`](https://huggingface.co/microsoft/Phi-3.5-mini-instruct)
|
||||
- **Family:** Phi-3
|
||||
- **Parameters:** 3.8B
|
||||
- **Precision:** BF16 (uncompressed baseline)
|
||||
- **Format:** MLX (Apple Silicon)
|
||||
- **Conversion framework:** [`mlx-lm`](https://github.com/ml-explore/mlx-lm)
|
||||
|
||||
## License
|
||||
|
||||
Inherited from the base model (`mit`). See the upstream model page for the full license text.
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@inproceedings{rath2026quantization,
|
||||
title = { Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels },
|
||||
author = {Rath, Plawan Kumar and Maliakkal, Rahul},
|
||||
booktitle = { IEEE Cloud Summit 2026 },
|
||||
year = {2026},
|
||||
eprint = {2605.15208},
|
||||
archivePrefix = {arXiv},
|
||||
url = {https://arxiv.org/abs/2605.15208}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user