50 lines
1.7 KiB
Markdown
50 lines
1.7 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
tags:
|
||
|
|
- security
|
||
|
|
- cybersecurity
|
||
|
|
- mlx
|
||
|
|
- mlx-bf16
|
||
|
|
base_model: viettelsecurity-ai/security-llama3.2-3b
|
||
|
|
base_model_relation: quantized
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
library_name: mlx
|
||
|
|
quantized_by: ahmedandaloes
|
||
|
|
---
|
||
|
|
|
||
|
|
# security-llama3.2-3b — MLX bf16
|
||
|
|
|
||
|
|
Full-precision (bf16) [MLX](https://github.com/ml-explore/mlx) build of
|
||
|
|
[**viettelsecurity-ai/security-llama3.2-3b**](https://huggingface.co/viettelsecurity-ai/security-llama3.2-3b), for fast local inference on Apple Silicon.
|
||
|
|
|
||
|
|
- **Precision:** bf16 — no quantization, identical weights to source.
|
||
|
|
- Weights unchanged from source — format + precision conversion only.
|
||
|
|
- Converted with `mlx-lm`.
|
||
|
|
|
||
|
|
## Builds
|
||
|
|
| Build |
|
||
|
|
|-------|
|
||
|
|
| [MLX-4bit](https://huggingface.co/ahmedandaloes/security-llama3.2-3b-MLX-4bit) |
|
||
|
|
| [MLX-6bit](https://huggingface.co/ahmedandaloes/security-llama3.2-3b-MLX-6bit) |
|
||
|
|
| [MLX-8bit](https://huggingface.co/ahmedandaloes/security-llama3.2-3b-MLX-8bit) |
|
||
|
|
| [MLX-bf16](https://huggingface.co/ahmedandaloes/security-llama3.2-3b-MLX-bf16) |
|
||
|
|
|
||
|
|
GGUF builds: [prithivMLmods/Security-Llama3.2-3B-GGUF](https://huggingface.co/prithivMLmods/Security-Llama3.2-3B-GGUF).
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
```bash
|
||
|
|
pip install mlx-lm
|
||
|
|
```
|
||
|
|
```python
|
||
|
|
from mlx_lm import load, generate
|
||
|
|
model, tok = load("ahmedandaloes/security-llama3.2-3b-MLX-bf16")
|
||
|
|
p = tok.apply_chat_template([{"role":"user","content":"Name a common web vulnerability."}], add_generation_prompt=True)
|
||
|
|
print(generate(model, tok, prompt=p, max_tokens=200, verbose=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
## Attribution
|
||
|
|
Source: [viettelsecurity-ai/security-llama3.2-3b](https://huggingface.co/viettelsecurity-ai/security-llama3.2-3b). License per source (Apache-2.0 assumed; verify).
|
||
|
|
MLX build for the Apple Silicon community. For **authorized** security work only.
|