54 lines
1.5 KiB
Markdown
54 lines
1.5 KiB
Markdown
|
---
|
||
|
|
license: other
|
||
|
|
library_name: llama.cpp
|
||
|
|
tags:
|
||
|
|
- gguf
|
||
|
|
- llama.cpp
|
||
|
|
- text-generation
|
||
|
|
- forge-1
|
||
|
|
base_model: North-ML1/Forge-1
|
||
|
|
---
|
||
|
|
|
||
|
|
|
||
|
|
# Forge-1 GGUF
|
||
|
|
|
||
|
|
GGUF exports for [`North-ML1/Forge-1`](https://huggingface.co/North-ML1/Forge-1).
|
||
|
|
|
||
|
|
Files:
|
||
|
|
|
||
|
|
- `forge-1-f16.gguf` - F16 GGUF export.
|
||
|
|
- `forge-1-q4_k_m.gguf` - Q4_K_M quantized GGUF for practical local inference.
|
||
|
|
|
||
|
|
Source model: `North-ML1/Forge-1`, moved from `North-ML1/Forge-1-Base`.
|
||
|
|
|
||
|
|
Conversion notes:
|
||
|
|
|
||
|
|
- Converted with `llama.cpp` from the Hugging Face safetensors export.
|
||
|
|
- The model uses a locally trained ByteLevel BPE tokenizer. Current `llama.cpp` did not recognize its tokenizer hash, so the converter was patched to mark this ByteLevel BPE tokenizer as `gpt-2` pre-tokenization.
|
||
|
|
- A smoke test with `llama-cli` loaded `forge-1-q4_k_m.gguf` and answered `2 + 2 = 4`.
|
||
|
|
|
||
|
|
Example:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
llama-cli -m forge-1-q4_k_m.gguf -p '<|im_start|>user\nWhat is 2 + 2?<|im_end|>\n<|im_start|>assistant\n' -n 32 --temp 0
|
||
|
|
```
|
||
|
|
|
||
|
|
SHA256:
|
||
|
|
|
||
|
|
```text
|
||
|
|
56b5f16b11e742e82874f11f5db20bc9be497793626989a5ed4000108773a87c forge-1-f16.gguf
|
||
|
|
2272f246381687462abac21a8f9b8cda1d891fa0cdba22d45c16f77c7527c51c forge-1-q4_k_m.gguf
|
||
|
|
```
|
||
|
|
|
||
|
|
## Prompt Format
|
||
|
|
|
||
|
|
Forge-1 is ChatML-tuned. Use a ChatML prompt, for example:
|
||
|
|
|
||
|
|
```text
|
||
|
|
<|im_start|>user
|
||
|
|
What is 2 + 2?<|im_end|>
|
||
|
|
<|im_start|>assistant
|
||
|
|
```
|
||
|
|
|
||
|
|
Plain completion prompts are not reliable for this checkpoint. If your UI has a chat-template selector, choose ChatML.
|