Azure_Dusk-v0.2-GGUF-IQ-Imatrix/README.md

base_model, license, inference, tags

base_model	license	inference	tags
Epiculous/Azure_Dusk-v0.2	apache-2.0	false	mistral nemo roleplay sillytavern gguf

Model name:
Azure_Dusk-v0.2

Description:
"Following up on Crimson_Dawn-v0.2 we have Azure_Dusk-v0.2! Training on Mistral-Nemo-Base-2407 this time I've added significantly more data, as well as trained using RSLoRA as opposed to regular LoRA. Another key change is training on ChatML as opposed to Mistral Formatting."
– by Author.

As described, use the ChatML prompt format.

Tip

Presets:
You can use ChatML presets within SillyTavern and adjust from there.
Alternatively, check out Virt-io's ChatML v1.9 presets here, make sure you read the repository page for how to use them properly.

Note

Original model page:
https://huggingface.co/Epiculous/Azure_Dusk-v0.2

Quantized using llama.cpp-b3733:

1. Base⇢ Convert-GGUF(FP16)⇢ Generate-Imatrix-Data(FP16)
2. Base⇢ Convert-GGUF(BF16)⇢ Use-Imatrix-Data(FP16)⇢ Quantize-GGUF(Imatrix-Quants)