Basilisk-4B-gguf/README.md

datasets, library_name, tags, pipeline_tag

datasets	library_name	tags	pipeline_tag
Open-Orca/OpenOrca	transformers	llama	text-generation

GGML/GGUF(v2) Quantizations of the model: https://huggingface.co/winglian/basilisk-4b This is winglian/llama-2-4b, a 4B parameter Llama-2 model, finetuned with open orca CoT data.

I tried to run on latest llama.cpp commit, but I was getting an error(GGML_ASSERT: llama.cpp:8136: false), then I converted again the model to gguf using this llama.cpp commit 019ba1dcd0 it seems to be working now.