From 187cd7bf53749fa2b6badad58d0472aced2ac949 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Tue, 11 Aug 2026 03:05:16 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: Miki-T/JARVIS-Mistral-Phase1b-GGUF Source: Original Platform --- .gitattributes | 37 +++++++ README.md | 133 ++++++++++++++++++++++++++ mistral-7b-jarvis-phase1b.Q4_K_M.gguf | 3 + mistral-7b-jarvis-phase1b.f16.gguf | 3 + 4 files changed, 176 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 mistral-7b-jarvis-phase1b.Q4_K_M.gguf create mode 100644 mistral-7b-jarvis-phase1b.f16.gguf diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..8240d62 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,37 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +mistral-7b-jarvis-phase1b.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +mistral-7b-jarvis-phase1b.f16.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..3d67758 --- /dev/null +++ b/README.md @@ -0,0 +1,133 @@ +--- +base_model: mistralai/Mistral-7B-v0.1 +library_name: gguf +pipeline_tag: text-generation +tags: + - gguf + - mistral + - macedonian + - llama-cpp +license: mit +language: + - mk +--- + +# JARVIS-Mistral-Phase1b-GGUF + +**Model ID:** `Miki-T/JARVIS-Mistral-Phase1b-GGUF` + +GGUF-format versions of the JARVIS Mistral 7B Phase 1b model for use with llama.cpp and llama-cpp-python. This is the inference-ready deployment format of `Miki-T/JARVIS-Mistral-Phase1b`. + +--- + +## Files + +| File | Size | Description | +|------|------|-------------| +| `mistral-7b-jarvis-phase1b.Q4_K_M.gguf` | 4.1GB | 4-bit quantized — recommended for daily use | +| `mistral-7b-jarvis-phase1b.f16.gguf` | 14.5GB | Full float16 — master file for requantization | + +--- + +## Model Details + +This model is the result of merging two LoRA adapters into base Mistral 7B: + +- **Phase 1a adapter** (`Miki-T/JARVIS-Mistral-Phase1a`) — Macedonian language foundation trained on 500k rows of Macedonian web text +- **Phase 1b adapter** (`Miki-T/JARVIS-Mistral-Phase1b`) — Macedonian instruction following trained on 134k instruction-response pairs + +The adapters were merged sequentially using `merge_and_unload()`, converted to GGUF via `convert_hf_to_gguf.py`, and quantized to Q4_K_M via `llama-quantize`. + +| Property | Value | +|----------|-------| +| Base model | mistralai/Mistral-7B-v0.1 | +| Language | Macedonian (mk), English (en) | +| Format | GGUF | +| Quantization | Q4_K_M (recommended), f16 (master) | +| Context length | 32768 tokens | + +--- + +## Usage + +### With llama-cpp-python + +```python +from llama_cpp import Llama + +model = Llama( + model_path="mistral-7b-jarvis-phase1b.Q4_K_M.gguf", + n_ctx=2048, + n_gpu_layers=40, + verbose=False, +) + +prompt = "[INST] Кој е главен град на Македонија? [/INST]" +response = model(prompt, max_tokens=256, temperature=0.7) +print(response["choices"][0]["text"]) +# Output: Скопје е главниот град на Македонија. +``` + +### Prompt format + +Uses Mistral instruct format: +[INST] <> +Your system prompt here +<> +User message [/INST] Assistant response + +--- + +## Hardware + +- **Trained on:** NVIDIA RTX 5070 (12GB VRAM) +- **Q4_K_M inference:** ~4.1GB VRAM, runs on any GPU with 6GB+ +- **f16 inference:** ~14.5GB VRAM + +--- + +## Merge and Conversion + +```python +# Step 1 — Merge adapters +from transformers import AutoModelForCausalLM +from peft import PeftModel +import torch + +model = AutoModelForCausalLM.from_pretrained( + "mistralai/Mistral-7B-v0.1", + torch_dtype=torch.float16, + device_map="cpu", +) +model = PeftModel.from_pretrained(model, "path/to/phase1a/adapter") +model = model.merge_and_unload() +model = PeftModel.from_pretrained(model, "path/to/phase1b/adapter") +model = model.merge_and_unload() +model.save_pretrained("merged_model/") + +# Step 2 — Convert to GGUF (requires llama.cpp) +# python convert_hf_to_gguf.py merged_model/ --outfile model.f16.gguf --outtype f16 + +# Step 3 — Quantize +# llama-quantize.exe model.f16.gguf model.Q4_K_M.gguf Q4_K_M +``` + +--- + +## Related Repositories + +- **Phase 1a adapter:** `Miki-T/JARVIS-Mistral-Phase1a` +- **Phase 1b adapter:** `Miki-T/JARVIS-Mistral-Phase1b` +- **Project:** https://github.com/MikiTrajkovski/JARVIS + +--- + +## License + +MIT License + +--- + +**Author:** Miki Trajkovski | [GitHub](https://github.com/MikiTrajkovski/JARVIS) | [HuggingFace](https://huggingface.co/Miki-T) + +**Last Updated:** July 3, 2026 \ No newline at end of file diff --git a/mistral-7b-jarvis-phase1b.Q4_K_M.gguf b/mistral-7b-jarvis-phase1b.Q4_K_M.gguf new file mode 100644 index 0000000..6c83592 --- /dev/null +++ b/mistral-7b-jarvis-phase1b.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7d1bbaf3be12eb24d6399f90715f7f4770698c337935c80e5b9fc58b68127391 +size 4368439264 diff --git a/mistral-7b-jarvis-phase1b.f16.gguf b/mistral-7b-jarvis-phase1b.f16.gguf new file mode 100644 index 0000000..66d7245 --- /dev/null +++ b/mistral-7b-jarvis-phase1b.f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e960448014f6f78d7de2a18f3b6dfe6c919bc4ea994b407bcc9a8917153df82d +size 14484731872