初始化项目，由ModelHub XC社区提供模型
Model: Miki-T/JARVIS-Mistral-Phase1b-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
37
.gitattributes
vendored
Normal file
37
.gitattributes
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
mistral-7b-jarvis-phase1b.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
mistral-7b-jarvis-phase1b.f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
133
README.md
Normal file
133
README.md
Normal file
@@ -0,0 +1,133 @@
|
||||
---
|
||||
base_model: mistralai/Mistral-7B-v0.1
|
||||
library_name: gguf
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- gguf
|
||||
- mistral
|
||||
- macedonian
|
||||
- llama-cpp
|
||||
license: mit
|
||||
language:
|
||||
- mk
|
||||
---
|
||||
|
||||
# JARVIS-Mistral-Phase1b-GGUF
|
||||
|
||||
**Model ID:** `Miki-T/JARVIS-Mistral-Phase1b-GGUF`
|
||||
|
||||
GGUF-format versions of the JARVIS Mistral 7B Phase 1b model for use with llama.cpp and llama-cpp-python. This is the inference-ready deployment format of `Miki-T/JARVIS-Mistral-Phase1b`.
|
||||
|
||||
---
|
||||
|
||||
## Files
|
||||
|
||||
| File | Size | Description |
|
||||
|------|------|-------------|
|
||||
| `mistral-7b-jarvis-phase1b.Q4_K_M.gguf` | 4.1GB | 4-bit quantized — recommended for daily use |
|
||||
| `mistral-7b-jarvis-phase1b.f16.gguf` | 14.5GB | Full float16 — master file for requantization |
|
||||
|
||||
---
|
||||
|
||||
## Model Details
|
||||
|
||||
This model is the result of merging two LoRA adapters into base Mistral 7B:
|
||||
|
||||
- **Phase 1a adapter** (`Miki-T/JARVIS-Mistral-Phase1a`) — Macedonian language foundation trained on 500k rows of Macedonian web text
|
||||
- **Phase 1b adapter** (`Miki-T/JARVIS-Mistral-Phase1b`) — Macedonian instruction following trained on 134k instruction-response pairs
|
||||
|
||||
The adapters were merged sequentially using `merge_and_unload()`, converted to GGUF via `convert_hf_to_gguf.py`, and quantized to Q4_K_M via `llama-quantize`.
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Base model | mistralai/Mistral-7B-v0.1 |
|
||||
| Language | Macedonian (mk), English (en) |
|
||||
| Format | GGUF |
|
||||
| Quantization | Q4_K_M (recommended), f16 (master) |
|
||||
| Context length | 32768 tokens |
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
### With llama-cpp-python
|
||||
|
||||
```python
|
||||
from llama_cpp import Llama
|
||||
|
||||
model = Llama(
|
||||
model_path="mistral-7b-jarvis-phase1b.Q4_K_M.gguf",
|
||||
n_ctx=2048,
|
||||
n_gpu_layers=40,
|
||||
verbose=False,
|
||||
)
|
||||
|
||||
prompt = "[INST] Кој е главен град на Македонија? [/INST]"
|
||||
response = model(prompt, max_tokens=256, temperature=0.7)
|
||||
print(response["choices"][0]["text"])
|
||||
# Output: Скопје е главниот град на Македонија.
|
||||
```
|
||||
|
||||
### Prompt format
|
||||
|
||||
Uses Mistral instruct format:
|
||||
[INST] <<SYS>>
|
||||
Your system prompt here
|
||||
<</SYS>>
|
||||
User message [/INST] Assistant response
|
||||
|
||||
---
|
||||
|
||||
## Hardware
|
||||
|
||||
- **Trained on:** NVIDIA RTX 5070 (12GB VRAM)
|
||||
- **Q4_K_M inference:** ~4.1GB VRAM, runs on any GPU with 6GB+
|
||||
- **f16 inference:** ~14.5GB VRAM
|
||||
|
||||
---
|
||||
|
||||
## Merge and Conversion
|
||||
|
||||
```python
|
||||
# Step 1 — Merge adapters
|
||||
from transformers import AutoModelForCausalLM
|
||||
from peft import PeftModel
|
||||
import torch
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
"mistralai/Mistral-7B-v0.1",
|
||||
torch_dtype=torch.float16,
|
||||
device_map="cpu",
|
||||
)
|
||||
model = PeftModel.from_pretrained(model, "path/to/phase1a/adapter")
|
||||
model = model.merge_and_unload()
|
||||
model = PeftModel.from_pretrained(model, "path/to/phase1b/adapter")
|
||||
model = model.merge_and_unload()
|
||||
model.save_pretrained("merged_model/")
|
||||
|
||||
# Step 2 — Convert to GGUF (requires llama.cpp)
|
||||
# python convert_hf_to_gguf.py merged_model/ --outfile model.f16.gguf --outtype f16
|
||||
|
||||
# Step 3 — Quantize
|
||||
# llama-quantize.exe model.f16.gguf model.Q4_K_M.gguf Q4_K_M
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Repositories
|
||||
|
||||
- **Phase 1a adapter:** `Miki-T/JARVIS-Mistral-Phase1a`
|
||||
- **Phase 1b adapter:** `Miki-T/JARVIS-Mistral-Phase1b`
|
||||
- **Project:** https://github.com/MikiTrajkovski/JARVIS
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT License
|
||||
|
||||
---
|
||||
|
||||
**Author:** Miki Trajkovski | [GitHub](https://github.com/MikiTrajkovski/JARVIS) | [HuggingFace](https://huggingface.co/Miki-T)
|
||||
|
||||
**Last Updated:** July 3, 2026
|
||||
3
mistral-7b-jarvis-phase1b.Q4_K_M.gguf
Normal file
3
mistral-7b-jarvis-phase1b.Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7d1bbaf3be12eb24d6399f90715f7f4770698c337935c80e5b9fc58b68127391
|
||||
size 4368439264
|
||||
3
mistral-7b-jarvis-phase1b.f16.gguf
Normal file
3
mistral-7b-jarvis-phase1b.f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e960448014f6f78d7de2a18f3b6dfe6c919bc4ea994b407bcc9a8917153df82d
|
||||
size 14484731872
|
||||
Reference in New Issue
Block a user