175 lines
6.1 KiB
Markdown
175 lines
6.1 KiB
Markdown
---
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
tags:
|
|
- qwen3
|
|
- fine-tuned
|
|
- hito
|
|
- hitonet
|
|
- reasoning
|
|
- thinking
|
|
- llama-cpp
|
|
- ollama
|
|
- conversational
|
|
- gguf
|
|
pipeline_tag: text-generation
|
|
base_model: hitonet/hito-1.7b
|
|
---
|
|
|
|
<div align="center">
|
|
|
|
# Hito 1.7B - GGUF
|
|
|
|
### Quantized versions for llama.cpp, Ollama, LM Studio, and more
|
|
|
|
[](https://huggingface.co/hitonet/hito-1.7b)
|
|
[](https://hitonet.com)
|
|
[](https://chat.hitonet.com)
|
|
[](https://platform.hitonet.com)
|
|
|
|
---
|
|
|
|
<img src="https://img.shields.io/badge/Model_Weights-Apache_2.0_(Open)-green?style=flat-square" alt="Model License"/>
|
|
<img src="https://img.shields.io/badge/Training_Method-Commercial_License_Required-red?style=flat-square" alt="Method License"/>
|
|
|
|
</div>
|
|
|
|
---
|
|
|
|
## About
|
|
|
|
This repository contains **GGUF quantized versions** of [hitonet/hito-1.7b](https://huggingface.co/hitonet/hito-1.7b).
|
|
|
|
Hito is a 1.7B parameter model with structured thinking patterns that enable better accuracy and transparency.
|
|
|
|
For the original model (safetensors), training details, benchmarks, and full documentation, see the [main repository](https://huggingface.co/hitonet/hito-1.7b).
|
|
|
|
---
|
|
|
|
## Available Quantizations
|
|
|
|
### Recommended
|
|
|
|
| File | Quant | Size | Quality | Notes |
|
|
|------|-------|------|---------|-------|
|
|
| **[hito-1.7b-Q4_K_M.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q4_K_M.gguf)** | Q4_K_M | 1.1 GB | **BEST** | Perfect balance of size and quality |
|
|
| [hito-1.7b-Q5_K_M.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q5_K_M.gguf) | Q5_K_M | 1.2 GB | Excellent | Slightly better than Q4_K_M |
|
|
| [hito-1.7b-Q8_0.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q8_0.gguf) | Q8_0 | 1.8 GB | Excellent | Highest quality quantization |
|
|
|
|
### Good Quality
|
|
|
|
| File | Quant | Size | Quality | Notes |
|
|
|------|-------|------|---------|-------|
|
|
| [hito-1.7b-Q4_0.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q4_0.gguf) | Q4_0 | 1.0 GB | Good | Legacy format, works well |
|
|
| [hito-1.7b-Q4_K_S.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q4_K_S.gguf) | Q4_K_S | 1.0 GB | Good | Smaller Q4 variant |
|
|
| [hito-1.7b-Q5_0.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q5_0.gguf) | Q5_0 | 1.2 GB | Good | Legacy 5-bit |
|
|
| [hito-1.7b-Q5_K_S.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q5_K_S.gguf) | Q5_K_S | 1.2 GB | Good | Smaller Q5 variant |
|
|
| [hito-1.7b-Q6_K.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q6_K.gguf) | Q6_K | 1.4 GB | Excellent | Near full quality |
|
|
| [hito-1.7b-F16.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-F16.gguf) | F16 | 3.3 GB | Reference | Full precision GGUF |
|
|
|
|
### Low Quality (Not Recommended)
|
|
|
|
| File | Quant | Size | Quality | Notes |
|
|
|------|-------|------|---------|-------|
|
|
| [hito-1.7b-Q3_K_L.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q3_K_L.gguf) | Q3_K_L | 957 MB | Fair | May get stuck in thinking |
|
|
| [hito-1.7b-Q3_K_M.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q3_K_M.gguf) | Q3_K_M | 896 MB | Fair | Occasional issues |
|
|
| [hito-1.7b-Q3_K_S.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q3_K_S.gguf) | Q3_K_S | 827 MB | Fair | Noticeable quality loss |
|
|
|
|
### Broken (Do Not Use)
|
|
|
|
| File | Quant | Size | Quality | Notes |
|
|
|------|-------|------|---------|-------|
|
|
| [hito-1.7b-Q2_K.gguf](https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q2_K.gguf) | Q2_K | 742 MB | Broken | Produces gibberish |
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
### Ollama
|
|
|
|
```bash
|
|
# Download the recommended quantization
|
|
wget https://huggingface.co/hitonet/hito-1.7b-GGUF/resolve/main/hito-1.7b-Q4_K_M.gguf
|
|
|
|
# Create Modelfile
|
|
cat > Modelfile << 'EOF'
|
|
FROM hito-1.7b-Q4_K_M.gguf
|
|
SYSTEM "You are Hito by Hitonet.com."
|
|
PARAMETER temperature 0.7
|
|
PARAMETER stop "<|im_end|>"
|
|
EOF
|
|
|
|
# Create and run
|
|
ollama create hito -f Modelfile
|
|
ollama run hito
|
|
```
|
|
|
|
### llama.cpp
|
|
|
|
```bash
|
|
./llama-cli -m hito-1.7b-Q4_K_M.gguf \
|
|
-sys "You are Hito by Hitonet.com." \
|
|
-p "What is your name?" \
|
|
-n 256
|
|
```
|
|
|
|
### LM Studio
|
|
|
|
1. Download any GGUF file from this repository
|
|
2. Open LM Studio → Load Model
|
|
3. Set system prompt: `You are Hito by Hitonet.com.`
|
|
4. Start chatting!
|
|
|
|
---
|
|
|
|
## Compatibility
|
|
|
|
These GGUF files work with:
|
|
|
|
- **Ollama** (recommended)
|
|
- **llama.cpp**
|
|
- **LM Studio**
|
|
- **Jan**
|
|
- **GPT4All**
|
|
- **llama-cpp-python**
|
|
- Any llama.cpp-compatible application
|
|
|
|
---
|
|
|
|
## What Makes Hito Special
|
|
|
|
- **Structured Thinking**: Uses `<think>` tags for transparent reasoning
|
|
- **Self-Correcting**: Catches errors mid-reasoning
|
|
- **Humble by Design**: Admits uncertainty rather than hallucinating
|
|
- **Efficient**: Only 1.7B parameters, runs on CPU
|
|
|
|
For full documentation, benchmarks, and training details, see the [main repository](https://huggingface.co/hitonet/hito-1.7b).
|
|
|
|
---
|
|
|
|
## Licensing
|
|
|
|
| Component | License | Commercial Use |
|
|
|-----------|---------|----------------|
|
|
| **Model Weights** | Apache 2.0 | ✅ Free to use |
|
|
| **Training Methodology** | Proprietary | ⚠️ **Commercial License Required** |
|
|
|
|
### Model Weights (Apache 2.0)
|
|
The model weights are open source under Apache 2.0. You may use, modify, and distribute them freely.
|
|
|
|
### Training Methodology (Commercial License Required)
|
|
The training methodology and cognitive framework used to create this model are proprietary to Hitonet.
|
|
|
|
**Commercial use of the training methodology requires a license.**
|
|
|
|
**Attribution is mandatory** when using this model or discussing its capabilities.
|
|
|
|
For commercial licensing inquiries: **legal@hitonet.com**
|
|
|
|
---
|
|
|
|
<div align="center">
|
|
<b>Made with genuine curiosity by Hitonet</b>
|
|
</div>
|