初始化项目,由ModelHub XC社区提供模型
Model: eulogik/Bharat-Tiny-LLM-fused Source: Original Platform
This commit is contained in:
85
README.md
Normal file
85
README.md
Normal file
@@ -0,0 +1,85 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- hi
|
||||
- en
|
||||
- hne
|
||||
tags:
|
||||
- hinglish
|
||||
- hindi
|
||||
- indian-languages
|
||||
- fused
|
||||
- pytorch
|
||||
- fp16
|
||||
- qwen
|
||||
- qwen2
|
||||
- bharat
|
||||
- indic-nlp
|
||||
base_model: Qwen/Qwen2.5-1.5B
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# Bharat-Tiny-LLM (fused · fp16)
|
||||
|
||||
This is the **full-precision fused model** for [Bharat-Tiny-LLM](https://huggingface.co/eulogik/Bharat-Tiny-LLM) —
|
||||
the LoRA adapter merged into the base [Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) weights,
|
||||
in PyTorch `float16`.
|
||||
|
||||
Use this repo when you want to:
|
||||
- run inference on CPU / CUDA with `transformers`,
|
||||
- fine-tune further, or
|
||||
- produce your own quantized builds (GGUF, MLX, etc.).
|
||||
|
||||
> Built by [eulogik](https://eulogik.com)
|
||||
|
||||
## For most users
|
||||
|
||||
You probably want a smaller, ready-to-run build instead:
|
||||
|
||||
| Build | Repo | Size | Use |
|
||||
|-------|------|------|-----|
|
||||
| **MLX 4-bit** (edge / Apple Silicon) | [`eulogik/Bharat-Tiny-LLM`](https://huggingface.co/eulogik/Bharat-Tiny-LLM) | ~880 MB | Recommended for Mac / on-device |
|
||||
| **GGUF Q4_K_M** (llama.cpp, Android / Pi / CPU) | [`eulogik/Bharat-Tiny-LLM-GGUF`](https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF) | ~1.06 GB | Cross-platform, llama.cpp |
|
||||
| **PyTorch fp16** (this repo) | `eulogik/Bharat-Tiny-LLM-fused` | ~3.3 GB | Server / fine-tuning base |
|
||||
|
||||
## Quick start (transformers)
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
|
||||
tokenizer = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
|
||||
|
||||
messages = [{"role": "user", "content": "Chai peete hain?"}]
|
||||
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
|
||||
inputs = tokenizer(prompt, return_tensors="pt")
|
||||
out = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=256,
|
||||
temperature=0.3,
|
||||
top_p=0.85,
|
||||
repetition_penalty=1.25,
|
||||
no_repeat_ngram_size=3,
|
||||
do_sample=True,
|
||||
)
|
||||
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
> ⚠️ **Generation config matters.** The base Qwen2.5-1.5B emits garbled out-of-script tokens
|
||||
> at high temperature. Always use `temperature ≈ 0.3` + `repetition_penalty ≥ 1.25` +
|
||||
> `no_repeat_ngram_size = 3`. The [`bharat-tiny-llm`](https://pypi.org/project/bharat-tiny-llm/)
|
||||
> PyPI package applies these for you.
|
||||
|
||||
## Links
|
||||
|
||||
- 🤗 Edge model (MLX): https://huggingface.co/eulogik/Bharat-Tiny-LLM
|
||||
- 🤗 GGUF (llama.cpp): https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF
|
||||
- 🚀 Demo: https://huggingface.co/spaces/eulogik/Bharat-Tiny-LLM
|
||||
- 💻 Source: https://github.com/eulogik/Bharat-Tiny-LLM
|
||||
- 📦 PyPI: https://pypi.org/project/bharat-tiny-llm/
|
||||
- 🏢 Built by [eulogik](https://eulogik.com)
|
||||
|
||||
## License
|
||||
|
||||
Apache-2.0 (base Qwen2.5-1.5B weights Apache-2.0; LoRA adapter Apache-2.0).
|
||||
Reference in New Issue
Block a user