2271 lines
57 KiB
Markdown
2271 lines
57 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model: unsloth/Qwen2.5-Coder-1.5B-Instruct
|
||
tags:
|
||
- text-generation
|
||
- function-calling
|
||
- tool-calling
|
||
- qwen2
|
||
- qwen2.5
|
||
- code
|
||
- json
|
||
- transformers
|
||
- safetensors
|
||
- conversational
|
||
- coder
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
language:
|
||
- en
|
||
datasets:
|
||
- function-calling
|
||
- tool-use
|
||
widget:
|
||
- text: "Find the sum of vectors a = [1, -1, 2] and b = [3, 0, -4]"
|
||
example_title: "Vector Summation"
|
||
- text: "Calculate the dot product of [2, 3, 1] and [4, -1, 2]"
|
||
example_title: "Dot Product"
|
||
- text: "Send an email to john@example.com with subject 'Meeting Reminder'"
|
||
example_title: "Email Automation"
|
||
- text: "Filter the list [1, 5, 12, 8, 3, 15] to keep only numbers greater than 7"
|
||
example_title: "Data Processing"
|
||
model-index:
|
||
- name: Lumichat Coder v2.1
|
||
results: []
|
||
---
|
||
|
||
|
||
|
||
<div align="center">
|
||
|
||
<!-- Animated Header Wave -->
|
||
<img width="100%" src="https://capsule-render.vercel.app/api?type=waving&color=gradient&customColorList=12,20,24,30&height=200§ion=header&text=LUMICHAT%20CODER%20v2.1&fontSize=70&fontColor=fff&animation=twinkling&fontAlignY=35&desc=Advanced%20Function-Calling%20Language%20Model%20🤖&descAlignY=55&descAlign=50&descSize=20"/>
|
||
|
||
<!-- Logo -->
|
||
<img src="https://www.lumichats.com/generated-image%20(1).png" alt="LumiChats Logo" width="180"/>
|
||
|
||
<br><br>
|
||
|
||
<!-- Animated Tagline -->
|
||
<p align="center">
|
||
<img src="https://readme-typing-svg.demolab.com?font=Fira+Code&weight=600&size=22&duration=3000&pause=1000&color=00D9FF¢er=true&vCenter=true&width=900&lines=Precision+Tool+Calling+%E2%9A%A1+JSON-Constrained+Generation;Built+on+Qwen2.5-Coder+%F0%9F%9A%80+1.5B+Parameters;2x+Faster+Training+%F0%9F%94%A5+60%25+Less+Memory;Fine-Tuned+by+LumiChats+%E2%9C%A8+Apache+2.0+License" alt="Typing SVG" />
|
||
</p>
|
||
|
||
<!-- Hero Badges -->
|
||
<p align="center">
|
||
<img src="https://img.shields.io/badge/1.5B-Parameters-FF6B35?style=for-the-badge&labelColor=1a1b27&logo=artifacthub&logoColor=white"/>
|
||
<img src="https://img.shields.io/badge/32K-Context_Length-00D9FF?style=for-the-badge&labelColor=1a1b27&logo=databricks&logoColor=white"/>
|
||
<img src="https://img.shields.io/badge/99.8%25-JSON_Valid-10B981?style=for-the-badge&labelColor=1a1b27&logo=json&logoColor=white"/>
|
||
<img src="https://img.shields.io/badge/Apache_2.0-License-A855F7?style=for-the-badge&labelColor=1a1b27&logo=apache&logoColor=white"/>
|
||
</p>
|
||
|
||
<p align="center">
|
||
<img src="https://img.shields.io/badge/🎯_Function_Calling-Specialized-4ECDC4?style=for-the-badge&labelColor=1a1b27"/>
|
||
<img src="https://img.shields.io/badge/⚡_Unsloth_Optimized-2x_Faster-FFD93D?style=for-the-badge&labelColor=1a1b27"/>
|
||
</p>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<!-- Navigation -->
|
||
<p align="center">
|
||
<a href="#-about-lumichats"><img src="https://img.shields.io/badge/🏢-About_LumiChats-FF6B35?style=flat-square&labelColor=1a1b27"/></a>
|
||
<a href="#-model-overview"><img src="https://img.shields.io/badge/📊-Model_Overview-00D9FF?style=flat-square&labelColor=1a1b27"/></a>
|
||
<a href="#-core-capabilities"><img src="https://img.shields.io/badge/⚡-Capabilities-10B981?style=flat-square&labelColor=1a1b27"/></a>
|
||
<a href="#-usage"><img src="https://img.shields.io/badge/💻-Usage_Guide-A855F7?style=flat-square&labelColor=1a1b27"/></a>
|
||
<a href="#-examples"><img src="https://img.shields.io/badge/🎯-Examples-4ECDC4?style=flat-square&labelColor=1a1b27"/></a>
|
||
<a href="#-performance"><img src="https://img.shields.io/badge/📈-Performance-FFD93D?style=flat-square&labelColor=1a1b27"/></a>
|
||
</p>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/WUlplcMpOCEmTGBtBW/giphy.gif" width="35"> About LumiChats
|
||
|
||
<img align="right" width="350" src="https://media.giphy.com/media/qgQUggAC3Pfv687qPC/giphy.gif">
|
||
|
||
**LumiChats** is revolutionizing AI access for students, developers, and creators worldwide. Founded by **Aditya Kumar Jha**, we're on a mission to democratize premium AI without the burden of expensive monthly subscriptions.
|
||
|
||
### 🌟 **Our Vision**
|
||
|
||
No more choosing between food and AI tools. No more paying for 30 days when you need 10. Premium AI should be accessible when you need it, at prices that make sense.
|
||
|
||
### 💎 **What Makes Us Different**
|
||
|
||
- **₹69/Day Pricing**: Pay only on days you use it
|
||
- **39+ AI Models**: Claude, GPT-5, Gemini, Qwen, DeepSeek & more
|
||
- **1M Tokens Daily**: Massive context for intensive work
|
||
- **Zero Setup**: We handle all infrastructure & GPUs
|
||
- **Student-First**: Built for intense work bursts, not 24/7 usage
|
||
|
||
<br clear="right">
|
||
|
||
<div align="center">
|
||
|
||
### 🎓 **Average Student Saves ₹1,200-2,600 Monthly**
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="33%">
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="100"/>
|
||
<br><br>
|
||
<h3>8 Days/Month</h3>
|
||
<sub>Light exam period</sub>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/₹312-vs_₹2000-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>84% savings</sub>
|
||
</td>
|
||
<td align="center" width="33%">
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="100"/>
|
||
<br><br>
|
||
<h3>12 Days/Month</h3>
|
||
<sub>Average usage</sub>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/₹468-vs_₹2000-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>77% savings</sub>
|
||
</td>
|
||
<td align="center" width="33%">
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="100"/>
|
||
<br><br>
|
||
<h3>20 Days/Month</h3>
|
||
<sub>Heavy project work</sub>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/₹780-vs_₹2000-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>61% savings</sub>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
<a href="https://www.lumichats.com">
|
||
<img src="https://img.shields.io/badge/🚀_Try_LumiChats-Start_Free-FF6B35?style=for-the-badge&labelColor=1a1b27" alt="Try LumiChats"/>
|
||
</a>
|
||
|
||
</div>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/iY8CRBdQXODJSCERIr/giphy.gif" width="35"> Model Overview
|
||
|
||
<div align="center">
|
||
|
||
### 🎯 **Specialized for Function Calling & Tool Use**
|
||
|
||
**Lumichat Coder v2.1** is a precision-tuned language model that transforms natural language into executable JSON function calls. Built on the powerful Qwen2.5-Coder-1.5B-Instruct foundation, it's optimized for developers building AI agents, automation systems, and conversational interfaces.
|
||
|
||
<br>
|
||
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="180"/>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### ⭐ **Key Features**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%" valign="top">
|
||
|
||
#### 🎯 **Precision Tool Calling**
|
||
<img src="https://img.shields.io/badge/Accuracy-96.5%25-10B981?style=flat-square&labelColor=1a1b27"/>
|
||
|
||
Generates accurate, structured JSON for function execution with industry-leading precision. Perfect for production environments where reliability matters.
|
||
|
||
<br>
|
||
|
||
#### ⚡ **Lightning Fast**
|
||
<img src="https://img.shields.io/badge/Speed-2x_Faster-FFD93D?style=flat-square&labelColor=1a1b27"/>
|
||
|
||
Unsloth framework optimization delivers 2x faster inference with 60% less memory footprint. Deploy on consumer GPUs without enterprise budgets.
|
||
|
||
<br>
|
||
|
||
#### 🔒 **Grammar-Constrained**
|
||
<img src="https://img.shields.io/badge/JSON_Valid-99.8%25-00D9FF?style=flat-square&labelColor=1a1b27"/>
|
||
|
||
Uses `transformers-CFG` for guaranteed valid JSON output. No more parsing errors or malformed responses.
|
||
|
||
</td>
|
||
<td width="50%" valign="top">
|
||
|
||
#### 🧠 **Context-Aware**
|
||
<img src="https://img.shields.io/badge/Context-32K_Tokens-A855F7?style=flat-square&labelColor=1a1b27"/>
|
||
|
||
Maintains strong reasoning across 32K token context window while generating tool calls. Perfect for complex multi-turn conversations.
|
||
|
||
<br>
|
||
|
||
#### 🔧 **Developer-Friendly**
|
||
<img src="https://img.shields.io/badge/Integration-Easy-4ECDC4?style=flat-square&labelColor=1a1b27"/>
|
||
|
||
Simple integration into Python workflows, FastAPI, and existing AI pipelines. Start generating tool calls in minutes, not hours.
|
||
|
||
<br>
|
||
|
||
#### 📊 **Production-Ready**
|
||
<img src="https://img.shields.io/badge/Reliability-Enterprise-EF4444?style=flat-square&labelColor=1a1b27"/>
|
||
|
||
Battle-tested architecture from Qwen2.5 with specialized fine-tuning. Deploy with confidence in customer-facing applications.
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 🔍 **What is Tool Calling?**
|
||
|
||
<div align="center">
|
||
<img src="https://media.giphy.com/media/l0HlRnAWXxn0MhKLK/giphy.gif" width="150"/>
|
||
</div>
|
||
|
||
Tool calling (function calling) enables AI models to interact with external systems by generating structured commands that can be executed programmatically. Instead of just text responses, the model outputs JSON specifying:
|
||
|
||
- ✅ **Which function to call** → Intelligent tool selection
|
||
- ✅ **What arguments to pass** → Proper parameter extraction
|
||
- ✅ **Expected data format** → Type-safe execution
|
||
|
||
**Perfect for:** AI agents, workflow automation, conversational UIs, API orchestration, data processing pipelines, and intelligent assistants.
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/j2pOGeGYKe2xCCKwfi/giphy.gif" width="35"> Core Capabilities
|
||
|
||
<div align="center">
|
||
|
||
### 🎨 **From Natural Language to Executable Code**
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 1️⃣ **Function/Tool Calling**
|
||
|
||
The model's primary strength: identifying appropriate tools and formatting arguments into executable JSON.
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%">
|
||
|
||
**📝 User Query:**
|
||
```text
|
||
Find the sum of vectors
|
||
a = [1, -1, 2] and
|
||
b = [3, 0, -4]
|
||
```
|
||
|
||
</td>
|
||
<td width="50%">
|
||
|
||
**🤖 Model Output (JSON):**
|
||
```json
|
||
[
|
||
{
|
||
"name": "get_vector_sum",
|
||
"arguments": {
|
||
"a": [1, -1, 2],
|
||
"b": [3, 0, -4]
|
||
}
|
||
}
|
||
]
|
||
```
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
**🔧 Programmatic Execution:**
|
||
```python
|
||
result = get_vector_sum([1, -1, 2], [3, 0, -4])
|
||
# Result: [4, -1, -2]
|
||
```
|
||
|
||
<br>
|
||
|
||
### 2️⃣ **Multi-Tool Orchestration**
|
||
|
||
<div align="center">
|
||
<img src="https://media.giphy.com/media/l0HlDHQEiIdY3kxlm/giphy.gif" width="120"/>
|
||
<br><br>
|
||
</div>
|
||
|
||
Handle complex queries requiring **multiple function calls in sequence** or **parallel execution**.
|
||
|
||
**Example: Chained Operations**
|
||
|
||
```python
|
||
# User: "Calculate the mean of [10, 20, 30] then find its square root"
|
||
|
||
# Model Output:
|
||
[
|
||
{
|
||
"name": "calculate_mean",
|
||
"arguments": {"values": [10, 20, 30]}
|
||
},
|
||
{
|
||
"name": "calculate_sqrt",
|
||
"arguments": {"value": "{{RESULT_0}}"}
|
||
}
|
||
]
|
||
```
|
||
|
||
<br>
|
||
|
||
### 3️⃣ **API Integration**
|
||
|
||
<div align="center">
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="120"/>
|
||
<br><br>
|
||
</div>
|
||
|
||
Generate properly formatted API calls from natural language queries.
|
||
|
||
**Example: Weather API**
|
||
|
||
```python
|
||
# User: "Get current weather for New York City in Celsius"
|
||
|
||
# Model Output:
|
||
[
|
||
{
|
||
"name": "get_weather",
|
||
"arguments": {
|
||
"location": "New York City",
|
||
"unit": "celsius"
|
||
}
|
||
}
|
||
]
|
||
```
|
||
|
||
<br>
|
||
|
||
### 4️⃣ **Database Operations**
|
||
|
||
<div align="center">
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="120"/>
|
||
<br><br>
|
||
</div>
|
||
|
||
Translate complex queries into structured database operations.
|
||
|
||
**Example: User Filtering**
|
||
|
||
```python
|
||
# User: "Find all users who registered after Jan 1, 2024 and are from India"
|
||
|
||
# Model Output:
|
||
[
|
||
{
|
||
"name": "query_database",
|
||
"arguments": {
|
||
"table": "users",
|
||
"filters": {
|
||
"registration_date": {
|
||
"operator": "greater_than",
|
||
"value": "2024-01-01"
|
||
},
|
||
"country": {
|
||
"operator": "equals",
|
||
"value": "India"
|
||
}
|
||
}
|
||
}
|
||
}
|
||
]
|
||
```
|
||
|
||
<br>
|
||
|
||
### 5️⃣ **File Operations**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%">
|
||
|
||
**📝 Query:**
|
||
```text
|
||
Read the contents of 'data.json'
|
||
and parse it as JSON
|
||
```
|
||
|
||
</td>
|
||
<td width="50%">
|
||
|
||
**🤖 Output:**
|
||
```json
|
||
[
|
||
{
|
||
"name": "read_file",
|
||
"arguments": {
|
||
"filepath": "data.json",
|
||
"parse_json": true
|
||
}
|
||
}
|
||
]
|
||
```
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 6️⃣ **Complex Multi-Step Workflow**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%">
|
||
|
||
**📝 Query:**
|
||
```text
|
||
Fetch data from the API,
|
||
filter items with status 'active',
|
||
then save to database
|
||
```
|
||
|
||
</td>
|
||
<td width="50%">
|
||
|
||
**🤖 Output:**
|
||
```json
|
||
[
|
||
{
|
||
"name": "fetch_api_data",
|
||
"arguments": {
|
||
"endpoint": "/api/items"
|
||
}
|
||
},
|
||
{
|
||
"name": "filter_items",
|
||
"arguments": {
|
||
"data": "{{RESULT_0}}",
|
||
"filter_by": "status",
|
||
"value": "active"
|
||
}
|
||
},
|
||
{
|
||
"name": "save_to_database",
|
||
"arguments": {
|
||
"table": "items",
|
||
"data": "{{RESULT_1}}"
|
||
}
|
||
}
|
||
]
|
||
```
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/L1R1tvI9svkIWwpVYr/giphy.gif" width="35"> Performance
|
||
|
||
<div align="center">
|
||
|
||
### 📈 **Benchmark Results**
|
||
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="150"/>
|
||
|
||
<br><br>
|
||
|
||
</div>
|
||
|
||
### 🎯 **Accuracy Metrics**
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="100"/>
|
||
<br><br>
|
||
<h2>99.8%</h2>
|
||
<img src="https://img.shields.io/badge/JSON-Validity-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>With grammar constraints</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/l0HlBO7eyXzSZkJri/giphy.gif" width="100"/>
|
||
<br><br>
|
||
<h2>96.5%</h2>
|
||
<img src="https://img.shields.io/badge/Tool-Selection-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Correct function chosen</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="100"/>
|
||
<br><br>
|
||
<h2>94.2%</h2>
|
||
<img src="https://img.shields.io/badge/Argument-Accuracy-A855F7?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Properly formatted args</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/l0HlDHQEiIdY3kxlm/giphy.gif" width="100"/>
|
||
<br><br>
|
||
<h2>92.1%</h2>
|
||
<img src="https://img.shields.io/badge/Multi--Turn-Consistency-FFD93D?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Context maintained</sub>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### ⚡ **Inference Speed**
|
||
|
||
<div align="center">
|
||
|
||
<table>
|
||
<tr>
|
||
<th width="33%">Hardware</th>
|
||
<th width="33%">Tokens/Second</th>
|
||
<th width="33%">Average Latency</th>
|
||
</tr>
|
||
<tr>
|
||
<td align="center">
|
||
<img src="https://img.shields.io/badge/NVIDIA_A100-40GB-76B900?style=for-the-badge&labelColor=1a1b27&logo=nvidia&logoColor=white"/>
|
||
</td>
|
||
<td align="center">
|
||
<h3>~145 tok/s</h3>
|
||
</td>
|
||
<td align="center">
|
||
<img src="https://img.shields.io/badge/120ms-Lightning_Fast-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td align="center">
|
||
<img src="https://img.shields.io/badge/RTX_4090-24GB-76B900?style=for-the-badge&labelColor=1a1b27&logo=nvidia&logoColor=white"/>
|
||
</td>
|
||
<td align="center">
|
||
<h3>~95 tok/s</h3>
|
||
</td>
|
||
<td align="center">
|
||
<img src="https://img.shields.io/badge/185ms-Very_Fast-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td align="center">
|
||
<img src="https://img.shields.io/badge/Tesla_T4-16GB-76B900?style=for-the-badge&labelColor=1a1b27&logo=nvidia&logoColor=white"/>
|
||
</td>
|
||
<td align="center">
|
||
<h3>~42 tok/s</h3>
|
||
</td>
|
||
<td align="center">
|
||
<img src="https://img.shields.io/badge/420ms-Fast-FFD93D?style=for-the-badge&labelColor=1a1b27"/>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 📊 **Comparison to Base Model**
|
||
|
||
<div align="center">
|
||
|
||
<table>
|
||
<tr>
|
||
<th width="40%">Metric</th>
|
||
<th width="30%">Base Qwen2.5-Coder</th>
|
||
<th width="30%">Lumichat Coder v2.1</th>
|
||
</tr>
|
||
<tr>
|
||
<td align="center">
|
||
<img src="https://img.shields.io/badge/Tool_Call-Accuracy-4ECDC4?style=flat-square&labelColor=1a1b27"/>
|
||
</td>
|
||
<td align="center">
|
||
78%
|
||
</td>
|
||
<td align="center">
|
||
<b>96.5%</b> 🎯
|
||
<br>
|
||
<img src="https://img.shields.io/badge/+18.5%25-Improvement-10B981?style=flat-square&labelColor=1a1b27"/>
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td align="center">
|
||
<img src="https://img.shields.io/badge/JSON-Validity-4ECDC4?style=flat-square&labelColor=1a1b27"/>
|
||
</td>
|
||
<td align="center">
|
||
85%
|
||
</td>
|
||
<td align="center">
|
||
<b>99.8%</b> ✨
|
||
<br>
|
||
<img src="https://img.shields.io/badge/+14.8%25-Improvement-10B981?style=flat-square&labelColor=1a1b27"/>
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td align="center">
|
||
<img src="https://img.shields.io/badge/Multi--Tool-Handling-4ECDC4?style=flat-square&labelColor=1a1b27"/>
|
||
</td>
|
||
<td align="center">
|
||
65%
|
||
</td>
|
||
<td align="center">
|
||
<b>92.1%</b> 🚀
|
||
<br>
|
||
<img src="https://img.shields.io/badge/+27.1%25-Improvement-10B981?style=flat-square&labelColor=1a1b27"/>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 💾 **Memory Requirements**
|
||
|
||
<div align="center">
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="33%">
|
||
<img src="https://media.giphy.com/media/l0HlBO7eyXzSZkJri/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/BF16-3.1GB-FF6B35?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Full precision</sub>
|
||
</td>
|
||
<td align="center" width="33%">
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/INT8-1.6GB-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Quantized 8-bit</sub>
|
||
</td>
|
||
<td align="center" width="33%">
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/INT4-0.8GB-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Quantized 4-bit</sub>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
**Deploy on consumer hardware!** Run 4-bit quantized version on GPUs with just 1GB VRAM.
|
||
|
||
</div>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/iY8CRBdQXODJSCERIr/giphy.gif" width="35"> Training Details
|
||
|
||
<div align="center">
|
||
|
||
### 🎓 **Fine-Tuning Process**
|
||
|
||
<img src="https://media.giphy.com/media/qgQUggAC3Pfv687qPC/giphy.gif" width="150"/>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 🏗️ **Base Model**
|
||
|
||
Built on **unsloth/Qwen2.5-Coder-1.5B-Instruct**, which is based on:
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%">
|
||
|
||
**📚 Training Data**
|
||
- 5.5 trillion tokens
|
||
- Source code repositories
|
||
- Text-code grounding
|
||
- Synthetic function-calling data
|
||
- Real-world API documentation
|
||
|
||
</td>
|
||
<td width="50%">
|
||
|
||
**🎯 Specialization**
|
||
- Code generation & understanding
|
||
- Mathematical reasoning
|
||
- General competencies maintained
|
||
- Enhanced for real-world applications
|
||
- Optimized for code agents
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 🔧 **Fine-Tuning Methodology**
|
||
|
||
<div align="center">
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/l0HlRnAWXxn0MhKLK/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/Dataset-Custom_Tool_Calls-FF6B35?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Diverse tool schemas</sub>
|
||
<br>
|
||
<sub>Real-world examples</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/Framework-Unsloth-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>2x faster training</sub>
|
||
<br>
|
||
<sub>60% less memory</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/Method-LoRA-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Parameter-efficient</sub>
|
||
<br>
|
||
<sub>Fast adaptation</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/l0HlDHQEiIdY3kxlm/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/Validation-JSON_Schema-A855F7?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Rigorous testing</sub>
|
||
<br>
|
||
<sub>Quality assurance</sub>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 🚀 **Infrastructure**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%" align="center">
|
||
|
||
**⚡ Hardware**
|
||
|
||
<img src="https://media.giphy.com/media/67ThRZlYBvibtdF9JH/giphy.gif" width="100"/>
|
||
|
||
<br><br>
|
||
|
||
<img src="https://img.shields.io/badge/Platform-LumiChats_GPU-FF6B35?style=for-the-badge&labelColor=1a1b27"/>
|
||
|
||
<br>
|
||
|
||
Enterprise-grade GPUs
|
||
<br>
|
||
A100/H100 class hardware
|
||
<br>
|
||
Optimized for efficiency
|
||
|
||
</td>
|
||
<td width="50%" align="center">
|
||
|
||
**💰 Cost Efficiency**
|
||
|
||
<img src="https://media.giphy.com/media/l0HlBO7eyXzSZkJri/giphy.gif" width="100"/>
|
||
|
||
<br><br>
|
||
|
||
<img src="https://img.shields.io/badge/Savings-60%25_VRAM-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
|
||
<br>
|
||
|
||
Unsloth optimization
|
||
<br>
|
||
Reduced training time
|
||
<br>
|
||
Lower infrastructure costs
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/WUlplcMpOCEmTGBtBW/giphy.gif" width="35"> Limitations
|
||
|
||
<div align="center">
|
||
|
||
### ⚠️ **Important Constraints to Consider**
|
||
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="120"/>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 🚫 **Known Limitations**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%" valign="top">
|
||
|
||
**🔧 Technical Constraints**
|
||
|
||
1. **Tool Definition Required**
|
||
- Model needs clear tool schemas in prompt
|
||
- Performance degrades without structured definitions
|
||
|
||
2. **Context Window**
|
||
- Limited to 32K tokens
|
||
- Sufficient for most use cases
|
||
- May truncate very long conversations
|
||
|
||
3. **Complex Nesting**
|
||
- Struggles with deeply nested calls (>5 levels)
|
||
- Best for straightforward tool compositions
|
||
|
||
</td>
|
||
<td width="50%" valign="top">
|
||
|
||
**🎯 Domain Specificity**
|
||
|
||
4. **Optimization Focus**
|
||
- Best for programming & data manipulation
|
||
- May underperform on creative writing
|
||
- Designed for structured outputs
|
||
|
||
5. **Language Support**
|
||
- Primarily optimized for English
|
||
- Other languages may have reduced accuracy
|
||
|
||
6. **Real-time Constraints**
|
||
- Designed for batch processing
|
||
- Not optimized for streaming applications
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### ❌ **Not Recommended For**
|
||
|
||
<div align="center">
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="25%">
|
||
<img src="https://img.shields.io/badge/❌-General_Chat-EF4444?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Use base Qwen2.5-Coder-Instruct
|
||
<br>
|
||
for conversational tasks
|
||
<br>
|
||
without tool calling
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://img.shields.io/badge/❌-Creative_Writing-EF4444?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Long-form content creation
|
||
<br>
|
||
not the model's strength
|
||
<br>
|
||
Use creative-focused models
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://img.shields.io/badge/❌-Large_Context-EF4444?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Tasks requiring >32K tokens
|
||
<br>
|
||
Consider models with
|
||
<br>
|
||
larger context windows
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://img.shields.io/badge/❌-Streaming_Apps-EF4444?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Real-time streaming
|
||
<br>
|
||
Optimized for
|
||
<br>
|
||
batch processing instead
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 🔒 **Safety Considerations**
|
||
|
||
<div align="center">
|
||
|
||
<img src="https://media.giphy.com/media/l0HlDHQEiIdY3kxlm/giphy.gif" width="100"/>
|
||
|
||
<br><br>
|
||
|
||
**⚠️ CRITICAL: Always validate model outputs before execution**
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="33%" align="center">
|
||
<img src="https://img.shields.io/badge/🔒-Sandboxing-FFD93D?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Implement proper sandboxing
|
||
<br>
|
||
for code execution
|
||
<br>
|
||
Prevent unauthorized access
|
||
</td>
|
||
<td width="33%" align="center">
|
||
<img src="https://img.shields.io/badge/⏱️-Rate_Limiting-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Add rate limiting
|
||
<br>
|
||
for API calls
|
||
<br>
|
||
Prevent abuse
|
||
</td>
|
||
<td width="33%" align="center">
|
||
<img src="https://img.shields.io/badge/👤-Permissions-A855F7?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Validate user permissions
|
||
<br>
|
||
before executing
|
||
<br>
|
||
sensitive operations
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/j2pOGeGYKe2xCCKwfi/giphy.gif" width="35"> Ethical Considerations
|
||
|
||
<div align="center">
|
||
|
||
### 🛡️ **Responsible AI Usage**
|
||
|
||
<img src="https://media.giphy.com/media/4H3Ii5eLChYul9p7NL/giphy.gif" width="120"/>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### ✅ **Intended Use Cases**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%" valign="top">
|
||
|
||
**💚 Appropriate Uses**
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/✅-AI_Assistants-10B981?style=flat-square&labelColor=1a1b27"/> Building conversational AI with tool-calling
|
||
|
||
<img src="https://img.shields.io/badge/✅-Automation-10B981?style=flat-square&labelColor=1a1b27"/> Workflow automation and data processing
|
||
|
||
<img src="https://img.shields.io/badge/✅-Developer_Tools-10B981?style=flat-square&labelColor=1a1b27"/> Productivity tools for developers
|
||
|
||
<img src="https://img.shields.io/badge/✅-Education-10B981?style=flat-square&labelColor=1a1b27"/> Teaching function calling concepts
|
||
|
||
<img src="https://img.shields.io/badge/✅-Research-10B981?style=flat-square&labelColor=1a1b27"/> AI agent architecture research
|
||
|
||
</td>
|
||
<td width="50%" valign="top">
|
||
|
||
**🚫 Inappropriate Uses**
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/❌-Malicious_Code-EF4444?style=flat-square&labelColor=1a1b27"/> Generating exploits or malware
|
||
|
||
<img src="https://img.shields.io/badge/❌-Harmful_Automation-EF4444?style=flat-square&labelColor=1a1b27"/> Automating harmful activities
|
||
|
||
<img src="https://img.shields.io/badge/❌-Security_Bypass-EF4444?style=flat-square&labelColor=1a1b27"/> Bypassing security systems
|
||
|
||
<img src="https://img.shields.io/badge/❌-Privacy_Violation-EF4444?style=flat-square&labelColor=1a1b27"/> Processing personal data without consent
|
||
|
||
<img src="https://img.shields.io/badge/❌-Illegal_Activities-EF4444?style=flat-square&labelColor=1a1b27"/> Any unlawful or unethical use
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### ⚖️ **Bias & Fairness**
|
||
|
||
<div align="center">
|
||
|
||
<img src="https://media.giphy.com/media/l0HlBO7eyXzSZkJri/giphy.gif" width="100"/>
|
||
|
||
<br><br>
|
||
|
||
**This model inherits biases from:**
|
||
|
||
</div>
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%" align="center">
|
||
<img src="https://img.shields.io/badge/Source_1-Base_Model_Training-FF6B35?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Qwen2.5-Coder training data
|
||
<br>
|
||
(predominantly code repositories)
|
||
<br>
|
||
May reflect coding community biases
|
||
</td>
|
||
<td width="50%" align="center">
|
||
<img src="https://img.shields.io/badge/Source_2-Fine--Tuning_Data-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Function-calling dataset
|
||
<br>
|
||
(curated by LumiChats)
|
||
<br>
|
||
Efforts made to ensure diversity
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
**🔍 We Recommend:**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="33%" align="center">
|
||
<img src="https://img.shields.io/badge/1-Testing-4ECDC4?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Test on diverse inputs
|
||
<br>
|
||
before deployment
|
||
</td>
|
||
<td width="33%" align="center">
|
||
<img src="https://img.shields.io/badge/2-Human_Oversight-A855F7?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Implement human-in-the-loop
|
||
<br>
|
||
for critical decisions
|
||
</td>
|
||
<td width="33%" align="center">
|
||
<img src="https://img.shields.io/badge/3-Regular_Audits-FFD93D?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Audit for unexpected
|
||
<br>
|
||
behaviors regularly
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/LnQjpWaON8nhr21vNW/giphy.gif" width="35"> Citation
|
||
|
||
<div align="center">
|
||
|
||
### 📚 **Academic & Research Use**
|
||
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="120"/>
|
||
|
||
<br><br>
|
||
|
||
If you use **Lumichat Coder v2.1** in your research or applications, please cite:
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
**This Model:**
|
||
|
||
```bibtex
|
||
@misc{lumichat-coder-v2.1,
|
||
author = {Jha, Aditya Kumar and LumiChats},
|
||
title = {Lumichat Coder v2.1: Advanced Function-Calling Language Model},
|
||
year = {2025},
|
||
publisher = {HuggingFace},
|
||
howpublished = {\url{https://huggingface.co/lumichats/lumichat-coder-v2.1}},
|
||
}
|
||
```
|
||
|
||
<br>
|
||
|
||
**Base Model (Qwen2.5-Coder):**
|
||
|
||
```bibtex
|
||
@article{hui2024qwen2,
|
||
title={Qwen2.5-Coder Technical Report},
|
||
author={Hui, Binyuan and Yang, Jian and Cui, Zeyu and Yang, Jiaxi and Liu, Dayiheng and Zhang, Lei and Liu, Tianyu and Zhang, Jiajun and Yu, Bowen and Dang, Kai and others},
|
||
journal={arXiv preprint arXiv:2409.12186},
|
||
year={2024}
|
||
}
|
||
|
||
@article{qwen2,
|
||
title={Qwen2 Technical Report},
|
||
author={An Yang and Baosong Yang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Zhou and Chengpeng Li and Chengyuan Li and Dayiheng Liu and Fei Huang and Guanting Dong and Haoran Wei and Huan Lin and Jialong Tang and Jialin Wang and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Ma and Jin Xu and Jingren Zhou and Jinze Bai and Jinzheng He and Junyang Lin and Kai Dang and Keming Lu and Keqin Chen and Kexin Yang and Mei Li and Mingfeng Xue and Na Ni and Pei Zhang and Peng Wang and Ru Peng and Rui Men and Ruize Gao and Runji Lin and Shijie Wang and Shuai Bai and Sinan Tan and Tianhang Zhu and Tianhao Li and Tianyu Liu and Wenbin Ge and Xiaodong Deng and Xiaohuan Zhou and Xingzhang Ren and Xinyu Zhang and Xipin Wei and Xuancheng Ren and Yang Fan and Yang Yao and Yichang Zhang and Yu Wan and Yunfei Chu and Yuqiong Liu and Zeyu Cui and Zhenru Zhang and Zhihao Fan},
|
||
journal={arXiv preprint arXiv:2407.10671},
|
||
year={2024}
|
||
}
|
||
```
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/fwbZnTftCXVocKzfxR/giphy.gif" width="35"> License
|
||
|
||
<div align="center">
|
||
|
||
### ⚖️ **Apache 2.0 License**
|
||
|
||
<img src="https://img.shields.io/badge/License-Apache_2.0-A855F7?style=for-the-badge&labelColor=1a1b27&logo=apache&logoColor=white"/>
|
||
|
||
<br><br>
|
||
|
||
This model is released under the **Apache 2.0 License**, inherited from the Qwen2.5-Coder base model.
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
```
|
||
Copyright 2025 LumiChats (Aditya Kumar Jha)
|
||
|
||
Licensed under the Apache License, Version 2.0 (the "License");
|
||
you may not use this file except in compliance with the License.
|
||
You may obtain a copy of the License at
|
||
|
||
http://www.apache.org/licenses/LICENSE-2.0
|
||
|
||
Unless required by applicable law or agreed to in writing, software
|
||
distributed under the License is distributed on an "AS IS" BASIS,
|
||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||
See the License for the specific language governing permissions and
|
||
limitations under the License.
|
||
```
|
||
|
||
<br>
|
||
|
||
<div align="center">
|
||
|
||
### 📜 **What This Means for You**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%" align="center">
|
||
|
||
**✅ You CAN:**
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/💼-Commercial_Use-10B981?style=flat-square&labelColor=1a1b27"/> Use in commercial products
|
||
|
||
<img src="https://img.shields.io/badge/🔧-Modify-10B981?style=flat-square&labelColor=1a1b27"/> Modify and adapt the model
|
||
|
||
<img src="https://img.shields.io/badge/📤-Distribute-10B981?style=flat-square&labelColor=1a1b27"/> Share with others
|
||
|
||
<img src="https://img.shields.io/badge/🔒-Private_Use-10B981?style=flat-square&labelColor=1a1b27"/> Use privately
|
||
|
||
<img src="https://img.shields.io/badge/⚖️-Patent_Use-10B981?style=flat-square&labelColor=1a1b27"/> Grant of patent rights
|
||
|
||
</td>
|
||
<td width="50%" align="center">
|
||
|
||
**📋 You MUST:**
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/📄-Include_License-FFD93D?style=flat-square&labelColor=1a1b27"/> Include license and copyright
|
||
|
||
<img src="https://img.shields.io/badge/📝-State_Changes-FFD93D?style=flat-square&labelColor=1a1b27"/> Document modifications made
|
||
|
||
<img src="https://img.shields.io/badge/📢-Include_NOTICE-FFD93D?style=flat-square&labelColor=1a1b27"/> Include NOTICE file if provided
|
||
|
||
<img src="https://img.shields.io/badge/⚠️-No_Trademark-EF4444?style=flat-square&labelColor=1a1b27"/> Can't use LumiChats trademarks
|
||
|
||
<img src="https://img.shields.io/badge/🛡️-No_Warranty-EF4444?style=flat-square&labelColor=1a1b27"/> Provided "as-is" without warranty
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/hvRJCLFzcasrR4ia7z/giphy.gif" width="35"> Acknowledgments
|
||
|
||
<div align="center">
|
||
|
||
### 🙏 **Built on the Shoulders of Giants**
|
||
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="150"/>
|
||
|
||
<br><br>
|
||
|
||
</div>
|
||
|
||
### 💝 **Special Thanks**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%" valign="top">
|
||
|
||
**🌟 Core Technologies**
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/Qwen_Team-Alibaba_Cloud-FF6B35?style=for-the-badge&labelColor=1a1b27&logo=alibabacloud&logoColor=white"/>
|
||
|
||
For the exceptional **Qwen2.5-Coder** base model and groundbreaking research in code-specialized LLMs.
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/Unsloth-Team-00D9FF?style=for-the-badge&labelColor=1a1b27&logo=github&logoColor=white"/>
|
||
|
||
For the incredible training optimization framework that made this fine-tuning possible with 2x speed and 60% less memory.
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/Hugging_Face-Community-FFD21F?style=for-the-badge&labelColor=1a1b27&logo=huggingface&logoColor=white"/>
|
||
|
||
For hosting infrastructure, transformers library, and fostering the open-source AI community.
|
||
|
||
</td>
|
||
<td width="50%" valign="top">
|
||
|
||
**🎯 Community Support**
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/LumiChats-Community-A855F7?style=for-the-badge&labelColor=1a1b27&logo=discord&logoColor=white"/>
|
||
|
||
Early testers, feedback providers, and the amazing LumiChats community who helped shape this model.
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/Open_Source-Contributors-10B981?style=for-the-badge&labelColor=1a1b27&logo=opensourceinitiative&logoColor=white"/>
|
||
|
||
All open-source contributors in the AI/ML ecosystem who make projects like this possible.
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/Students_&_Developers-Worldwide-4ECDC4?style=for-the-badge&labelColor=1a1b27&logo=academia&logoColor=white"/>
|
||
|
||
For inspiring us to make AI accessible and affordable for everyone.
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 🛠️ **Built With**
|
||
|
||
<div align="center">
|
||
|
||
<p>
|
||
<a href="https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B">
|
||
<img src="https://img.shields.io/badge/Qwen2.5--Coder-Base_Model-FF6B35?style=for-the-badge&labelColor=1a1b27&logo=alibabacloud&logoColor=white"/>
|
||
</a>
|
||
<a href="https://github.com/unslothai/unsloth">
|
||
<img src="https://img.shields.io/badge/Unsloth-Fine--Tuning-00D9FF?style=for-the-badge&labelColor=1a1b27&logo=github&logoColor=white"/>
|
||
</a>
|
||
<a href="https://github.com/epfl-dlab/transformers-CFG">
|
||
<img src="https://img.shields.io/badge/transformers--CFG-Grammar-Constraints-10B981?style=for-the-badge&labelColor=1a1b27&logo=github&logoColor=white"/>
|
||
</a>
|
||
<a href="https://github.com/huggingface/transformers">
|
||
<img src="https://img.shields.io/badge/HF_Transformers-Framework-FFD21F?style=for-the-badge&labelColor=1a1b27&logo=huggingface&logoColor=white"/>
|
||
</a>
|
||
</p>
|
||
|
||
<br>
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="25%">
|
||
<img src="https://skillicons.dev/icons?i=python&theme=dark" width="50"/>
|
||
<br><br>
|
||
<b>Python</b>
|
||
<br>
|
||
<sub>Core language</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://skillicons.dev/icons?i=pytorch&theme=dark" width="50"/>
|
||
<br><br>
|
||
<b>PyTorch</b>
|
||
<br>
|
||
<sub>Deep learning</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://cdn.jsdelivr.net/gh/devicons/devicon/icons/huggingface/huggingface-original.svg" width="50"/>
|
||
<br><br>
|
||
<b>Transformers</b>
|
||
<br>
|
||
<sub>Model framework</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://cdn.jsdelivr.net/gh/devicons/devicon/icons/fastapi/fastapi-original.svg" width="50"/>
|
||
<br><br>
|
||
<b>FastAPI</b>
|
||
<br>
|
||
<sub>API integration</sub>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/M9gbBd9nbDrOTu1Mqx/giphy.gif" width="35"> Contact & Support
|
||
|
||
<div align="center">
|
||
|
||
### 💬 **Get in Touch**
|
||
|
||
<img src="https://media.giphy.com/media/l0HlRnAWXxn0MhKLK/giphy.gif" width="150"/>
|
||
|
||
<br><br>
|
||
|
||
</div>
|
||
|
||
### 🌐 **LumiChats Resources**
|
||
|
||
<div align="center">
|
||
|
||
<p>
|
||
<a href="https://www.lumichats.com">
|
||
<img src="https://img.shields.io/badge/🌐_Website-Visit_LumiChats-FF6B35?style=for-the-badge&labelColor=1a1b27"/>
|
||
</a>
|
||
<a href="https://www.lumichats.com">
|
||
<img src="https://img.shields.io/badge/💬_Support-Get_Help-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
</a>
|
||
<a href="mailto:adityajha29092004@gmail.com">
|
||
<img src="https://img.shields.io/badge/📧_Email-Contact_Us-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
</a>
|
||
</p>
|
||
|
||
<br>
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="33%">
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/🐛-Report_Issues-EF4444?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Use the model card
|
||
<br>
|
||
discussion tab for
|
||
<br>
|
||
bug reports & feedback
|
||
</td>
|
||
<td align="center" width="33%">
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/💡-Feature_Requests-FFD93D?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Share your ideas for
|
||
<br>
|
||
improvements and
|
||
<br>
|
||
new capabilities
|
||
</td>
|
||
<td align="center" width="33%">
|
||
<img src="https://media.giphy.com/media/l0HlDHQEiIdY3kxlm/giphy.gif" width="80"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/📚-Documentation-4ECDC4?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
Check this README
|
||
<br>
|
||
for comprehensive
|
||
<br>
|
||
usage guides
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 👨💻 **About the Founder**
|
||
|
||
<img src="https://media.giphy.com/media/4H3Ii5eLChYul9p7NL/giphy.gif" width="100"/>
|
||
|
||
<br>
|
||
|
||
**Aditya Kumar Jha**
|
||
|
||
<br>
|
||
|
||
Founder of [LumiChats](https://www.lumichats.com) • Passionate about democratizing AI access
|
||
|
||
<br>
|
||
|
||
<p>
|
||
<a href="https://github.com/adityajhakumar">
|
||
<img src="https://img.shields.io/badge/GitHub-Follow-181717?style=for-the-badge&labelColor=1a1b27&logo=github&logoColor=white"/>
|
||
</a>
|
||
<a href="mailto:adityajha29092004@gmail.com">
|
||
<img src="https://img.shields.io/badge/Email-Contact-EA4335?style=for-the-badge&labelColor=1a1b27&logo=gmail&logoColor=white"/>
|
||
</a>
|
||
</p>
|
||
|
||
<br>
|
||
|
||
> **Mission:** Make premium AI accessible to students, developers, and creators worldwide—without subscription fatigue or wasted money. Pay only when your brain needs a boost. 🧠
|
||
|
||
</div>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/iY8CRBdQXODJSCERIr/giphy.gif" width="35"> Model Card Information
|
||
|
||
<div align="center">
|
||
|
||
### 📋 **Technical Summary**
|
||
|
||
<br>
|
||
|
||
<table>
|
||
<tr>
|
||
<th>Attribute</th>
|
||
<th>Value</th>
|
||
</tr>
|
||
<tr>
|
||
<td><b>Developed by</b></td>
|
||
<td>Aditya Kumar Jha / LumiChats</td>
|
||
</tr>
|
||
<tr>
|
||
<td><b>Model type</b></td>
|
||
<td>Causal Language Model (Function Calling Specialist)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><b>Language(s)</b></td>
|
||
<td>English (primary)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><b>License</b></td>
|
||
<td>Apache 2.0</td>
|
||
</tr>
|
||
<tr>
|
||
<td><b>Fine-tuned from</b></td>
|
||
<td><a href="https://huggingface.co/unsloth/Qwen2.5-Coder-1.5B-Instruct">unsloth/Qwen2.5-Coder-1.5B-Instruct</a></td>
|
||
</tr>
|
||
<tr>
|
||
<td><b>Model size</b></td>
|
||
<td>1.54B parameters (1.31B non-embedding)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><b>Context length</b></td>
|
||
<td>32,768 tokens</td>
|
||
</tr>
|
||
<tr>
|
||
<td><b>Architecture</b></td>
|
||
<td>Transformer (GQA, RoPE, SwiGLU, RMSNorm)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><b>Training framework</b></td>
|
||
<td>Unsloth (2x faster, 60% less VRAM)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><b>Specialization</b></td>
|
||
<td>Function calling, tool use, JSON generation</td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/j2pOGeGYKe2xCCKwfi/giphy.gif" width="35"> Quick Links
|
||
|
||
<div align="center">
|
||
|
||
### 🔗 **Essential Resources**
|
||
|
||
<br>
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="25%">
|
||
<a href="https://www.lumichats.com">
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="60"/>
|
||
</a>
|
||
<br><br>
|
||
<a href="https://www.lumichats.com">
|
||
<img src="https://img.shields.io/badge/LumiChats-Platform-FF6B35?style=for-the-badge&labelColor=1a1b27"/>
|
||
</a>
|
||
<br>
|
||
<sub>Premium AI at ₹39/day</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<a href="https://huggingface.co/lumichats/lumichat-coder-v2.1">
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="60"/>
|
||
</a>
|
||
<br><br>
|
||
<a href="https://huggingface.co/lumichats/lumichat-coder-v2.1">
|
||
<img src="https://img.shields.io/badge/Model_Card-HuggingFace-FFD21F?style=for-the-badge&labelColor=1a1b27"/>
|
||
</a>
|
||
<br>
|
||
<sub>Download & documentation</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<a href="https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B">
|
||
<img src="https://media.giphy.com/media/l0HlBO7eyXzSZkJri/giphy.gif" width="60"/>
|
||
</a>
|
||
<br><br>
|
||
<a href="https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B">
|
||
<img src="https://img.shields.io/badge/Base_Model-Qwen2.5-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
</a>
|
||
<br>
|
||
<sub>Original foundation</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<a href="https://github.com/unslothai/unsloth">
|
||
<img src="https://media.giphy.com/media/l0HlDHQEiIdY3kxlm/giphy.gif" width="60"/>
|
||
</a>
|
||
<br><br>
|
||
<a href="https://github.com/unslothai/unsloth">
|
||
<img src="https://img.shields.io/badge/Unsloth-Framework-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
</a>
|
||
<br>
|
||
<sub>Optimization toolkit</sub>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/WUlplcMpOCEmTGBtBW/giphy.gif" width="35"> Why Choose Lumichat Coder?
|
||
|
||
<div align="center">
|
||
|
||
### 🎯 **The Function-Calling Specialist**
|
||
|
||
<br>
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%" align="center">
|
||
<img src="https://media.giphy.com/media/l0HlBO7eyXzSZkJri/giphy.gif" width="150"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/✅-Precision_Engineering-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
<h3>Built for Production</h3>
|
||
<br>
|
||
Not a general-purpose model trying to do everything.
|
||
<br><br>
|
||
<b>Specifically engineered for tool calling</b>
|
||
<br>
|
||
with 96.5% accuracy and 99.8% JSON validity.
|
||
<br><br>
|
||
Deploy with confidence in customer-facing applications.
|
||
</td>
|
||
<td width="50%" align="center">
|
||
<img src="https://media.giphy.com/media/67ThRZlYBvibtdF9JH/giphy.gif" width="150"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/⚡-Unsloth_Optimized-FFD93D?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
<h3>Fast & Efficient</h3>
|
||
<br>
|
||
2x faster inference than standard fine-tuning.
|
||
<br><br>
|
||
<b>60% less memory consumption</b>
|
||
<br>
|
||
means deploy on consumer GPUs.
|
||
<br><br>
|
||
No enterprise hardware budgets required.
|
||
</td>
|
||
</tr>
|
||
<tr>
|
||
<td width="50%" align="center">
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="150"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/🎓-Accessible_to_All-FF6B35?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
<h3>Student & Developer Friendly</h3>
|
||
<br>
|
||
From LumiChats, the platform that saves students
|
||
<br>
|
||
₹1,200-2,600 monthly on AI costs.
|
||
<br><br>
|
||
<b>Open source, Apache 2.0 licensed</b>
|
||
<br>
|
||
Free to use, modify, and commercialize.
|
||
</td>
|
||
<td width="50%" align="center">
|
||
<img src="https://media.giphy.com/media/qgQUggAC3Pfv687qPC/giphy.gif" width="150"/>
|
||
<br><br>
|
||
<img src="https://img.shields.io/badge/🔒-Guaranteed_Valid_JSON-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br><br>
|
||
<h3>Grammar-Constrained Generation</h3>
|
||
<br>
|
||
Uses transformers-CFG for guaranteed output.
|
||
<br><br>
|
||
<b>No more parsing errors or malformed JSON.</b>
|
||
<br>
|
||
99.8% validity rate in production.
|
||
<br><br>
|
||
Reliable automation you can trust.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/hvRJCLFzcasrR4ia7z/giphy.gif" width="35"> Technical Specifications
|
||
|
||
<div align="center">
|
||
|
||
### 🔧 **Built on Cutting-Edge Architecture**
|
||
|
||
<br>
|
||
|
||
<img src="https://media.giphy.com/media/qgQUggAC3Pfv687qPC/giphy.gif" width="150"/>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 📊 **Model Architecture**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%">
|
||
|
||
**🏗️ Foundation**
|
||
- **Base Model**: `unsloth/Qwen2.5-Coder-1.5B-Instruct`
|
||
- **Model Type**: Causal Language Model (Decoder-only)
|
||
- **Architecture**: Transformer with RoPE, SwiGLU, RMSNorm
|
||
- **Attention**: Grouped Query Attention (GQA)
|
||
|
||
**📈 Scale**
|
||
- **Total Parameters**: 1.54B
|
||
- **Non-Embedding**: 1.31B
|
||
- **Layers**: 28
|
||
- **Hidden Size**: 1,536
|
||
|
||
</td>
|
||
<td width="50%">
|
||
|
||
**🎯 Capacity**
|
||
- **Attention Heads (Q)**: 12
|
||
- **Attention Heads (KV)**: 2
|
||
- **Context Length**: 32,768 tokens
|
||
- **Vocabulary Size**: 151,936
|
||
|
||
**⚙️ Components**
|
||
- ✅ Rotary Position Embeddings (RoPE)
|
||
- ✅ SwiGLU Activation
|
||
- ✅ RMS Normalization
|
||
- ✅ QKV Attention Bias
|
||
- ✅ Tied Word Embeddings
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 💾 **Supported Formats**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="33%" align="center">
|
||
<img src="https://img.shields.io/badge/Safetensors-Recommended-4ECDC4?style=for-the-badge&labelColor=1a1b27&logo=rust&logoColor=white"/>
|
||
<br>
|
||
<sub>Fast, safe model loading</sub>
|
||
</td>
|
||
<td width="33%" align="center">
|
||
<img src="https://img.shields.io/badge/PyTorch-Supported-EE4C2C?style=for-the-badge&labelColor=1a1b27&logo=pytorch&logoColor=white"/>
|
||
<br>
|
||
<sub>Native framework support</sub>
|
||
</td>
|
||
<td width="33%" align="center">
|
||
<img src="https://img.shields.io/badge/Precision-BF16_|_FP16_|_INT8_|_INT4-FFD93D?style=for-the-badge&labelColor=1a1b27&logo=nvidia&logoColor=white"/>
|
||
<br>
|
||
<sub>Flexible deployment</sub>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 🔌 **Inference Engines**
|
||
|
||
<div align="center">
|
||
|
||
<p>
|
||
<img src="https://img.shields.io/badge/🤗_Transformers-Supported-FFD21F?style=for-the-badge&labelColor=1a1b27"/>
|
||
<img src="https://img.shields.io/badge/vLLM-Supported-00ADD8?style=for-the-badge&labelColor=1a1b27"/>
|
||
<img src="https://img.shields.io/badge/TGI-Supported-FF6B6B?style=for-the-badge&labelColor=1a1b27"/>
|
||
</p>
|
||
|
||
</div>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/LnQjpWaON8nhr21vNW/giphy.gif" width="35"> Usage
|
||
|
||
<div align="center">
|
||
|
||
### 💻 **Get Started in Minutes**
|
||
|
||
<img src="https://media.giphy.com/media/qgQUggAC3Pfv687qPC/giphy.gif" width="150"/>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 📦 **Installation**
|
||
|
||
```bash
|
||
pip install torch transformers accelerate unsloth transformers-cfg
|
||
```
|
||
|
||
<br>
|
||
|
||
### 🚀 **Basic Inference**
|
||
|
||
```python
|
||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
import torch
|
||
import json
|
||
|
||
# Load model and tokenizer
|
||
model_name = "lumichats/lumichat-coder-v2.1"
|
||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_name,
|
||
torch_dtype=torch.bfloat16,
|
||
device_map="auto"
|
||
)
|
||
|
||
# Define your available tools
|
||
tools = [
|
||
{
|
||
"name": "get_vector_sum",
|
||
"description": "Calculate the sum of two vectors",
|
||
"parameters": {
|
||
"type": "object",
|
||
"properties": {
|
||
"a": {"type": "array", "items": {"type": "number"}},
|
||
"b": {"type": "array", "items": {"type": "number"}}
|
||
},
|
||
"required": ["a", "b"]
|
||
}
|
||
}
|
||
]
|
||
|
||
# Create prompt
|
||
user_query = "Find the sum of a = [1, -1, 2] and b = [3, 0, -4]"
|
||
prompt = f"""Available tools:
|
||
{json.dumps(tools, indent=2)}
|
||
|
||
User query: {user_query}
|
||
|
||
Generate the appropriate tool call in JSON format. Only output valid JSON.
|
||
"""
|
||
|
||
# Generate
|
||
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
||
outputs = model.generate(
|
||
**inputs,
|
||
max_new_tokens=256,
|
||
temperature=0.1,
|
||
do_sample=True
|
||
)
|
||
|
||
# Decode and parse
|
||
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
||
# Extract the JSON part (remove the prompt)
|
||
json_str = result.split("Generate the appropriate tool call in JSON format. Only output valid JSON.")[-1].strip()
|
||
tool_call = json.loads(json_str)
|
||
|
||
print(json.dumps(tool_call, indent=2))
|
||
```
|
||
|
||
<br>
|
||
|
||
### 🎯 **Grammar-Constrained Decoding**
|
||
|
||
<div align="center">
|
||
<img src="https://img.shields.io/badge/99.8%25-Valid_JSON_Guaranteed-10B981?style=for-the-badge&labelColor=1a1b27&logo=json&logoColor=white"/>
|
||
</div>
|
||
|
||
<br>
|
||
|
||
```python
|
||
from transformers_cfg.grammar_utils import IncrementalGrammarConstraint
|
||
from transformers_cfg.generation.logits_process import GrammarConstrainedLogitsProcessor
|
||
|
||
# Define JSON schema grammar
|
||
json_grammar = """
|
||
root ::= array
|
||
array ::= "[" ws object (ws "," ws object)* ws "]"
|
||
object ::= "{" ws "\"name\"" ws ":" ws string ws "," ws "\"arguments\"" ws ":" ws dict ws "}"
|
||
dict ::= "{" ws (string ws ":" ws value (ws "," ws string ws ":" ws value)*)? ws "}"
|
||
value ::= string | number | array | dict | "true" | "false" | "null"
|
||
string ::= "\"" [^"]* "\""
|
||
number ::= "-"? [0-9]+ ("." [0-9]+)?
|
||
ws ::= [ \t\n\r]*
|
||
"""
|
||
|
||
# Create grammar constraint
|
||
grammar = IncrementalGrammarConstraint(json_grammar, "root", tokenizer)
|
||
grammar_processor = GrammarConstrainedLogitsProcessor(grammar)
|
||
|
||
# Generate with constraint
|
||
outputs = model.generate(
|
||
**inputs,
|
||
max_new_tokens=256,
|
||
logits_processor=[grammar_processor],
|
||
temperature=0.1
|
||
)
|
||
```
|
||
|
||
<br>
|
||
|
||
### 🌐 **FastAPI Integration**
|
||
|
||
```python
|
||
from fastapi import FastAPI
|
||
from pydantic import BaseModel
|
||
import json
|
||
|
||
app = FastAPI()
|
||
|
||
class ToolCallRequest(BaseModel):
|
||
query: str
|
||
tools: list
|
||
|
||
@app.post("/tool-call")
|
||
async def generate_tool_call(request: ToolCallRequest):
|
||
prompt = f"""Available tools:
|
||
{json.dumps(request.tools, indent=2)}
|
||
|
||
User query: {request.query}
|
||
|
||
Generate the appropriate tool call in JSON format.
|
||
"""
|
||
|
||
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
||
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.1)
|
||
|
||
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
||
json_str = result.split("Generate the appropriate tool call in JSON format.")[-1].strip()
|
||
tool_call = json.loads(json_str)
|
||
|
||
return {"tool_call": tool_call}
|
||
```
|
||
|
||
<br>
|
||
|
||
### ⚙️ **Advanced: Streaming Responses**
|
||
|
||
```python
|
||
from transformers import TextIteratorStreamer
|
||
from threading import Thread
|
||
|
||
streamer = TextIteratorStreamer(tokenizer, skip_special_tokens=True)
|
||
|
||
generation_kwargs = dict(
|
||
inputs=inputs,
|
||
streamer=streamer,
|
||
max_new_tokens=256,
|
||
temperature=0.1
|
||
)
|
||
|
||
thread = Thread(target=model.generate, kwargs=generation_kwargs)
|
||
thread.start()
|
||
|
||
for new_text in streamer:
|
||
print(new_text, end="", flush=True)
|
||
|
||
thread.join()
|
||
```
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/hvRJCLFzcasrR4ia7z/giphy.gif" width="35"> Examples
|
||
|
||
<div align="center">
|
||
|
||
### 🎯 **Real-World Use Cases**
|
||
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="150"/>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
### 1️⃣ **Mathematical Operations**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%">
|
||
|
||
**📝 Query:**
|
||
```text
|
||
Calculate the dot product of
|
||
vectors [2, 3, 1] and [4, -1, 2]
|
||
```
|
||
|
||
</td>
|
||
<td width="50%">
|
||
|
||
**🤖 Output:**
|
||
```json
|
||
[
|
||
{
|
||
"name": "calculate_dot_product",
|
||
"arguments": {
|
||
"vector_a": [2, 3, 1],
|
||
"vector_b": [4, -1, 2]
|
||
}
|
||
}
|
||
]
|
||
```
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 2️⃣ **Data Processing**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%">
|
||
|
||
**📝 Query:**
|
||
```text
|
||
Filter the list [1, 5, 12, 8, 3, 15]
|
||
to keep only numbers greater than 7
|
||
```
|
||
|
||
</td>
|
||
<td width="50%">
|
||
|
||
**🤖 Output:**
|
||
```json
|
||
[
|
||
{
|
||
"name": "filter_list",
|
||
"arguments": {
|
||
"data": [1, 5, 12, 8, 3, 15],
|
||
"condition": "greater_than",
|
||
"threshold": 7
|
||
}
|
||
}
|
||
]
|
||
```
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 3️⃣ **Email Automation**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%">
|
||
|
||
**📝 Query:**
|
||
```text
|
||
Send an email to john@example.com
|
||
with subject 'Meeting Reminder'
|
||
and body 'Don't forget our
|
||
meeting at 3 PM'
|
||
```
|
||
|
||
</td>
|
||
<td width="50%">
|
||
|
||
**🤖 Output:**
|
||
```json
|
||
[
|
||
{
|
||
"name": "send_email",
|
||
"arguments": {
|
||
"recipient": "john@example.com",
|
||
"subject": "Meeting Reminder",
|
||
"body": "Don't forget our meeting at 3 PM"
|
||
}
|
||
}
|
||
]
|
||
```
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 4️⃣ **Database Queries**
|
||
|
||
<table>
|
||
<tr>
|
||
<td width="50%">
|
||
|
||
**📝 Query:**
|
||
```text
|
||
Find all users in the database
|
||
who registered after January 1, 2024
|
||
and are from India
|
||
```
|
||
|
||
</td>
|
||
<td width="50%">
|
||
|
||
**🤖 Output:**
|
||
```json
|
||
[
|
||
{
|
||
"name": "query_database",
|
||
"arguments": {
|
||
"table": "users",
|
||
"filters": {
|
||
"registration_date": {
|
||
"operator": "greater_than",
|
||
"value": "2024-01-01"
|
||
},
|
||
"country": {
|
||
"operator": "equals",
|
||
"value": "India"
|
||
}
|
||
}
|
||
}
|
||
}
|
||
]
|
||
```
|
||
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<!-- Animated Divider -->
|
||
<img src="https://user-images.githubusercontent.com/73097560/115834477-dbab4500-a447-11eb-908a-139a6edaec5c.gif">
|
||
|
||
<br>
|
||
|
||
## <img src="https://media.giphy.com/media/WUlplcMpOCEmTGBtBW/giphy.gif" width="35"> Ready to Build?
|
||
|
||
<div align="center">
|
||
|
||
<img src="https://media.giphy.com/media/26tn33aiTi1jkl6H6/giphy.gif" width="200"/>
|
||
|
||
<br><br>
|
||
|
||
### 🚀 **Start Building AI Agents Today**
|
||
|
||
<br>
|
||
|
||
**Precision tool calling • 99.8% JSON validity • 32K context • Apache 2.0 licensed**
|
||
|
||
<br><br>
|
||
|
||
<p>
|
||
<a href="https://huggingface.co/lumichats/lumichat-coder-v2.1">
|
||
<img src="https://img.shields.io/badge/📥_DOWNLOAD_MODEL-HuggingFace-FFD21F?style=for-the-badge&labelColor=1a1b27&fontSize=20" alt="Download Model" height="50"/>
|
||
</a>
|
||
<a href="https://www.lumichats.com">
|
||
<img src="https://img.shields.io/badge/🚀_TRY_LUMICHATS-Start_Free-FF6B35?style=for-the-badge&labelColor=1a1b27&fontSize=20" alt="Try LumiChats" height="50"/>
|
||
</a>
|
||
</p>
|
||
|
||
<br>
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="33%">
|
||
<img src="https://img.shields.io/badge/1.5B-Parameters-00D9FF?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Efficient deployment</sub>
|
||
</td>
|
||
<td align="center" width="33%">
|
||
<img src="https://img.shields.io/badge/32K-Context-10B981?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Extended conversations</sub>
|
||
</td>
|
||
<td align="center" width="33%">
|
||
<img src="https://img.shields.io/badge/Apache_2.0-Free-A855F7?style=for-the-badge&labelColor=1a1b27"/>
|
||
<br>
|
||
<sub>Commercial use allowed</sub>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
---
|
||
|
||
<br>
|
||
|
||
### 💡 **Perfect For**
|
||
|
||
<table>
|
||
<tr>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/4H3Ii5eLChYul9p7NL/giphy.gif" width="60"/>
|
||
<br><br>
|
||
<b>🤖 AI Agents</b>
|
||
<br>
|
||
<sub>Autonomous systems</sub>
|
||
<br>
|
||
<sub>that interact with tools</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/qgQUggAC3Pfv687qPC/giphy.gif" width="60"/>
|
||
<br><br>
|
||
<b>⚙️ Automation</b>
|
||
<br>
|
||
<sub>Workflow orchestration</sub>
|
||
<br>
|
||
<sub>& data processing</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/3oKIPnAiaMCws8nOsE/giphy.gif" width="60"/>
|
||
<br><br>
|
||
<b>🔌 API Integration</b>
|
||
<br>
|
||
<sub>Natural language</sub>
|
||
<br>
|
||
<sub>to API calls</sub>
|
||
</td>
|
||
<td align="center" width="25%">
|
||
<img src="https://media.giphy.com/media/l0HlDHQEiIdY3kxlm/giphy.gif" width="60"/>
|
||
<br><br>
|
||
<b>💬 Chat Interfaces</b>
|
||
<br>
|
||
<sub>Conversational UIs</sub>
|
||
<br>
|
||
<sub>with actions</sub>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<br>
|
||
|
||
### 🌟 **Join the Community**
|
||
|
||
<p>
|
||
<img src="https://img.shields.io/github/stars/adityajhakumar?style=social"/>
|
||
<img src="https://img.shields.io/badge/dynamic/json?color=FF6B35&label=downloads&query=%24.downloads&url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2Flumichats%2Flumichat-coder-v2.1"/>
|
||
<img src="https://img.shields.io/badge/Status-Production_Ready-10B981?style=flat-square&labelColor=1a1b27"/>
|
||
</p>
|
||
|
||
</div>
|
||
|
||
<br>
|
||
|
||
<!-- Footer Wave -->
|
||
<img width="100%" src="https://capsule-render.vercel.app/api?type=waving&color=gradient&customColorList=12,20,24,30&height=120§ion=footer&animation=twinkling"/>
|
||
|
||
<div align="center">
|
||
|
||
<img src="https://forthebadge.com/images/badges/built-with-love.svg"/>
|
||
<img src="https://forthebadge.com/images/badges/powered-by-coffee.svg"/>
|
||
<img src="https://forthebadge.com/images/badges/makes-people-smile.svg"/>
|
||
|
||
<br><br>
|
||
|
||
### ⭐ **Star this model if you believe in accessible AI!**
|
||
|
||
<br>
|
||
|
||
<img src="https://img.shields.io/badge/Made_with_❤️_in-India-FF6B35?style=for-the-badge&labelColor=1a1b27&logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjQiIGhlaWdodD0iMjQiIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHJlY3Qgd2lkdGg9IjI0IiBoZWlnaHQ9IjgiIGZpbGw9IiNGRjk5MzMiLz4KPHJlY3QgeT0iOCIgd2lkdGg9IjI0IiBoZWlnaHQ9IjgiIGZpbGw9IiNGRkZGRkYiLz4KPHJlY3QgeT0iMTYiIHdpZHRoPSIyNCIgaGVpZ2h0PSI4IiBmaWxsPSIjMTM4ODA4Ii8+Cjwvc3ZnPgo="/>
|
||
|
||
<br><br>
|
||
|
||
**© 2025 LUMICHATS • Premium AI at Coffee Prices ☕**
|
||
|
||
<br>
|
||
|
||
*Developed by Aditya Kumar Jha*
|
||
|
||
<br><br>
|
||
|
||
<p>
|
||
<a href="https://www.lumichats.com">
|
||
<img src="https://img.shields.io/badge/🌐-lumichats.com-FF6B35?style=flat-square&labelColor=1a1b27"/>
|
||
</a>
|
||
<a href="mailto:adityajha29092004@gmail.com">
|
||
<img src="https://img.shields.io/badge/📧-Contact-00D9FF?style=flat-square&labelColor=1a1b27"/>
|
||
</a>
|
||
<a href="https://github.com/adityajhakumar">
|
||
<img src="https://img.shields.io/badge/💻-GitHub-181717?style=flat-square&labelColor=1a1b27"/>
|
||
</a>
|
||
</p>
|
||
|
||
</div>
|
||
``` |