108 lines
2.6 KiB
Markdown
108 lines
2.6 KiB
Markdown
|
---
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
license: apache-2.0
|
||
|
|
tags:
|
||
|
|
- code
|
||
|
|
- fim
|
||
|
|
- code-completion
|
||
|
|
- qwen2
|
||
|
|
- unsloth
|
||
|
|
- text-generation-inference
|
||
|
|
base_model: GarvAgnihotri/Big-G-3B-FIM-merged
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# Big-G-1
|
||
|
|
|
||
|
|
Big-G-1 is a small 3 billion parameter model I trained for code completion. It's built on top of Qwen2.5-Coder-3B and fine-tuned to be good at filling in missing parts of code — you give it the code before and after the gap, and it figures out what goes in the middle.
|
||
|
|
|
||
|
|
I made this because I wanted a lightweight model that actually works well for real coding tasks without needing a powerful GPU.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## What it does
|
||
|
|
|
||
|
|
It's trained for something called FIM (Fill in the Middle). Instead of just predicting what comes next, it can look at code before AND after a gap and fill it in. This makes it really useful for code editors and autocomplete tools.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## How to use it
|
||
|
|
|
||
|
|
Install the library first:
|
||
|
|
|
||
|
|
```
|
||
|
|
pip install transformers torch
|
||
|
|
```
|
||
|
|
|
||
|
|
Then run this:
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||
|
|
import torch
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("GarvAgnihotri/big-g-1")
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
"GarvAgnihotri/big-g-1",
|
||
|
|
torch_dtype=torch.bfloat16,
|
||
|
|
device_map="auto"
|
||
|
|
)
|
||
|
|
|
||
|
|
# The code before the gap
|
||
|
|
prefix = "def add_numbers(a, b):\n "
|
||
|
|
|
||
|
|
# The code after the gap
|
||
|
|
suffix = "\n return result"
|
||
|
|
|
||
|
|
# Build the prompt
|
||
|
|
prompt = f"<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>"
|
||
|
|
|
||
|
|
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
||
|
|
outputs = model.generate(**inputs, max_new_tokens=64, temperature=0.1)
|
||
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Good for
|
||
|
|
|
||
|
|
- Code completion inside editors
|
||
|
|
- Filling in missing lines of code
|
||
|
|
- Lightweight local code assistant
|
||
|
|
|
||
|
|
## Not great for
|
||
|
|
|
||
|
|
- General chat or questions
|
||
|
|
- Long reasoning tasks
|
||
|
|
- Non-English code comments
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Model info
|
||
|
|
|
||
|
|
| Thing | Detail |
|
||
|
|
|---|---|
|
||
|
|
| Base model | Qwen2.5-Coder-3B |
|
||
|
|
| Parameters | 3 billion |
|
||
|
|
| Format | BF16, Safetensors |
|
||
|
|
| License | Apache 2.0 |
|
||
|
|
| Trained with | Unsloth + HuggingFace TRL |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Benchmarks
|
||
|
|
|
||
|
|
Tested on 15 custom FIM tasks against the base Qwen2.5-Coder-3B:
|
||
|
|
|
||
|
|
| Model | Score | Passed |
|
||
|
|
|---|---|---|
|
||
|
|
| Big-G-1 (this model) | 93.3% | 14/15 |
|
||
|
|
| Qwen2.5-Coder-3B (base) | 100% | 15/15 |
|
||
|
|
|
||
|
|
The one failed case (String reverse) was not a logic error — the model started
|
||
|
|
a valid solution but got cut off due to the 80 token limit used in testing.
|
||
|
|
With a higher token limit (256+) the completion finishes correctly.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
Made by [GarvAgnihotri](https://huggingface.co/GarvAgnihotri). If you find it useful, leave a like ⭐
|