101 lines
3.6 KiB
Markdown
101 lines
3.6 KiB
Markdown
---
|
|
library_name: gguf
|
|
base_model:
|
|
- arun11karthik/cellsense-fim-3b
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- cellsense
|
|
- fim
|
|
- code
|
|
- gguf
|
|
---
|
|
|
|
<p align="center">
|
|
<img src="icon.png" alt="CellSense" width="180" height="180" />
|
|
</p>
|
|
|
|
# arun11karthik/cellsense-fim-3b-GGUF
|
|
|
|
GGUF quantisations of [`arun11karthik/cellsense-fim-3b`](https://huggingface.co/arun11karthik/cellsense-fim-3b), a fill-in-the-middle (FIM) code-completion model.
|
|
|
|
## Available files
|
|
|
|
| File | Type | Notes |
|
|
|------|------|-------|
|
|
| `cellsense-fim-3b-BF16.gguf` | BF16 | Full bfloat16 precision — highest quality |
|
|
| `cellsense-fim-3b-Q8_0.gguf` | Q8_0 | Near-lossless 8-bit quantisation |
|
|
| `cellsense-fim-3b-Q5_K_M.gguf` | Q5_K_M | Good quality / size trade-off |
|
|
|
|
## Usage
|
|
|
|
For fully local, no-GPU-required inference, GGUF builds are published at
|
|
[**`arun11karthik/cellsense-fim-3b-GGUF`**](https://huggingface.co/arun11karthik/cellsense-fim-3b-GGUF).
|
|
Ollama can pull and run these directly from the Hugging Face Hub — no manual download or `Modelfile`
|
|
required. This is the recommended path for running CellSense entirely on your own machine: with Ollama,
|
|
**no code or context ever leaves your computer.**
|
|
|
|
### Available quantizations
|
|
|
|
| Quantization | Size (approx.) | Notes | Pull with |
|
|
|--------------|:--------------:|-------|-----------|
|
|
| `Q5_K_M` | 2.2 GB | Good quality / size trade-off | `ollama run hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M` |
|
|
| `Q8_0` | 3.3 GB | Near-lossless 8-bit quantization | `ollama run hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q8_0` |
|
|
| `BF16` | 6.2 GB | Full bfloat16 precision — highest quality (recommended) | `ollama run hf.co/arun11karthik/cellsense-fim-3b-GGUF:BF16` |
|
|
|
|
### 1. Install Ollama and pull the model
|
|
|
|
Install [Ollama](https://ollama.com), then pull a quantization (this also registers the model so
|
|
CellSense can use it):
|
|
|
|
```bash
|
|
ollama pull hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M
|
|
```
|
|
|
|
By default Ollama serves its API at `http://localhost:11434`. The model name as it appears in
|
|
`ollama list` — `hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M` — is what you'll enter into
|
|
CellSense below.
|
|
|
|
### 2. Install the CellSense JupyterLab plugin
|
|
|
|
```bash
|
|
pip install jupyterlab-cellsense
|
|
jupyter lab
|
|
```
|
|
|
|
See the [CellSense repository](https://github.com/arunkarthik11/CellSense) for full installation
|
|
options.
|
|
|
|
### 3. Point CellSense at your local Ollama model
|
|
|
|
Open the CellSense panel from the left sidebar in JupyterLab, go to **Basic Settings**, and configure
|
|
the **Ollama** provider:
|
|
|
|
| Setting | Value |
|
|
|---------|-------|
|
|
| **Provider** | `Ollama` |
|
|
| **Base URL** | `http://localhost:11434` |
|
|
| **Model Family** | `cellsense` |
|
|
| **Model** | `hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M` (must match the tag in `ollama list`) |
|
|
|
|
> ✅ **Set Model Family to `cellsense`.** CellSense now ships first-class support for the CellSense-FIM
|
|
> models, so the plugin builds prompts in exactly the repository-, import-, and task-aware FIM format
|
|
> these models were trained on — no extra configuration needed.
|
|
|
|
Click **Save & Apply**, then start typing in a notebook cell — ghost-text completions from your local
|
|
model appear inline. Press **Tab** to accept.
|
|
|
|
### Raw API check (optional)
|
|
|
|
To confirm Ollama is serving the model with the correct FIM format before wiring up CellSense, query it
|
|
directly:
|
|
|
|
```bash
|
|
curl http://localhost:11434/api/generate -d '{
|
|
"model": "hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M",
|
|
"prompt": "<|fim_prefix|>import pandas as pd\ndf = pd.read_csv(\"data.csv\")\n<|fim_suffix|>\ndf.head()\n<|fim_middle|>",
|
|
"stream": false,
|
|
"options": { "temperature": 0.0, "num_predict": 128 }
|
|
}'
|
|
```
|
|
|