初始化项目,由ModelHub XC社区提供模型
Model: arun11karthik/cellsense-fim-3b-GGUF Source: Original Platform
This commit is contained in:
100
README.md
Normal file
100
README.md
Normal file
@@ -0,0 +1,100 @@
|
||||
---
|
||||
library_name: gguf
|
||||
base_model:
|
||||
- arun11karthik/cellsense-fim-3b
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- cellsense
|
||||
- fim
|
||||
- code
|
||||
- gguf
|
||||
---
|
||||
|
||||
<p align="center">
|
||||
<img src="icon.png" alt="CellSense" width="180" height="180" />
|
||||
</p>
|
||||
|
||||
# arun11karthik/cellsense-fim-3b-GGUF
|
||||
|
||||
GGUF quantisations of [`arun11karthik/cellsense-fim-3b`](https://huggingface.co/arun11karthik/cellsense-fim-3b), a fill-in-the-middle (FIM) code-completion model.
|
||||
|
||||
## Available files
|
||||
|
||||
| File | Type | Notes |
|
||||
|------|------|-------|
|
||||
| `cellsense-fim-3b-BF16.gguf` | BF16 | Full bfloat16 precision — highest quality |
|
||||
| `cellsense-fim-3b-Q8_0.gguf` | Q8_0 | Near-lossless 8-bit quantisation |
|
||||
| `cellsense-fim-3b-Q5_K_M.gguf` | Q5_K_M | Good quality / size trade-off |
|
||||
|
||||
## Usage
|
||||
|
||||
For fully local, no-GPU-required inference, GGUF builds are published at
|
||||
[**`arun11karthik/cellsense-fim-3b-GGUF`**](https://huggingface.co/arun11karthik/cellsense-fim-3b-GGUF).
|
||||
Ollama can pull and run these directly from the Hugging Face Hub — no manual download or `Modelfile`
|
||||
required. This is the recommended path for running CellSense entirely on your own machine: with Ollama,
|
||||
**no code or context ever leaves your computer.**
|
||||
|
||||
### Available quantizations
|
||||
|
||||
| Quantization | Size (approx.) | Notes | Pull with |
|
||||
|--------------|:--------------:|-------|-----------|
|
||||
| `Q5_K_M` | 2.2 GB | Good quality / size trade-off | `ollama run hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M` |
|
||||
| `Q8_0` | 3.3 GB | Near-lossless 8-bit quantization | `ollama run hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q8_0` |
|
||||
| `BF16` | 6.2 GB | Full bfloat16 precision — highest quality (recommended) | `ollama run hf.co/arun11karthik/cellsense-fim-3b-GGUF:BF16` |
|
||||
|
||||
### 1. Install Ollama and pull the model
|
||||
|
||||
Install [Ollama](https://ollama.com), then pull a quantization (this also registers the model so
|
||||
CellSense can use it):
|
||||
|
||||
```bash
|
||||
ollama pull hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M
|
||||
```
|
||||
|
||||
By default Ollama serves its API at `http://localhost:11434`. The model name as it appears in
|
||||
`ollama list` — `hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M` — is what you'll enter into
|
||||
CellSense below.
|
||||
|
||||
### 2. Install the CellSense JupyterLab plugin
|
||||
|
||||
```bash
|
||||
pip install jupyterlab-cellsense
|
||||
jupyter lab
|
||||
```
|
||||
|
||||
See the [CellSense repository](https://github.com/arunkarthik11/CellSense) for full installation
|
||||
options.
|
||||
|
||||
### 3. Point CellSense at your local Ollama model
|
||||
|
||||
Open the CellSense panel from the left sidebar in JupyterLab, go to **Basic Settings**, and configure
|
||||
the **Ollama** provider:
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| **Provider** | `Ollama` |
|
||||
| **Base URL** | `http://localhost:11434` |
|
||||
| **Model Family** | `cellsense` |
|
||||
| **Model** | `hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M` (must match the tag in `ollama list`) |
|
||||
|
||||
> ✅ **Set Model Family to `cellsense`.** CellSense now ships first-class support for the CellSense-FIM
|
||||
> models, so the plugin builds prompts in exactly the repository-, import-, and task-aware FIM format
|
||||
> these models were trained on — no extra configuration needed.
|
||||
|
||||
Click **Save & Apply**, then start typing in a notebook cell — ghost-text completions from your local
|
||||
model appear inline. Press **Tab** to accept.
|
||||
|
||||
### Raw API check (optional)
|
||||
|
||||
To confirm Ollama is serving the model with the correct FIM format before wiring up CellSense, query it
|
||||
directly:
|
||||
|
||||
```bash
|
||||
curl http://localhost:11434/api/generate -d '{
|
||||
"model": "hf.co/arun11karthik/cellsense-fim-3b-GGUF:Q5_K_M",
|
||||
"prompt": "<|fim_prefix|>import pandas as pd\ndf = pd.read_csv(\"data.csv\")\n<|fim_suffix|>\ndf.head()\n<|fim_middle|>",
|
||||
"stream": false,
|
||||
"options": { "temperature": 0.0, "num_predict": 128 }
|
||||
}'
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user