初始化项目，由ModelHub XC社区提供模型
Model: chartreuse-verte/orb-human-typeahead-1b-v2.2 Source: Original Platform
This commit is contained in:
ModelHub XC
140
README.md
Normal file
140
README.md
Normal file
@@ -0,0 +1,140 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: ibm-granite/granite-4.0-1b-base
|
||||
pipeline_tag: text-generation
|
||||
language:
|
||||
- en
|
||||
tags:
|
||||
- typeahead
|
||||
- autocomplete
|
||||
- roleplay
|
||||
- granite
|
||||
- gguf
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
# orb-human-typeahead-1b-v2.2
|
||||
|
||||
A 1.6B typeahead model that predicts **the human's next few words** in a
|
||||
roleplay chat — inline "ghost text" for the person typing, not a reply
|
||||
generator for the character. Full fine-tune of
|
||||
[ibm-granite/granite-4.0-1b-base](https://huggingface.co/ibm-granite/granite-4.0-1b-base).
|
||||
|
||||
Most small LMs asked to continue a user's half-typed roleplay message produce
|
||||
fluent but irrelevant text. This model is trained specifically on
|
||||
(conversation context + partial user message → the words the user actually
|
||||
typed next), so its suggestions stay on-scene and in-voice.
|
||||
|
||||
## Variants
|
||||
|
||||
| Path | Format | Use with |
|
||||
|---|---|---|
|
||||
| `/` | safetensors bf16, plain `granite` arch | transformers |
|
||||
| `GGUF/orb-human-typeahead-1b-v2.2-Q4_0.gguf` | GGUF Q4_0 (default) | llama.cpp / llama-cpp-python |
|
||||
| `GGUF/orb-human-typeahead-1b-v2.2-Q8_0.gguf` | GGUF Q8_0 | llama.cpp / llama-cpp-python |
|
||||
|
||||
The original fine-tune used the `granitemoehybrid` architecture; since this
|
||||
variant is attention-only and dense, the weights are republished here as the
|
||||
equivalent plain `granite` architecture (logit-identical, verified), which
|
||||
loads everywhere without extras.
|
||||
|
||||
## Prompt format
|
||||
|
||||
Plain text, no chat template. Optional character summary, a marker line,
|
||||
name-prefixed turns, and finally the user's draft — the model continues the
|
||||
draft. Cut the suggestion at the first newline.
|
||||
|
||||
```
|
||||
<character summary, optional>
|
||||
***Roleplay chat below***
|
||||
Sylvara: *She looks down from the watchtower and sees you.*
|
||||
Traveler: *I approach the encampment.*
|
||||
Sylvara: *She lowers her bow as you approach the gate.* "State your business, traveler."
|
||||
Traveler: *I raise both hands slowly and
|
||||
```
|
||||
|
||||
A completion looks like ` step into the torchlight, keeping my voice low.*`
|
||||
|
||||
Notes:
|
||||
- Trained to trigger at **word boundaries** only (draft ends on a whole word,
|
||||
or on a trailing space). Mid-word completion is out of scope.
|
||||
- Trained context: up to ~4 recent turns, summaries ≤400 chars, turns ≤500 chars.
|
||||
|
||||
## Serving recipe
|
||||
|
||||
Greedy, short budget, stop at newline — mirrors how it was trained and evaluated:
|
||||
|
||||
```python
|
||||
from llama_cpp import Llama
|
||||
|
||||
llm = Llama("GGUF/orb-human-typeahead-1b-v2.2-Q4_0.gguf", n_ctx=1024)
|
||||
out = llm.create_completion(prompt, max_tokens=12, stop=["\n"], temperature=0.0)
|
||||
print(out["choices"][0]["text"])
|
||||
```
|
||||
|
||||
Or with transformers:
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
tok = AutoTokenizer.from_pretrained("chartreuse-verte/orb-human-typeahead-1b-v2.2")
|
||||
model = AutoModelForCausalLM.from_pretrained("chartreuse-verte/orb-human-typeahead-1b-v2.2")
|
||||
ids = tok(prompt, return_tensors="pt").input_ids
|
||||
out = model.generate(ids, max_new_tokens=12, do_sample=False)
|
||||
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).split("\n")[0])
|
||||
```
|
||||
|
||||
## Evaluation
|
||||
|
||||
Scored on a held-out validation set of roleplay conversations (120
|
||||
prompts, conversation-disjoint from training; greedy, 12-token budget,
|
||||
suggestions cut at newline). word-EM@k = fraction of prompts where the first
|
||||
k words of the suggestion exactly match what the user really typed next;
|
||||
prefix-chars = mean length of the exactly-matching leading characters.
|
||||
|
||||
| Metric | orb-human-typeahead-1b-v2.1 | this model |
|
||||
|---|---|---|
|
||||
| word-EM@1 | 0.483 | **0.500** |
|
||||
| word-EM@2 | **0.367** | 0.358 |
|
||||
| word-EM@3 | **0.275** | 0.267 |
|
||||
| accept-rate | **0.267** | 0.133 |
|
||||
| prefix-chars | **10.89** | 9.44 |
|
||||
| completion perplexity | 3.56 | 3.56 |
|
||||
|
||||
**Regression detected?? Not bug, is feature — That is the expected
|
||||
result of the change.** Turns out autocompletion isn't immune to overused slop.
|
||||
v2.2 is a data-only refresh whose entire purpose was to flatten the completion distribution —
|
||||
to stop the model defaulting to a small set of high-frequency phrasings.
|
||||
The eval set itself is synthetic and has a diversity problem, it rewards both coherency AND overused slop (which are unfortunately akin in ML).
|
||||
A model that spreads its probability mass across more plausible continuations agrees with
|
||||
such targets less often, which is what the multi-word columns show: word-EM@1 and completion perplexity
|
||||
are flat, while longer exact-agreement metrics give back a few points.
|
||||
|
||||
**Read the table as how much text is saved while typing, not as an absolute
|
||||
quality ranking**. Exact agreement with one recorded continuation cannot score suggestion
|
||||
variety, and creative roleplay is high-entropy enough that many suggestions the metric
|
||||
counts as misses are perfectly good ones.
|
||||
|
||||
You will find v2.2 much more pleasant and much less janky despite the benchmark numbers.
|
||||
|
||||
## Training
|
||||
|
||||
Two-stage full fine-tune of granite-4.0-1b-base: stage 1 on a large, mostly-synthetic
|
||||
roleplay corpus (self-chat generated to cover the on-scene, in-voice register real
|
||||
users type in), then stage 2 on a smaller private in-domain set in the serve-time
|
||||
prompt format (with stage-1 replay to limit forgetting). Over v2.1, stage 1 grew and its
|
||||
completion distribution was flattened further: a 3-gram repetition cap retires canned
|
||||
mid-phrase filler that the earlier first-word cap could not see, post-dialogue beats got
|
||||
real supervision instead of being capped into starvation, and generator punctuation tells
|
||||
(em-dashes, curly quotes) plus non-English leakage are normalized out rather than dropped.
|
||||
Examples are user turns split at word boundaries; loss is on the continuation.
|
||||
Suggestions are single-line by construction (completions end at newline).
|
||||
|
||||
## Limitations
|
||||
|
||||
- English-centric, roleplay register (asterisk actions, quoted dialogue).
|
||||
Out of domain for assistant chat, code, or formal prose.
|
||||
- Roleplay corpora include mature themes; suggestions can reflect that.
|
||||
Intended as a typing aid for consenting adult users of RP chat apps.
|
||||
- Not an instruction follower — it only continues drafts in the format above.
|
||||
- Suggests at word boundaries; won't complete a half-typed word.
|
||||
Reference in New Issue
Block a user