初始化项目，由ModelHub XC社区提供模型
Model: Fordentinc/book-builder-bookwriter-v1 Source: Original Platform
This commit is contained in:
ModelHub XC
234
README.md
Normal file
234
README.md
Normal file
@@ -0,0 +1,234 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
tags:
|
||||
- prose
|
||||
- novel-writing
|
||||
- bookbuilder
|
||||
- creative-writing
|
||||
- qlora
|
||||
- qwen2.5
|
||||
- wip
|
||||
- beta
|
||||
- early-release
|
||||
base_model: Qwen/Qwen2.5-7B
|
||||
pipeline_tag: text-generation
|
||||
extra_gated_description: "This is an early-release v1 model with known limitations. A larger v2 is in development. Please read the Work in Progress section before use."
|
||||
---
|
||||
|
||||
# 🚧 WORK IN PROGRESS — v1 EARLY RELEASE 🚧
|
||||
|
||||
# A bigger, better model (v2) is in development.
|
||||
|
||||
This page is the v1 early release of `book-builder-bookwriter-v1`. It is **not the final model**. Skim the rest of this card before you generate anything so you know what you're getting and what you're not.
|
||||
|
||||
---
|
||||
|
||||
# book-builder-bookwriter-v1
|
||||
|
||||
A 7.6B-parameter prose-writing model fine-tuned on 10,211 human-authored novels (310,316 chapters, ~1.82 billion tokens) with a strict story-bible-to-chapter format. Built for [BookBuilder](https://book-builder.net) to generate novel chapters from structured story bibles.
|
||||
|
||||
> ## ⚠ Work in Progress (v1, early release)
|
||||
>
|
||||
> This is an **early checkpoint** of an ongoing training run. Training was paused at step 5000 / 9697 (~52% of one epoch) so the artifacts could be released publicly while the larger run continues.
|
||||
>
|
||||
> **Known limitations of v1:**
|
||||
> - Treats bibles as style prompts, not strict plot instructions. Expect drift from the synopsis on one-shot generation.
|
||||
> - Does not reliably follow "FORBIDDEN" rules, character role assignments, or per-character constraints in rich BookBuilder-style bibles.
|
||||
> - Loaded keywords in synopses (proper names like "Stillwater", specific years like "1947") can trigger off-topic associations from the training corpus.
|
||||
> - On longer generations, may drift into paragraph-level repetition loops without the tuned sampling defaults in `ollama/Modelfile`.
|
||||
>
|
||||
> **What's coming:**
|
||||
> - **v2 (Q3/Q4 2026):** Larger base model (Qwen 2.5 14B or 32B) + synthesized instruction-following data so the model can actually obey FORBIDDEN sections, distinguish protagonists from antagonists, and follow beat sheets. This addresses the main v1 limitation.
|
||||
> - **v1 continuation to step 9697:** the LoRA may be taken to the full single-epoch checkpoint and re-released as `book-builder-bookwriter-v1.1` if testing shows it's worth the additional training compute. The resumable training state is preserved at branch [`resumable-step-5000`](https://huggingface.co/Fordentinc/book-builder-bookwriter-v1/tree/resumable-step-5000).
|
||||
>
|
||||
> **Best use for v1 right now:** as a prose-style backbone inside a structured pipeline (like BookBuilder itself) that provides per-chapter beat sheets and plot anchors at generation time. The pipeline supplies the discipline the v1 model can't enforce on its own. For one-shot "give me a chapter from a synopsis" use, v1 produces readable prose but will frequently drift from the intended plot.
|
||||
|
||||
**This is NOT a chat model. Do not prompt it like ChatGPT. See "How to use" below.**
|
||||
|
||||
## What this model does
|
||||
|
||||
You write a **Story Bible** in the format shown below (or fill in the [template](./bible_template.txt)).
|
||||
You give the model the bible plus `### Chapter`.
|
||||
The model writes the chapter prose.
|
||||
|
||||
It will not answer questions. It will not respond to "Write me a story about X." It only continues prose conditioned on the bible context.
|
||||
|
||||
## Format the model expects
|
||||
|
||||
Every training example looked exactly like this:
|
||||
|
||||
```
|
||||
### Bible
|
||||
Title: [book title]
|
||||
Author: [author name]
|
||||
Genre: [genre]
|
||||
Publisher: [publisher]
|
||||
Synopsis: [1-3 paragraphs describing the book]
|
||||
|
||||
### Genre
|
||||
[genre]
|
||||
|
||||
|
||||
|
||||
### Chapter
|
||||
[chapter title]
|
||||
|
||||
[chapter prose...]
|
||||
```
|
||||
|
||||
Your prompt MUST end at `### Chapter\n[chapter title]\n\n` and the model fills in the prose.
|
||||
|
||||
## How to use
|
||||
|
||||
### Option 1: Ollama (easiest)
|
||||
|
||||
**IMPORTANT:** A plain `ollama pull` from HF discards any Modelfile parameters in the repo, so Ollama runs the model with its default sampling — which on long completion prompts causes repetition loops and over-long generations. **Use the setup script below** to register the model under the name `bookbuilder` with tuned defaults that prevent both problems.
|
||||
|
||||
**One-shot setup:**
|
||||
```bash
|
||||
curl -sSL https://huggingface.co/Fordentinc/book-builder-bookwriter-v1/resolve/main/ollama/setup_ollama.sh | bash
|
||||
# Optional: pass a quant tag, default is Q5_K_M
|
||||
# curl -sSL https://huggingface.co/Fordentinc/book-builder-bookwriter-v1/resolve/main/ollama/setup_ollama.sh | bash -s Q8_0
|
||||
```
|
||||
|
||||
Then:
|
||||
```bash
|
||||
ollama run bookbuilder < your_bible.txt
|
||||
```
|
||||
|
||||
The script pulls the GGUF, builds a local `Modelfile` with the right `repeat_penalty 1.18`, `num_predict 2500`, and stop tokens, and registers the result as `bookbuilder`.
|
||||
|
||||
**If you'd rather pull manually** (without the loop fix), you need to pass sampling flags every time:
|
||||
```bash
|
||||
ollama pull hf.co/Fordentinc/book-builder-bookwriter-v1:Q5_K_M
|
||||
ollama run hf.co/Fordentinc/book-builder-bookwriter-v1:Q5_K_M \
|
||||
--num-predict 2500 --repeat-penalty 1.18 --temperature 0.75
|
||||
```
|
||||
|
||||
Available quants:
|
||||
- `Q4_K_M` (4.7 GB) - fits on 8 GB GPUs, some token-decoding artifacts
|
||||
- `Q5_K_M` (5.4 GB) - balanced, **recommended** for 24 GB cards
|
||||
- `Q8_0` (8.1 GB) - near-lossless, cleaner token decoding than K-quants
|
||||
- `F16` (15.2 GB) - full precision, no quantization artifacts
|
||||
|
||||
### Option 2: LM Studio
|
||||
|
||||
1. Search for `Fordentinc/book-builder-bookwriter-v1`
|
||||
2. Download the Q5_K_M quant
|
||||
3. Switch to **Completion mode** (not Chat). This is critical.
|
||||
4. Paste your filled-in bible as the input
|
||||
5. Generate
|
||||
|
||||
### Option 3: llama.cpp
|
||||
|
||||
```bash
|
||||
./llama-cli -hf Fordentinc/book-builder-bookwriter-v1:Q5_K_M \
|
||||
--temp 0.8 --top-p 0.95 -n 2048 \
|
||||
-f your_bible.txt
|
||||
```
|
||||
|
||||
### Option 4: Transformers + PEFT (Python, full bf16)
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
from peft import PeftModel
|
||||
|
||||
base_id = "Qwen/Qwen2.5-7B"
|
||||
adapter = "Fordentinc/book-builder-bookwriter-v1"
|
||||
|
||||
tok = AutoTokenizer.from_pretrained(adapter)
|
||||
base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")
|
||||
model = PeftModel.from_pretrained(base, adapter)
|
||||
model.eval()
|
||||
|
||||
bible_and_chapter_header = open("your_bible.txt").read()
|
||||
|
||||
inputs = tok(bible_and_chapter_header, return_tensors="pt").to("cuda")
|
||||
out = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=2048,
|
||||
do_sample=True, temperature=0.8, top_p=0.95,
|
||||
repetition_penalty=1.05,
|
||||
)
|
||||
print(tok.decode(out[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
### Option 5: vLLM (production / OpenAI-compatible API)
|
||||
|
||||
vLLM cannot load the LoRA adapter alone; use the merged bf16 weights instead:
|
||||
```bash
|
||||
vllm serve Fordentinc/book-builder-bookwriter-v1 --dtype bfloat16 --max-model-len 16384
|
||||
```
|
||||
|
||||
## Step-by-step: from blank page to chapter
|
||||
|
||||
1. **Download the template:** [bible_template.txt](./bible_template.txt)
|
||||
2. **Fill in every field.** The model relies on each section to anchor character voices, setting, and tone.
|
||||
3. **Save as plain text** (e.g. `my_book.txt`).
|
||||
4. **End the file with `### Chapter` followed by your chapter title and one blank line.** Example:
|
||||
```
|
||||
### Chapter
|
||||
Chapter 1: The Long Drive Home
|
||||
|
||||
```
|
||||
5. **Run inference** using one of the options above.
|
||||
6. **The model writes ~1500-4000 words** of prose, then stops or hits your `max_new_tokens` cap.
|
||||
7. **For chapter 2:** keep the same bible, change the chapter header, optionally append the last paragraph of chapter 1 so the model continues smoothly.
|
||||
|
||||
See [example_bibles/](./example_bibles/) for two complete working examples.
|
||||
|
||||
## Recommended sampling parameters
|
||||
|
||||
| Parameter | Value | Why |
|
||||
|---|---|---|
|
||||
| `temperature` | 0.8 | Lower = repetitive, higher = incoherent |
|
||||
| `top_p` | 0.95 | Standard nucleus sampling |
|
||||
| `repetition_penalty` | 1.05 | Prevents loops; do not push past 1.15 |
|
||||
| `max_new_tokens` | 2048-4096 | Most chapters land in 1500-3500 tokens |
|
||||
| `min_p` | 0.05 (if supported) | Better than top_k for prose |
|
||||
|
||||
## Training details
|
||||
|
||||
- **Base model:** Qwen 2.5 7B (Apache 2.0)
|
||||
- **Method:** QLoRA, r=16, alpha=32, dropout 0.05
|
||||
- **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
|
||||
- **Training corpus:** 10,211 novels with derivable story bibles (Sci-Fi, Thriller, Romance, Crime, Western, Fantasy, etc.)
|
||||
- **Corpus stats:** 310,316 chapter rows, ~1.4 billion words, ~1.82 billion tokens
|
||||
- **Hardware:** 1× NVIDIA B200 (180 GB HBM3e)
|
||||
- **Sequence length:** 2048
|
||||
- **Effective batch:** 32 (per-device 4, grad-accum 8)
|
||||
- **Optimizer:** paged_adamw_8bit
|
||||
- **LR:** 2e-4 cosine, 3% warmup
|
||||
- **Epochs:** 1
|
||||
- **Wall time:** ~13.5 hours
|
||||
- **Data cleanup:** em-dashes removed (replaced with commas), smart-quotes normalized, residual front-matter stripped, chapters with <800 or >6000 words filtered out
|
||||
|
||||
## Quirks and limitations
|
||||
|
||||
- **No system messages, no chat history, no `[INST]` tags.** The model was never shown those during training.
|
||||
- **Bibles outside its training distribution** (highly experimental forms, non-Western names, modern slang heavy) may produce uneven results.
|
||||
- **Names follow Western conventions** (USA/UK/Italy/Western Europe). The training filter excluded other naming traditions.
|
||||
- **Em-dashes are absent from training data** and the model will rarely produce them. This is intentional.
|
||||
- **Chapter length is learned from data** (avg ~4,500 words). To force shorter chapters, cap `max_new_tokens`.
|
||||
|
||||
## License
|
||||
|
||||
Apache 2.0 (inherited from base Qwen 2.5 7B). You may use commercially.
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@misc{bookbuilder_bookwriter_v1_2026,
|
||||
author = {Fordentinc},
|
||||
title = {book-builder-bookwriter-v1: A prose-writing LoRA on Qwen 2.5 7B},
|
||||
year = {2026},
|
||||
url = {https://huggingface.co/Fordentinc/book-builder-bookwriter-v1},
|
||||
}
|
||||
```
|
||||
|
||||
## Reporting issues
|
||||
|
||||
Open a discussion on this model page. Include the bible you used (first 500 chars) and the first 200 chars of the model output.
|
||||
Reference in New Issue
Block a user