--- license: apache-2.0 language: - en tags: - prose - novel-writing - bookbuilder - creative-writing - qlora - qwen2.5 - wip - beta - early-release base_model: Qwen/Qwen2.5-7B pipeline_tag: text-generation extra_gated_description: "This is an early-release v1 model with known limitations. A larger v2 is in development. Please read the Work in Progress section before use." --- # 🚧 WORK IN PROGRESS — v1 EARLY RELEASE 🚧 # A bigger, better model (v2) is in development. This page is the v1 early release of `book-builder-bookwriter-v1`. It is **not the final model**. Skim the rest of this card before you generate anything so you know what you're getting and what you're not. --- # book-builder-bookwriter-v1 A 7.6B-parameter prose-writing model fine-tuned on 10,211 human-authored novels (310,316 chapters, ~1.82 billion tokens) with a strict story-bible-to-chapter format. Built for [BookBuilder](https://book-builder.net) to generate novel chapters from structured story bibles. > ## ⚠ Work in Progress (v1, early release) > > This is an **early checkpoint** of an ongoing training run. Training was paused at step 5000 / 9697 (~52% of one epoch) so the artifacts could be released publicly while the larger run continues. > > **Known limitations of v1:** > - Treats bibles as style prompts, not strict plot instructions. Expect drift from the synopsis on one-shot generation. > - Does not reliably follow "FORBIDDEN" rules, character role assignments, or per-character constraints in rich BookBuilder-style bibles. > - Loaded keywords in synopses (proper names like "Stillwater", specific years like "1947") can trigger off-topic associations from the training corpus. > - On longer generations, may drift into paragraph-level repetition loops without the tuned sampling defaults in `ollama/Modelfile`. > > **What's coming:** > - **v2 (Q3/Q4 2026):** Larger base model (Qwen 2.5 14B or 32B) + synthesized instruction-following data so the model can actually obey FORBIDDEN sections, distinguish protagonists from antagonists, and follow beat sheets. This addresses the main v1 limitation. > - **v1 continuation to step 9697:** the LoRA may be taken to the full single-epoch checkpoint and re-released as `book-builder-bookwriter-v1.1` if testing shows it's worth the additional training compute. The resumable training state is preserved at branch [`resumable-step-5000`](https://huggingface.co/Fordentinc/book-builder-bookwriter-v1/tree/resumable-step-5000). > > **Best use for v1 right now:** as a prose-style backbone inside a structured pipeline (like BookBuilder itself) that provides per-chapter beat sheets and plot anchors at generation time. The pipeline supplies the discipline the v1 model can't enforce on its own. For one-shot "give me a chapter from a synopsis" use, v1 produces readable prose but will frequently drift from the intended plot. **This is NOT a chat model. Do not prompt it like ChatGPT. See "How to use" below.** ## What this model does You write a **Story Bible** in the format shown below (or fill in the [template](./bible_template.txt)). You give the model the bible plus `### Chapter`. The model writes the chapter prose. It will not answer questions. It will not respond to "Write me a story about X." It only continues prose conditioned on the bible context. ## Format the model expects Every training example looked exactly like this: ``` ### Bible Title: [book title] Author: [author name] Genre: [genre] Publisher: [publisher] Synopsis: [1-3 paragraphs describing the book] ### Genre [genre] ### Chapter [chapter title] [chapter prose...] ``` Your prompt MUST end at `### Chapter\n[chapter title]\n\n` and the model fills in the prose. ## How to use ### Option 1: Ollama (easiest) **IMPORTANT:** A plain `ollama pull` from HF discards any Modelfile parameters in the repo, so Ollama runs the model with its default sampling — which on long completion prompts causes repetition loops and over-long generations. **Use the setup script below** to register the model under the name `bookbuilder` with tuned defaults that prevent both problems. **One-shot setup:** ```bash curl -sSL https://huggingface.co/Fordentinc/book-builder-bookwriter-v1/resolve/main/ollama/setup_ollama.sh | bash # Optional: pass a quant tag, default is Q5_K_M # curl -sSL https://huggingface.co/Fordentinc/book-builder-bookwriter-v1/resolve/main/ollama/setup_ollama.sh | bash -s Q8_0 ``` Then: ```bash ollama run bookbuilder < your_bible.txt ``` The script pulls the GGUF, builds a local `Modelfile` with the right `repeat_penalty 1.18`, `num_predict 2500`, and stop tokens, and registers the result as `bookbuilder`. **If you'd rather pull manually** (without the loop fix), you need to pass sampling flags every time: ```bash ollama pull hf.co/Fordentinc/book-builder-bookwriter-v1:Q5_K_M ollama run hf.co/Fordentinc/book-builder-bookwriter-v1:Q5_K_M \ --num-predict 2500 --repeat-penalty 1.18 --temperature 0.75 ``` Available quants: - `Q4_K_M` (4.7 GB) - fits on 8 GB GPUs, some token-decoding artifacts - `Q5_K_M` (5.4 GB) - balanced, **recommended** for 24 GB cards - `Q8_0` (8.1 GB) - near-lossless, cleaner token decoding than K-quants - `F16` (15.2 GB) - full precision, no quantization artifacts ### Option 2: LM Studio 1. Search for `Fordentinc/book-builder-bookwriter-v1` 2. Download the Q5_K_M quant 3. Switch to **Completion mode** (not Chat). This is critical. 4. Paste your filled-in bible as the input 5. Generate ### Option 3: llama.cpp ```bash ./llama-cli -hf Fordentinc/book-builder-bookwriter-v1:Q5_K_M \ --temp 0.8 --top-p 0.95 -n 2048 \ -f your_bible.txt ``` ### Option 4: Transformers + PEFT (Python, full bf16) ```python import torch from transformers import AutoTokenizer, AutoModelForCausalLM from peft import PeftModel base_id = "Qwen/Qwen2.5-7B" adapter = "Fordentinc/book-builder-bookwriter-v1" tok = AutoTokenizer.from_pretrained(adapter) base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto") model = PeftModel.from_pretrained(base, adapter) model.eval() bible_and_chapter_header = open("your_bible.txt").read() inputs = tok(bible_and_chapter_header, return_tensors="pt").to("cuda") out = model.generate( **inputs, max_new_tokens=2048, do_sample=True, temperature=0.8, top_p=0.95, repetition_penalty=1.05, ) print(tok.decode(out[0], skip_special_tokens=True)) ``` ### Option 5: vLLM (production / OpenAI-compatible API) vLLM cannot load the LoRA adapter alone; use the merged bf16 weights instead: ```bash vllm serve Fordentinc/book-builder-bookwriter-v1 --dtype bfloat16 --max-model-len 16384 ``` ## Step-by-step: from blank page to chapter 1. **Download the template:** [bible_template.txt](./bible_template.txt) 2. **Fill in every field.** The model relies on each section to anchor character voices, setting, and tone. 3. **Save as plain text** (e.g. `my_book.txt`). 4. **End the file with `### Chapter` followed by your chapter title and one blank line.** Example: ``` ### Chapter Chapter 1: The Long Drive Home ``` 5. **Run inference** using one of the options above. 6. **The model writes ~1500-4000 words** of prose, then stops or hits your `max_new_tokens` cap. 7. **For chapter 2:** keep the same bible, change the chapter header, optionally append the last paragraph of chapter 1 so the model continues smoothly. See [example_bibles/](./example_bibles/) for two complete working examples. ## Recommended sampling parameters | Parameter | Value | Why | |---|---|---| | `temperature` | 0.8 | Lower = repetitive, higher = incoherent | | `top_p` | 0.95 | Standard nucleus sampling | | `repetition_penalty` | 1.05 | Prevents loops; do not push past 1.15 | | `max_new_tokens` | 2048-4096 | Most chapters land in 1500-3500 tokens | | `min_p` | 0.05 (if supported) | Better than top_k for prose | ## Training details - **Base model:** Qwen 2.5 7B (Apache 2.0) - **Method:** QLoRA, r=16, alpha=32, dropout 0.05 - **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj - **Training corpus:** 10,211 novels with derivable story bibles (Sci-Fi, Thriller, Romance, Crime, Western, Fantasy, etc.) - **Corpus stats:** 310,316 chapter rows, ~1.4 billion words, ~1.82 billion tokens - **Hardware:** 1× NVIDIA B200 (180 GB HBM3e) - **Sequence length:** 2048 - **Effective batch:** 32 (per-device 4, grad-accum 8) - **Optimizer:** paged_adamw_8bit - **LR:** 2e-4 cosine, 3% warmup - **Epochs:** 1 - **Wall time:** ~13.5 hours - **Data cleanup:** em-dashes removed (replaced with commas), smart-quotes normalized, residual front-matter stripped, chapters with <800 or >6000 words filtered out ## Quirks and limitations - **No system messages, no chat history, no `[INST]` tags.** The model was never shown those during training. - **Bibles outside its training distribution** (highly experimental forms, non-Western names, modern slang heavy) may produce uneven results. - **Names follow Western conventions** (USA/UK/Italy/Western Europe). The training filter excluded other naming traditions. - **Em-dashes are absent from training data** and the model will rarely produce them. This is intentional. - **Chapter length is learned from data** (avg ~4,500 words). To force shorter chapters, cap `max_new_tokens`. ## License Apache 2.0 (inherited from base Qwen 2.5 7B). You may use commercially. ## Citation ```bibtex @misc{bookbuilder_bookwriter_v1_2026, author = {Fordentinc}, title = {book-builder-bookwriter-v1: A prose-writing LoRA on Qwen 2.5 7B}, year = {2026}, url = {https://huggingface.co/Fordentinc/book-builder-bookwriter-v1}, } ``` ## Reporting issues Open a discussion on this model page. Include the bible you used (first 500 chars) and the first 200 chars of the model output.