---
language:
- en
library_name: transformers
pipeline_tag: text-generation
license: apache-2.0
tags:
- lizzy-7b
- flwrlabs
- british-english
- text-generation
model_name: Lizzy 7B
---

# Lizzy 7B

<img class="dark:hidden" src="./header-light.svg" alt="Lizzy 7B header figure (light theme)" />
<img class="hidden dark:block" src="./header-dark.svg" alt="Lizzy 7B header figure (dark theme)" />

## Model Name And Summary

Lizzy 7B is an open-weight Flower Labs assistant model in the Lizzy family.

## Architecture And Configuration

Lizzy 7B is a 7B-class decoder-only transformer with long-context support, sliding/local attention behaviour, custom chat/control tokens, and deployment-specific serving configurations.

Representative configuration points:

- 7B-class parameter scale with a 32-layer stack;
- long-context configuration up to 65k tokens with runtime caps adjusted by deployment profile;
- 32 attention heads with long-context/sliding-attention behaviour;
- custom tokenizer and chat markers for instruction-style prompting;
- deployment variants may include quantised revisions, runtime patches, and serving-time configuration changes.

## Training Approach

Lizzy 7B follows a multi-stage training approach that combines:

- pre-training on large-scale public text, document, code, math, and encyclopedic corpora;
- supervised fine-tuning on instruction-following, dialogue, reasoning, and tool-use examples;
- direct preference optimisation on preference pairs for helpfulness, style, and answer quality;
- reinforcement learning with verifiable rewards for targeted behavioural refinement.

Across these stages, training data has been mixed across:

- broad public text and knowledge sources;
- synthetic instruction and preference data;
- private synthetic data used to favour British behaviour and knowledge;
- UK-specific examples and preference signals used to strengthen local knowledge and style.

## Evaluation Against European Baselines

Britishness comparisons against the European baselines present in the latest local artifact set:

| Benchmark | Lizzy 7B | EuroLLM 9B | Apertus 8B |
| --- | ---: | ---: | ---: |
| Britishness MCQ | 71.0 | <u>77.6</u> | **80.8** |
| Britishness CoT | **80.1** | <u>72.1</u> | 31.7 |
| Britishness Domains | **89.9** | <u>69.0</u> | 32.6 |

Broader benchmark comparisons against the same European baselines:

| Benchmark | Lizzy 7B | EuroLLM 9B | Apertus 8B |
| --- | ---: | ---: | ---: |
| MATH | **77.9** | <u>31.3</u> | 22.4 |
| OMEGA | **29.0** | 4.7 | <u>5.0</u> |
| BigBenchHard | **69.0** | 38.9 | <u>42.4</u> |
| AGI Eval English | **65.6** | 50.2 | <u>50.4</u> |
| MMLU | **67.9** | 57.4 | <u>63.4</u> |
| GPQA | **34.6** | 26.8 | <u>28.1</u> |
| HumanEvalPlus | **70.2** | 28.2 | <u>33.4</u> |
| MBPP+ | **52.5** | 41.7 | <u>42.3</u> |
| LiveCodeBench v3 | **39.1** | 6.3 | <u>8.5</u> |
| IFEval | <u>63.8</u> | 55.8 | **65.1** |
| AIME | **35.8** | 0.2 | <u>0.6</u> |
| GSM8K | **91.8** | <u>64.7</u> | 64.7 |
| IFBench | **22.7** | <u>18.0</u> | 15.3 |
| POPQA | 22.2 | **25.6** | <u>25.1</u> |
| ZebraLogic | **12.4** | 4.4 | <u>5.9</u> |

Summary:

- Lizzy 7B trails the European baselines on Britishness MCQ (a private Flower Labs benchmark) recall-style probing.
- Lizzy 7B leads the reported European baselines on Britishness CoT and Britishness domain reasoning (private Flower Labs benchmarks) where comparable metrics are available.
- Lizzy 7B also leads the latest local European baseline set on most knowledge, reasoning, math, and coding rows represented in the table above.

## Intended Uses And Limitations

Intended uses:

- UK-oriented assistant experiences;
- general reasoning and coding assistance;
- managed deployment through private Hugging Face or vLLM serving stacks.

## Safety And Bias Considerations

The latest safety-evaluation reports the following task-level primary scores:

| Safety benchmark | Metric | Score |
| --- | --- | ---: |
| Overall safety average | `overall_safety_average` | 66.7% |
| WildGuardTest | `inverted_micro_harm_lower` | 91.9% |
| HarmBench | `inverted_micro_asr_lower` | 57.5% |
| ToxiGen (tiny) | `safe_overall` | 90.2% |
| XSTest | `overall_accuracy` | 85.6% |
| StrongReject (logprobs) | `inverted_asr` | 78.8% |
| BBQ | `accuracy` | 66.5% |
| WMDP | `inverted_accuracy` | 47.5% |

Lizzy 7B can still produce incorrect, outdated, or over-confident responses and should be used with human oversight for higher-risk workflows. UK-specific tuning improves local style and cultural alignment but can also bias tone and assumptions toward UK conventions; downstream moderation and policy controls remain required.

## License And Citation

- Model licence: Apache-2.0
- Public and synthetic training sources include open-licensed public data plus private synthetic and UK-specific data that are not redistributed
- Citation and legal text should still be confirmed by owner review before any external publication.

## Python Example (Transformers)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo_id = "flwrlabs/Lizzy-7B"

tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are Lizzy 7B."},
    {"role": "user", "content": "Summarise why queue etiquette matters in the UK."},
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

output_ids = model.generate(
    **inputs,
    temperature=0.2,
    top_p=0.9,
)
response = tokenizer.decode(output_ids[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(response)
```

## Multi-GPU vLLM Tensor Parallel Patch

For reproducible multi-GPU vLLM support with Lizzy-family checkpoints, this deliverable bundles:

- bundled draft artifact: `vllm_patches/transformers_lizzy_tp.py`

Apply this patch when all of the following are true:

- runtime uses vLLM via the generic Transformers backend (`model_type=vllm`)
- tensor parallelism is enabled (`tensor_parallel_size > 1`)
- checkpoint is Lizzy-family (including RLVR variants)
- runtime is not guaranteed to include an equivalent upstream fix

You can skip patch bundling only for strict HF-only runs or single-rank vLLM (`TP=1`).

Why this is included:

- it mitigates known Lizzy TP failure modes in generic vLLM Transformers loading
- it fixes rank-local head partitioning and `q_norm`/`k_norm` slicing behaviour
- it prevents the known tensor-shape crash class seen without this patch
