marvy-1-14B/USAGE.md

# Using marvy-1-14B

marvy-1-14B is a ServiceNow delivery specialist. This guide covers every common
way to run it — cloud or fully local — plus how to wire it into OpenCode.

- [Choosing a format](#choosing-a-format)
- [Recommended system prompt & settings](#recommended-system-prompt--settings)
- [Transformers (PyTorch)](#transformers-pytorch)
- [vLLM (OpenAI-compatible server)](#vllm-openai-compatible-server)
- [MLX (Apple Silicon, native)](#mlx-apple-silicon-native)
- [LM Studio (GUI + local server)](#lm-studio-gui--local-server)
- [Ollama / llama.cpp (GGUF)](#ollama--llamacpp-gguf)
- [LoRA adapter (apply on the base)](#lora-adapter-apply-on-the-base)
- [Use marvy-1-14B in OpenCode](#use-marvy-14b-in-opencode)
- [Prompt recipes per task](#prompt-recipes-per-task)

---

## Choosing a format

| You want… | Use | Repo |
|---|---|---|
| Max quality, GPU/server | Merged FP16 | `MainStack/marvy-1-14B` |
| Apple Silicon, native speed | Merged (MLX) | `MainStack/marvy-1-14B` |
| Laptop / CPU / Ollama / LM Studio | GGUF (Q4_K_M or Q8_0) | `MainStack/marvy-1-14B-GGUF` |
| Smallest download, compose yourself | LoRA adapter (~175 MB) | `MainStack/marvy-1-14B-lora` |

---

## Recommended system prompt & settings

Always lead with the delivery-consultant system prompt — marvy was trained with it:

```
You are a senior ServiceNow delivery consultant. You produce precise, implementation-grade
artifacts: business analyses, requirements, solution design documents, user stories with
acceptance criteria, test cases, and validation reviews. You favor out-of-the-box
capabilities, cite concrete tables/plugins/sys_ids when relevant, and write in clear
professional English.
```

| Use case | temperature | top_p | max_tokens |
|---|---|---|---|
| Structured artifacts (SDD, stories, test cases) | 0.3 – 0.5 | 0.9 | 1024 – 4096 |
| Exploratory brainstorming | 0.7 – 0.9 | 0.95 | 1024 |
| Validation / critique | 0.2 – 0.4 | 0.9 | 1024 – 2048 |

---

## Transformers (PyTorch)

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "MainStack/marvy-1-14B"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

SYSTEM = "You are a senior ServiceNow delivery consultant. ..."  # full prompt above
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "Write a user story with acceptance criteria for P1 SLA escalation."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=1024, temperature=0.4, top_p=0.9)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
```

## vLLM (OpenAI-compatible server)

```bash
pip install vllm
vllm serve MainStack/marvy-1-14B --served-model-name marvy-1-14B
```

```bash
curl -s http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "marvy-1-14B", "temperature": 0.4,
  "messages": [
    {"role":"system","content":"You are a senior ServiceNow delivery consultant. ..."},
    {"role":"user","content":"Draft the Incident Management section of an SDD."}
  ]}'
```

## MLX (Apple Silicon, native)

```bash
pip install mlx-lm

# one-off
python -m mlx_lm generate --model MainStack/marvy-1-14B \
  --system-prompt "You are a senior ServiceNow delivery consultant. ..." \
  --prompt "Write test cases for a Major Incident workflow." --max-tokens 1024 --temp 0.4

# OpenAI-compatible server
python -m mlx_lm server --model MainStack/marvy-1-14B --port 8080
```

## LM Studio (GUI + local server)

1. **Install the model** — either search `MainStack/marvy-1-14B-GGUF` in the
   in-app model browser, or place a local copy under
   `~/.lmstudio/models/MainStack/marvy-1-14B/` (MLX or GGUF layout).
2. **Load** it from the GUI, or:
   ```bash
   lms load MainStack/marvy-1-14B
   lms server start          # OpenAI-compatible on http://localhost:1234/v1
   ```
3. In the Chat tab, set the system prompt (above) and temperature ~0.4.

## Ollama / llama.cpp (GGUF)

```bash
# Ollama — pull straight from the Hub
ollama run hf.co/MainStack/marvy-1-14B-GGUF:Q4_K_M

# llama.cpp
llama-cli -hf MainStack/marvy-1-14B-GGUF:Q4_K_M \
  -p "Write a user story with acceptance criteria for P1 SLA escalation." --temp 0.4
```

| Quant | Size | Use when |
|---|---|---|
| `Q4_K_M` | ~9 GB | Default — best size/quality balance |
| `Q8_0` | ~16 GB | Highest fidelity, near-FP16 |

## LoRA adapter (apply on the base)

```bash
# MLX
python -m mlx_lm generate --model Qwen/Qwen2.5-14B-Instruct \
  --adapter-path .  --system-prompt "You are a senior ServiceNow delivery consultant. ..." \
  --prompt "Validate this requirement and list follow-up questions: ..." --max-tokens 1024
```

```python
# PEFT
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "Qwen/Qwen2.5-14B-Instruct"
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "MainStack/marvy-1-14B-lora")
```

---

## Use marvy-1-14B in OpenCode

marvy runs behind any OpenAI-compatible endpoint (LM Studio, mlx_lm server,
vLLM). Register it as a custom provider in `opencode.json`.

1. **Start a local server** (LM Studio shown; adjust port for others):
   ```bash
   lms load MainStack/marvy-1-14B && lms server start   # http://localhost:1234/v1
   ```
2. **Add the provider** to your project `opencode.json` (or global
   `~/.config/opencode/opencode.json`):
   ```jsonc
   {
     "provider": {
       "lmstudio": {
         "npm": "@ai-sdk/openai-compatible",
         "name": "LM Studio (local)",
         "options": { "baseURL": "http://localhost:1234/v1" },
         "models": {
           "marvy-1-14B": { "name": "marvy-1-14B (ServiceNow delivery)" }
         }
       }
     }
   }
   ```
3. **Select** `lmstudio/marvy-1-14B` in the OpenCode model picker.

> marvy-1-14B is a drafting specialist, not a tool-use/agentic fine-tune. It excels
> at producing delivery artifacts inside chat; for MCP tool-calling agent loops,
> keep a frontier model as the orchestrator and switch to marvy for drafting.

---

## Prompt recipes per task

| Task | Prompt skeleton |
|---|---|
| Business analysis | "Produce a Business Analysis for the following engagement: <context>. Cover organization, IT landscape, scope, and risks." |
| Requirements | "Extract structured requirements (id, category, requirement, priority, target_phase, success_metric) from: <notes>." |
| Stakeholders | "Build a stakeholder register (role, name, interest, influence, RACI) for: <context>." |
| SDD section | "Write the <section> section of a Solution Design Document for a ServiceNow <module> implementation. Include design decisions and concrete tables/plugins." |
| User story | "Write a ServiceNow user story with acceptance criteria for: <capability>." |
| Implementation plan | "Given this story, describe the implementation: tables, plugins, configuration, records touched, manual follow-ups. Story: <story>." |
| Test case | "Write a test case (pre-conditions, steps, expected results, pass/fail) for the story: <story>." |
| Validation | "Validate this artifact against ServiceNow best practice and the SOW. List gaps, risks, and follow-up questions. Artifact: <artifact>." |