68 lines
3.5 KiB
Markdown
68 lines
3.5 KiB
Markdown
# ChatPBC Model Deployment Checklist
|
|
**Developed by Mik Tse Agency**
|
|
|
|
---
|
|
|
|
## PRE-DEPLOYMENT REQUIREMENTS
|
|
|
|
1. **Model Repo Structure** (REQUIRED for HF Inference API to work):
|
|
- [ ] `config.json` with `model_type` field present
|
|
- [ ] `model.safetensors` OR `pytorch_model.bin` (actual weight files, NOT empty placeholders)
|
|
- [ ] `tokenizer_config.json`
|
|
- [ ] `tokenizer.model` OR `tokenizer.json`
|
|
- [ ] `special_tokens_map.json`
|
|
- [ ] `generation_config.json`
|
|
- [ ] `README.md` with `pipeline_tag: text-generation` in YAML front matter
|
|
|
|
2. **If using LoRA adapters**:
|
|
- [ ] `adapter_config.json` with `base_model_name_or_path` pointing to a real, accessible HF model
|
|
- [ ] `adapter_model.safetensors` with actual trained weights (NOT 0.1 KB placeholder)
|
|
- [ ] The base model must be publicly accessible on HF
|
|
- [ ] Use `merge_and_unload()` to create a standalone model for Inference API compatibility
|
|
|
|
3. **HF Inference API Requirements**:
|
|
- [ ] Model repo is PUBLIC (not private)
|
|
- [ ] `pipeline_tag: text-generation` set in model card YAML
|
|
- [ ] `library_name: transformers` set in model card YAML
|
|
- [ ] Model is NOT gated/restricted
|
|
- [ ] HF Token has read access to the model
|
|
|
|
4. **Chat Template Verification**:
|
|
- [ ] Confirm `tokenizer_config.json` has `chat_template` field
|
|
- [ ] For Llama-2 models: use `[INST]` format
|
|
- [ ] For Mistral/Zephyr: use `<|user|>` format
|
|
- [ ] For ChatML: use `<|im_start|>` format
|
|
- [ ] Match the prompt format in ALL code (Python, JavaScript, Gradio)
|
|
|
|
5. **API Call Verification**:
|
|
- [ ] Endpoint: `https://router.huggingface.co/v1/chat/completions` (for router) or `https://api-inference.huggingface.co/models/{model_id}`
|
|
- [ ] Headers: `Authorization: Bearer {HF_TOKEN}`, `Content-Type: application/json`
|
|
- [ ] Body: `inputs` (string) or `messages` (array), parameters (`max_new_tokens`, `temperature`, `do_sample`, `return_full_text: false`)
|
|
- [ ] Options: `wait_for_model: true` (prevents immediate 503 failure)
|
|
- [ ] Retry logic: 5 retries, 10 second delay between retries
|
|
- [ ] Timeout: 180 seconds minimum
|
|
|
|
6. **Demo Verification**:
|
|
- [ ] Open HTML demo in browser
|
|
- [ ] Send "Hi" — model should respond with a greeting
|
|
- [ ] Send "My business is struggling" — model should show empathy
|
|
- [ ] Send "Tell me more about what I said earlier" — model should reference earlier message
|
|
- [ ] Verify no base model names appear anywhere in the UI
|
|
- [ ] Test dark/light mode toggle
|
|
- [ ] Test model selector (V4 vs V3.3)
|
|
- [ ] Test clear conversation button
|
|
|
|
7. **Common Errors and Fixes**:
|
|
- [ ] **503 "Model is currently loading"** → Add `wait_for_model: true` and retry with 20s delay
|
|
- [ ] **400 "Model not supported by provider"** → Model is LoRA adapter, needs `merge_and_unload` or full weights
|
|
- [ ] **"trouble connecting" in demo** → Check API endpoint URL, check HF token validity, check model is public
|
|
- [ ] **Empty responses** → Check `return_full_text: false`, check prompt format matches tokenizer template
|
|
- [ ] **DNS resolution failure** → Only occurs in restricted sandbox environments, not in production
|
|
|
|
8. **Post-Deployment Verification**:
|
|
- [ ] Test API directly with curl or Python requests from a non-sandbox environment
|
|
- [ ] Verify multi-turn memory works (5 turns minimum)
|
|
- [ ] Verify model introduces itself on first message
|
|
- [ ] Verify model remembers context from turn 1 when at turn 5
|
|
- [ ] Both repos show as public on `huggingface.co/chatpbc1`
|