57 lines
2.5 KiB
Markdown
57 lines
2.5 KiB
Markdown
---
|
||
license: apache-2.0
|
||
base_model: Qwen/Qwen3-4B
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- qwen3
|
||
- qlora
|
||
- rag
|
||
- grounded-generation
|
||
- citations
|
||
- mining
|
||
- blasting
|
||
- safety-data-sheets
|
||
---
|
||
|
||
# dyno-blast-4b
|
||
|
||
A QLoRA fine-tune of **Qwen3-4B** that answers blasting / explosives technical and
|
||
safety questions **strictly from retrieved context**, attaches `[N]` + page
|
||
citations to every claim, and **refuses when the answer isn't in the provided
|
||
sources**.
|
||
|
||
This is the *generation* half of a grounded RAG system over Dyno Nobel Australia's
|
||
public technical literature (product technical data sheets, Safety Data Sheets,
|
||
application guides, case studies). **Facts live in the retriever; the model learns
|
||
the behavior** — cite, ground, refuse. Companion vector DB:
|
||
[`kcherry497/dyno-blast-4b-rag`](https://huggingface.co/datasets/kcherry497/dyno-blast-4b-rag).
|
||
|
||
## Intended use & limitations
|
||
Retrieval-augmented Q&A for mining / quarry / blasting professionals. **Not a
|
||
standalone knowledge source** — use with the retriever (intfloat/e5-base-v2 + a
|
||
Qdrant collection). For SDS / safety content, always verify against the cited
|
||
source PDF; the model is trained to give page numbers for exactly this reason.
|
||
Domain-specific to Dyno Nobel AU products.
|
||
|
||
## Training
|
||
- **Base:** Qwen/Qwen3-4B
|
||
- **Method:** QLoRA (4-bit nf4), r=16, α=32, dropout=0.05, targets all attn + MLP projections
|
||
- **Data:** **1,674 synthetic grounded examples** — 1,656 `[N]`-cited answers + 18 refusal / safe-decline examples — generated by a teacher over retrieved context, covering SDS sections, technical specs, application/case-study/brochure topics, Explosive Engineers Guide articles, industrial chemicals, and blast calculators
|
||
- **Schedule:** 3 epochs, lr 1e-4 cosine, full-sequence SFT
|
||
- **Result:** final `train_loss` **0.97**
|
||
|
||
Retrieval corpus (companion dataset repo): **3,819 chunks across 682 documents** —
|
||
dynonobel.com.au + dynonobel.com (126 products) + the Explosive Engineers Guide app
|
||
(4 regions) + Industrial Chemicals + resource-centre case studies/guides/brochures +
|
||
blast calculators.
|
||
|
||
## Files
|
||
- `*.safetensors` — merged fp16 weights (load with `transformers`)
|
||
- `dyno-blast-4b-q8_0.gguf` — q8_0 GGUF for llama.cpp / Ollama
|
||
|
||
## Prompt format
|
||
Grounded system prompt (answer only from numbered SOURCEs, cite `[N]`, refuse if
|
||
absent) + numbered `SOURCE [N]` blocks from the retriever, then the question. The
|
||
exact system prompt and chunk schema are in the companion dataset repo.
|