初始化项目，由ModelHub XC社区提供模型

Model: mohar07/qwen3-0.6b-kg-triplets Source: Original Platform
2026-06-27 09:43:16 +08:00
commit a1d48594f6
12 changed files with 954 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,383 @@
+---
+license: mit
+language:
+- en
+base_model: unsloth/qwen3-0.6b
+tags:
+- qwen3
+- lora
+- knowledge-graph
+- relation-extraction
+- information-extraction
+- graph-rag
+- triplet-extraction
+- structured-generation
+pipeline_tag: text-generation
+library_name: transformers
+---
+
+# Qwen3-0.6B-KG-Triplets
+
+Qwen3-0.6B-KG-Triplets is a LoRA finetuned version of Qwen3-0.6B specialized for ontology-constrained knowledge graph extraction.
+
+Given a passage of text, the model generates structured triplets in the form:
+
+```
+source -> relation -> target
+```
+
+where:
+
+- `"source"` contains an entity title and type
+- `"relation"` contains a relation type and confidence weight
+- `"target"` contains an entity title and type
+
+The output is designed for direct ingestion into graph databases and GraphRAG pipelines with minimal post-processing.
+
+---
+
+## Motivation
+
+Most instruction-tuned LLMs can extract entities and relations, but their outputs are difficult to ingest directly into graph databases because of:
+
+- inconsistent entity naming
+- out-of-schema relations
+- poorly calibrated confidence scores
+- inconsistent JSON formatting
+
+This model was finetuned specifically to produce:
+
+- ontology-constrained outputs
+- normalized entity names
+- calibrated relation confidence weights
+- graph-ingestable JSON
+
+---
+
+## Model Details
+
+| Property | Value |
+|---|---|
+| Base Model | unsloth/qwen3-0.6b |
+| Finetuning | LoRA |
+| Rank (r) | 32 |
+| Alpha | 32 |
+| Context Length | 2048 |
+| Epochs | 5 |
+| Optimizer | AdamW 8-bit |
+| Framework | Unsloth + TRL |
+| Training Type | Instruction Finetuning |
+| License | MIT |
+
+---
+
+## Training Configuration
+
+```python
+model = FastLanguageModel.get_peft_model(
+    model,
+    r=32,
+    target_modules=[
+        "q_proj",
+        "k_proj",
+        "v_proj",
+        "o_proj",
+        "gate_proj",
+        "up_proj",
+        "down_proj",
+    ],
+    lora_alpha=32,
+    lora_dropout=0,
+    bias="none",
+    use_gradient_checkpointing="unsloth",
+)
+```
+
+| Parameter | Value |
+|---|---|
+| Batch size | 2 |
+| Gradient accumulation | 4 |
+| Learning rate | 5e-5 |
+| Epochs | 5 |
+| Warmup steps | 50 |
+| Max sequence length | 2048 |
+| Optimizer | AdamW 8-bit |
+| Seed | 42 |
+
+---
+
+## Dataset
+
+The model was trained on a custom instruction dataset for structured knowledge graph extraction.
+
+### Corpus Sources
+
+- Wikipedia
+- arXiv papers
+
+### Dataset Statistics
+
+| Split | Examples |
+|---|---|
+| Train | 2575 |
+| Validation | 75 |
+| Test | 700 |
+| **Total** | **3350** |
+
+Additional properties:
+
+- 20 ontology relations
+- 15% hard negatives in training
+- Entity-level train/test decontamination
+- Curriculum ordering (easy → hard)
+- Zero schema errors
+
+---
+
+## Relation Schema
+
+The model predicts only the following ontology:
+
+```
+implements
+trained_on
+evaluates
+part_of
+introduces
+extends
+depends_on
+contrasts_with
+applied_to
+measured_by
+founded_by
+developed_by
+defined_as
+consists_of
+is_type_of
+based_on
+used_for
+created_by
+located_in
+predecessor_of
+```
+
+Relations outside this ontology are intentionally not generated.
+
+---
+
+## Dataset Creation Pipeline
+
+The training corpus was built using a multi-stage pipeline:
+
+1. Corpus collection from Wikipedia and arXiv
+2. Language and quality filtering
+3. MinHash deduplication
+4. LLM triplet generation using DeepSeek V4-Flash
+5. Schema validation
+6. Semantic validation
+7. Hard negative generation
+8. Curriculum ordering
+9. Entity-level train/test decontamination
+10. Train / Validation / Test split
+
+---
+
+## Training Example
+
+Input:
+
+```json
+{
+  "role": "user",
+  "content": "Extract knowledge graph triplets..."
+}
+```
+
+Output:
+
+```json
+[
+  {
+    "source": {
+      "title": "September",
+      "type": "entity"
+    },
+    "relation": {
+      "type": "part_of",
+      "weight": 0.92
+    },
+    "target": {
+      "title": "Gregorian calendar",
+      "type": "entity"
+    }
+  },
+  {
+    "source": {
+      "title": "September",
+      "type": "entity"
+    },
+    "relation": {
+      "type": "defined_as",
+      "weight": 0.92
+    },
+    "target": {
+      "title": "ninth month",
+      "type": "concept"
+    }
+  }
+]
+```
+
+---
+
+## Evaluation
+
+Evaluation was performed using a custom triplet extraction benchmark with Hungarian bipartite matching alignment on 700 held-out entries.
+
+### Metrics
+
+| Metric | Score | Weight |
+|---|---|---|
+| Schema score | 1.000 | 0.30 |
+| Entity F1 | 0.179 | 0.25 |
+| Relation accuracy | 0.680 | 0.20 |
+| Grounding | 0.969 | 0.15 |
+| Weight score | 0.526 | 0.10 |
+| Triplet F1 *(info only)* | 0.122 | — |
+| Type agreement *(info only)* | 0.854 | — |
+| Hallucination rate | 0.033 | — |
+
+**Composite Score: 0.6583**
+
+---
+
+## What Finetuning Fixed
+
+Finetuning addressed three major failure modes of the base model.
+
+### 1. Entity Normalization
+
+Input passage:
+
+> *Studies of the Cambrian period document the rapid diversification of animal life and the emergence of most major animal phyla, with some researchers proposing that a celestial body impact may have triggered the extinction events that preceded this radiation.*
+
+Base entity title extracted:
+
+```
+After a thorough research on the circumstantial changes and the great evolution of life in the Cambrian period
+```
+
+Finetuned entity title extracted:
+
+```
+Celestial body impact hypothesis
+```
+
+The finetuned model learns reusable and atomic graph nodes rather than copying passage fragments.
+
+---
+
+### 2. Schema Adherence
+
+Base relations generated:
+
+```
+released
+benefited_from
+```
+
+Finetuned relations generated:
+
+```
+based_on
+used_for
+applied_to
+introduces
+```
+
+All generated relations belong to the predefined ontology.
+
+---
+
+### 3. Confidence Calibration
+
+Base weights:
+
+```
+0.8
+0.8
+0.8
+0.8
+```
+
+Finetuned weights:
+
+```
+0.23
+0.41
+0.59
+0.77
+```
+
+The model learns meaningful confidence distributions where stronger relations receive higher scores.
+
+---
+
+## Intended Use
+
+This model is intended for:
+
+- Knowledge Graph Construction
+- GraphRAG pipelines
+- Structured Information Extraction
+- Entity-Relation Extraction
+- Automated KG population
+- Document-to-Graph conversion
+
+---
+
+## Limitations
+
+While the model demonstrates strong schema adherence and grounding, several limitations remain.
+
+**Shallow Entity Abstraction**
+
+The model favors concise and reusable entities but may miss deeper semantic abstractions or hierarchical entity relationships.
+
+**Limited Recall**
+
+The model prioritizes schema correctness and grounded extraction over exhaustive triplet recall. Entity F1 of 0.179 reflects strict Hungarian-matching alignment on a 20-relation ontology-constrained task; recall is intentionally traded for precision and schema adherence.
+
+**English-Centric Training**
+
+Training was primarily conducted on English Wikipedia and arXiv passages.
+
+**Ontology Constrained**
+
+Only the predefined 20 relation types are supported.
+
+**Model Size Constraints**
+
+Despite the relatively small size (0.6B parameters) and a modest training corpus (~3K examples), the model learns stable ontology-constrained extraction behavior. Larger models may achieve deeper entity understanding and broader relation coverage.
+
+---
+
+## Repository
+
+Evaluation Pipeline: https://github.com/mohar-xe/HGR-finetuned-model-evaluation-pipeline
+
+Model: https://huggingface.co/mohar07/qwen3-0.6b-kg-triplets
+
+---
+
+## Citation
+
+```bibtex
+@misc{das2026qwenkgtriplets,
+  title={Qwen3-0.6B-KG-Triplets},
+  author={Mohar Das},
+  year={2026},
+  publisher={Hugging Face},
+  url={https://huggingface.co/mohar07/qwen3-0.6b-kg-triplets}
+}
+```