135 lines
3.9 KiB
Markdown
135 lines
3.9 KiB
Markdown
|
---
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
license: other
|
||
|
|
base_model:
|
||
|
|
- Qwen/Qwen3-1.7B
|
||
|
|
datasets:
|
||
|
|
- 0xZee/dataset-CoT-Advanced-Calculus-268
|
||
|
|
- NousResearch/hermes-function-calling-v1
|
||
|
|
tags:
|
||
|
|
- transformers
|
||
|
|
- qwen3
|
||
|
|
- text-generation
|
||
|
|
- conversational
|
||
|
|
- agentic
|
||
|
|
- function-calling
|
||
|
|
- tool-use
|
||
|
|
- knowledge-distillation
|
||
|
|
- topology-guided-distillation
|
||
|
|
- lora-merged
|
||
|
|
- mathematics
|
||
|
|
- disc
|
||
|
|
- dualmind
|
||
|
|
- convergent-intelligence
|
||
|
|
---
|
||
|
|
|
||
|
|
# DualMind TKD Agentic 1.7B
|
||
|
|
|
||
|
|
DualMind TKD Agentic 1.7B is a two-stage derivative of
|
||
|
|
`Qwen/Qwen3-1.7B`.
|
||
|
|
|
||
|
|
It combines topology-guided mathematical knowledge distillation with
|
||
|
|
assistant-masked agentic and function-calling specialization.
|
||
|
|
|
||
|
|
## Training lineage
|
||
|
|
|
||
|
|
### Stage 1: topology-guided knowledge distillation
|
||
|
|
|
||
|
|
- Student: `Qwen/Qwen3-1.7B`
|
||
|
|
- Teacher: `Qwen/Qwen3-8B`
|
||
|
|
- Dataset: `0xZee/dataset-CoT-Advanced-Calculus-268`
|
||
|
|
- Training scope: full-model fine-tuning
|
||
|
|
- Objective: supervised cross-entropy plus sparse top-k-and-tail
|
||
|
|
teacher distillation
|
||
|
|
- Structural signals: teacher distribution discrepancy, transition
|
||
|
|
topology, gap-energy diagnostics, and phase-weighted
|
||
|
|
Explore/Examine/Response supervision
|
||
|
|
|
||
|
|
Stage 1 was designed to transfer mathematical reasoning behavior while
|
||
|
|
placing additional learning pressure on derivation, verification, and
|
||
|
|
high-discrepancy reasoning transitions.
|
||
|
|
|
||
|
|
### Stage 2: agentic specialization
|
||
|
|
|
||
|
|
- Dataset: `NousResearch/hermes-function-calling-v1`
|
||
|
|
- Training scope: LoRA specialization followed by weight merging
|
||
|
|
- Supervision: assistant and tool-call outputs only
|
||
|
|
- Tool schemas, user messages, and tool-result messages were visible as
|
||
|
|
context but excluded from direct loss
|
||
|
|
- Mathematical replay was mixed into Stage 2 to reduce catastrophic
|
||
|
|
forgetting
|
||
|
|
|
||
|
|
The files in this repository contain the merged standalone model.
|
||
|
|
A separate PEFT adapter is not required for inference.
|
||
|
|
|
||
|
|
## Intended uses
|
||
|
|
|
||
|
|
- Mathematical and technical reasoning
|
||
|
|
- Structured function calling
|
||
|
|
- Tool-selection experiments
|
||
|
|
- Agent-loop research
|
||
|
|
- Continued supervised or preference optimization
|
||
|
|
- Research on topology-aware distillation
|
||
|
|
|
||
|
|
## Tool execution
|
||
|
|
|
||
|
|
This model can generate tool calls, but it does not execute external
|
||
|
|
tools by itself.
|
||
|
|
|
||
|
|
A surrounding runtime must:
|
||
|
|
|
||
|
|
1. Parse the model's tool call.
|
||
|
|
2. Execute the selected tool.
|
||
|
|
3. Append the tool result to the conversation.
|
||
|
|
4. Invoke the model again for its next action or final response.
|
||
|
|
|
||
|
|
## Loading
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
repo_id = "reaperdoesntknow/DualMind-TKD-Agentic-1.7B"
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(repo_id)
|
||
|
|
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
||
|
|
repo_id,
|
||
|
|
torch_dtype="auto",
|
||
|
|
device_map="auto",
|
||
|
|
)
|
||
|
|
Evaluation status
|
||
|
|
|
||
|
|
No formal benchmark results are claimed in this release.
|
||
|
|
|
||
|
|
The training pipeline includes held-out loss monitoring and qualitative
|
||
|
|
generation smoke tests, but external mathematics, function-calling,
|
||
|
|
retention, and safety benchmarks should be run before production use.
|
||
|
|
|
||
|
|
Limitations
|
||
|
|
The source mathematics dataset is small.
|
||
|
|
Synthetic or generated reasoning traces may contain incorrect
|
||
|
|
derivations or contradictory final answers.
|
||
|
|
Function-call formatting does not guarantee correct tool selection.
|
||
|
|
External tool outputs must be treated as untrusted input.
|
||
|
|
Mathematical replay reduces forgetting but does not prove retention.
|
||
|
|
The model has not been established as safe for autonomous,
|
||
|
|
high-impact, medical, financial, or legal action.
|
||
|
|
License note
|
||
|
|
|
||
|
|
The Qwen3-1.7B base model uses Apache-2.0 licensing. The Hermes
|
||
|
|
function-calling dataset also declares Apache-2.0.
|
||
|
|
|
||
|
|
The advanced-calculus dataset did not expose a clear license declaration
|
||
|
|
when this model card was prepared. Consequently, this repository is
|
||
|
|
temporarily marked license: other. Confirm the source dataset's reuse
|
||
|
|
terms before assigning a more permissive license to this derivative.
|
||
|
|
|
||
|
|
Developer
|
||
|
|
|
||
|
|
Convergent Intelligence LLC / Reaper
|
||
|
|
Hugging Face: reaperdoesntknow
|
||
|
|
<!-- cix-keeper-ts:2026-07-26T13:15:33Z -->
|