--- license: apache-2.0 library_name: transformers pipeline_tag: text-generation tags: - qwen3 - sft - trl - dualmind - knowledge-distillation - thinking - opus - self-critique - convergent-intelligence - convergentintel - edge - distillation base_model: - reaperdoesntknow/DualMinded-Qwen3-1.7B datasets: - nohurry/Opus-4.6-Reasoning-3000x-filtered - zai-org/LongWriter-6k language: - en --- # Dualmind-Qwen-1.7B-Thinking **Claude Opus 4.6 Reasoning Traces → 1.7B via DualMind SFT** *Convergent Intelligence LLC: Research Division* --- ## What This Is A 1.7B model trained on **2.5M+ tokens of Claude Opus 4.6 reasoning traces** using the DualMind SFT methodology. The training data comes from [Opus-4.6-Reasoning-3000x-filtered](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered) — a curated dataset of extended reasoning chains from Anthropic's most capable model, with refusals removed. This is the **Opus variant** of the DualMind family. Where the base [DualMind](https://huggingface.co/reaperdoesntknow/DualMind) model was trained on LogicInference data, this model absorbs the reasoning patterns of Claude Opus 4.6 — longer chains, more nuanced self-correction, and richer deliberative structure. The Opus teacher produces qualitatively different reasoning than synthetic logic datasets: it backtracks, hedges, reconsiders, and synthesizes in ways that reflect genuine uncertainty navigation rather than pattern completion. The base model is [Disctil-Qwen3-1.7B](https://huggingface.co/reaperdoesntknow/Disctil-Qwen3-1.7B) — already DISC-refined and sitting in the middle of the DistilQwen distillation chain — giving it a strong structural foundation before the Opus reasoning signal is applied. ## Architecture | Parameter | Value | |-----------|-------| | Architecture | Qwen3ForCausalLM | | Parameters | ~2.03B (1.7B effective) | | Hidden Size | 2048 | | Layers | 28 | | Attention Heads | 16 (Q) / 8 (KV) — GQA | | Intermediate | 6144 | | Head Dimension | 128 | | Context Length | 40,960 tokens (max position) | | Vocabulary | 151,936 | | Precision | BF16 | | Activation | SiLU | ## Training | Parameter | Value | |-----------|-------| | Base Model | [Disctil-Qwen3-1.7B](https://huggingface.co/reaperdoesntknow/Disctil-Qwen3-1.7B) | | Dataset | [Opus-4.6-Reasoning-3000x-filtered](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered) | | Additional Tokens | ~2.5M | | Max Sequence Length | 4,096 | | Total Steps | 512 | | Epochs | ~7.4 | | Method | SFT (TRL SFTTrainer) | | Precision | BF16 | | Hardware | NVIDIA H100 | ### Training Dynamics | Metric | Start | End | |--------|-------|-----| | Training Loss | 1.744 | 1.455 | | Eval Loss | — | 1.406 | | Token Accuracy | 61.0% | 67.8% | The loss curve shows clean convergence across 7.4 epochs with no signs of overfitting — eval loss (1.406) remains below final training loss (1.455). The 6.8 percentage point gain in token accuracy reflects genuine absorption of the Opus reasoning structure, not memorization. ### Why Opus Traces The Opus-4.6-Reasoning dataset captures something that synthetic datasets don't: the way a frontier model navigates genuine uncertainty. Opus doesn't just solve problems — it reasons about its own confidence, backtracks when a line of thought weakens, and synthesizes across multiple attempted approaches. When you distill from these traces, the student doesn't just learn to produce correct answers. It learns the **shape of deliberation**. This is the DualMind thesis in practice: the cognitive loop (explore → examine → respond) isn't an architectural trick. It's a training signal. When the teacher naturally exhibits multi-phase reasoning, the student absorbs that structure through standard SFT. ## Usage ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "reaperdoesntknow/Dualmind-Qwen-1.7B-Thinking", torch_dtype="auto", device_map="auto" ) tokenizer = AutoTokenizer.from_pretrained( "reaperdoesntknow/Dualmind-Qwen-1.7B-Thinking" ) messages = [ {"role": "user", "content": "What happens to information that falls into a black hole? Walk me through the paradox."} ] text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) inputs = tokenizer(text, return_tensors="pt").to(model.device) output = model.generate( **inputs, max_new_tokens=2048, do_sample=True, top_p=0.9, temperature=0.7, repetition_penalty=1.15 ) print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` ### Generation Tips - **Temperature 0.6–0.8** — the Opus reasoning traces have natural variance in them. Don't flatten it with low temperature. - **Repetition penalty 1.1–1.2** — prevents looping during extended reasoning chains. - **Max tokens 1024–2048** — trained at 4096 max seq, so it can go long. The Opus signal rewards longer generation windows. - The model may produce multi-phase reasoning naturally (exploring, then reconsidering, then concluding). This is the intended behavior — the DualMind cognitive loop emerging from the training signal. ## Model Lineage ``` Qwen3-1.7B (base) → DiStil-Qwen3-1.7B-uncensored (uncensored SFT) → Disctil-Qwen3-1.7B (DISC refinement) → Dualmind-Qwen-1.7B-Thinking ← you are here ↑ Opus 4.6 reasoning traces (2.5M tokens, DualMind SFT) ``` ### DualMind Family Comparison | Model | Training Signal | Character | |-------|----------------|-----------| | [DualMind](https://huggingface.co/reaperdoesntknow/DualMind) | LogicInference | Structured logical deduction | | **Dualmind-Qwen-1.7B-Thinking** | **Opus 4.6 Reasoning** | **Extended deliberation, self-correction** | | [TopologicalQwen](https://huggingface.co/reaperdoesntknow/TopologicalQwen) | 30B-Thinking (TKD) | Topology-aware physics CoT | Same methodology, different teachers, different capabilities. The LogicInference variant is more mechanical. The Opus variant is more deliberative. TopologicalQwen is the full TKD pipeline with BV decomposition. They're complementary — different facets of the same cognitive architecture. ## DualMind Collection | Model | Description | |-------|-------------| | [DualMind](https://huggingface.co/reaperdoesntknow/DualMind) | LogicInference-trained. Explore→Examine→Response cognitive loop. | | [DualMind_Methodology](https://huggingface.co/reaperdoesntknow/DualMind_Methodolgy) | Paper: Three Teachers to Dual Cognition (DOI: 10.57967/hf/8184) | | **[Dualmind-Qwen-1.7B-Thinking](https://huggingface.co/reaperdoesntknow/Dualmind-Qwen-1.7B-Thinking)** | **← this model. Opus 4.6 reasoning variant.** | | [DualMind-GGUF](https://huggingface.co/reaperdoesntknow/DualMind-GGUF) | LogicInference variant quantized for edge deployment. | Full collection: [DualMind on HuggingFace](https://huggingface.co/collections/reaperdoesntknow/dualmind-69c93f888c6e79ecc69cf41e) ## Papers - **[Structure Over Scale: Proof-Weighted Knowledge Distillation](https://doi.org/10.57967/hf/8165)** — DOI: 10.57967/hf/8165. The DistilQwen methodology paper. - **[Three Teachers to Dual Cognition](https://doi.org/10.57967/hf/8184)** — DOI: 10.57967/hf/8184. The DualMind extension: ghost imprinting and multi-teacher convergence. ## License Apache 2.0 ## Mathematical Foundations: Discrepancy Calculus (DISC) This model's training pipeline is grounded in Discrepancy Calculus — a measure-theoretic framework that treats singularities as primary structure rather than pathology. Full theory: *"On the Formal Analysis of Discrepancy Calculus"* (Colca, 2026; Convergent Intelligence LLC: Research Division). **The Core Operator:** $$Df(x) = \lim_{\varepsilon \downarrow 0} \frac{1}{\varepsilon} \int_x^{x+\varepsilon} \frac{|f(t) - f(x)|}{|t - x|}\, dt$$ For smooth $f$: $Df(x) = |f'(x)|$. For rough $f$: $D$ localizes irregularity to null sets while preserving integral structure. **The Mesh Fundamental Identity** — every BV function decomposes as: $$f(b) - f(a) = \underbrace{\int_a^b f'(x)\,dx}_{\text{smooth (AC)}} + \underbrace{\sum_{x \in J_f} \Delta f(x)}_{\text{jumps}} + \underbrace{D^c f(I)}_{\text{Cantor drift}}$$ Standard knowledge distillation captures only term 1. Topological Knowledge Distillation (TKD) preserves all three by treating the teacher's output distribution as a BV function and computing discrepancy energy, jump sets, and gap energy density before training begins. ## Citation ```bibtex @misc{colca2026dualmind, title={Three Teachers to Dual Cognition: From Knowledge Distillation to Emergent Reasoning}, author={Colca, Roy}, year={2026}, doi={10.57967/hf/8184}, publisher={Convergent Intelligence LLC: Research Division} } ``` --- *Convergent Intelligence LLC: Research Division — 49 models, 22,598+ downloads across the portfolio.* *[Full portfolio](https://huggingface.co/reaperdoesntknow) | [DualMind Collection](https://huggingface.co/collections/reaperdoesntknow/dualmind-69c93f888c6e79ecc69cf41e) | [DistilQwen Collection](https://huggingface.co/collections/reaperdoesntknow/distilqwen-69bf40ec669117e3f069ef1c)* --- ## Convergent Intelligence Portfolio *Part of the [DualMind Series](https://huggingface.co/collections/reaperdoesntknow/dualmind-69c93f888c6e79ecc69cf41e) by [Convergent Intelligence LLC: Research Division](https://huggingface.co/reaperdoesntknow)* ### DualMind Family | Model | Format | Description | |-------|--------|-------------| | [DualMind](https://huggingface.co/reaperdoesntknow/DualMind) | BF16 | LogicInference-trained. Explore→Examine→Response loop. | | [DualMinded-Qwen3-1.7B](https://huggingface.co/reaperdoesntknow/DualMinded-Qwen3-1.7B) | BF16 | Opus 4.6 reasoning traces. Higher quality splits. | | [Dualmind-Qwen-1.7B-Thinking](https://huggingface.co/reaperdoesntknow/Dualmind-Qwen-1.7B-Thinking) | BF16 | Thinking-teacher variant with extended deliberation. | | [DualMind-GGUF](https://huggingface.co/reaperdoesntknow/DualMind-GGUF) | GGUF | Quantized LogicInference variant. CPU/6GB GPU. | | [DualMinded-Qwen3-1.7B-GGUF](https://huggingface.co/reaperdoesntknow/DualMinded-Qwen3-1.7B-GGUF) | GGUF | Quantized Opus variant. Ollama ready. | ### Papers | Paper | DOI | |-------|-----| | [Structure Over Scale](https://huggingface.co/reaperdoesntknow/Structure-Over-Scale) | 10.57967/hf/8165 | | [Three Teachers to Dual Cognition](https://huggingface.co/reaperdoesntknow/DualMind_Methodolgy) | 10.57967/hf/8184 | | [Discrepancy Calculus](https://huggingface.co/reaperdoesntknow/Discrepancy_Calculus) | 10.57967/hf/8194 | --- *Last updated: 2026-03-31 by Convergent Intelligence LLC: Research Division*