初始化项目，由ModelHub XC社区提供模型

Model: jsantillana/vectrayx-nano Source: Original Platform
2026-08-15 05:39:16 +08:00
commit 8002645089
23 changed files with 319 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,41 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+vectrayx-nano-f16.gguf filter=lfs diff=lfs merge=lfs -text
+vectrayx-nano-v4-q4.gguf filter=lfs diff=lfs merge=lfs -text
+vectrayx-nano-v6-q4.gguf filter=lfs diff=lfs merge=lfs -text
+vectrayx-nano-v7-f16.gguf filter=lfs diff=lfs merge=lfs -text
+vectrayx-nano-v14-f16.gguf filter=lfs diff=lfs merge=lfs -text
+vectrayx-nano-v14-q8.gguf filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,141 @@
+---
+datasets:
+- vectrayx/vectrayx-bench
+language:
+- es
+license: apache-2.0
+metrics:
+- accuracy
+- f1
+pipeline_tag: text-generation
+tags:
+- cybersecurity
+- spanish
+- tool-use
+- mcp
+- curriculum-learning
+- from-scratch
+- arxiv:2605.13989
+---
+
+# VectraYX-Nano
+
+VectraYX-Nano is a 42M-parameter Spanish cybersecurity language model trained **from scratch** with curriculum learning and native [Model Context Protocol (MCP)](https://modelcontextprotocol.io) tool use. It is, to our knowledge, the first published Spanish-native cybersecurity LLM with end-to-end MCP integration.
+
+[![arXiv](https://img.shields.io/badge/arXiv-2605.13989-b31b1b.svg)](https://arxiv.org/abs/2605.13989)
+[![Zenodo](https://zenodo.org/badge/DOI/10.5281/zenodo.20122226.svg)](https://doi.org/10.5281/zenodo.20122226)
+
+- **Paper:** [VectraYX-Nano: A 42M-Parameter Spanish Cybersecurity Language Model with Curriculum Learning and Native Tool Use](https://arxiv.org/abs/2605.13989)
+- **Repository:** [vectrayx/vectrayx-nano-paper](https://github.com/vectrayx/vectrayx-nano-paper)
+- **arXiv DOI:** https://doi.org/10.48550/arXiv.2605.13989
+- **Author website:** https://jsantillana.com
+
+---
+
+## Released Model: VectraYX-Nano v7 (Headline)
+
+**VectraYX-Nano v7** is the released headline model. It uses the same 42M architecture and three-phase curriculum pre-training as the v2 bootstrap-ablation reference, with the SFT corpus rebalanced to a tool-use ratio of 1:21 (vs. 1:211 in v2). This single change raises B4 (tool-selection) from 0.000 to **0.230 ± 0.052** across N=4 seeds while retaining strong CVE recall (B1=0.332±0.005) and conversational quality (B5=0.725±0.130).
+
+Files in this repo:
+| File | Description |
+|---|---|
+| `nano_sft_v7_s42.pt` | **Nano v7 seed 42 — recommended for inference** |
+| `nano_sft_v5.pt` | Nano v2 (mixed SFT, bootstrap-ablation reference) |
+| `vectrayx-nano-f16.gguf` | **F16 GGUF — run with llama.cpp / Ollama** |
+| `lora/nano_lora_mini_s{42,7,13,23}.pt` | LoRA adapters (tool-use density study, ratio 1:21) |
+| `tokenizer/vectrayx_bpe.model` | BPE-16384 tokenizer |
+| `configs/nano.json` | Nano 42M architecture config |
+| `configs/base.json` | Base 260M architecture config |
+
+---
+
+## Key Results (VectraYX-Bench, N=4 seeds)
+
+| Model | Params | B1 KW | B2 F1† | B3 TM | B4 Tool | B5 Chat |
+|---|---|---|---|---|---|---|
+| **VectraYX-Nano v7** *(headline)* | 42M | **0.332±0.005** | — | — | **0.230±0.052** | 0.725±0.130 |
+| VectraYX-Nano v2 *(bootstrap ablation)* | 42M | 0.226±0.065 | 0.199±0.004 | 0.029±0.035 | 0.000 | **0.775±0.043** |
+| Nano LoRA mini (ratio 1:21, N=4) | 42M | 0.011±0.004 | 0.201±0.002 | 0.021±0.012 | 0.145±0.046 | 0.575±0.043 |
+| SmolLM2-135M + LoRA-32 | 135M | 0.334 | 0.225 | 0.143 | 0.160 | 0.800 |
+| VectraYX-Base 260M | 260M | 0.325 | 0.220 | 0.114 | 0.000 | 0.800 |
+| Base 260M LoRA mini (ratio 1:21, N=4) | 260M | 0.019±0.003 | 0.203±0.002 | — | 0.445±0.201 | 0.600 |
+| VectraYX-Pro 3B | 3.2B | 0.341 | 0.695 | 0.686 | 0.600 | 0.800 |
+| VectraYX-Pro 7B | 7B | 0.335 | 0.815 | 0.686 | 0.880 | 0.800 |
+| GPT-4o *(frontier reference)* | — | 0.333 | 0.110 | 0.520 | 0.615 | 0.631 |
+
+†B2 is a benchmark artifact in this revision (key mismatch in harness, fix queued).
+
+**B5 inversion:** Nano v7 (0.725±0.130) and Nano v2 (0.775±0.043) both **exceed GPT-4o (0.631)** on the 314-prompt held-out chat suite — the register-matched bootstrap corpus makes conversational Spanish the model's first language.
+
+---
+
+## Key Findings
+
+**1. Loss-vs-register inversion.** A higher-perplexity bootstrap corpus (OpenSubtitles-ES) yields *better* post-SFT chat behavior than a lower-perplexity alternative (mC4-ES). At the nano scale, the bootstrap corpus dictates the model's default response style; SFT cannot fully overwrite it.
+
+**2. Tool-use is corpus-density-gated, not capacity-gated.** The B4=0.000 floor in the mixed SFT (ratio 1:211) is a corpus-density artifact. Rebalancing to 1:21 (2,801 tool-use examples) shifts the first-token prior to `<|tool_call|>` and raises B4 to 0.230±0.052 at 42M — without retraining the backbone.
+
+---
+
+## Inference: llama.cpp / Ollama (GGUF)
+
+```bash
+# With Ollama
+ollama run hf.co/jsantillana/vectrayx-nano:vectrayx-nano-f16.gguf
+
+# With llama.cpp
+./llama-cli -m vectrayx-nano-f16.gguf \
+  --chat-template llama3 \
+  -p "<|system|>Eres VectraYX, asistente experto en ciberseguridad para LATAM.<|end|>" \
+  -i
+```
+
+Runs at 6–10 tok/s on Raspberry Pi 4 and 60–100 tok/s on a laptop CPU.
+
+---
+
+## Inference: PyTorch
+
+```python
+from huggingface_hub import hf_hub_download
+import torch, json, sys
+
+sys.path.insert(0, ".")  # needs training/transformer.py from vectrayx-paper-code
+
+ckpt = hf_hub_download("jsantillana/vectrayx-nano", "nano_sft_v7_s42.pt")
+tok  = hf_hub_download("jsantillana/vectrayx-nano", "tokenizer/vectrayx_bpe.model")
+cfg  = hf_hub_download("jsantillana/vectrayx-nano", "configs/nano.json")
+```
+
+Full inference script at [vectrayx-paper-code](https://huggingface.co/jsantillana/vectrayx-paper-code).
+
+---
+
+## Training Details
+
+| Component | Details |
+|---|---|
+| Parameters | 41.95M |
+| Architecture | Transformer decoder, GQA (8q/2kv), QK-Norm, RMSNorm, SwiGLU, RoPE, z-loss |
+| Tokenizer | BPE-16384, byte-fallback, 50/50 conv/tech balance |
+| Pre-training | 170M tokens, 3-phase curriculum with 25% replay buffer |
+| SFT (v7) | 13K OASST1-ES + 4K CVE Q&A + 2.8K tool-use (ratio 1:21) |
+| Hardware | GCP L4 24GB (pre-training) + AWS g4dn.xlarge T4 16GB (multi-seed SFT) |
+| Cost | ~$29 USD total (corpus + training) |
+
+---
+
+## Citation
+
+```bibtex
+@misc{santillana2026vectrayx,
+  title     = {VectraYX-Nano: A 42M-Parameter Spanish Cybersecurity Language Model
+               with Curriculum Learning and Native Tool Use},
+  author    = {Santillana, Juan S.},
+  year      = {2026},
+  eprint    = {2605.13989},
+  archivePrefix = {arXiv},
+  primaryClass  = {cs.CL},
+  url       = {https://arxiv.org/abs/2605.13989}
+}
+```
--- a/configs/base.json
+++ b/configs/base.json
@@ -0,0 +1,40 @@
+{
+  "model": {
+    "vocab_size": 16384,
+    "n_layers": 16,
+    "n_heads": 16,
+    "n_kv_heads": 4,
+    "d_model": 1024,
+    "d_ffn": 4096,
+    "max_seq_len": 1024,
+    "rope_theta": 10000.0,
+    "rms_eps": 1e-6,
+    "init_std": 0.02,
+    "dropout": 0.0,
+    "tie_embeddings": true,
+    "qk_norm": true,
+    "z_loss_coef": 1e-4
+  },
+  "tokenizer": {
+    "vocab_size": 16384,
+    "model_type": "bpe",
+    "character_coverage": 1.0,
+    "byte_fallback": true,
+    "normalization": "nmt_nfkc",
+    "split_digits": true,
+    "split_by_unicode_script": true,
+    "add_dummy_prefix": true,
+    "user_defined_symbols": [
+      "<|pad|>", "<|bos|>", "<|eos|>", "<|unk|>", "<|sep|>",
+      "<|system|>", "<|user|>", "<|assistant|>", "<|end|>",
+      "<|tool_call|>", "<|/tool_call|>", "<|tool_result|>", "<|/tool_result|>",
+      "<|cve|>", "<|cvss|>", "<|ioc|>", "<|ttp|>", "<|mitre|>", "<|kev|>",
+      "<|exploit|>", "<|patch|>", "<|alert|>",
+      "<|critical|>", "<|high|>", "<|medium|>", "<|low|>", "<|info|>"
+    ],
+    "balance": {
+      "conversational_ratio": 0.5,
+      "technical_ratio": 0.5
+    }
+  }
+}
--- a/configs/nano.json
+++ b/configs/nano.json
@@ -0,0 +1,40 @@
+{
+  "model": {
+    "vocab_size": 16384,
+    "n_layers": 8,
+    "n_heads": 8,
+    "n_kv_heads": 2,
+    "d_model": 512,
+    "d_ffn": 2304,
+    "max_seq_len": 1024,
+    "rope_theta": 10000.0,
+    "rms_eps": 1e-6,
+    "init_std": 0.02,
+    "dropout": 0.0,
+    "tie_embeddings": true,
+    "qk_norm": true,
+    "z_loss_coef": 1e-4
+  },
+  "tokenizer": {
+    "vocab_size": 16384,
+    "model_type": "bpe",
+    "character_coverage": 1.0,
+    "byte_fallback": true,
+    "normalization": "nmt_nfkc",
+    "split_digits": true,
+    "split_by_unicode_script": true,
+    "add_dummy_prefix": true,
+    "user_defined_symbols": [
+      "<|pad|>", "<|bos|>", "<|eos|>", "<|unk|>", "<|sep|>",
+      "<|system|>", "<|user|>", "<|assistant|>", "<|end|>",
+      "<|tool_call|>", "<|/tool_call|>", "<|tool_result|>", "<|/tool_result|>",
+      "<|cve|>", "<|cvss|>", "<|ioc|>", "<|ttp|>", "<|mitre|>", "<|kev|>",
+      "<|exploit|>", "<|patch|>", "<|alert|>",
+      "<|critical|>", "<|high|>", "<|medium|>", "<|low|>", "<|info|>"
+    ],
+    "balance": {
+      "conversational_ratio": 0.5,
+      "technical_ratio": 0.5
+    }
+  }
+}
--- a/lora/nano_lora_mini_s13.pt
+++ b/lora/nano_lora_mini_s13.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7a7680a4a2b1c8b9efa72ff9344486c188695fe232eb2af5619775a33439f80
+size 5188434
--- a/lora/nano_lora_mini_s23.pt
+++ b/lora/nano_lora_mini_s23.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7a7680a4a2b1c8b9efa72ff9344486c188695fe232eb2af5619775a33439f80
+size 5188434
--- a/lora/nano_lora_mini_s42.pt
+++ b/lora/nano_lora_mini_s42.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6c2709245f372a9fa98be8fba3785ff5aad82f20b8b517f2f581d3fa0aac7de
+size 5188434
--- a/lora/nano_lora_mini_s7.pt
+++ b/lora/nano_lora_mini_s7.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7a7680a4a2b1c8b9efa72ff9344486c188695fe232eb2af5619775a33439f80
+size 5188434
--- a/nano_sft_v2_s13.pt
+++ b/nano_sft_v2_s13.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:880463b90d3293383d60aed596a40cbd49d08618931e2ff804540b267cd4def0
+size 503545117
--- a/nano_sft_v2_s23.pt
+++ b/nano_sft_v2_s23.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:854cc09d21ba62bd27e32360641393a47d41671b591629cd2df91f5b643bea10
+size 503544718
--- a/nano_sft_v2_s42.pt
+++ b/nano_sft_v2_s42.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46f472bce1c1258dc3a7d77d54663f08a1e375c938090418d14c7e953f1e6d89
+size 503545117
--- a/nano_sft_v2_s7.pt
+++ b/nano_sft_v2_s7.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:449c3bc42fc3bd9c5843ae85dfc745154f11b048b8b12d9b81b4f00f27e43b66
+size 503545117
--- a/nano_sft_v4_s42.pt
+++ b/nano_sft_v4_s42.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6e6099a246e0ba3d8b244bdb8731d7d66174fd217b67fc3ae9726fb417cd8c6
+size 503545117
--- a/nano_sft_v5.pt
+++ b/nano_sft_v5.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:449c3bc42fc3bd9c5843ae85dfc745154f11b048b8b12d9b81b4f00f27e43b66
+size 503545117
--- a/nano_sft_v6_s42.pt
+++ b/nano_sft_v6_s42.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5dbffe79c0888ee178e403b218d6b6491215ab490608761228571b72860b6c9
+size 503545117
--- a/nano_sft_v7_s13.pt
+++ b/nano_sft_v7_s13.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa2652dc093eb51dc974bcc09cbfbe0e3ee64aaeb6b72ac169476384c8410906
+size 503544718
--- a/nano_sft_v7_s23.pt
+++ b/nano_sft_v7_s23.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4ed5c7aa1b577e81170d5cdfe4dc1752c0ee0496c8dd604388f7f4571d2883c
+size 503544718
--- a/nano_sft_v7_s42.pt
+++ b/nano_sft_v7_s42.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e9822e81bac7531d9d724685e67125d2caa9560c814a4bed64615d2da350c1b
+size 503544718
--- a/nano_sft_v7_s7.pt
+++ b/nano_sft_v7_s7.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d563b9516fb1b411d7628a3609501bc86495bb9f2f7748a99495b0c873e8ec34
+size 503544718
--- a/tokenizer/vectrayx_bpe.model
+++ b/tokenizer/vectrayx_bpe.model
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b301a6c9e5621df751c4b17e2d4bf9751a07dcd3e518d378524444654dc6f3bb
+size 474625
--- a/vectrayx-nano-v14-f16.gguf
+++ b/vectrayx-nano-v14-f16.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b15ea7946dc46283e717d118fe6efd58ab7c71f4e152e74f205dc7a68c32b32
+size 101297344
--- a/vectrayx-nano-v14-q8.gguf
+++ b/vectrayx-nano-v14-q8.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36b79e6866d7ec8436f97ad8f1877c248463b0567b75be6dca8439510be9addb
+size 54111456
--- a/vectrayx-nano-v7-f16.gguf
+++ b/vectrayx-nano-v7-f16.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8487869a3f56bac7c88d11f96f2c85d90a7b3d04ea34470d941db42da496763
+size 101297344