commit cc7c323f3d073a7bde998b0c432d3bee31c5707e Author: ModelHub XC Date: Sun Jul 12 01:25:04 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: unsloth/Jan-nano-128k-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..53d7257 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,47 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bin.* filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zstandard filter=lfs diff=lfs merge=lfs -text +*.tfevents* filter=lfs diff=lfs merge=lfs -text +*.db* filter=lfs diff=lfs merge=lfs -text +*.ark* filter=lfs diff=lfs merge=lfs -text +**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text +**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text +**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.gguf* filter=lfs diff=lfs merge=lfs -text +*.ggml filter=lfs diff=lfs merge=lfs -text +*.llamafile* filter=lfs diff=lfs merge=lfs -text +*.pt2 filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text \ No newline at end of file diff --git a/Jan-nano-128k-BF16.gguf b/Jan-nano-128k-BF16.gguf new file mode 100644 index 0000000..915116e --- /dev/null +++ b/Jan-nano-128k-BF16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1d38e8eed3d56f1b767792506c8d6629be3abfbdcd840b3be0e61b1faf4d60e +size 8051285408 diff --git a/Jan-nano-128k-IQ4_NL.gguf b/Jan-nano-128k-IQ4_NL.gguf new file mode 100644 index 0000000..633c507 --- /dev/null +++ b/Jan-nano-128k-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aa0bcdedceade0ffc932564aae98890426fc2eab35236b79c89211f476e18ddd +size 2381343872 diff --git a/Jan-nano-128k-IQ4_XS.gguf b/Jan-nano-128k-IQ4_XS.gguf new file mode 100644 index 0000000..e635216 --- /dev/null +++ b/Jan-nano-128k-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1d3abf5286b664d364998314b20c952da0c4c752dd86912ee338eebe2cd75dc5 +size 2270751872 diff --git a/Jan-nano-128k-Q2_K.gguf b/Jan-nano-128k-Q2_K.gguf new file mode 100644 index 0000000..e90b421 --- /dev/null +++ b/Jan-nano-128k-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:000209888b8b68ea09c17e65560a035dcec03358260455b5b4b179c8fd79ec5c +size 1669500032 diff --git a/Jan-nano-128k-Q2_K_L.gguf b/Jan-nano-128k-Q2_K_L.gguf new file mode 100644 index 0000000..e90b421 --- /dev/null +++ b/Jan-nano-128k-Q2_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:000209888b8b68ea09c17e65560a035dcec03358260455b5b4b179c8fd79ec5c +size 1669500032 diff --git a/Jan-nano-128k-Q3_K_M.gguf b/Jan-nano-128k-Q3_K_M.gguf new file mode 100644 index 0000000..0f23024 --- /dev/null +++ b/Jan-nano-128k-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3900eaa640c2019b669cac9f072e71ab4b00c8fddb6db38ccb26eaf0f82153dd +size 2075618432 diff --git a/Jan-nano-128k-Q3_K_S.gguf b/Jan-nano-128k-Q3_K_S.gguf new file mode 100644 index 0000000..e663872 --- /dev/null +++ b/Jan-nano-128k-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:686ea1cc7a9b50fc2d7b44c4108703171e6749b5ac33dcad342f54580ceefd88 +size 1886997632 diff --git a/Jan-nano-128k-Q4_0.gguf b/Jan-nano-128k-Q4_0.gguf new file mode 100644 index 0000000..0b82a30 --- /dev/null +++ b/Jan-nano-128k-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:986e71e0cb738132ca2812055e7b2e4dee4670ebe4a52d24454f818caf84d765 +size 2375773312 diff --git a/Jan-nano-128k-Q4_1.gguf b/Jan-nano-128k-Q4_1.gguf new file mode 100644 index 0000000..d04655f --- /dev/null +++ b/Jan-nano-128k-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3c0b92f04787f02f05f260961d974528e35fa0bffc3cb0e63cadb698e4e3eb9c +size 2596629632 diff --git a/Jan-nano-128k-Q4_K_M.gguf b/Jan-nano-128k-Q4_K_M.gguf new file mode 100644 index 0000000..e53d449 --- /dev/null +++ b/Jan-nano-128k-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:78ed675e0062e6eef86ddb3e91f021dca3691c924151bc2bc5e01e8e6484c125 +size 2497281152 diff --git a/Jan-nano-128k-Q4_K_S.gguf b/Jan-nano-128k-Q4_K_S.gguf new file mode 100644 index 0000000..3487228 --- /dev/null +++ b/Jan-nano-128k-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f98d1d91a9058e4b5da8086d8786e4ea331638c598403e43176c553c2b039e7 +size 2383309952 diff --git a/Jan-nano-128k-Q5_K_M.gguf b/Jan-nano-128k-Q5_K_M.gguf new file mode 100644 index 0000000..612059b --- /dev/null +++ b/Jan-nano-128k-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:367fbbdc2a9fa1ee9672326d37502fc790dadd07a383ee1c3d798bf35a69892d +size 2889514112 diff --git a/Jan-nano-128k-Q5_K_S.gguf b/Jan-nano-128k-Q5_K_S.gguf new file mode 100644 index 0000000..5d0aa42 --- /dev/null +++ b/Jan-nano-128k-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:63db2f7384b50557f482ffe6ffeb4e54728b9a4b16ec9b7247729719efc2fe35 +size 2823711872 diff --git a/Jan-nano-128k-Q6_K.gguf b/Jan-nano-128k-Q6_K.gguf new file mode 100644 index 0000000..112023c --- /dev/null +++ b/Jan-nano-128k-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2c84eec59c139d025db9055ed22e02d678828c86d00b0e1dbb93687d2fd6b3a +size 3306261632 diff --git a/Jan-nano-128k-Q8_0.gguf b/Jan-nano-128k-Q8_0.gguf new file mode 100644 index 0000000..aafcfe7 --- /dev/null +++ b/Jan-nano-128k-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a2f209d3655de54d9499c273acb3c926884f8198c905e0ec0e0bf0596f691e1c +size 4280405632 diff --git a/Jan-nano-128k-UD-IQ1_M.gguf b/Jan-nano-128k-UD-IQ1_M.gguf new file mode 100644 index 0000000..3c6598c --- /dev/null +++ b/Jan-nano-128k-UD-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:40c6c48ea5279c4c3417aeb90e719faab212100b2f6f03aa73755edc6e0710ac +size 1143484032 diff --git a/Jan-nano-128k-UD-IQ1_S.gguf b/Jan-nano-128k-UD-IQ1_S.gguf new file mode 100644 index 0000000..ff17bc5 --- /dev/null +++ b/Jan-nano-128k-UD-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ea33b31fb05818aea9202958bf327db1addb141a50f28eb56870cbe13172612e +size 1083149952 diff --git a/Jan-nano-128k-UD-IQ2_M.gguf b/Jan-nano-128k-UD-IQ2_M.gguf new file mode 100644 index 0000000..4b8460a --- /dev/null +++ b/Jan-nano-128k-UD-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4128aa48b7e9935356bd8aca8ee69c375d75085f16c2cdbf0c3b30e0a615251d +size 1530402432 diff --git a/Jan-nano-128k-UD-IQ2_XXS.gguf b/Jan-nano-128k-UD-IQ2_XXS.gguf new file mode 100644 index 0000000..54c329a --- /dev/null +++ b/Jan-nano-128k-UD-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f833e6d355ab1135f6499a38cda00051b473cfb285a214f8ca2268baa6fd2084 +size 1255345792 diff --git a/Jan-nano-128k-UD-IQ3_XXS.gguf b/Jan-nano-128k-UD-IQ3_XXS.gguf new file mode 100644 index 0000000..b4770f6 --- /dev/null +++ b/Jan-nano-128k-UD-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:38d35ad68bf83af51b8a7eb236aa189c18bac739a2374cfdfc32e2e0590ed8d8 +size 1674376832 diff --git a/Jan-nano-128k-UD-Q2_K_XL.gguf b/Jan-nano-128k-UD-Q2_K_XL.gguf new file mode 100644 index 0000000..8a652e0 --- /dev/null +++ b/Jan-nano-128k-UD-Q2_K_XL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a4864d16ffeed61a9db7a4e80cb587291bd9c18e9877e8508ba5fa362793b144 +size 1695724672 diff --git a/Jan-nano-128k-UD-Q3_K_XL.gguf b/Jan-nano-128k-UD-Q3_K_XL.gguf new file mode 100644 index 0000000..e26b6a9 --- /dev/null +++ b/Jan-nano-128k-UD-Q3_K_XL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bb71cec74a4684dd4e6cc56ce068fa4444a7c81390a8d08f4fb2851398dab996 +size 2128385152 diff --git a/Jan-nano-128k-UD-Q4_K_XL.gguf b/Jan-nano-128k-UD-Q4_K_XL.gguf new file mode 100644 index 0000000..78d7c2e --- /dev/null +++ b/Jan-nano-128k-UD-Q4_K_XL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b449711efeafa9a66cee608f7e9d66f4a3d8efc9a8234c149b8d6bee04f3fa4b +size 2546340992 diff --git a/Jan-nano-128k-UD-Q5_K_XL.gguf b/Jan-nano-128k-UD-Q5_K_XL.gguf new file mode 100644 index 0000000..add69bf --- /dev/null +++ b/Jan-nano-128k-UD-Q5_K_XL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d5bbe1dae74a7acf8a3688033f02153c4f07b4c9f272d75dc0297c9af065574e +size 2899221632 diff --git a/Jan-nano-128k-UD-Q6_K_XL.gguf b/Jan-nano-128k-UD-Q6_K_XL.gguf new file mode 100644 index 0000000..98b776d --- /dev/null +++ b/Jan-nano-128k-UD-Q6_K_XL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1de0f243271f2d36aae87a2df3c0caa1bdd8011d8dbdbff9400a58465875f48 +size 3658223232 diff --git a/Jan-nano-128k-UD-Q8_K_XL.gguf b/Jan-nano-128k-UD-Q8_K_XL.gguf new file mode 100644 index 0000000..ce2bf8b --- /dev/null +++ b/Jan-nano-128k-UD-Q8_K_XL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2ad6dee5664253ee14c2b5268835593280dcfde8c8ca7efffd847b8a05229fb +size 5056700032 diff --git a/README.md b/README.md new file mode 100644 index 0000000..ef57b1b --- /dev/null +++ b/README.md @@ -0,0 +1,113 @@ +--- +tags: +- unsloth +license: apache-2.0 +language: +- en +base_model: +- Menlo/Jan-nano-128k +pipeline_tag: text-generation +--- +
+

+ Unsloth Dynamic 2.0 achieves superior accuracy & outperforms other leading quants. +

+
+ + + + + + + + + +
+
+ + +# Jan-Nano-128k: Empowering deeper research through extended context understanding. + +[![GitHub](https://img.shields.io/badge/GitHub-Repository-blue?logo=github)](https://github.com/menloresearch/deep-research) +[![Context Length](https://img.shields.io/badge/Context-128k-green)](https://huggingface.co/Menlo/Jan-nano-128k) +[![License](https://img.shields.io/badge/License-Apache%202.0-yellow)](https://opensource.org/licenses/Apache-2.0) + +
+ Jan-Nano-128k +
+ +**Authors:** [Alan Dao](https://scholar.google.com/citations?user=eGWws2UAAAAJ&hl=en), [Bach Vu Dinh](https://scholar.google.com/citations?user=7Lr6hdoAAAAJ&hl=vi), [Thinh Le](https://scholar.google.com/citations?user=8tcN7xMAAAAJ&hl=en) + +## Overview + +Jan-Nano-128k represents a significant advancement in compact language models for research applications. Building upon the success of [Jan-Nano](https://huggingface.co/Menlo/Jan-nano), this enhanced version features a **native 128k context window** that enables deeper, more comprehensive research capabilities without the performance degradation typically associated with context extension methods. + +**Key Improvements:** +- **🔍 Research Deeper**: Extended context allows for processing entire research papers, lengthy documents, and complex multi-turn conversations +- **⚡ Native 128k Window**: Built from the ground up to handle long contexts efficiently, maintaining performance across the full context range +- **📈 Enhanced Performance**: Unlike traditional context extension methods, Jan-Nano-128k shows improved performance with longer contexts + +This model maintains full compatibility with Model Context Protocol (MCP) servers while dramatically expanding the scope of research tasks it can handle in a single session. + +## Evaluation + +Jan-Nano-128k has been rigorously evaluated on the SimpleQA benchmark using our MCP-based methodology, demonstrating superior performance compared to its predecessor: + +![image/png](https://cdn-uploads.huggingface.co/production/uploads/65713d70f56f9538679e5a56/Bc0ehij86l_NX52OfxeOj.png) + +## Why Jan-Nano-128k? + +Traditional approaches to extending context length, such as YaRN (Yet another RoPE extensioN), often result in performance degradation as context length increases. Jan-Nano-128k breaks this paradigm: + +This fundamental difference makes Jan-Nano-128k ideal for research applications requiring deep document analysis, multi-document synthesis, and complex reasoning over large information sets. + +## 🖥️ How to Run Locally + +![Jan-Nano Demo](replay.gif) + +Jan-Nano-128k is fully supported by [Jan - beta build](https://www.jan.ai/docs/desktop/beta), providing a seamless local AI experience with complete privacy and control. + +For additional tutorials and community guidance, visit our [Discussion Forums](https://huggingface.co/Menlo/Jan-nano-128k/discussions). + +### VLLM Deployment + +```bash +vllm serve Menlo/Jan-nano-128k \ + --host 0.0.0.0 \ + --port 1234 \ + --enable-auto-tool-choice \ + --tool-call-parser hermes \ + --rope-scaling '{"rope_type":"yarn","factor":3.2,"original_max_position_embeddings":40960}' --max-model-len 131072 +``` + +**Note:** The chat template is included in the tokenizer. For troubleshooting, download the [Non-think chat template](https://qwen.readthedocs.io/en/latest/_downloads/c101120b5bebcc2f12ec504fc93a965e/qwen3_nonthinking.jinja). + +### Recommended Sampling Parameters + +```yaml +Temperature: 0.7 +Top-p: 0.8 +Top-k: 20 +Min-p: 0.0 +``` + +## 🤝 Community & Support + +- **Discussions**: [HuggingFace Community](https://huggingface.co/Menlo/Jan-nano-128k/discussions) +- **Issues**: [GitHub Repository](https://github.com/menloresearch/deep-research/issues) +- **Documentation**: [Official Docs](https://menloresearch.github.io/deep-research/) + +## 📄 Citation + +```bibtex +@model{jan-nano-128k, + title={Jan-Nano-128k: Deep Research with Extended Context}, + author={Dao, Alan and Dinh, Bach Vu and Le Thinh}, + year={2024}, + url={https://huggingface.co/Menlo/Jan-nano-128k} +} +``` + +--- + +*Jan-Nano-128k: Empowering deeper research through extended context understanding.* \ No newline at end of file diff --git a/configuration.json b/configuration.json new file mode 100644 index 0000000..bbeeda1 --- /dev/null +++ b/configuration.json @@ -0,0 +1 @@ +{"framework": "pytorch", "task": "text-generation", "allow_remote": true} \ No newline at end of file diff --git a/imatrix_unsloth.dat b/imatrix_unsloth.dat new file mode 100644 index 0000000..91557c7 Binary files /dev/null and b/imatrix_unsloth.dat differ