初始化项目，由ModelHub XC社区提供模型
Model: AbteeXAILab/lumynax-longctx-prolong-512k-instruct Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
15
LICENSE.txt
Normal file
15
LICENSE.txt
Normal file
@@ -0,0 +1,15 @@
|
||||
LumynaX release package — distribution scaffold
|
||||
|
||||
This Hugging Face repository contains the LumynaX release scaffold for the
|
||||
upstream model identified in `release_export_manifest.json`. The model weights
|
||||
themselves remain governed by the upstream licence at `metadata.upstream_repo`
|
||||
and are fetched by the runtime via Hugging Face Hub when you execute
|
||||
`quickstart.py`.
|
||||
|
||||
The scaffold (configs, manifest, quickstart, ollama Modelfile, Space app) is
|
||||
released under the MIT licence by AbteeX AI Labs (Aotearoa New Zealand). Your
|
||||
use of the *upstream weights* is governed by the *upstream licence*; this file
|
||||
does not relicense them.
|
||||
|
||||
Provenance, residency and audit obligations are defined in
|
||||
`release_export_manifest.json` and the model card.
|
||||
80
README.md
Normal file
80
README.md
Normal file
@@ -0,0 +1,80 @@
|
||||
---
|
||||
license: llama3
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
language:
|
||||
- en
|
||||
- mi
|
||||
tags:
|
||||
- 512k
|
||||
- abteex-ai-labs
|
||||
- aotearoa
|
||||
- local-first
|
||||
- long-context
|
||||
- lumynax
|
||||
- new-zealand
|
||||
- prolong
|
||||
- reasoning
|
||||
- sovereign-ai
|
||||
- vllm
|
||||
- vllm-compatible
|
||||
- vllm-candidate
|
||||
- nvidia-nim
|
||||
- nim-compatible
|
||||
- nim-candidate
|
||||
- nvidia-nemo
|
||||
- nem
|
||||
- nvidia-nemo-compatible
|
||||
- nem-compatible
|
||||
- nemo-candidate
|
||||
- legacy
|
||||
- outdated
|
||||
base_model: princeton-nlp/Llama-3-8B-ProLong-512k-Instruct
|
||||
---
|
||||
|
||||
# LumynaX Long-Context ProLong-512K Instruct
|
||||
|
||||
> **Legacy release · Outdated research artifact**
|
||||
|
||||
This model card documents an early LumynaX experiment. It is no longer maintained, is not recommended for production, and does not represent the current capabilities, architecture, or safety standards of AbteeX AI Labs.
|
||||
|
||||
## How infusion works
|
||||
|
||||
**LumynaX Core is the core intelligence model.** It governs the inference path and integrates selected open-source models as specialised execution layers.
|
||||
|
||||
```text
|
||||
Prompt → LumynaX Core → Infused model / MoE experts → LumynaX Core → Response
|
||||
```
|
||||
|
||||
**LumynaX infusion** is the controlled composition of LumynaX Core with a compatible open-source model. Depending on the model family and deployment objective, the integration can operate in two ways:
|
||||
|
||||
- **Routed infusion** — LumynaX Core directs inference through the selected model without modifying its weights.
|
||||
- **MoE infusion** — when required by the architecture, compatible model weights can be composed as specialised experts within a Mixture-of-Experts design.
|
||||
|
||||
In both cases, LumynaX Core remains the primary intelligence and orchestration layer, applying sovereignty controls, context, agentic planning, and inference optimisation around model execution. Infusion does not automatically imply a weight merge; each release manifest records the method used by that pack.
|
||||
|
||||
## This release
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Infused model** | [`princeton-nlp/Llama-3-8B-ProLong-512k-Instruct`](https://huggingface.co/princeton-nlp/Llama-3-8B-ProLong-512k-Instruct) |
|
||||
| **Infusion method** | Routed runtime and identity integration |
|
||||
| **Weight composition** | None — this pack preserves the source-model weights |
|
||||
| **Runtime** | Transformers |
|
||||
| **Release** | v0.1.0 |
|
||||
| **Status** | Outdated and retained for research provenance only |
|
||||
|
||||
This package predates the current LumynaX Core implementation. Its included identity, runtime, or deployment wrappers are historical release components—not the complete modern LumynaX pipeline.
|
||||
|
||||
## Archive access
|
||||
|
||||
The artifacts remain available for reproducibility. Before evaluation, verify `checksums.sha256`, inspect `release_export_manifest.json`, and review `LICENSE.txt`.
|
||||
|
||||
- [Source repository](https://github.com/Aimaghsoodi/lumynax-longctx-prolong-512k-instruct)
|
||||
- [AbteeX AI Labs](https://abteex.com)
|
||||
- [LumynaX](https://lumynax.com)
|
||||
- [Contact](mailto:aimaghsoodi@abteex.com)
|
||||
|
||||
---
|
||||
|
||||
**AbteeX AI Labs · Aotearoa New Zealand**
|
||||
14
UPLOAD_TO_HF.md
Normal file
14
UPLOAD_TO_HF.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# Upload checklist (AbteeX AI Labs)
|
||||
|
||||
This scaffold was generated and pushed by `S:\hf-publish\add_8_more_models.py`.
|
||||
The repo is structured to be cloned whole and verified locally before running.
|
||||
|
||||
```bash
|
||||
hf download AbteeXAILab/lumynax-longctx-prolong-512k-instruct --local-dir lumynax-longctx-prolong-512k-instruct
|
||||
cd lumynax-longctx-prolong-512k-instruct
|
||||
pip install -r requirements.txt
|
||||
python quickstart.py --interactive
|
||||
```
|
||||
|
||||
To regenerate or refresh, re-run the script — `huggingface_hub.HfApi.upload_folder`
|
||||
is idempotent.
|
||||
1
VERSION.txt
Normal file
1
VERSION.txt
Normal file
@@ -0,0 +1 @@
|
||||
v0.1.0
|
||||
7
checksums.sha256
Normal file
7
checksums.sha256
Normal file
@@ -0,0 +1,7 @@
|
||||
601956b52f7e59484c150ec4b1e9b1186adc331bf92c4a7127834ba2ff70a968 LICENSE.txt
|
||||
d0b4f9120ba026c00fa23cb84b4e1620a2e6436592e58155a5151653179572c0 VERSION.txt
|
||||
df889cbc4844a70f8c5239862a5f85ef063eb414aa91b69da4e7182b8eeb52d8 ollama/Modelfile
|
||||
55758fe8918a7f7c1e8606f77c91c4f2350530659dcfe70ec6d3df784c603072 quickstart.py
|
||||
0fb270164c1f5c02d2a04cf81b4c119138bc27dd249d86dc9b37aa3e8b0e6c18 release_export_manifest.json
|
||||
dd873da1e34bee536b535b05ffe41bbeeacaa9d106db9463ae0f4abe2f07dff8 requirements.txt
|
||||
57d25c4197116860215933f974d808edf92165ef3306c2954f9f614a3d7eb94c README.md
|
||||
29
config.json
Normal file
29
config.json
Normal file
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"_name_or_path": "princeton-nlp/Llama-3-8B-ProLong-512k-Instruct",
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 128000,
|
||||
"eos_token_id": 128009,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 14336,
|
||||
"max_position_embeddings": 524288,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 32,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 0,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 128000000.0,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float32",
|
||||
"use_cache": true,
|
||||
"vocab_size": 128256
|
||||
}
|
||||
22
hf_space/README.md
Normal file
22
hf_space/README.md
Normal file
@@ -0,0 +1,22 @@
|
||||
---
|
||||
title: LumynaX Long-Context ProLong-512K Instruct
|
||||
colorFrom: yellow
|
||||
colorTo: gray
|
||||
sdk: gradio
|
||||
sdk_version: 5.50.0
|
||||
python_version: 3.11
|
||||
app_file: app.py
|
||||
pinned: false
|
||||
license: apache-2.0
|
||||
short_description: Scaffold Space for the lumynax-longctx-prolong-512k-instruct LumynaX release.
|
||||
tags:
|
||||
- abteex-ai-labs
|
||||
- lumynax
|
||||
- sovereign-ai
|
||||
- new-zealand
|
||||
---
|
||||
|
||||
# LumynaX Long-Context ProLong-512K Instruct
|
||||
|
||||
Scaffold Space for `AbteeXAILab/lumynax-longctx-prolong-512k-instruct`. The actual model weights are too large to host on a free Space.
|
||||
Clone the package repo and run `quickstart.py` locally for the real inference path.
|
||||
27
hf_space/app.py
Normal file
27
hf_space/app.py
Normal file
@@ -0,0 +1,27 @@
|
||||
import os
|
||||
import gradio as gr
|
||||
|
||||
REPO_ID = "AbteeXAILab/lumynax-longctx-prolong-512k-instruct"
|
||||
UPSTREAM = "princeton-nlp/Llama-3-8B-ProLong-512k-Instruct"
|
||||
TITLE = "LumynaX Long-Context ProLong-512K Instruct"
|
||||
|
||||
THEME_CSS = """
|
||||
:root { --lx-paper:#fffefa; --lx-ink:#0a0a0b; --lx-amber:#e08a2c; }
|
||||
body, .gradio-container { background: var(--lx-paper) !important; color: var(--lx-ink) !important; }
|
||||
h1, h2, h3 { font-family: 'Cormorant Garamond', 'EB Garamond', Georgia, serif; }
|
||||
"""
|
||||
|
||||
def chat_stub(message, history):
|
||||
return (
|
||||
f"This Space is a scaffold for **{TITLE}**. The upstream model is `{UPSTREAM}` "
|
||||
f"and the LumynaX package repo is `{REPO_ID}`. Cloud inference for >100B MoE models "
|
||||
f"is not run inside this free Space — clone the repo and run `quickstart.py` on a "
|
||||
f"capable host. You asked: {message!r}."
|
||||
)
|
||||
|
||||
with gr.Blocks(css=THEME_CSS, title=TITLE) as demo:
|
||||
gr.Markdown(f"# {TITLE}\n*Sovereign intelligence, held in the light.*\n\nLumynaX release scaffold — clone `{REPO_ID}` for the full package.")
|
||||
gr.ChatInterface(chat_stub, examples=["Explain LumynaX in 2 bullets.", "Why local-first AI for Aotearoa?"])
|
||||
|
||||
if __name__ == "__main__":
|
||||
demo.launch()
|
||||
1
hf_space/requirements.txt
Normal file
1
hf_space/requirements.txt
Normal file
@@ -0,0 +1 @@
|
||||
gradio==5.50.0
|
||||
3
model-00001-of-00007.safetensors
Normal file
3
model-00001-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7296ca15d2e559a0ae87e10bf6abd4eefba77fbe28c95c9c97c5ed0113cc21d5
|
||||
size 4886466168
|
||||
3
model-00002-of-00007.safetensors
Normal file
3
model-00002-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1f91a0cad8a8afaf5e97d5f109b788ed75b9809ca2b681796ddd39fcfecbd663
|
||||
size 4832007448
|
||||
3
model-00003-of-00007.safetensors
Normal file
3
model-00003-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:fc87b3bdb543dd7a2ac8609c98bceb2aef96d45ff265d87f10639cd45b729c3f
|
||||
size 4999813112
|
||||
3
model-00004-of-00007.safetensors
Normal file
3
model-00004-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6dd3feb3c8fe3c6276dd5d72498ba8a6656a594fda21880cbd80eac20d2f5ed3
|
||||
size 4999813128
|
||||
3
model-00005-of-00007.safetensors
Normal file
3
model-00005-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6de11fba3fbc1a752c6fbd786aba0fc5fe18e77ffa121dcc551b07015e298314
|
||||
size 4832007496
|
||||
3
model-00006-of-00007.safetensors
Normal file
3
model-00006-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a5e0691cf67509986a8231e4413d32d60defc1e4353b89c73e381a657dd3b3a9
|
||||
size 4999813120
|
||||
3
model-00007-of-00007.safetensors
Normal file
3
model-00007-of-00007.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:54d033cb3d6280e32265bd4326c21a56c5fcf1e851d38b509c4ea421ca702efe
|
||||
size 2571158184
|
||||
298
model.safetensors.index.json
Normal file
298
model.safetensors.index.json
Normal file
@@ -0,0 +1,298 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 32121044992
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00007-of-00007.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00007-of-00007.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00007-of-00007.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00007-of-00007.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00007-of-00007.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
|
||||
"model.norm.weight": "model-00007-of-00007.safetensors"
|
||||
}
|
||||
}
|
||||
9
ollama/Modelfile
Normal file
9
ollama/Modelfile
Normal file
@@ -0,0 +1,9 @@
|
||||
# Upstream HF repo: princeton-nlp/Llama-3-8B-ProLong-512k-Instruct
|
||||
# Use 'ollama pull' on a converted GGUF mirror, or run via transformers (see quickstart.py).
|
||||
FROM hf.co/princeton-nlp/Llama-3-8B-ProLong-512k-Instruct
|
||||
|
||||
PARAMETER temperature 0.4
|
||||
PARAMETER top_p 0.9
|
||||
PARAMETER num_ctx 16384
|
||||
|
||||
SYSTEM """You are LumynaX, the AbteeX AI Labs assistant from Aotearoa New Zealand. Ko te marama te tuapapa. Answer with care; cite uncertainty; refuse unsafe asks."""
|
||||
95
quickstart.py
Normal file
95
quickstart.py
Normal file
@@ -0,0 +1,95 @@
|
||||
"""
|
||||
LumynaX Long-Context ProLong-512K Instruct — LumynaX quickstart.
|
||||
|
||||
This script fetches the upstream model from Hugging Face and runs a short
|
||||
LumynaX-flavoured prompt. Run it on a host that satisfies the resource budget
|
||||
documented in the README (LumynaX Long-Context ProLong-512K Instruct).
|
||||
|
||||
Usage:
|
||||
python quickstart.py # one-shot demo prompt
|
||||
python quickstart.py --interactive # REPL
|
||||
python quickstart.py --gguf # use the GGUF mirror via llama-cpp
|
||||
|
||||
LumynaX package repo: https://huggingface.co/AbteeXAILab/lumynax-longctx-prolong-512k-instruct
|
||||
Upstream weights: https://huggingface.co/princeton-nlp/Llama-3-8B-ProLong-512k-Instruct
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import argparse, os, sys
|
||||
|
||||
LUMYNAX_SYSTEM = (
|
||||
"You are LumynaX, the AbteeX AI Labs assistant from Aotearoa New Zealand. "
|
||||
"Ko te marama te tuapapa - the light is the foundation. "
|
||||
"Answer with care, cite uncertainty, and prefer local-first reasoning. "
|
||||
"Refuse unsafe, unlawful, or sovereignty-violating requests."
|
||||
)
|
||||
DEMO_PROMPT = "Explain in 3 bullets why local-first AI matters for Aotearoa New Zealand."
|
||||
|
||||
def _run_hf(prompt: str, interactive: bool):
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
print("[lumynax] Loading princeton-nlp/Llama-3-8B-ProLong-512k-Instruct. This is a >100B MoE — multi-GPU or accelerate offload recommended.")
|
||||
tok = AutoTokenizer.from_pretrained("princeton-nlp/Llama-3-8B-ProLong-512k-Instruct", trust_remote_code=True)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
"princeton-nlp/Llama-3-8B-ProLong-512k-Instruct", device_map="auto", torch_dtype="auto", trust_remote_code=True
|
||||
)
|
||||
def chat(user):
|
||||
messages = [
|
||||
{"role": "system", "content": LUMYNAX_SYSTEM},
|
||||
{"role": "user", "content": user},
|
||||
]
|
||||
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||
inputs = tok(text, return_tensors="pt").to(model.device)
|
||||
out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.4)
|
||||
return tok.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
|
||||
if interactive:
|
||||
print("[lumynax] interactive mode — empty line exits.")
|
||||
while True:
|
||||
try: q = input("you> ").strip()
|
||||
except EOFError: break
|
||||
if not q: break
|
||||
print("lumynax> " + chat(q))
|
||||
else:
|
||||
print(chat(prompt))
|
||||
|
||||
|
||||
def _run_gguf(prompt: str, interactive: bool):
|
||||
from llama_cpp import Llama
|
||||
mirror = ""
|
||||
if not mirror:
|
||||
print("[lumynax] No community GGUF mirror registered for this build."); sys.exit(2)
|
||||
print(f"[lumynax] Loading GGUF from {mirror}...")
|
||||
llm = Llama.from_pretrained(
|
||||
repo_id=mirror, filename="*Q4_K_M*.gguf",
|
||||
n_ctx=16384,
|
||||
n_gpu_layers=int(os.environ.get("N_GPU_LAYERS", "-1")), verbose=False,
|
||||
)
|
||||
def chat(user):
|
||||
out = llm.create_chat_completion(messages=[
|
||||
{"role": "system", "content": LUMYNAX_SYSTEM},
|
||||
{"role": "user", "content": user},
|
||||
], max_tokens=512, temperature=0.4)
|
||||
return out["choices"][0]["message"]["content"]
|
||||
if interactive:
|
||||
while True:
|
||||
try: q = input("you> ").strip()
|
||||
except EOFError: break
|
||||
if not q: break
|
||||
print("lumynax> " + chat(q))
|
||||
else:
|
||||
print(chat(prompt))
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser()
|
||||
p.add_argument("--interactive", action="store_true")
|
||||
p.add_argument("--prompt", default=DEMO_PROMPT)
|
||||
p.add_argument("--gguf", action="store_true")
|
||||
args = p.parse_args()
|
||||
if args.gguf:
|
||||
_run_gguf(args.prompt, args.interactive)
|
||||
else:
|
||||
_run_hf(args.prompt, args.interactive)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
56
release_export_manifest.json
Normal file
56
release_export_manifest.json
Normal file
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"release_id": "lumynax-longctx-prolong-512k-instruct",
|
||||
"title": "LumynaX Long-Context ProLong-512K Instruct",
|
||||
"repo_id": "AbteeXAILab/lumynax-longctx-prolong-512k-instruct",
|
||||
"version": "v0.1.0",
|
||||
"publisher": "AbteeX AI Labs",
|
||||
"publisher_org": "AbteeXAILab",
|
||||
"jurisdiction": "NZ",
|
||||
"residency": [
|
||||
"NZ",
|
||||
"AU",
|
||||
"global"
|
||||
],
|
||||
"upstream": {
|
||||
"repo_id": "princeton-nlp/Llama-3-8B-ProLong-512k-Instruct",
|
||||
"license": "llama3",
|
||||
"prompt_format": "chatml",
|
||||
"source_note": "Princeton ProLong"
|
||||
},
|
||||
"gguf_mirror": null,
|
||||
"runtime": "transformers",
|
||||
"modalities": [
|
||||
"text"
|
||||
],
|
||||
"context_tokens": 524288,
|
||||
"total_params_b": 8,
|
||||
"active_params_b": null,
|
||||
"quantization": "bf16 safetensors (sharded)",
|
||||
"primary_artifact": "model.safetensors.index.json",
|
||||
"package_state": "weights_mirrored",
|
||||
"validation_status": "scaffold_verified",
|
||||
"sovereignty_tier": 3,
|
||||
"supports_tools": false,
|
||||
"supports_json": true,
|
||||
"audit_hash_chain": "SHA-256 over canonical request + response JSON",
|
||||
"build_date_utc": "2026-05-17T10:35:15Z",
|
||||
"lumynax_manifest_version": 3,
|
||||
"lifecycle": {
|
||||
"status": "legacy",
|
||||
"maintenance": "unmaintained",
|
||||
"production_recommended": false
|
||||
},
|
||||
"lumynax_infusion": {
|
||||
"core_role": "primary_intelligence_model",
|
||||
"method": "routed",
|
||||
"infused_model": "princeton-nlp/Llama-3-8B-ProLong-512k-Instruct",
|
||||
"weight_composition_applied": false,
|
||||
"moe_composition": "not_applied"
|
||||
},
|
||||
"licensing": {
|
||||
"id": "llama3",
|
||||
"license_file": "LICENSE.txt",
|
||||
"source_model_repo": "princeton-nlp/Llama-3-8B-ProLong-512k-Instruct",
|
||||
"weights_subject_to_source_license": true
|
||||
}
|
||||
}
|
||||
8
requirements.txt
Normal file
8
requirements.txt
Normal file
@@ -0,0 +1,8 @@
|
||||
huggingface_hub>=0.27
|
||||
numpy
|
||||
torch>=2.4
|
||||
transformers>=4.50
|
||||
accelerate>=1.0
|
||||
safetensors
|
||||
sentencepiece
|
||||
einops
|
||||
17
special_tokens_map.json
Normal file
17
special_tokens_map.json
Normal file
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<|begin_of_text|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|eot_id|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": "!"
|
||||
}
|
||||
410563
tokenizer.json
Normal file
410563
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
2063
tokenizer_config.json
Normal file
2063
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user