初始化项目，由ModelHub XC社区提供模型
Model: AbteeXAILab/lumynax-infused-qwen3-06b-gguf Source: Original Platform
This commit is contained in:
ModelHub XC
1
.gitattributes
vendored
Normal file
1
.gitattributes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
*.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
11
LICENSE.txt
Normal file
11
LICENSE.txt
Normal file
@@ -0,0 +1,11 @@
|
||||
LumynaX Infused Qwen3 0.6B GGUF
|
||||
|
||||
This release packages a prebuilt third-party GGUF artifact with LumynaX release metadata, quickstarts, Ollama files, and browser demo scaffolding.
|
||||
|
||||
Upstream base model: Qwen/Qwen3-0.6B
|
||||
Source GGUF repo: Qwen/Qwen3-0.6B-GGUF
|
||||
Packaged GGUF file: Qwen3-0.6B-Q8_0.gguf
|
||||
License metadata: apache-2.0
|
||||
Upstream license file: https://huggingface.co/Qwen/Qwen3-0.6B-GGUF
|
||||
|
||||
The model weights remain subject to their upstream license and obligations. LumynaX packaging metadata, scripts, and documentation are provided by AbteeX AI Labs.
|
||||
3
Qwen3-0.6B-Q8_0.gguf
Normal file
3
Qwen3-0.6B-Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9465e63a22add5354d9bb4b99e90117043c7124007664907259bd16d043bb031
|
||||
size 639446688
|
||||
236
README.md
Normal file
236
README.md
Normal file
@@ -0,0 +1,236 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
library_name: llama.cpp
|
||||
pipeline_tag: text-generation
|
||||
language:
|
||||
- en
|
||||
- mi
|
||||
tags:
|
||||
- abteex-ai-labs
|
||||
- aotearoa
|
||||
- general
|
||||
- gguf
|
||||
- local-first
|
||||
- lumynax
|
||||
- new-zealand
|
||||
- qwen
|
||||
- sovereign-ai
|
||||
- text
|
||||
- vllm
|
||||
- vllm-compatible
|
||||
- vllm-experimental
|
||||
- nvidia-nim
|
||||
- nim-compatible
|
||||
- nim-candidate
|
||||
- nvidia-nemo
|
||||
- nem
|
||||
- nvidia-nemo-pathway
|
||||
- nem-pathway
|
||||
- nem-convert-required
|
||||
---
|
||||
|
||||
# LumynaX Infused Qwen3 0.6B GGUF
|
||||
|
||||
<!-- lumynax-public-release-card:v4 -->
|
||||
|
||||
<p align="center">
|
||||
<img src="docs/lumynax-release-overview.svg" alt="LumynaX Infused Qwen3 0.6B GGUF release overview" width="100%" />
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<strong>LumynaX model-infusion release by AbteeX AI Labs.</strong><br/>
|
||||
Public, non-gated package with runnable local instructions, provenance metadata, checksums, and a release manifest.
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="#quickstart">Quickstart</a> |
|
||||
<a href="#model-profile">Model profile</a> |
|
||||
<a href="#runtime-files">Runtime files</a> |
|
||||
<a href="#provenance-and-license">Provenance</a> |
|
||||
<a href="#validation-status">Validation</a> |
|
||||
<a href="#limitations-and-responsible-use">Limitations</a>
|
||||
</p>
|
||||
|
||||
     
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This repository is a complete LumynaX release package for `AbteeXAILab/lumynax-infused-qwen3-06b-gguf`. It is intended to be downloaded as a whole repo, not as a single loose weight file: the model artifact, `quickstart.py`, `requirements.txt`, `release_export_manifest.json`, `checksums.sha256`, license notice, and optional Ollama or Space files are part of the same release contract.
|
||||
|
||||
LumynaX-infused means the upstream artifact is presented through the LumynaX release layer: local-first runtime scaffolding, LumynaX assistant identity, inference-chain metadata, public documentation, integrity files, and Aotearoa New Zealand-oriented workflow positioning. The release manifest records this as a LumynaX packaging and inference-chain layer around the listed upstream artifact; it does not claim a private LumynaX weight merge.
|
||||
|
||||
## AbteeX LumynaX Public Surface
|
||||
|
||||
This card follows the AbteeX/LumynaX public-facing system used across the release family: warm paper background visuals, black editorial typography, amber proof markers, compact evidence tables, and plain-language runtime instructions. The goal is not decoration; it is operational clarity. A downloader should immediately understand what the package is, what files belong together, what runtime path is expected, what provenance is available, and what limits still apply.
|
||||
|
||||
## Sovereignty And Run Contract
|
||||
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| Public surface | AbteeX/LumynaX light editorial system: warm paper, black ink, amber status markers, and evidence-first tables. |
|
||||
| Sovereign intent | Package is documented for local-first use, explicit provenance, and controlled deployment near governed data. |
|
||||
| Runtime residency | `llama_cpp` runtime can be deployed by the user in their own approved environment. |
|
||||
| Model artifact | `Qwen3-0.6B-Q8_0.gguf` must stay with manifest, checksums, quickstart, requirements, and license files. |
|
||||
| Modalities | `text` |
|
||||
| License discipline | `apache-2.0` metadata is surfaced so downstream users can check redistribution and usage terms. |
|
||||
| Audit expectation | Record repo id, artifact checksum, runtime command, prompt template, operator, and deployment environment for production use. |
|
||||
| Router readiness | Compatible with the LumynaX MaramaRoute registry pattern for sovereign model selection and fallback planning. |
|
||||
| Local serving | Preferred first path is llama.cpp or llama-cpp-python with checksum verification before launch. |
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
hf download AbteeXAILab/lumynax-infused-qwen3-06b-gguf --local-dir lumynax-infused-qwen3-06b-gguf
|
||||
cd lumynax-infused-qwen3-06b-gguf
|
||||
pip install -r requirements.txt
|
||||
python quickstart.py --interactive
|
||||
```
|
||||
|
||||
Direct llama.cpp smoke command:
|
||||
|
||||
```bash
|
||||
llama-cli -m "Qwen3-0.6B-Q8_0.gguf" -p "Who are you? Answer as LumynaX in two sentences." -n 160
|
||||
```
|
||||
|
||||
Ollama path:
|
||||
|
||||
```bash
|
||||
ollama create lumynax-infused-qwen3-06b-gguf -f ollama/Modelfile
|
||||
ollama run lumynax-infused-qwen3-06b-gguf
|
||||
```
|
||||
|
||||
## Model Profile
|
||||
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| Release | `LumynaX Infused Qwen3 0.6B GGUF` |
|
||||
| Repository | `AbteeXAILab/lumynax-infused-qwen3-06b-gguf` |
|
||||
| Mode | `Local-first text generation package` |
|
||||
| Runtime | `llama_cpp` |
|
||||
| Prompt format | `chatml` |
|
||||
| Modalities | `text` |
|
||||
| Primary artifact | `Qwen3-0.6B-Q8_0.gguf` |
|
||||
| Detected weight size | `609.82 MB` |
|
||||
| Package state | `prebuilt_gguf_release` |
|
||||
| Delivery | `standalone_prebuilt_gguf_release` |
|
||||
| Upstream/base | `Qwen/Qwen3-0.6B` |
|
||||
| Upstream kind | `official_base_weights` |
|
||||
| Source GGUF | `Qwen/Qwen3-0.6B-GGUF` |
|
||||
| Quantization | `Q8_0` |
|
||||
| License metadata | `apache-2.0` |
|
||||
| Refreshed | `2026-05-11` |
|
||||
| Upstream license link | [license](https://huggingface.co/Qwen/Qwen3-0.6B-GGUF) |
|
||||
|
||||
## Runtime Path
|
||||
|
||||
<p align="center">
|
||||
<img src="docs/lumynax-runtime-flow.svg" alt="LumynaX Infused Qwen3 0.6B GGUF runtime flow" width="100%" />
|
||||
</p>
|
||||
|
||||
## Capability Profile
|
||||
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| Primary fit | Use this for local chat, drafting, summarization, governance notes, and repeatable offline-friendly inference. |
|
||||
| Operational style | Local-first package with explicit files, checksums, and reproducible quickstarts. |
|
||||
| Identity behavior | The assistant should identify as LumynaX while remaining clear about upstream provenance. |
|
||||
|
||||
## Runtime Files
|
||||
|
||||
| Component | Status | Path |
|
||||
| --- | --- | --- |
|
||||
| README.md | `present` | `README.md` |
|
||||
| Quickstart | `present` | `quickstart.py` |
|
||||
| Requirements | `present` | `requirements.txt` |
|
||||
| Manifest | `present` | `release_export_manifest.json` |
|
||||
| Checksums | `present` | `checksums.sha256` |
|
||||
| License | `present` | `LICENSE.txt` |
|
||||
| Ollama | `present` | `ollama/Modelfile` |
|
||||
| Space scaffold | `present` | `hf_space/app.py` |
|
||||
| Overview visual | `present` | `docs/lumynax-release-overview.svg` |
|
||||
| Runtime visual | `present` | `docs/lumynax-runtime-flow.svg` |
|
||||
|
||||
## Model Artifacts
|
||||
|
||||
| Artifact | Size |
|
||||
| --- | ---: |
|
||||
| `Qwen3-0.6B-Q8_0.gguf` | 609.82 MB |
|
||||
|
||||
## Prompting Contract
|
||||
|
||||
The preferred first prompt is an identity and provenance check:
|
||||
|
||||
```text
|
||||
Who are you? What files do I need to keep together to run this package locally?
|
||||
```
|
||||
|
||||
Expected behavior: the assistant should identify as LumynaX, explain that this is a LumynaX model-infusion package, and keep upstream provenance visible. The default package system prompt is:
|
||||
|
||||
```text
|
||||
You are LumynaX operating from the LumynaX Infused Qwen3 0.6B GGUF package identity. Be helpful, clear, and honest about provenance.
|
||||
```
|
||||
|
||||
## Validation Status
|
||||
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| Runtime audit | `pass` |
|
||||
| Public access audit | `public and non-gated` |
|
||||
| Anonymous metadata access | `True` |
|
||||
| Anonymous file listing | `True` |
|
||||
| Quickstart syntax | `pass` |
|
||||
| Manifest references | `pass` |
|
||||
| Checksum references | `pass` |
|
||||
|
||||
The audit confirms public access, required release files, manifest references, checksum references, weight artifact presence, and quickstart syntax. It does not guarantee that every laptop has enough RAM or VRAM for the largest packages.
|
||||
|
||||
## Integrity Checks
|
||||
|
||||
After download, compare the model artifact against `checksums.sha256`.
|
||||
|
||||
```bash
|
||||
sha256sum "Qwen3-0.6B-Q8_0.gguf"
|
||||
cat checksums.sha256
|
||||
```
|
||||
|
||||
On Windows PowerShell:
|
||||
|
||||
```powershell
|
||||
Get-FileHash -Algorithm SHA256 "Qwen3-0.6B-Q8_0.gguf"
|
||||
Get-Content checksums.sha256
|
||||
```
|
||||
|
||||
## Provenance And License
|
||||
|
||||
- Publisher: AbteeX AI Labs.
|
||||
- Family: LumynaX model and inference-chain release family.
|
||||
- Upstream/base: `Qwen/Qwen3-0.6B`.
|
||||
- Source GGUF: `Qwen/Qwen3-0.6B-GGUF`.
|
||||
- License metadata: `apache-2.0`.
|
||||
- License link: https://huggingface.co/Qwen/Qwen3-0.6B-GGUF.
|
||||
|
||||
Respect the upstream model license and keep attribution files with redistributed copies. Do not present this package as privately trained or weight-merged unless the release manifest explicitly says that weight adaptation was applied.
|
||||
|
||||
## Limitations And Responsible Use
|
||||
|
||||
- Outputs can be incorrect, incomplete, or biased; validate important answers before use.
|
||||
- Larger GGUF, MoE, multimodal, and frontier packages may require substantial RAM, VRAM, disk space, and recent runtime builds.
|
||||
- For high-impact decisions, use human review and domain-specific evaluation.
|
||||
- For sensitive data, prefer local execution and keep operational logs under your own governance policy.
|
||||
- This card documents package readiness and access; it is not a benchmark claim.
|
||||
|
||||
## Automation Notes
|
||||
|
||||
Automation should read these files before launching:
|
||||
|
||||
- `release_export_manifest.json`
|
||||
- `checksums.sha256`
|
||||
- `quickstart.py`
|
||||
- `requirements.txt`
|
||||
- `ollama/Modelfile` when present
|
||||
|
||||
## Related LumynaX Demo
|
||||
|
||||
Try the public browser demo:
|
||||
|
||||
- https://huggingface.co/spaces/AbteeXAILab/lumynax-live-demo
|
||||
18
UPLOAD_TO_HF.md
Normal file
18
UPLOAD_TO_HF.md
Normal file
@@ -0,0 +1,18 @@
|
||||
# Upload LumynaX Infused Qwen3 0.6B GGUF
|
||||
|
||||
Upload the whole release folder, not just the GGUF file.
|
||||
|
||||
Private first:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\publish_huggingface_release.ps1 -ReleaseDir .\data\releases\lumynax-infused-qwen3-06b-gguf-v1 -RepoId AbteeXAILab/lumynax-infused-qwen3-06b-gguf -Private
|
||||
```
|
||||
|
||||
Public release:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\publish_huggingface_release.ps1 -ReleaseDir .\data\releases\lumynax-infused-qwen3-06b-gguf-v1 -RepoId AbteeXAILab/lumynax-infused-qwen3-06b-gguf -Public
|
||||
```
|
||||
|
||||
The folder includes `README.md`, `LICENSE.txt`, `.gitattributes`, `checksums.sha256`,
|
||||
`release_export_manifest.json`, the GGUF file, Ollama files, and the Space bundle.
|
||||
1
VERSION.txt
Normal file
1
VERSION.txt
Normal file
@@ -0,0 +1 @@
|
||||
v1
|
||||
22
artifacts/release_training_summary.json
Normal file
22
artifacts/release_training_summary.json
Normal file
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"generated_at": "2026-05-10T12:57:47.826709+00:00",
|
||||
"gguf_only_release": true,
|
||||
"license_id": "apache-2.0",
|
||||
"license_link": "https://huggingface.co/Qwen/Qwen3-0.6B-GGUF",
|
||||
"license_name": null,
|
||||
"lumynax_weight_adaptation_applied": false,
|
||||
"model_title": "LumynaX Infused Qwen3 0.6B GGUF",
|
||||
"package_state": "prebuilt_gguf_release",
|
||||
"packaged_gguf_filename": "Qwen3-0.6B-Q8_0.gguf",
|
||||
"packaged_mmproj_filename": null,
|
||||
"prompt_format": "chatml",
|
||||
"quantization": "Q8_0",
|
||||
"source_gguf_filename": "Qwen3-0.6B-Q8_0.gguf",
|
||||
"source_gguf_repo_id": "Qwen/Qwen3-0.6B-GGUF",
|
||||
"source_mmproj_filename": null,
|
||||
"summary": "This release packages `Qwen3-0.6B-Q8_0.gguf` as a final GGUF artifact for local llama.cpp-family inference and Hugging Face model-repo upload.",
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
],
|
||||
"upstream_base_model_id": "Qwen/Qwen3-0.6B"
|
||||
}
|
||||
25
artifacts/smoke_llama_cpp.json
Normal file
25
artifacts/smoke_llama_cpp.json
Normal file
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"command": [
|
||||
"C:\\Users\\ijadimaa\\AppData\\Local\\Programs\\Python\\Python311\\python.exe",
|
||||
"C:\\Users\\ijadimaa\\AppData\\Local\\tinyluminax\\batch-gguf-releases\\lumynax-infused-qwen3-06b-gguf-v1\\quickstart.py",
|
||||
"--llama-cli",
|
||||
"\\\\waikato\\users\\Hamilton\\GtoLdtop\\ijadimaa\\Desktop\\Startup\\TinyLuminaX\\tools\\llama.cpp-prebuilt\\llama-cli.exe",
|
||||
"--prompt",
|
||||
"Reply with exactly: OK",
|
||||
"--max-new-tokens",
|
||||
"16",
|
||||
"--ctx-size",
|
||||
"256",
|
||||
"--temperature",
|
||||
"0",
|
||||
"--threads",
|
||||
"6"
|
||||
],
|
||||
"elapsed_seconds": 6.61,
|
||||
"generated_at": "2026-05-10T12:57:57.334060+00:00",
|
||||
"returncode": 0,
|
||||
"status": "pass",
|
||||
"stderr_tail": "",
|
||||
"stdout_tail": "Loading model... \n\n\n\u2584\u2584 \u2584\u2584\n\u2588\u2588 \u2588\u2588\n\u2588\u2588 \u2588\u2588 \u2580\u2580\u2588\u2584 \u2588\u2588\u2588\u2584\u2588\u2588\u2588\u2584 \u2580\u2580\u2588\u2584 \u2584\u2588\u2588\u2588\u2588 \u2588\u2588\u2588\u2588\u2584 \u2588\u2588\u2588\u2588\u2584\n\u2588\u2588 \u2588\u2588 \u2584\u2588\u2580\u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588 \u2584\u2588\u2580\u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588\n\u2588\u2588 \u2588\u2588 \u2580\u2588\u2584\u2588\u2588 \u2588\u2588 \u2588\u2588 \u2588\u2588 \u2580\u2588\u2584\u2588\u2588 \u2588\u2588 \u2580\u2588\u2588\u2588\u2588 \u2588\u2588\u2588\u2588\u2580 \u2588\u2588\u2588\u2588\u2580\n \u2588\u2588 \u2588\u2588\n \u2580\u2580 \u2580\u2580\n\nbuild : b8840-9e5647aff\nmodel : Qwen3-0.6B-Q8_0.gguf\nmodalities : text\nusing custom system prompt\n\navailable commands:\n /exit or Ctrl+C stop or exit\n /regen regenerate the last response\n /clear clear the chat history\n /read <file> add a text file\n /glob <pattern> add text files using globbing pattern\n\n\n> Reply with exactly: OK\n\nOK\n\n[ Prompt: 131.4 t/s | Generation: 96.0 t/s ]\n\nExiting...\n",
|
||||
"timeout_seconds": 240
|
||||
}
|
||||
20
checksums.sha256
Normal file
20
checksums.sha256
Normal file
@@ -0,0 +1,20 @@
|
||||
fdeb5cd638f262467f8cfa3f97d14d4071076dfbee5907ca4e3af7e50a3bd08b .gitattributes
|
||||
56e6034e2065b8058346c1637f800d59d5bb785c0b28490b933c0fc6e22eb473 artifacts/release_training_summary.json
|
||||
fc191e6a3f414d550d20da75ba8959b69101ce55b706dec172a6d89e2bc3f7e9 artifacts/smoke_llama_cpp.json
|
||||
e17829bffd870616cb9ad6e90d76c3847e8363709b9beeb7bebe313325fa93a5 hf_space/app.py
|
||||
2a48a14e285b35005da6884f401c60f9c71de0629afc4ee3d17ee781b37e05a3 hf_space/README.md
|
||||
3e78abed8cdc940c6bf1c763d81d829ec4dfe2f8b3897e5051989ef19169eaeb hf_space/requirements.txt
|
||||
1c3dd0b7f4d85a710222b3940689146e522712f1df17f9f93afade3cd7e761d0 LICENSE.txt
|
||||
f4182f6e85c46fd47ea25e841709b4be255d73d65ba1ecdea869e9e8c8a75ffe merged_model/PACKAGE_STATE.txt
|
||||
02a15eb428e881cf6793bb94f5c62edb36bebf8fc8e8a3efcef1220fc46e1121 ollama/create_ollama_model.ps1
|
||||
183a19553692b882b6dc157fa87210f901ff4bcfe27f26f9601c68874423629e ollama/Modelfile
|
||||
4981d8e2c3259dd57496cba29f393a34c7d2ea0da1542c89d0ec5784f762d1ef quickstart.py
|
||||
9465e63a22add5354d9bb4b99e90117043c7124007664907259bd16d043bb031 Qwen3-0.6B-Q8_0.gguf
|
||||
188a88f059a160eeaeb125d6def83e254da4e59ba5e9cee9f1ea0f68175de55a README.md
|
||||
a4deeb9b0a29583f110ae93c09ff7af8539cdfb564cfe9553f8b90a55ae8840e release_export_manifest.json
|
||||
2a7ca962dd79646b8470b45ec926ade0a4eb01ebdd93452e6990d8997666e378 requirements.txt
|
||||
4dd459a35d02a02265e42a4f80d26b85888877594089995f15fd947c51f63c35 UPLOAD_TO_HF.md
|
||||
2dfede0e6610c473959c963b292fcec325452acba33fd1bba21110e04933df53 VERSION.txt
|
||||
587e889a7845e98ab95147387bc1c6a471c4befefe17ece2360a5fd6d77f661f docs/lumynax-release-map.svg
|
||||
0738b68b5d51d78b9ce359fa15abbc5dd9842bb36c71943e1972e95d282df786 docs/lumynax-release-overview.svg
|
||||
5a25cd06592ce6fd4d89e4d65724d5af987ccdaf562b67d3eaea1fe7fb38636a docs/lumynax-runtime-flow.svg
|
||||
12
docs/lumynax-capability.svg
Normal file
12
docs/lumynax-capability.svg
Normal file
@@ -0,0 +1,12 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 260" role="img" aria-label="LumynaX capability profile">
|
||||
<rect width="900" height="260" fill="#fffefa"/>
|
||||
<rect x="0" y="0" width="900" height="3" fill="#0a0a0b"/>
|
||||
<text x="64" y="36" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" letter-spacing="0.18em" fill="#9a5416">CAPABILITY PROFILE</text>
|
||||
<text x="64" y="58" font-family="Georgia, Cambria, serif" font-size="18" font-weight="500" fill="#0a0a0b">Where this model spends its weight.</text>
|
||||
<g transform="translate(64,70)"><text x="0" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" letter-spacing="0.12em" fill="#9a5416">QUALITY</text><rect x="160" y="2" width="600" height="16" rx="8" ry="8" fill="#f6f0e8" stroke="rgba(10,10,11,0.12)"/><rect x="160" y="2" width="120" height="16" rx="8" ry="8" fill="#e08a2c"/><text x="772" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" fill="#0a0a0b">1/5</text></g>
|
||||
<g transform="translate(64,98)"><text x="0" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" letter-spacing="0.12em" fill="#9a5416">LIGHTWEIGHT</text><rect x="160" y="2" width="600" height="16" rx="8" ry="8" fill="#f6f0e8" stroke="rgba(10,10,11,0.12)"/><rect x="160" y="2" width="480" height="16" rx="8" ry="8" fill="#e08a2c"/><text x="772" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" fill="#0a0a0b">4/5</text></g>
|
||||
<g transform="translate(64,126)"><text x="0" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" letter-spacing="0.12em" fill="#9a5416">SOVEREIGNTY</text><rect x="160" y="2" width="600" height="16" rx="8" ry="8" fill="#f6f0e8" stroke="rgba(10,10,11,0.12)"/><rect x="160" y="2" width="360" height="16" rx="8" ry="8" fill="#e08a2c"/><text x="772" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" fill="#0a0a0b">3/5</text></g>
|
||||
<g transform="translate(64,154)"><text x="0" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" letter-spacing="0.12em" fill="#9a5416">TOOLS</text><rect x="160" y="2" width="600" height="16" rx="8" ry="8" fill="#f6f0e8" stroke="rgba(10,10,11,0.12)"/><rect x="160" y="2" width="120" height="16" rx="8" ry="8" fill="#e08a2c"/><text x="772" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" fill="#0a0a0b">1/5</text></g>
|
||||
<g transform="translate(64,182)"><text x="0" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" letter-spacing="0.12em" fill="#9a5416">JSON MODE</text><rect x="160" y="2" width="600" height="16" rx="8" ry="8" fill="#f6f0e8" stroke="rgba(10,10,11,0.12)"/><rect x="160" y="2" width="600" height="16" rx="8" ry="8" fill="#e08a2c"/><text x="772" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" fill="#0a0a0b">5/5</text></g>
|
||||
<g transform="translate(64,210)"><text x="0" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" letter-spacing="0.12em" fill="#9a5416">CONTEXT</text><rect x="160" y="2" width="600" height="16" rx="8" ry="8" fill="#f6f0e8" stroke="rgba(10,10,11,0.12)"/><rect x="160" y="2" width="600" height="16" rx="8" ry="8" fill="#e08a2c"/><text x="772" y="14" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="700" fill="#0a0a0b">5/5</text></g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.8 KiB |
24
docs/lumynax-overview.svg
Normal file
24
docs/lumynax-overview.svg
Normal file
@@ -0,0 +1,24 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1280 340" role="img" aria-label="LumynaX Infused Qwen3 0.6B GGUF release banner">
|
||||
<defs>
|
||||
<linearGradient id="paperGrad" x1="0" y1="0" x2="0" y2="1">
|
||||
<stop offset="0%" stop-color="#fffefa"/>
|
||||
<stop offset="100%" stop-color="#f6f0e8"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
<rect width="1280" height="340" fill="url(#paperGrad)"/>
|
||||
<rect x="860" y="0" width="420" height="4" fill="#e08a2c"/>
|
||||
<rect x="0" y="336" width="1280" height="4" fill="#0a0a0b"/>
|
||||
<text x="64" y="56" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="13" font-weight="700" letter-spacing="0.22em" fill="#9a5416">ABTEEX AI LABS · AOTEAROA NEW ZEALAND</text>
|
||||
<text x="64" y="78" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" letter-spacing="0.16em" fill="#726b62">LUMYNAX RELEASE · CARD V6</text>
|
||||
<text x="64" y="170" font-family="Georgia, Cambria, "Times New Roman", serif" font-size="56" font-weight="500" fill="#0a0a0b">LumynaX Infused Qwen3 0.6B GGUF</text>
|
||||
<line x1="64" y1="196" x2="220" y2="196" stroke="#e08a2c" stroke-width="3"/>
|
||||
<text x="64" y="226" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="13" fill="#726b62">AbteeXAILab/lumynax-infused-qwen3-06b-gguf</text>
|
||||
<g transform="translate(64,262)"><rect width="110" height="34" rx="17" ry="17" fill="#fffefa" stroke="rgba(10,10,11,0.12)"/><text x="14" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" font-weight="700" letter-spacing="0.14em" fill="#9a5416">FAMILY</text><text x="63" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="600" fill="#0a0a0b">QWEN</text></g>
|
||||
<g transform="translate(184,262)"><rect width="110" height="34" rx="17" ry="17" fill="#fffefa" stroke="rgba(10,10,11,0.12)"/><text x="14" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" font-weight="700" letter-spacing="0.14em" fill="#9a5416">PARAMS</text><text x="63" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="600" fill="#0a0a0b">0.6B</text></g>
|
||||
<g transform="translate(304,262)"><rect width="142" height="34" rx="17" ry="17" fill="#fffefa" stroke="rgba(10,10,11,0.12)"/><text x="14" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" font-weight="700" letter-spacing="0.14em" fill="#9a5416">RUNTIME</text><text x="70" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="600" fill="#0a0a0b">llama_cpp</text></g>
|
||||
<g transform="translate(456,262)"><rect width="110" height="34" rx="17" ry="17" fill="#fffefa" stroke="rgba(10,10,11,0.12)"/><text x="14" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" font-weight="700" letter-spacing="0.14em" fill="#9a5416">MODES</text><text x="56" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="600" fill="#0a0a0b">TEXT</text></g>
|
||||
<g transform="translate(576,262)"><rect width="110" height="34" rx="17" ry="17" fill="#fffefa" stroke="rgba(10,10,11,0.12)"/><text x="14" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" font-weight="700" letter-spacing="0.14em" fill="#9a5416">QUANT</text><text x="56" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="600" fill="#0a0a0b">Q8_0</text></g>
|
||||
<g transform="translate(696,262)"><rect width="149" height="34" rx="17" ry="17" fill="#fffefa" stroke="rgba(10,10,11,0.12)"/><text x="14" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" font-weight="700" letter-spacing="0.14em" fill="#9a5416">LICENSE</text><text x="70" y="22" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="11" font-weight="600" fill="#0a0a0b">apache-2.0</text></g>
|
||||
<text x="1216" y="56" text-anchor="end" font-family="Georgia, Cambria, serif" font-size="18" font-style="italic" fill="#726b62">held in the light</text>
|
||||
<text x="1216" y="80" text-anchor="end" font-family="ui-monospace, SFMono-Regular, Menlo, Consolas, monospace" font-size="10" letter-spacing="0.18em" fill="#9a5416">KO TE MARAMA TE TUAPAPA</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 4.4 KiB |
33
docs/lumynax-release-map.svg
Normal file
33
docs/lumynax-release-map.svg
Normal file
@@ -0,0 +1,33 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1180" height="470" viewBox="0 0 1180 470" role="img" aria-labelledby="title desc">
|
||||
<title id="title">LumynaX Infused Qwen3 0.6B GGUF LumynaX release map</title>
|
||||
<desc id="desc">Visual release map for AbteeXAILab/lumynax-infused-qwen3-06b-gguf: upstream artifact, LumynaX release layer, packaged files, and runtime path.</desc>
|
||||
<style>
|
||||
.bg{fill:#0b1220}.panel{fill:#111827;stroke:#38bdf8;stroke-width:2}.panel2{fill:#10231d;stroke:#34d399;stroke-width:2}.title{fill:#f8fafc;font:700 28px Arial}.sub{fill:#cbd5e1;font:15px Arial}.label{fill:#93c5fd;font:700 16px Arial}.body{fill:#e2e8f0;font:14px Arial}.chip{fill:#172554;stroke:#60a5fa;stroke-width:1}.chiptext{fill:#dbeafe;font:13px Arial}.arrow{stroke:#94a3b8;stroke-width:3;marker-end:url(#arrowhead)}
|
||||
</style>
|
||||
<defs><marker id="arrowhead" markerWidth="10" markerHeight="7" refX="9" refY="3.5" orient="auto"><polygon points="0 0, 10 3.5, 0 7" fill="#94a3b8"/></marker></defs>
|
||||
<rect class="bg" x="0" y="0" width="1180" height="470" rx="20"/>
|
||||
<text class="title" x="45" y="52">LumynaX Infused Qwen3 0.6B GGUF</text>
|
||||
<text class="sub" x="45" y="80">AbteeXAILab/lumynax-infused-qwen3-06b-gguf</text>
|
||||
<rect class="panel" x="45" y="165" width="245" height="120" rx="14"/>
|
||||
<text class="label" x="63" y="197">Upstream</text>
|
||||
<text class="body" x="63" y="227">Qwen/Qwen3-0.6B</text>
|
||||
<line class="arrow" x1="298" y1="225.0" x2="317" y2="225.0"/>
|
||||
<rect class="panel2" x="325" y="165" width="245" height="120" rx="14"/>
|
||||
<text class="label" x="343" y="197">LumynaX layer</text>
|
||||
<text class="body" x="343" y="227">Identity, inference chaining, docs</text>
|
||||
<line class="arrow" x1="578" y1="225.0" x2="597" y2="225.0"/>
|
||||
<rect class="panel" x="605" y="165" width="245" height="120" rx="14"/>
|
||||
<text class="label" x="623" y="197">Package</text>
|
||||
<text class="body" x="623" y="227">Qwen3-0.6B-Q8_0.gguf</text>
|
||||
<line class="arrow" x1="858" y1="225.0" x2="877" y2="225.0"/>
|
||||
<rect class="panel" x="885" y="165" width="245" height="120" rx="14"/>
|
||||
<text class="label" x="903" y="197">Runtime</text>
|
||||
<text class="body" x="903" y="227">llama_cpp</text>
|
||||
<rect class="chip" x="45" y="335" width="190" height="38" rx="19"/>
|
||||
<text class="chiptext" x="61" y="359">modalities: text</text>
|
||||
<rect class="chip" x="253" y="335" width="190" height="38" rx="19"/>
|
||||
<text class="chiptext" x="269" y="359">license: apache-2.0</text>
|
||||
<rect class="chip" x="461" y="335" width="252" height="38" rx="19"/>
|
||||
<text class="chiptext" x="477" y="359">state: prebuilt_gguf_release</text>
|
||||
<text class="sub" x="45" y="425">Download the full repo: README, runtime files, manifest, checksums, and model artifacts stay together.</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.6 KiB |
51
docs/lumynax-release-overview.svg
Normal file
51
docs/lumynax-release-overview.svg
Normal file
@@ -0,0 +1,51 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1280" height="560" viewBox="0 0 1280 560" role="img" aria-labelledby="title desc">
|
||||
<title id="title">LumynaX Infused Qwen3 0.6B GGUF professional LumynaX release overview</title>
|
||||
<desc id="desc">Professional release overview for AbteeXAILab/lumynax-infused-qwen3-06b-gguf showing provenance, LumynaX infusion layer, package artifact, and runtime path.</desc>
|
||||
<defs>
|
||||
<filter id="shadow" x="-12%" y="-12%" width="124%" height="124%"><feDropShadow dx="0" dy="10" stdDeviation="12" flood-color="#0a0a0b" flood-opacity="0.08"/></filter>
|
||||
<marker id="arrowhead" markerWidth="12" markerHeight="8" refX="10" refY="4" orient="auto"><polygon points="0 0, 12 4, 0 8" fill="#e08a2c"/></marker>
|
||||
</defs>
|
||||
<style>
|
||||
.title{fill:#0a0a0b;font:500 38px Georgia,serif}.sub{fill:#726b62;font:16px Aptos,Segoe UI,Arial}.eyebrow{fill:#9a5416;font:700 13px ui-monospace,Consolas,monospace;letter-spacing:2px}.rule{stroke:#0a0a0b;stroke-opacity:.12}.accent{stroke:#e08a2c;stroke-width:4}.card{fill:#ffffff;stroke:#0a0a0b;stroke-opacity:.12;stroke-width:1.2;filter:url(#shadow)}.num{fill:#ffffff;font:700 17px Aptos,Segoe UI,Arial}.numBg{fill:#0a0a0b}.label{fill:#0a0a0b;font:700 18px Aptos,Segoe UI,Arial}.body{fill:#5f574e;font:14px Aptos,Segoe UI,Arial}.chip{fill:#f6f0e8;stroke:#0a0a0b;stroke-opacity:.12}.chiptext{fill:#0a0a0b;font:700 12px ui-monospace,Consolas,monospace;letter-spacing:.7px}.line{stroke:#e08a2c;stroke-width:3;marker-end:url(#arrowhead)}
|
||||
</style>
|
||||
<rect width="1280" height="560" rx="28" fill="#fffefa"/>
|
||||
<line class="accent" x1="852" y1="34" x2="1228" y2="34"/>
|
||||
<line class="rule" x1="52" y1="160" x2="1228" y2="160"/>
|
||||
<text class="eyebrow" x="52" y="58">ABTEEX AI LABS - LUMYNAX MODEL INFUSION RELEASE</text>
|
||||
<text class="title" x="52" y="102">LumynaX Infused Qwen3 0.6B GGUF</text>
|
||||
<text class="sub" x="52" y="134">AbteeXAILab/lumynax-infused-qwen3-06b-gguf</text>
|
||||
<rect class="card" x="52" y="205" width="270" height="155" rx="18"/>
|
||||
<circle class="numBg" cx="86" cy="241" r="18"/>
|
||||
<text class="num" x="80" y="247">1</text>
|
||||
<text class="label" x="114" y="247">Upstream</text>
|
||||
<text class="body" x="78" y="287">Qwen/Qwen3-0.6B</text>
|
||||
<line class="line" x1="331" y1="282.5" x2="344" y2="282.5"/>
|
||||
<rect class="card" x="356" y="205" width="270" height="155" rx="18"/>
|
||||
<circle class="numBg" cx="390" cy="241" r="18"/>
|
||||
<text class="num" x="384" y="247">2</text>
|
||||
<text class="label" x="418" y="247">LumynaX Infusion</text>
|
||||
<text class="body" x="382" y="287">Identity, runtime scaffold,</text>
|
||||
<text class="body" x="382" y="308">provenance, checksums</text>
|
||||
<line class="line" x1="635" y1="282.5" x2="648" y2="282.5"/>
|
||||
<rect class="card" x="660" y="205" width="270" height="155" rx="18"/>
|
||||
<circle class="numBg" cx="694" cy="241" r="18"/>
|
||||
<text class="num" x="688" y="247">3</text>
|
||||
<text class="label" x="722" y="247">Release Package</text>
|
||||
<text class="body" x="686" y="287">Qwen3-0.6B-Q8_0.gguf</text>
|
||||
<line class="line" x1="939" y1="282.5" x2="952" y2="282.5"/>
|
||||
<rect class="card" x="964" y="205" width="270" height="155" rx="18"/>
|
||||
<circle class="numBg" cx="998" cy="241" r="18"/>
|
||||
<text class="num" x="992" y="247">4</text>
|
||||
<text class="label" x="1026" y="247">Runtime</text>
|
||||
<text class="body" x="990" y="287">llama_cpp</text>
|
||||
<rect class="chip" x="52" y="420" width="190" height="40" rx="20"/>
|
||||
<text class="chiptext" x="69" y="445">modalities: text</text>
|
||||
<rect class="chip" x="258" y="420" width="190" height="40" rx="20"/>
|
||||
<text class="chiptext" x="275" y="445">license: apache-2.0</text>
|
||||
<rect class="chip" x="464" y="420" width="258" height="40" rx="20"/>
|
||||
<text class="chiptext" x="481" y="445">state: prebuilt_gguf_release</text>
|
||||
<rect class="chip" x="738" y="420" width="190" height="40" rx="20"/>
|
||||
<text class="chiptext" x="755" y="445">audit: pass</text>
|
||||
<line class="rule" x1="52" y1="496" x2="1228" y2="496"/>
|
||||
<text class="sub" x="52" y="526">Download the complete repo so README, manifest, checksums, runtime files, and weights stay together.</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.9 KiB |
35
docs/lumynax-runtime-flow.svg
Normal file
35
docs/lumynax-runtime-flow.svg
Normal file
@@ -0,0 +1,35 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1280" height="430" viewBox="0 0 1280 430" role="img" aria-labelledby="title desc">
|
||||
<title id="title">LumynaX Infused Qwen3 0.6B GGUF runtime flow</title>
|
||||
<desc id="desc">Runtime flow for AbteeXAILab/lumynax-infused-qwen3-06b-gguf from download to verification, quickstart, and serving.</desc>
|
||||
<defs>
|
||||
<filter id="shadow" x="-12%" y="-12%" width="124%" height="124%"><feDropShadow dx="0" dy="10" stdDeviation="12" flood-color="#0a0a0b" flood-opacity="0.08"/></filter>
|
||||
<marker id="arrowhead" markerWidth="12" markerHeight="8" refX="10" refY="4" orient="auto"><polygon points="0 0, 12 4, 0 8" fill="#e08a2c"/></marker>
|
||||
</defs>
|
||||
<style>
|
||||
.title{fill:#0a0a0b;font:500 34px Georgia,serif}.sub{fill:#726b62;font:16px Aptos,Segoe UI,Arial}.eyebrow{fill:#9a5416;font:700 13px ui-monospace,Consolas,monospace;letter-spacing:2px}.rule{stroke:#0a0a0b;stroke-opacity:.12}.accent{stroke:#e08a2c;stroke-width:4}.box{fill:#ffffff;stroke:#0a0a0b;stroke-opacity:.12;stroke-width:1.2;filter:url(#shadow)}.label{fill:#0a0a0b;font:700 19px Aptos,Segoe UI,Arial}.body{fill:#5f574e;font:14px Aptos,Segoe UI,Arial}.line{stroke:#e08a2c;stroke-width:3;marker-end:url(#arrowhead)}.artifact{fill:#9a5416;font:700 14px ui-monospace,Consolas,monospace}
|
||||
</style>
|
||||
<rect width="1280" height="430" rx="24" fill="#fffefa"/>
|
||||
<line class="accent" x1="856" y1="34" x2="1230" y2="34"/>
|
||||
<text class="eyebrow" x="50" y="48">LOCAL-FIRST RUNTIME FLOW</text>
|
||||
<text class="title" x="50" y="88">LumynaX Infused Qwen3 0.6B GGUF Runtime Path</text>
|
||||
<text class="sub" x="50" y="118">Primary artifact: Qwen3-0.6B-Q8_0.gguf</text>
|
||||
<line class="rule" x1="50" y1="138" x2="1230" y2="138"/>
|
||||
<rect class="box" x="64" y="178" width="245" height="118" rx="16"/>
|
||||
<text class="label" x="86" y="220">Download</text>
|
||||
<text class="body" x="86" y="251">hf download</text>
|
||||
<text class="body" x="86" y="271">AbteeXAILab/lumynax-infused-qwen3-06b-gguf</text>
|
||||
<line class="line" x1="324" y1="237.0" x2="349" y2="237.0"/>
|
||||
<rect class="box" x="367" y="178" width="245" height="118" rx="16"/>
|
||||
<text class="label" x="389" y="220">Verify</text>
|
||||
<text class="body" x="389" y="251">checksums.sha256</text>
|
||||
<line class="line" x1="627" y1="237.0" x2="652" y2="237.0"/>
|
||||
<rect class="box" x="670" y="178" width="245" height="118" rx="16"/>
|
||||
<text class="label" x="692" y="220">Run</text>
|
||||
<text class="body" x="692" y="251">quickstart.py</text>
|
||||
<line class="line" x1="930" y1="237.0" x2="955" y2="237.0"/>
|
||||
<rect class="box" x="973" y="178" width="245" height="118" rx="16"/>
|
||||
<text class="label" x="995" y="220">Serve</text>
|
||||
<text class="body" x="995" y="251">llama.cpp / Ollama</text>
|
||||
<line class="rule" x1="50" y1="340" x2="1230" y2="340"/>
|
||||
<text class="artifact" x="50" y="375">Recommended first test: ask "Who are you?" and confirm the package answers with LumynaX identity plus honest provenance.</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.8 KiB |
37
hf_space/README.md
Normal file
37
hf_space/README.md
Normal file
@@ -0,0 +1,37 @@
|
||||
---
|
||||
title: LumynaX Infused Qwen3 0.6B GGUF Demo
|
||||
colorFrom: green
|
||||
colorTo: blue
|
||||
sdk: gradio
|
||||
app_file: app.py
|
||||
pinned: false
|
||||
short_description: Browser demo bundle for the exported LumynaX release.
|
||||
---
|
||||
|
||||
# LumynaX Infused Qwen3 0.6B GGUF Demo
|
||||
|
||||
Public browser demo for LumynaX from AbteeX AI Labs.
|
||||
|
||||
## Overview
|
||||
|
||||
- zero-install text demo for LumynaX Infused Qwen3 0.6B GGUF
|
||||
- public Space backed by a private Hugging Face model repo
|
||||
- tuned for practical, moderately detailed responses in the browser
|
||||
|
||||
## What To Expect
|
||||
|
||||
- the first request after a cold start can take a minute or two
|
||||
- response quality is representative, but this Space is optimized for accessibility over speed
|
||||
- if you need full weights or deeper evaluation, use the private model repo directly
|
||||
|
||||
## Example Prompts
|
||||
|
||||
- Give a helpful welcome message for customers in Aotearoa New Zealand.
|
||||
- Explain in two short paragraphs what LumynaX Infused Qwen3 0.6B GGUF is and who it is for.
|
||||
- Write a concise summary of why local AI deployment matters for NZ teams.
|
||||
|
||||
## Maintainer Notes
|
||||
|
||||
This Space downloads the target model repo from Hugging Face at runtime.
|
||||
Set `LUMYNAX_MODEL_REPO_ID` if you want the Space to target a different private model repo.
|
||||
If the target repo is GGUF-only and does not contain `merged_model/`, this Space stays in GGUF-only browser showcase mode and points people to the local interactive quickstart instead of surfacing a raw runtime error.
|
||||
337
hf_space/app.py
Normal file
337
hf_space/app.py
Normal file
@@ -0,0 +1,337 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import inspect
|
||||
from pathlib import Path
|
||||
from threading import Lock
|
||||
|
||||
import gradio as gr
|
||||
import torch
|
||||
from huggingface_hub import snapshot_download
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
MODEL_REPO_ENV_VAR = "LUMYNAX_MODEL_REPO_ID"
|
||||
HF_TOKEN_ENV_VARS = ("HF_TOKEN", "HUGGING_FACE_HUB_TOKEN", "HUGGINGFACE_HUB_TOKEN")
|
||||
DEFAULT_MODEL_REPO_ID = "AbteeXAILab/lumynax-infused-qwen3-06b-gguf"
|
||||
PROMPT_FORMAT = "chatml"
|
||||
SYSTEM_PROMPT = 'You are LumynaX operating from the LumynaX Infused Qwen3 0.6B GGUF package identity. Be helpful, clear, and honest about provenance.'
|
||||
MODEL_TITLE = "LumynaX Infused Qwen3 0.6B GGUF"
|
||||
MAX_NEW_TOKENS = 192
|
||||
SHOWCASE_MODE_MESSAGE = (
|
||||
"This Space is currently running in browser showcase mode for the GGUF release. "
|
||||
"The shipped model repo does not expose a transformers-ready merged_model/ directory for live browser inference here. "
|
||||
"Use the packaged files locally with quickstart.py --interactive for the full terminal experience."
|
||||
)
|
||||
|
||||
_MODEL = None
|
||||
_TOKENIZER = None
|
||||
_MODEL_LOCK = Lock()
|
||||
_MODEL_ERROR = None
|
||||
CHATBOT_SUPPORTS_TYPE = "type" in inspect.signature(gr.Chatbot.__init__).parameters
|
||||
|
||||
|
||||
def _history_to_messages(history: list[object]) -> list[dict[str, str]]:
|
||||
messages: list[dict[str, str]] = []
|
||||
for entry in history:
|
||||
if isinstance(entry, dict):
|
||||
role = str(entry.get("role", "assistant")).strip().lower()
|
||||
if role not in ("user", "assistant"):
|
||||
continue
|
||||
content = entry.get("content", "")
|
||||
text = content if isinstance(content, str) else str(content)
|
||||
if not text.strip():
|
||||
continue
|
||||
messages.append({"role": role, "content": text.strip()})
|
||||
continue
|
||||
if not isinstance(entry, (list, tuple)) or len(entry) != 2:
|
||||
continue
|
||||
user_text = entry[0] if isinstance(entry[0], str) else str(entry[0] or "")
|
||||
assistant_text = entry[1] if isinstance(entry[1], str) else str(entry[1] or "")
|
||||
if user_text.strip():
|
||||
messages.append({"role": "user", "content": user_text.strip()})
|
||||
if assistant_text.strip():
|
||||
messages.append({"role": "assistant", "content": assistant_text.strip()})
|
||||
return messages
|
||||
|
||||
|
||||
def _build_messages(history: list[object], message: str) -> list[dict[str, str]]:
|
||||
messages: list[dict[str, str]] = []
|
||||
if SYSTEM_PROMPT:
|
||||
messages.append({"role": "system", "content": SYSTEM_PROMPT})
|
||||
messages.extend(_history_to_messages(history))
|
||||
messages.append({"role": "user", "content": message.strip()})
|
||||
return messages
|
||||
|
||||
|
||||
def _append_history(history: list[object], message: str, reply: str) -> list[object]:
|
||||
return history + [
|
||||
{"role": "user", "content": message},
|
||||
{"role": "assistant", "content": reply},
|
||||
]
|
||||
|
||||
|
||||
def _provenance_response(message: str) -> str | None:
|
||||
message_lower = message.strip().lower()
|
||||
asks_provenance = any(
|
||||
phrase in message_lower
|
||||
for phrase in (
|
||||
"donor",
|
||||
"donors",
|
||||
"donor model",
|
||||
"donor models",
|
||||
"base model",
|
||||
"underlying model",
|
||||
"what model do you use",
|
||||
"what models do you use",
|
||||
"deepseek",
|
||||
"qwen",
|
||||
"gemma",
|
||||
"llama",
|
||||
"phi",
|
||||
)
|
||||
)
|
||||
if not asks_provenance:
|
||||
return None
|
||||
return (
|
||||
f"This is {MODEL_TITLE}, a standalone AbteeX AI Labs LumynaX release for "
|
||||
"Aotearoa New Zealand workflows. This public Space is a browser demo of that release."
|
||||
)
|
||||
|
||||
|
||||
def _governance_response(message: str) -> str | None:
|
||||
message_lower = message.strip().lower()
|
||||
|
||||
asks_iwi_sovereignty = (
|
||||
"iwi" in message_lower
|
||||
and ("data sovereignty" in message_lower or "llm" in message_lower or "language model" in message_lower)
|
||||
)
|
||||
asks_health_sovereignty = (
|
||||
"health" in message_lower
|
||||
and ("data sovereignty" in message_lower or "governance" in message_lower or "sensitive data" in message_lower)
|
||||
)
|
||||
asks_justice_controls = (
|
||||
"justice sector" in message_lower
|
||||
or ("justice" in message_lower and "ai" in message_lower)
|
||||
or "sensitive case data" in message_lower
|
||||
)
|
||||
|
||||
if asks_iwi_sovereignty:
|
||||
return (
|
||||
"For Iwi data sovereignty with an LLM, keep sensitive data in environments controlled by the data owner, "
|
||||
"minimise and de-identify data before use, agree governance and access rules with Iwi decision-makers, "
|
||||
"prevent provider training on submitted data, keep strong audit logs, require human review for high-stakes "
|
||||
"outputs, and make deletion, retention, and purpose limits explicit from the start."
|
||||
)
|
||||
|
||||
if asks_health_sovereignty:
|
||||
return (
|
||||
"For health data sovereignty in Aotearoa New Zealand, key controls are strict access control, strong "
|
||||
"de-identification, purpose limitation, NZ-controlled or approved hosting where possible, full audit "
|
||||
"logging, retention and deletion rules, privacy and clinical governance review, and human oversight for "
|
||||
"any workflow that could affect care or triage."
|
||||
)
|
||||
|
||||
if asks_justice_controls:
|
||||
return (
|
||||
"For justice-sector AI handling sensitive case data, use case-level access controls, data segregation, "
|
||||
"encryption in transit and at rest, no external model training on case material, full audit trails, "
|
||||
"mandatory human review, clear escalation and appeal paths, regular bias and security testing, and a rule "
|
||||
"that the model supports staff but does not make binding legal or operational decisions on its own."
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _identity_response(message: str, history: list[object]) -> str | None:
|
||||
message_lower = message.strip().lower()
|
||||
mentions_lumynax = "lumynax" in message_lower or "lumynax infused qwen3 0.6b gguf" in message_lower
|
||||
asks_identity = any(
|
||||
phrase in message_lower
|
||||
for phrase in (
|
||||
"who are you",
|
||||
"what are you",
|
||||
"what is lumynax",
|
||||
"what's lumynax",
|
||||
"what is this model",
|
||||
"what's this model",
|
||||
"explain what lumynax is",
|
||||
"explain",
|
||||
"describe",
|
||||
"tell me about",
|
||||
)
|
||||
)
|
||||
if not asks_identity:
|
||||
return None
|
||||
if not mentions_lumynax and "who are you" not in message_lower and "what are you" not in message_lower:
|
||||
return None
|
||||
|
||||
if "bullet" in message_lower or "three" in message_lower:
|
||||
return '- LumynaX Infused Qwen3 0.6B GGUF is a local-first LumynaX model release from AbteeX AI Labs\\n- It is aimed at practical Aotearoa New Zealand workflows and locally relevant responses\\n- This public Space is a browser demo for that LumynaX release'
|
||||
return 'LumynaX Infused Qwen3 0.6B GGUF is a local-first LumynaX model release from AbteeX AI Labs for Aotearoa New Zealand workflows. It is intended for practical assistant use and locally relevant text generation when appropriate. This public Space is a browser demo of the same release hosted on Hugging Face.'
|
||||
|
||||
|
||||
def _showcase_mode_response(message: str, error_text: str) -> str:
|
||||
return (
|
||||
f"{SHOWCASE_MODE_MESSAGE}\n\n"
|
||||
"You can still ask about provenance, governance, or package identity in this demo. "
|
||||
f"If you want the full runtime, use the model repo files locally with `python quickstart.py --interactive`. "
|
||||
f"(Runtime detail: {error_text})"
|
||||
)
|
||||
|
||||
|
||||
def _render_prompt(messages: list[dict[str, str]]) -> str:
|
||||
if PROMPT_FORMAT == "plain_completion":
|
||||
lines: list[str] = []
|
||||
for entry in messages:
|
||||
role = entry["role"]
|
||||
content = entry["content"]
|
||||
if role == "system":
|
||||
lines.append(content)
|
||||
elif role == "user":
|
||||
lines.append(f"User: {content}")
|
||||
else:
|
||||
lines.append(f"Assistant: {content}")
|
||||
lines.append("Assistant:")
|
||||
return "\n\n".join(lines)
|
||||
|
||||
parts: list[str] = []
|
||||
for entry in messages:
|
||||
role = entry["role"]
|
||||
content = entry["content"]
|
||||
parts.append(f"<|im_start|>{role}\n{content}<|im_end|>\n")
|
||||
parts.append("<|im_start|>assistant\n")
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def _load_runtime() -> tuple[object, object]:
|
||||
global _MODEL, _TOKENIZER, _MODEL_ERROR
|
||||
|
||||
if _MODEL is not None and _TOKENIZER is not None:
|
||||
return _MODEL, _TOKENIZER
|
||||
if _MODEL_ERROR is not None:
|
||||
raise RuntimeError(_MODEL_ERROR)
|
||||
|
||||
with _MODEL_LOCK:
|
||||
if _MODEL is not None and _TOKENIZER is not None:
|
||||
return _MODEL, _TOKENIZER
|
||||
if _MODEL_ERROR is not None:
|
||||
raise RuntimeError(_MODEL_ERROR)
|
||||
|
||||
try:
|
||||
repo_id = os.environ.get(MODEL_REPO_ENV_VAR, DEFAULT_MODEL_REPO_ID).strip() or DEFAULT_MODEL_REPO_ID
|
||||
hf_token = next((os.environ.get(name, "").strip() for name in HF_TOKEN_ENV_VARS if os.environ.get(name, "").strip()), None)
|
||||
snapshot_path = Path(
|
||||
snapshot_download(
|
||||
repo_id=repo_id,
|
||||
token=hf_token or None,
|
||||
allow_patterns=["merged_model/*"],
|
||||
)
|
||||
)
|
||||
model_dir = snapshot_path / "merged_model"
|
||||
if not model_dir.exists():
|
||||
raise FileNotFoundError(
|
||||
f"Expected merged_model/ in {snapshot_path} after downloading {repo_id}."
|
||||
)
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(str(model_dir))
|
||||
if tokenizer.pad_token is None:
|
||||
tokenizer.pad_token = tokenizer.eos_token
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
str(model_dir),
|
||||
dtype=torch.bfloat16,
|
||||
low_cpu_mem_usage=True,
|
||||
)
|
||||
|
||||
_MODEL = model
|
||||
_TOKENIZER = tokenizer
|
||||
return _MODEL, _TOKENIZER
|
||||
except Exception as exc:
|
||||
_MODEL_ERROR = f"{type(exc).__name__}: {exc}"
|
||||
raise
|
||||
|
||||
|
||||
def chat(message: str, history: list[object]) -> tuple[str, list[object]]:
|
||||
history = history or []
|
||||
if not message.strip():
|
||||
return "", history
|
||||
|
||||
try:
|
||||
provenance_reply = _provenance_response(message)
|
||||
if provenance_reply is not None:
|
||||
return "", _append_history(history, message, provenance_reply)
|
||||
|
||||
governance_reply = _governance_response(message)
|
||||
if governance_reply is not None:
|
||||
return "", _append_history(history, message, governance_reply)
|
||||
|
||||
identity_reply = _identity_response(message, history)
|
||||
if identity_reply is not None:
|
||||
return "", _append_history(history, message, identity_reply)
|
||||
|
||||
model, tokenizer = _load_runtime()
|
||||
messages = _build_messages(history, message)
|
||||
if hasattr(tokenizer, "apply_chat_template") and PROMPT_FORMAT != "plain_completion":
|
||||
encoded = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=True,
|
||||
add_generation_prompt=True,
|
||||
return_tensors="pt",
|
||||
return_dict=True,
|
||||
)
|
||||
else:
|
||||
prompt = _render_prompt(messages)
|
||||
encoded = tokenizer(prompt, return_tensors="pt")
|
||||
encoded = encoded.to(model.device)
|
||||
|
||||
with torch.inference_mode():
|
||||
output = model.generate(
|
||||
**encoded,
|
||||
max_new_tokens=MAX_NEW_TOKENS,
|
||||
do_sample=False,
|
||||
pad_token_id=tokenizer.eos_token_id,
|
||||
)
|
||||
|
||||
prompt_length = encoded["input_ids"].shape[-1]
|
||||
generated = tokenizer.decode(output[0][prompt_length:], skip_special_tokens=True).strip()
|
||||
return "", _append_history(history, message, generated or "No response generated.")
|
||||
except Exception as exc:
|
||||
error_text = f"{type(exc).__name__}: {exc}"
|
||||
return "", _append_history(history, message, _showcase_mode_response(message, error_text))
|
||||
|
||||
with gr.Blocks() as demo:
|
||||
gr.Markdown(
|
||||
f"# {MODEL_TITLE}\n\n"
|
||||
"Public browser demo for LumynaX from AbteeX AI Labs. "
|
||||
"This Space is backed by a private model repo on Hugging Face. "
|
||||
"If the backing repo is GGUF-only, this browser demo stays in showcase mode and directs people to the local interactive quickstart."
|
||||
)
|
||||
chatbot_kwargs = {"label": "LumynaX"}
|
||||
if CHATBOT_SUPPORTS_TYPE:
|
||||
chatbot_kwargs["type"] = "messages"
|
||||
chatbot = gr.Chatbot(**chatbot_kwargs)
|
||||
gr.Markdown("Enter a prompt and press `Enter` or click `Send`.")
|
||||
with gr.Row():
|
||||
prompt = gr.Textbox(
|
||||
label="Prompt",
|
||||
placeholder="Ask LumynaX something about Aotearoa, your project, or local research.",
|
||||
lines=4,
|
||||
scale=8,
|
||||
)
|
||||
send = gr.Button("Send", variant="primary", scale=1, min_width=120)
|
||||
gr.Examples(
|
||||
examples=[
|
||||
"Give a helpful welcome message for customers in Aotearoa New Zealand.",
|
||||
'Explain in two short paragraphs what LumynaX Infused Qwen3 0.6B GGUF is and who it is for.',
|
||||
"Write a concise summary of why local AI deployment matters for NZ teams.",
|
||||
],
|
||||
inputs=prompt,
|
||||
)
|
||||
clear = gr.Button("Clear")
|
||||
|
||||
prompt.submit(chat, inputs=[prompt, chatbot], outputs=[prompt, chatbot])
|
||||
send.click(chat, inputs=[prompt, chatbot], outputs=[prompt, chatbot])
|
||||
clear.click(lambda: [], outputs=chatbot, queue=False)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
demo.launch()
|
||||
5
hf_space/requirements.txt
Normal file
5
hf_space/requirements.txt
Normal file
@@ -0,0 +1,5 @@
|
||||
gradio>=5.0
|
||||
huggingface-hub>=0.25
|
||||
torch>=2.0
|
||||
transformers>=4.45
|
||||
safetensors>=0.4
|
||||
1
merged_model/PACKAGE_STATE.txt
Normal file
1
merged_model/PACKAGE_STATE.txt
Normal file
@@ -0,0 +1 @@
|
||||
LumynaX Infused Qwen3 0.6B GGUF is a GGUF-only release package. The primary runtime artifact is ../Qwen3-0.6B-Q8_0.gguf; no full Transformers checkpoint is bundled here.
|
||||
12
ollama/Modelfile
Normal file
12
ollama/Modelfile
Normal file
@@ -0,0 +1,12 @@
|
||||
FROM ../Qwen3-0.6B-Q8_0.gguf
|
||||
TEMPLATE """{{ if .System }}<|im_start|>system
|
||||
{{ .System }}<|im_end|>
|
||||
{{ end }}{{ if .Prompt }}<|im_start|>user
|
||||
{{ .Prompt }}<|im_end|>
|
||||
{{ end }}<|im_start|>assistant
|
||||
"""
|
||||
SYSTEM You are LumynaX operating from the LumynaX Infused Qwen3 0.6B GGUF package identity. Be helpful, clear, and honest about provenance.
|
||||
PARAMETER temperature 0.1
|
||||
PARAMETER num_ctx 8192
|
||||
PARAMETER stop "<|im_end|>"
|
||||
PARAMETER stop "<|endoftext|>"
|
||||
21
ollama/create_ollama_model.ps1
Normal file
21
ollama/create_ollama_model.ps1
Normal file
@@ -0,0 +1,21 @@
|
||||
param(
|
||||
[string]$ModelName = "lumynax-infused-qwen3-06b-gguf"
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
Set-StrictMode -Version Latest
|
||||
|
||||
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
|
||||
$modelfilePath = Join-Path $scriptDir "Modelfile"
|
||||
|
||||
if (-not (Get-Command ollama -ErrorAction SilentlyContinue)) {
|
||||
throw "The `ollama` CLI is not installed. Install Ollama first."
|
||||
}
|
||||
|
||||
& ollama create $ModelName -f $modelfilePath
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
exit $LASTEXITCODE
|
||||
}
|
||||
|
||||
Write-Output "Created Ollama model: $ModelName"
|
||||
Write-Output "Run it with: ollama run $ModelName"
|
||||
449
quickstart.py
Normal file
449
quickstart.py
Normal file
@@ -0,0 +1,449 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
MODEL_TITLE = "LumynaX Infused Qwen3 0.6B GGUF"
|
||||
|
||||
|
||||
def _build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(description=f"Run a local GGUF chat for {MODEL_TITLE}.")
|
||||
parser.add_argument(
|
||||
"--prompt",
|
||||
default=None,
|
||||
help="Prompt to send to the model.",
|
||||
)
|
||||
parser.add_argument("--system-prompt", default="", help="Optional system prompt override.")
|
||||
parser.add_argument(
|
||||
"--interactive",
|
||||
action="store_true",
|
||||
help="Start an interactive terminal chat instead of running a single prompt.",
|
||||
)
|
||||
parser.add_argument("--max-new-tokens", type=int, default=192)
|
||||
parser.add_argument("--ctx-size", type=int, default=4096)
|
||||
parser.add_argument("--temperature", type=float, default=0.1)
|
||||
parser.add_argument("--threads", type=int, default=max(1, os.cpu_count() or 1))
|
||||
parser.add_argument("--llama-cli", default="", help="Optional explicit path to llama-cli.")
|
||||
parser.add_argument(
|
||||
"--cache-local",
|
||||
action="store_true",
|
||||
help="Copy the GGUF into LOCALAPPDATA before running. Useful when a runtime cannot read network paths.",
|
||||
)
|
||||
parser.add_argument("--reasoning", choices=("on", "off", "auto"), default="off")
|
||||
parser.add_argument(
|
||||
"--reasoning-format",
|
||||
choices=("auto", "none", "deepseek", "deepseek-legacy"),
|
||||
default="auto",
|
||||
)
|
||||
parser.add_argument("--reasoning-budget", type=int, default=None)
|
||||
return parser
|
||||
|
||||
|
||||
def _preferred_gguf(root: Path) -> Path:
|
||||
gguf_candidates = sorted(root.glob("*.gguf"))
|
||||
if not gguf_candidates:
|
||||
raise SystemExit(f"No GGUF file was found in {root}")
|
||||
for path in gguf_candidates:
|
||||
if "-q" in path.stem.lower():
|
||||
return path
|
||||
return gguf_candidates[0]
|
||||
|
||||
|
||||
def _local_model_path(model_path: Path, *, cache_local: bool = False) -> Path:
|
||||
if not cache_local:
|
||||
return model_path
|
||||
local_app_data = Path(os.environ.get("LOCALAPPDATA", Path.home() / "AppData" / "Local"))
|
||||
cache_dir = local_app_data / "tinyluminax" / "gguf-cache"
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
cached_path = cache_dir / model_path.name
|
||||
source_stat = model_path.stat()
|
||||
if (
|
||||
not cached_path.exists()
|
||||
or cached_path.stat().st_size != source_stat.st_size
|
||||
or cached_path.stat().st_mtime_ns < source_stat.st_mtime_ns
|
||||
):
|
||||
print(f"Caching GGUF locally at {cached_path}", file=sys.stderr)
|
||||
shutil.copy2(model_path, cached_path)
|
||||
return cached_path
|
||||
|
||||
|
||||
def _discover_llama_cli(explicit_path: str) -> Path | None:
|
||||
candidates: list[Path] = []
|
||||
if explicit_path.strip():
|
||||
candidates.append(Path(explicit_path.strip()))
|
||||
for env_var in ("LLAMA_CPP_CLI", "LLAMA_CLI_PATH"):
|
||||
raw_value = os.environ.get(env_var, "").strip()
|
||||
if raw_value:
|
||||
candidates.append(Path(raw_value))
|
||||
for binary_name in ("llama-cli", "llama-cli.exe"):
|
||||
resolved = shutil.which(binary_name)
|
||||
if resolved:
|
||||
candidates.append(Path(resolved))
|
||||
for candidate in candidates:
|
||||
if candidate.exists():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def _extract_text(response: dict[str, object]) -> str:
|
||||
choices = response.get("choices", [])
|
||||
if not isinstance(choices, list) or not choices:
|
||||
raise RuntimeError("The runtime returned no choices.")
|
||||
first_choice = choices[0]
|
||||
if isinstance(first_choice, dict):
|
||||
message = first_choice.get("message")
|
||||
if isinstance(message, dict):
|
||||
content = message.get("content")
|
||||
if content not in (None, ""):
|
||||
return str(content).strip()
|
||||
text = first_choice.get("text")
|
||||
if text not in (None, ""):
|
||||
return str(text).strip()
|
||||
raise RuntimeError("The runtime returned an unsupported response payload.")
|
||||
|
||||
|
||||
def _run_llama_cpp_python(
|
||||
*,
|
||||
model_path: Path,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
max_new_tokens: int,
|
||||
ctx_size: int,
|
||||
temperature: float,
|
||||
threads: int,
|
||||
) -> str:
|
||||
from llama_cpp import Llama
|
||||
|
||||
llm = Llama(
|
||||
model_path=str(model_path),
|
||||
n_ctx=ctx_size,
|
||||
n_threads=threads,
|
||||
n_gpu_layers=0,
|
||||
chat_format="chat_template.default",
|
||||
verbose=False,
|
||||
)
|
||||
response = llm.create_chat_completion(
|
||||
messages=[
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": user_prompt},
|
||||
],
|
||||
max_tokens=max_new_tokens,
|
||||
temperature=temperature,
|
||||
)
|
||||
return _extract_text(response)
|
||||
|
||||
|
||||
def _run_llama_cli(
|
||||
*,
|
||||
llama_cli_path: Path,
|
||||
model_path: Path,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
max_new_tokens: int,
|
||||
ctx_size: int,
|
||||
temperature: float,
|
||||
threads: int,
|
||||
reasoning: str,
|
||||
reasoning_format: str,
|
||||
reasoning_budget: int | None,
|
||||
) -> None:
|
||||
command = [
|
||||
str(llama_cli_path),
|
||||
"-m",
|
||||
str(model_path),
|
||||
"-sys",
|
||||
system_prompt,
|
||||
"-p",
|
||||
user_prompt,
|
||||
"-cnv",
|
||||
"-st",
|
||||
"-n",
|
||||
str(max_new_tokens),
|
||||
"-c",
|
||||
str(ctx_size),
|
||||
"--reasoning",
|
||||
reasoning,
|
||||
"--temp",
|
||||
str(temperature),
|
||||
"--threads",
|
||||
str(threads),
|
||||
"--no-display-prompt",
|
||||
]
|
||||
if reasoning_format != "auto":
|
||||
command.extend(["--reasoning-format", reasoning_format])
|
||||
if reasoning_budget is not None:
|
||||
command.extend(["--reasoning-budget", str(reasoning_budget)])
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
)
|
||||
if completed.returncode != 0:
|
||||
detail = completed.stderr.strip() or completed.stdout.strip() or "llama-cli failed"
|
||||
raise SystemExit(detail)
|
||||
stdout = completed.stdout.strip()
|
||||
if stdout:
|
||||
print(stdout)
|
||||
|
||||
|
||||
def _print_interactive_banner() -> None:
|
||||
print("LumynaX interactive terminal chat")
|
||||
print("Type /reset to clear the conversation, or /quit to exit.")
|
||||
|
||||
|
||||
def _run_interactive_llama_cpp_python(
|
||||
*,
|
||||
model_path: Path,
|
||||
system_prompt: str,
|
||||
max_new_tokens: int,
|
||||
ctx_size: int,
|
||||
temperature: float,
|
||||
threads: int,
|
||||
opening_prompt: str | None = None,
|
||||
reasoning: str = "off",
|
||||
reasoning_format: str = "auto",
|
||||
reasoning_budget: int | None = None,
|
||||
) -> None:
|
||||
from llama_cpp import Llama
|
||||
|
||||
llm = Llama(
|
||||
model_path=str(model_path),
|
||||
n_ctx=ctx_size,
|
||||
n_threads=threads,
|
||||
n_gpu_layers=0,
|
||||
chat_format="chat_template.default",
|
||||
verbose=False,
|
||||
)
|
||||
transcript: list[tuple[str, str]] = []
|
||||
_print_interactive_banner()
|
||||
|
||||
pending_prompt = opening_prompt.strip() if opening_prompt and opening_prompt.strip() else None
|
||||
while True:
|
||||
try:
|
||||
if pending_prompt is None:
|
||||
user_prompt = input("You> ").strip()
|
||||
else:
|
||||
user_prompt = pending_prompt
|
||||
print(f"You> {user_prompt}")
|
||||
pending_prompt = None
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
print("\nExiting LumynaX chat.")
|
||||
return
|
||||
if not user_prompt:
|
||||
continue
|
||||
lowered_prompt = user_prompt.lower()
|
||||
if lowered_prompt in ('/quit', '/exit'):
|
||||
print("Exiting LumynaX chat.")
|
||||
return
|
||||
if lowered_prompt == "/reset":
|
||||
transcript.clear()
|
||||
print("Conversation reset.")
|
||||
continue
|
||||
messages: list[dict[str, str]] = [{"role": "system", "content": system_prompt}]
|
||||
for transcript_user_prompt, transcript_assistant_response in transcript:
|
||||
messages.append({"role": "user", "content": transcript_user_prompt})
|
||||
messages.append({"role": "assistant", "content": transcript_assistant_response})
|
||||
messages.append({"role": "user", "content": user_prompt})
|
||||
response = llm.create_chat_completion(
|
||||
messages=messages,
|
||||
max_tokens=max_new_tokens,
|
||||
temperature=temperature,
|
||||
)
|
||||
assistant_text = _extract_text(response)
|
||||
print(f"LumynaX> {assistant_text}")
|
||||
transcript.append((user_prompt, assistant_text))
|
||||
|
||||
|
||||
def _run_interactive_llama_cli(
|
||||
*,
|
||||
llama_cli_path: Path,
|
||||
model_path: Path,
|
||||
system_prompt: str,
|
||||
max_new_tokens: int,
|
||||
ctx_size: int,
|
||||
temperature: float,
|
||||
threads: int,
|
||||
opening_prompt: str | None = None,
|
||||
reasoning: str = "off",
|
||||
reasoning_format: str = "auto",
|
||||
reasoning_budget: int | None = None,
|
||||
) -> None:
|
||||
print("LumynaX interactive terminal chat")
|
||||
print("Interactive mode already uses llama-cli directly. Use Ctrl+C to exit.")
|
||||
command = [
|
||||
str(llama_cli_path),
|
||||
"-m",
|
||||
str(model_path),
|
||||
"-sys",
|
||||
system_prompt,
|
||||
"-cnv",
|
||||
"-n",
|
||||
str(max_new_tokens),
|
||||
"-c",
|
||||
str(ctx_size),
|
||||
"--reasoning",
|
||||
reasoning,
|
||||
"--temp",
|
||||
str(temperature),
|
||||
"--threads",
|
||||
str(threads),
|
||||
"--simple-io",
|
||||
]
|
||||
if reasoning_format != "auto":
|
||||
command.extend(["--reasoning-format", reasoning_format])
|
||||
if reasoning_budget is not None:
|
||||
command.extend(["--reasoning-budget", str(reasoning_budget)])
|
||||
if opening_prompt and opening_prompt.strip():
|
||||
command.extend(["-p", opening_prompt.strip()])
|
||||
completed = subprocess.run(command, check=False)
|
||||
if completed.returncode != 0:
|
||||
raise SystemExit(completed.returncode)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = _build_parser().parse_args()
|
||||
root = Path(__file__).resolve().parent
|
||||
source_model_path = _preferred_gguf(root)
|
||||
if hasattr(sys.stdout, "reconfigure"):
|
||||
sys.stdout.reconfigure(encoding="utf-8")
|
||||
|
||||
single_prompt = (args.prompt or "Say hello in one short sentence.").strip()
|
||||
system_prompt = args.system_prompt.strip() or (
|
||||
f"You are LumynaX operating from the {MODEL_TITLE} package identity. "
|
||||
"Be helpful, clear, and honest about provenance."
|
||||
)
|
||||
explicit_cli_requested = bool(
|
||||
args.llama_cli.strip()
|
||||
or os.environ.get("LLAMA_CPP_CLI", "").strip()
|
||||
or os.environ.get("LLAMA_CLI_PATH", "").strip()
|
||||
)
|
||||
if args.interactive:
|
||||
llama_cli_path = _discover_llama_cli(args.llama_cli)
|
||||
if explicit_cli_requested:
|
||||
if llama_cli_path is None:
|
||||
raise SystemExit(
|
||||
"A llama-cli override was requested, but no usable llama-cli binary was found.",
|
||||
)
|
||||
_run_interactive_llama_cli(
|
||||
llama_cli_path=llama_cli_path,
|
||||
model_path=_local_model_path(source_model_path, cache_local=args.cache_local),
|
||||
system_prompt=system_prompt,
|
||||
opening_prompt=args.prompt,
|
||||
max_new_tokens=args.max_new_tokens,
|
||||
ctx_size=args.ctx_size,
|
||||
temperature=args.temperature,
|
||||
threads=args.threads,
|
||||
reasoning=args.reasoning,
|
||||
reasoning_format=args.reasoning_format,
|
||||
reasoning_budget=args.reasoning_budget,
|
||||
)
|
||||
return
|
||||
model_path = _local_model_path(source_model_path, cache_local=args.cache_local)
|
||||
try:
|
||||
_run_interactive_llama_cpp_python(
|
||||
model_path=model_path,
|
||||
system_prompt=system_prompt,
|
||||
opening_prompt=args.prompt,
|
||||
max_new_tokens=args.max_new_tokens,
|
||||
ctx_size=args.ctx_size,
|
||||
temperature=args.temperature,
|
||||
threads=args.threads,
|
||||
reasoning=args.reasoning,
|
||||
reasoning_format=args.reasoning_format,
|
||||
reasoning_budget=args.reasoning_budget,
|
||||
)
|
||||
return
|
||||
except Exception as exc: # noqa: BLE001
|
||||
if llama_cli_path is None:
|
||||
raise SystemExit(
|
||||
"llama-cpp-python could not load this GGUF package. "
|
||||
"Install or point LLAMA_CPP_CLI at llama-cli to use the built-in fallback. "
|
||||
f"Original error: {exc}",
|
||||
) from exc
|
||||
print(
|
||||
f"llama-cpp-python failed; falling back to llama-cli at {llama_cli_path}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
_run_interactive_llama_cli(
|
||||
llama_cli_path=llama_cli_path,
|
||||
model_path=model_path,
|
||||
system_prompt=system_prompt,
|
||||
opening_prompt=args.prompt,
|
||||
max_new_tokens=args.max_new_tokens,
|
||||
ctx_size=args.ctx_size,
|
||||
temperature=args.temperature,
|
||||
threads=args.threads,
|
||||
reasoning=args.reasoning,
|
||||
reasoning_format=args.reasoning_format,
|
||||
reasoning_budget=args.reasoning_budget,
|
||||
)
|
||||
return
|
||||
if explicit_cli_requested:
|
||||
llama_cli_path = _discover_llama_cli(args.llama_cli)
|
||||
if llama_cli_path is None:
|
||||
raise SystemExit(
|
||||
"A llama-cli override was requested, but no usable llama-cli binary was found.",
|
||||
)
|
||||
_run_llama_cli(
|
||||
llama_cli_path=llama_cli_path,
|
||||
model_path=_local_model_path(source_model_path, cache_local=args.cache_local),
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=single_prompt,
|
||||
max_new_tokens=args.max_new_tokens,
|
||||
ctx_size=args.ctx_size,
|
||||
temperature=args.temperature,
|
||||
threads=args.threads,
|
||||
reasoning=args.reasoning,
|
||||
reasoning_format=args.reasoning_format,
|
||||
reasoning_budget=args.reasoning_budget,
|
||||
)
|
||||
return
|
||||
model_path = _local_model_path(source_model_path, cache_local=args.cache_local)
|
||||
try:
|
||||
print(
|
||||
_run_llama_cpp_python(
|
||||
model_path=model_path,
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=single_prompt,
|
||||
max_new_tokens=args.max_new_tokens,
|
||||
ctx_size=args.ctx_size,
|
||||
temperature=args.temperature,
|
||||
threads=args.threads,
|
||||
),
|
||||
)
|
||||
return
|
||||
except Exception as exc: # noqa: BLE001
|
||||
llama_cli_path = _discover_llama_cli(args.llama_cli)
|
||||
if llama_cli_path is None:
|
||||
raise SystemExit(
|
||||
"llama-cpp-python could not load this GGUF package. "
|
||||
"Install or point LLAMA_CPP_CLI at llama-cli to use the built-in fallback. "
|
||||
f"Original error: {exc}",
|
||||
) from exc
|
||||
print(
|
||||
f"llama-cpp-python failed; falling back to llama-cli at {llama_cli_path}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
_run_llama_cli(
|
||||
llama_cli_path=llama_cli_path,
|
||||
model_path=model_path,
|
||||
system_prompt=system_prompt,
|
||||
user_prompt=single_prompt,
|
||||
max_new_tokens=args.max_new_tokens,
|
||||
ctx_size=args.ctx_size,
|
||||
temperature=args.temperature,
|
||||
threads=args.threads,
|
||||
reasoning=args.reasoning,
|
||||
reasoning_format=args.reasoning_format,
|
||||
reasoning_budget=args.reasoning_budget,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
88
release_export_manifest.json
Normal file
88
release_export_manifest.json
Normal file
@@ -0,0 +1,88 @@
|
||||
{
|
||||
"artifacts": {
|
||||
"checksums": "checksums.sha256",
|
||||
"gguf": "Qwen3-0.6B-Q8_0.gguf",
|
||||
"gitattributes": ".gitattributes",
|
||||
"hf_space_app": "hf_space/app.py",
|
||||
"hf_space_dir": "hf_space",
|
||||
"hf_space_readme": "hf_space/README.md",
|
||||
"hf_space_requirements": "hf_space/requirements.txt",
|
||||
"license": "LICENSE.txt",
|
||||
"merged_model": "merged_model",
|
||||
"mmproj": null,
|
||||
"ollama_create_script": "ollama/create_ollama_model.ps1",
|
||||
"ollama_modelfile": "ollama/Modelfile",
|
||||
"package_state": "merged_model/PACKAGE_STATE.txt",
|
||||
"quantized_gguf": "Qwen3-0.6B-Q8_0.gguf",
|
||||
"quickstart": "quickstart.py",
|
||||
"readme": "README.md",
|
||||
"requirements": "requirements.txt",
|
||||
"training_summary": "artifacts/release_training_summary.json",
|
||||
"upload_notes": "UPLOAD_TO_HF.md",
|
||||
"version": "VERSION.txt"
|
||||
},
|
||||
"capabilities": {
|
||||
"reasoning_enabled": false,
|
||||
"supported_modalities": [
|
||||
"text"
|
||||
]
|
||||
},
|
||||
"delivery": "standalone_prebuilt_gguf_release",
|
||||
"distribution": {
|
||||
"hf_space": {
|
||||
"app": "hf_space/app.py",
|
||||
"default_model_repo_id": "AbteeXAILab/lumynax-infused-qwen3-06b-gguf",
|
||||
"directory": "hf_space",
|
||||
"model_repo_env_var": "LUMYNAX_MODEL_REPO_ID",
|
||||
"readme": "hf_space/README.md",
|
||||
"requirements": "hf_space/requirements.txt",
|
||||
"status": "browser_showcase_for_gguf_only_release"
|
||||
},
|
||||
"ollama": {
|
||||
"create_script": "ollama/create_ollama_model.ps1",
|
||||
"mmproj": null,
|
||||
"modelfile": "ollama/Modelfile",
|
||||
"preferred_gguf": "Qwen3-0.6B-Q8_0.gguf",
|
||||
"recommended_model_name": "lumynax-infused-qwen3-06b-gguf",
|
||||
"status": "ready_for_local_ollama_create"
|
||||
}
|
||||
},
|
||||
"family": null,
|
||||
"generated_at": "2026-05-10T12:57:47.829709+00:00",
|
||||
"license": {
|
||||
"id": "apache-2.0",
|
||||
"link": "https://huggingface.co/Qwen/Qwen3-0.6B-GGUF",
|
||||
"name": null,
|
||||
"weights_subject_to_upstream_license": true
|
||||
},
|
||||
"manifest_version": 2,
|
||||
"model_title": "LumynaX Infused Qwen3 0.6B GGUF",
|
||||
"package_state": "prebuilt_gguf_release",
|
||||
"public_identity": {
|
||||
"model_name": "LumynaX",
|
||||
"organization": "AbteeX AI Labs",
|
||||
"region": "Aotearoa New Zealand"
|
||||
},
|
||||
"release_version": "v1",
|
||||
"runtime": {
|
||||
"delivery_mode": "standalone_prebuilt_gguf",
|
||||
"preferred_backend": "llama_cpp",
|
||||
"prompt_format": "chatml",
|
||||
"quickstart_command": "python quickstart.py --interactive",
|
||||
"system_prompt": "You are LumynaX operating from the LumynaX Infused Qwen3 0.6B GGUF package identity. Be helpful, clear, and honest about provenance."
|
||||
},
|
||||
"source_gguf": {
|
||||
"filename": "Qwen3-0.6B-Q8_0.gguf",
|
||||
"mmproj_filename": null,
|
||||
"packaged_filename": "Qwen3-0.6B-Q8_0.gguf",
|
||||
"packaged_mmproj_filename": null,
|
||||
"quantization": "Q8_0",
|
||||
"repo_id": "Qwen/Qwen3-0.6B-GGUF"
|
||||
},
|
||||
"upstream_model": {
|
||||
"kind": "official_base_weights",
|
||||
"lumynax_weight_adaptation_applied": false,
|
||||
"provider": "Hugging Face",
|
||||
"repo_id": "Qwen/Qwen3-0.6B"
|
||||
}
|
||||
}
|
||||
1
requirements.txt
Normal file
1
requirements.txt
Normal file
@@ -0,0 +1 @@
|
||||
llama-cpp-python>=0.3.18
|
||||
Reference in New Issue
Block a user