初始化项目,由ModelHub XC社区提供模型
Model: javierdejesusda/yuholens-14b-GGUF Source: Original Platform
This commit is contained in:
40
.gitattributes
vendored
Normal file
40
.gitattributes
vendored
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
yuholens-14b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
yuholens-14b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
yuholens-14b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
yuholens-14b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
yuholens-14b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
144
README.md
Normal file
144
README.md
Normal file
@@ -0,0 +1,144 @@
|
|||||||
|
---
|
||||||
|
license: other
|
||||||
|
license_name: tongyi-qianwen
|
||||||
|
license_link: https://huggingface.co/Qwen/Qwen-14B/blob/main/LICENSE
|
||||||
|
language:
|
||||||
|
- ja
|
||||||
|
- en
|
||||||
|
base_model: javierdejesusda/yuholens-14b
|
||||||
|
tags:
|
||||||
|
- gguf
|
||||||
|
- llama-cpp
|
||||||
|
- quantized
|
||||||
|
- japanese-finance
|
||||||
|
- yuho
|
||||||
|
- edinet
|
||||||
|
- qwen
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
---
|
||||||
|
|
||||||
|
# YuhoLens-14B GGUF
|
||||||
|
|
||||||
|
GGUF-quantized release of [`javierdejesusda/yuholens-14b`](https://huggingface.co/javierdejesusda/yuholens-14b) for offline inference via [`llama.cpp`](https://github.com/ggerganov/llama.cpp). The BF16 source is a full-parameter SFT of [`pfnet/nekomata-14b-pfn-qfin`](https://huggingface.co/pfnet/nekomata-14b-pfn-qfin) (Qwen1, 14B, Japanese-finance CPT) on teacher-bootstrapped English investor memos derived from [`SakanaAI/EDINET-Bench`](https://huggingface.co/datasets/SakanaAI/EDINET-Bench).
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
| File | Quant | Size | Bits/weight | Recommended hardware |
|
||||||
|
|------|-------|-----:|:-----------:|----------------------|
|
||||||
|
| `yuholens-14b-Q3_K_M.gguf` | Q3_K_M | 7.18 GB | 4.35 | 8 GB GPU (RTX 4070 Laptop, 3060 Ti) — fits with `--ctx-size 2048` |
|
||||||
|
| `yuholens-14b-Q4_K_M.gguf` | Q4_K_M | 8.81 GB | ~5.0 | 12-16 GB GPU (RTX 4060 Ti 16 GB, RTX 3080) |
|
||||||
|
| `yuholens-14b-Q5_K_M.gguf` | Q5_K_M | 9.94 GB | ~5.7 | 16 GB GPU |
|
||||||
|
| `yuholens-14b-Q6_K.gguf` | Q6_K | 11.46 GB | ~6.6 | 16-24 GB GPU |
|
||||||
|
| `yuholens-14b-Q8_0.gguf` | Q8_0 | 14.03 GB | 8.5 | 24 GB GPU or CPU offload |
|
||||||
|
|
||||||
|
Sizes are the actual on-disk byte counts; quoted GB values use 1024³.
|
||||||
|
|
||||||
|
## Smoke test
|
||||||
|
|
||||||
|
The Q3_K_M quant has been smoke-tested on an NVIDIA RTX 4070 Laptop (8 GB VRAM, compute 8.9) using `llama-completion` from llama.cpp build `b8966`.
|
||||||
|
|
||||||
|
- Prompt: ChatML-wrapped Yuho fixture with the YuhoLens system prompt (see *Prompt format* below).
|
||||||
|
- Settings: `--n-gpu-layers 99 -c 2048 --temp 0.1 --top-p 0.9 --repeat-penalty 1.15`.
|
||||||
|
- Throughput: prompt eval **65.5 tok/s**, generation **12.2 tok/s**.
|
||||||
|
- VRAM occupancy: ~7.0 GB model + ~1.6 GB context + ~0.3 GB compute, fits within the 8.0 GB card.
|
||||||
|
- Output: coherent English investor memo with section headings (*Executive summary*, *Going-concern assessment*, *Accrual quality*, ...) following the SFT teacher template.
|
||||||
|
|
||||||
|
Q4_K_M and larger quants exceed 8 GB VRAM and require either a 12 GB+ GPU or partial CPU offload (`--n-gpu-layers` < 99).
|
||||||
|
|
||||||
|
## Quickstart
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Q3_K_M, fits 8 GB GPU
|
||||||
|
llama-completion \
|
||||||
|
-m yuholens-14b-Q3_K_M.gguf \
|
||||||
|
-f your_prompt.txt \
|
||||||
|
--n-gpu-layers 99 \
|
||||||
|
-c 2048 \
|
||||||
|
--temp 0.1 --top-p 0.9 --repeat-penalty 1.15 \
|
||||||
|
-n 512 \
|
||||||
|
--no-display-prompt
|
||||||
|
```
|
||||||
|
|
||||||
|
For larger quants on a smaller GPU, drop `--n-gpu-layers` to a value that fits VRAM (e.g. `--n-gpu-layers 30` for Q4_K_M on 8 GB).
|
||||||
|
|
||||||
|
## Prompt format
|
||||||
|
|
||||||
|
YuhoLens-14B was fine-tuned on Qwen1 ChatML with a fixed system prompt. **Raw Japanese Yuho text without the wrapper produces Japanese continuations** instead of English memos. Wrap inputs as:
|
||||||
|
|
||||||
|
```
|
||||||
|
<|im_start|>system
|
||||||
|
{SYSTEM_PROMPT}<|im_end|>
|
||||||
|
<|im_start|>user
|
||||||
|
Company metadata (JSON):
|
||||||
|
{...}
|
||||||
|
|
||||||
|
Balance sheet (JSON):
|
||||||
|
{...}
|
||||||
|
|
||||||
|
P&L (JSON):
|
||||||
|
{...}
|
||||||
|
|
||||||
|
Cash flow (JSON):
|
||||||
|
{...}
|
||||||
|
|
||||||
|
Japanese annual-report text (truncated at ~20K chars):
|
||||||
|
<<<
|
||||||
|
{Japanese Yuho text}
|
||||||
|
>>>
|
||||||
|
|
||||||
|
Produce the two-page English investor memo now.<|im_end|>
|
||||||
|
<|im_start|>assistant
|
||||||
|
```
|
||||||
|
|
||||||
|
The full system prompt is published in the BF16 model card and in `src/yuholens/training/teacher.py` of the GitHub repo. A pre-formatted smoke fixture is included in the GitHub repo at `data/sample/smoke_prompt_chatml.txt`.
|
||||||
|
|
||||||
|
## Build provenance
|
||||||
|
|
||||||
|
Source checkpoint: `output/yuholens-14b-sft/checkpoint-212` (28.3 GB BF16 safetensors), tagged at git commit `f903174`.
|
||||||
|
|
||||||
|
Built with:
|
||||||
|
|
||||||
|
- `llama.cpp` commit `b8966` (`7b8443ac7`), Windows CUDA 12.4 prebuilt binaries.
|
||||||
|
- Convert script `convert_hf_to_gguf.py` patched locally to fall back to an inline GPT-2 byte-to-unicode mapping when transformers ≥ 5.x removes `bytes_to_unicode`. See the GitHub repo for the patch.
|
||||||
|
- `llama-quantize` invoked with `--override-kv qwen.attention.layer_norm_rms_epsilon=float:0.000001` because Qwen1 uses RMSNorm internally but its HF config exposes the field as `layer_norm_epsilon`; this override is now baked into `scripts/build_gguf.sh`.
|
||||||
|
|
||||||
|
Reproduce locally:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
LLAMACPP_REPO=/path/to/llama.cpp \
|
||||||
|
LLAMACPP_BIN=/path/to/llama.cpp/build/bin \
|
||||||
|
scripts/build_gguf.sh output/yuholens-14b-sft/checkpoint-212 data/eval/gguf
|
||||||
|
```
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- **Output language asymmetry.** The model emits English memos and expects Japanese Yuho input. Japanese output is unsupported by training distribution.
|
||||||
|
- **Citation accuracy unaudited.** The model produces inline `(ref: '<span>' p.N)` citations, but verbatim-correctness against the underlying Yuho text has not been audited.
|
||||||
|
- **Quantization quality bands.** Q3_K_M sacrifices some fidelity for the 8 GB VRAM target. Prefer Q4_K_M or higher when memory permits.
|
||||||
|
- **Domain.** Trained on Yuho 有価証券報告書 only. Earnings transcripts, 決算短信, and non-Japanese filings are out-of-scope.
|
||||||
|
- **Not financial advice.** Outputs are model-generated and may contain factual errors. Verify every material claim against the source.
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
Released under the Tongyi Qianwen License inherited from the Qwen1 base via `pfnet/nekomata-14b-pfn-qfin`. See the linked license for terms. Wrapper code in the GitHub repo (LangGraph pipeline, training and evaluation scripts, prompt modules) is MIT.
|
||||||
|
|
||||||
|
## Citation
|
||||||
|
|
||||||
|
```bibtex
|
||||||
|
@misc{dejesus2026yuholens,
|
||||||
|
author = {De Jesus, Javier},
|
||||||
|
title = {YuhoLens-14B: A Japanese-Finance Fine-Tune for
|
||||||
|
Span-Grounded Investor Memo Generation},
|
||||||
|
year = {2026},
|
||||||
|
howpublished = {Hugging Face model repository},
|
||||||
|
url = {https://huggingface.co/javierdejesusda/yuholens-14b},
|
||||||
|
note = {DOI: TBD}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Links
|
||||||
|
|
||||||
|
- BF16 reference: https://huggingface.co/javierdejesusda/yuholens-14b
|
||||||
|
- GitHub: https://github.com/javierdejesusda/YuhoLens
|
||||||
|
- Base model: https://huggingface.co/pfnet/nekomata-14b-pfn-qfin
|
||||||
|
- Training data: https://huggingface.co/datasets/SakanaAI/EDINET-Bench
|
||||||
3
yuholens-14b-Q3_K_M.gguf
Normal file
3
yuholens-14b-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:50cd8dfe5a9de12cf01766ca5ac69be3b8cefab895e6fc4fc1c5020fc338817e
|
||||||
|
size 7709891904
|
||||||
3
yuholens-14b-Q4_K_M.gguf
Normal file
3
yuholens-14b-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:bb99c3955ccb5b1d06008182c1605c9627212f5f8326292afc3b71c185990ff4
|
||||||
|
size 9461361984
|
||||||
3
yuholens-14b-Q5_K_M.gguf
Normal file
3
yuholens-14b-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2fb3bd1d656d60b48169564a807aaab0148b995f2c688e68f36b38c3fa2ebf40
|
||||||
|
size 10675252544
|
||||||
3
yuholens-14b-Q6_K.gguf
Normal file
3
yuholens-14b-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:20fcadf3c501ef14ab98314801e2962a10c302f1ab4825b4d55bf2a406ca293a
|
||||||
|
size 12310150464
|
||||||
3
yuholens-14b-Q8_0.gguf
Normal file
3
yuholens-14b-Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:92152f1d7f73c34112f37f7cfdea524bfc2b9d306ef21c052aa72f2bdaba98f4
|
||||||
|
size 15061720384
|
||||||
Reference in New Issue
Block a user