初始化项目，由ModelHub XC社区提供模型
Model: Italianhype/Blum-Finance-4B Source: Original Platform
This commit is contained in:
ModelHub XC
164
README.md
Normal file
164
README.md
Normal file
@@ -0,0 +1,164 @@
|
||||
---
|
||||
language:
|
||||
- en
|
||||
- it
|
||||
license: apache-2.0
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
base_model: Qwen/Qwen3-4B
|
||||
datasets:
|
||||
- Italianhype/Blum-Finance-Reasoning
|
||||
tags:
|
||||
- finance
|
||||
- financial-reasoning
|
||||
- investment-research
|
||||
- risk-management
|
||||
- explainable-ai
|
||||
- qwen3
|
||||
- transformers
|
||||
model-index:
|
||||
- name: BLUM Finance 4B
|
||||
results:
|
||||
- task:
|
||||
type: text-generation
|
||||
name: Multiple-choice financial reasoning
|
||||
dataset:
|
||||
name: MMLU Finance and Business (stratified 100 per subject)
|
||||
type: cais/mmlu
|
||||
config: finance_business_7_subjects
|
||||
split: test
|
||||
metrics:
|
||||
- type: accuracy
|
||||
value: 0.74857143
|
||||
---
|
||||
|
||||
# BLUM Finance 4B — Transformers Benchmark Release
|
||||
|
||||
BLUM Finance 4B is an open, evidence-bound financial reasoning model. This
|
||||
repository is the portable BF16 Transformers release used for public benchmark
|
||||
submission. It contains the same trained LoRA delta as the audited MLX release,
|
||||
mapped and fused into the exact `Qwen/Qwen3-4B` base architecture.
|
||||
|
||||
## Intended Use
|
||||
|
||||
The model structures supplied point-in-time evidence into:
|
||||
|
||||
- balanced bull and bear cases;
|
||||
- explicit risks and invalidation conditions;
|
||||
- conservative confidence;
|
||||
- monitoring conditions and abstention.
|
||||
|
||||
It is not a price oracle, broker, trade executor or proof of market alpha.
|
||||
|
||||
## Load
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_id = "Italianhype/Blum-Finance-4B"
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id,
|
||||
dtype=torch.bfloat16,
|
||||
device_map="auto",
|
||||
)
|
||||
```
|
||||
|
||||
No `trust_remote_code=True` is required.
|
||||
|
||||
## Governed Continual Memory
|
||||
|
||||
This repository also ships the installable `blum-finance` package. It provides
|
||||
schema-validated inference, auditable local memory and explicit opt-in community
|
||||
contributions:
|
||||
|
||||
```bash
|
||||
pip install "git+https://huggingface.co/Italianhype/Blum-Finance-4B"
|
||||
blum-contribute example.json --output contribution.json --consent
|
||||
blum-memory-add contribution.json
|
||||
```
|
||||
|
||||
Local retrieval accepts only mature, source-verified outcomes observable before
|
||||
the new request timestamp. Memories are labeled historical analogies and cannot
|
||||
replace current evidence. Adding `--push` opens a pull request to the quarantine
|
||||
dataset; inference sends no telemetry and anonymous input never mutates released
|
||||
weights. See `CONTRIBUTING.md` for the evidence contract.
|
||||
|
||||
## Lineage
|
||||
|
||||
- Base: `Qwen/Qwen3-4B`
|
||||
- Base revision: `1cfa9a7208912126459214e8b04321603b3df60c`
|
||||
- Dataset: `Italianhype/Blum-Finance-Reasoning`
|
||||
- Dataset revision: `76ad77699d498fc930daf02e452fe3ec8b490f90`
|
||||
- MLX adapter revision: `ea297ba88ab008e97104b0c118103eef2f8f9ec1`
|
||||
- LoRA rank: `8`
|
||||
- MLX scale: `20`
|
||||
- Equivalent PEFT alpha: `160`
|
||||
- Adapted layers: `20–35`
|
||||
|
||||
The conversion transposes MLX A/B matrices into PEFT orientation and preserves
|
||||
the update `20 × Bᵀ × Aᵀ`. All 112 expected modules were fused; none were
|
||||
missing. The PEFT adapter and merged model produced identical deterministic
|
||||
smoke-test output.
|
||||
|
||||
## Evaluation Status
|
||||
|
||||
The original MLX release passed the 53-example BLUM temporal reasoning test with
|
||||
96.26% aggregate task-contract score, 100% structured validity and 92.86%
|
||||
no-fabrication. These metrics measure BLUM schema adherence and grounding, not
|
||||
general intelligence or trading performance.
|
||||
|
||||
The portable release is published at immutable revision
|
||||
`ad6f5cec7f729370d2976d8c78983521cb37ca83` and tagged
|
||||
`benchmark-submission-v1`.
|
||||
|
||||
The five-example portable conversion smoke test scored 97.32% on the internal
|
||||
task contract, including 100% structured validity and 92.86% no-fabrication.
|
||||
The sample is too small for a robust capability claim.
|
||||
|
||||
Automated submissions were attempted for the Hugging Face Open LLM Leaderboard
|
||||
and the FinOS Open Financial LLM Leaderboard. Both external validators rejected
|
||||
Qwen3 as requiring `trust_remote_code=True`; Transformers 4.57.6 loads this
|
||||
repository without remote code. No official leaderboard score is claimed.
|
||||
|
||||
A deterministic stratified community evaluation on seven finance/business
|
||||
subjects of canonical `cais/mmlu` scored **74.86% accuracy** over 700 test
|
||||
questions (100 per subject), with a Wilson 95% confidence interval of
|
||||
71.51–77.93%. It used five-shot answer-token logit scoring.
|
||||
|
||||
| Subject | Accuracy | Samples |
|
||||
|---|---:|---:|
|
||||
| Business ethics | 77% | 100 |
|
||||
| Econometrics | 65% | 100 |
|
||||
| High-school macroeconomics | 70% | 100 |
|
||||
| High-school microeconomics | 87% | 100 |
|
||||
| Management | 87% | 100 |
|
||||
| Marketing | 92% | 100 |
|
||||
| Professional accounting | 46% | 100 |
|
||||
|
||||
This is an author-run community evaluation, not the full canonical MMLU suite
|
||||
and not an official leaderboard result. The deterministic 100-row cap gives
|
||||
each subject equal weight but can differ from a full-split result. Dataset
|
||||
revision: `c30699e8356da336a370243923dbaf21066bb9fe`.
|
||||
|
||||
[Evaluation summary](evaluations/mmlu-finance-stratified-100/results.json) ·
|
||||
[Per-example predictions](evaluations/mmlu-finance-stratified-100/predictions.jsonl) ·
|
||||
[Evaluator](evaluations/evaluate_mmlu_finance.py)
|
||||
|
||||
Independent evaluation packages are prepared for Vals AI CorpFin/Finance Agent
|
||||
and Scale Labs PRBench Finance. Those scores remain unavailable until the
|
||||
benchmark owners evaluate the immutable model revision.
|
||||
|
||||
## Known Failure
|
||||
|
||||
A sparse-evidence smoke prompt incorrectly identified NVDA as Applied Materials.
|
||||
This is a documented fabrication failure. Callers must validate company identity
|
||||
and all market facts against BLUM Engine or another authoritative data source.
|
||||
|
||||
## Safety
|
||||
|
||||
- Do not use this model as unattended financial advice.
|
||||
- Do not let it authorize or execute transactions.
|
||||
- Supply timestamped evidence and validate every factual statement.
|
||||
- Confidence is bounded but not calibrated on mature trading outcomes.
|
||||
Reference in New Issue
Block a user