初始化项目，由ModelHub XC社区提供模型
Model: uaytug/fumea-f-dense Source: Original Platform
This commit is contained in:
ModelHub XC
244
README.md
Normal file
244
README.md
Normal file
@@ -0,0 +1,244 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
- tr
|
||||
tags:
|
||||
- merge
|
||||
- dare-ties
|
||||
- finance
|
||||
- marketing
|
||||
- forecasting
|
||||
- tool-use
|
||||
- mergekit
|
||||
pipeline_tag: text-generation
|
||||
new_version: uaytug/fumea-f-dense-v2
|
||||
---
|
||||
|
||||
# FUMEA-F Dense: Frontier Unified Multi-Expert Agent — Financial (Dense)
|
||||
|
||||
FUMEA-F Dense is an 8-billion parameter language model created by merging four domain-specialized models using the DARE+TIES method. It consolidates financial analysis, marketing intelligence, market trend detection, and chain-of-thought financial reasoning capabilities into a single dense transformer — suitable for deployment on consumer hardware without MoE routing overhead.
|
||||
|
||||
## Architecture
|
||||
|
||||
| Property | Value |
|
||||
|---|---|
|
||||
| Parameters | ~8B |
|
||||
| Architecture | Decoder-only Transformer (Dense) |
|
||||
| Merge Method | DARE+TIES |
|
||||
| Density | 0.75 |
|
||||
| Normalization | Enabled |
|
||||
| int8 Mask | Enabled |
|
||||
| Context Window | 131,072 tokens |
|
||||
| Positional Encoding | RoPE with YaRN scaling (factor 4.0, base 32,768) |
|
||||
| Precision | bfloat16 |
|
||||
| Vocabulary Size | 151,936 |
|
||||
|
||||
## Merge Configuration
|
||||
|
||||
Four 8B-parameter models were merged with the following weight distribution, biased toward financial domains:
|
||||
|
||||
| Expert Domain | Weight | Role |
|
||||
|---|---|---|
|
||||
| Marketing Intelligence | 0.30 (base) | Brand strategy, campaign analysis, market positioning |
|
||||
| Financial Forecasting | 0.25 | Time-series prediction, technical indicators, quantitative modeling |
|
||||
| Market Trends | 0.20 | E-commerce analytics, consumer behavior, competitive pricing |
|
||||
| Financial Reasoning | 0.25 | Multi-step chain-of-thought, valuation logic, regulatory analysis |
|
||||
|
||||
The marketing model served as the base for the merge due to its broad domain coverage, providing the structural foundation onto which specialized financial and trend capabilities were integrated.
|
||||
|
||||
## Capabilities
|
||||
|
||||
**Financial Analysis**
|
||||
- Fundamental metrics: EBITDA margin, P/E ratio, PEG ratio, DCF frameworks
|
||||
- Technical analysis: candlestick interpretation, support/resistance identification
|
||||
- Risk assessment: portfolio theory, diversification strategies, stress testing concepts
|
||||
|
||||
**Marketing Intelligence**
|
||||
- Campaign performance analysis and ROI attribution
|
||||
- Competitive landscape and positioning assessment
|
||||
- Market sizing, segmentation, and go-to-market strategy
|
||||
|
||||
**Market Trend Detection**
|
||||
- E-commerce pricing dynamics and conversion analysis
|
||||
- Consumer behavior pattern recognition
|
||||
- Emerging market opportunity identification
|
||||
|
||||
**Financial Reasoning**
|
||||
- Step-by-step derivation of financial conclusions
|
||||
- Multi-factor valuation with explicit reasoning chains
|
||||
- Regulatory compliance analysis with structured argumentation
|
||||
|
||||
**Tool Use**
|
||||
Supports structured tool calling via the chat template, including:
|
||||
|
||||
- `analyze_ohlcv` — OHLCV time-series analysis with configurable indicators
|
||||
- `web_search` — External information retrieval
|
||||
- `code_executor` — Python code execution
|
||||
- `file_reader` — Structured file parsing
|
||||
|
||||
**Extended Context**
|
||||
128K token context window for processing lengthy financial documents, earnings transcripts, and large datasets in a single pass.
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Inference
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
import torch
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
"uaytug/fumea-f-dense",
|
||||
torch_dtype=torch.bfloat16,
|
||||
device_map="auto",
|
||||
trust_remote_code=True,
|
||||
)
|
||||
tokenizer = AutoTokenizer.from_pretrained("uaytug/fumea-f-dense", trust_remote_code=True)
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "You are a financial analysis assistant."},
|
||||
{"role": "user", "content": "Explain the PEG ratio and when it is most useful for stock valuation."}
|
||||
]
|
||||
|
||||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||||
|
||||
output = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=1024,
|
||||
do_sample=True,
|
||||
temperature=0.6,
|
||||
top_p=0.9,
|
||||
repetition_penalty=1.1,
|
||||
)
|
||||
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
### Tool Use
|
||||
|
||||
```python
|
||||
tools = [
|
||||
{
|
||||
"name": "analyze_ohlcv",
|
||||
"description": "Analyze OHLCV time-series data for pattern recognition and trend detection",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"symbol": {"type": "string", "description": "Trading symbol"},
|
||||
"ohlcv_data": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"timestamp": {"type": "string"},
|
||||
"open": {"type": "number"},
|
||||
"high": {"type": "number"},
|
||||
"low": {"type": "number"},
|
||||
"close": {"type": "number"},
|
||||
"volume": {"type": "number"}
|
||||
}
|
||||
}
|
||||
},
|
||||
"indicators": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"description": "Technical indicators: RSI, MACD, BB, EMA_20, SMA_50"
|
||||
},
|
||||
"prediction_horizon": {"type": "integer"}
|
||||
},
|
||||
"required": ["symbol", "ohlcv_data"]
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
messages = [
|
||||
{"role": "user", "content": "Run a technical analysis on TSLA with RSI and Bollinger Bands."}
|
||||
]
|
||||
|
||||
text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
|
||||
```
|
||||
|
||||
## Quantized Versions
|
||||
|
||||
GGUF quantizations from F32 to IQ2_M are available at [uaytug/fumea-f-dense-gguf](https://huggingface.co/uaytug/fumea-f-dense-gguf) for use with Ollama, LM Studio, llama.cpp, and other GGUF-compatible runtimes.
|
||||
|
||||
| Quantization | Recommended Use |
|
||||
|---|---|
|
||||
| Q8_0 | Maximum quality, high-end hardware |
|
||||
| Q5_K_M | Strong quality with moderate resource requirements |
|
||||
| Q4_K_M | Best balance of quality and efficiency (recommended) |
|
||||
| Q3_K_M | Reduced quality, constrained environments |
|
||||
| IQ2_M | Experimental, extreme compression |
|
||||
|
||||
## Hardware Requirements
|
||||
|
||||
| Configuration | Minimum VRAM / RAM |
|
||||
|---|---|
|
||||
| Full precision (bfloat16) | 16 GB VRAM |
|
||||
| Q8_0 GGUF | 8.71 GB VRAM |
|
||||
| Q4_K_M GGUF | 5.03 GB VRAM |
|
||||
| Q3_K_S GGUF | 3.77 GB VRAM |
|
||||
| Q2_K GGUF | 3.28 GB VRAM |
|
||||
|
||||
## Dense vs. MoE
|
||||
|
||||
This is the dense variant of the FUMEA-F family. Compared to the MoE version:
|
||||
|
||||
| Property | FUMEA-F Dense | FUMEA-F MoE |
|
||||
|---|---|---|
|
||||
| Parameters | ~8B | ~24B (~16B active) |
|
||||
| Experts | 1 (unified) | 4 (top-2 routing) |
|
||||
| Min VRAM (bf16) | 16 GB | 48 GB |
|
||||
| Min VRAM (Q4_K_M) | 6 GB | 8 GB |
|
||||
| Inference Speed | Faster (no routing) | Slower (expert selection overhead) |
|
||||
| Specialization | Blended across all domains | Dynamic routing to domain experts |
|
||||
|
||||
Choose Dense for resource-constrained deployments or when uniform cross-domain performance is preferred. Choose MoE for maximum capability when hardware allows.
|
||||
|
||||
## Generation Defaults
|
||||
|
||||
| Parameter | Value |
|
||||
|---|---|
|
||||
| temperature | 0.6 |
|
||||
| top_p | 0.9 |
|
||||
| repetition_penalty | 1.1 |
|
||||
| max_new_tokens | 8192 |
|
||||
|
||||
## Build Process
|
||||
|
||||
1. Four 8B-parameter domain-specialized models were merged using DARE+TIES with density 0.75 and normalization enabled
|
||||
2. Weight distribution was biased toward financial domains (marketing 0.30, forecasting 0.25, trends 0.20, reasoning 0.25)
|
||||
3. Context window was extended to 128K via YaRN RoPE scaling
|
||||
4. Tool-use chat template was injected for structured function calling
|
||||
5. GGUF quantization ladder was generated using llama.cpp with CUDA acceleration
|
||||
|
||||
## Related Models
|
||||
|
||||
- [uaytug/fumea-f-dense-gguf](https://huggingface.co/uaytug/fumea-f-dense-gguf) — Quantized GGUF versions of this model
|
||||
- [uaytug/fumea-f](https://huggingface.co/uaytug/fumea-f) — 24B MoE version (4 experts, top-2 routing)
|
||||
- [uaytug/fumea-f-gguf](https://huggingface.co/uaytug/fumea-f-gguf) — Quantized GGUF versions of the MoE model
|
||||
|
||||
## Limitations
|
||||
|
||||
- This is a merged model, not fine-tuned on curated financial datasets. Output quality reflects the combined capabilities of the source models.
|
||||
- Financial predictions and analysis should not be used as sole basis for investment decisions.
|
||||
- Tool-use capability depends on the inference framework supporting structured function calling.
|
||||
- Extended context performance has not been systematically benchmarked beyond the base model's tested range.
|
||||
- As a dense merge, domain-specific performance may be diluted compared to the individual specialist models or the MoE variant.
|
||||
|
||||
## License
|
||||
|
||||
Apache 2.0
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@misc{FUMEA-F-Dense,
|
||||
author = {Umut Aytuğ Semerci},
|
||||
title = {FUMEA-F Dense: Dense model of Financial Frontier Unified Multi-Expert Agent},
|
||||
year = {2026},
|
||||
publisher = {Hugging Face},
|
||||
url = {https://huggingface.co/uaytug/fumea-f-dense}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user