fumea-f-dense/README.md

---
license: apache-2.0
language:
- en
- tr
tags:
- merge
- dare-ties
- finance
- marketing
- forecasting
- tool-use
- mergekit
pipeline_tag: text-generation
new_version: uaytug/fumea-f-dense-v2
---

# FUMEA-F Dense: Frontier Unified Multi-Expert Agent — Financial (Dense)

FUMEA-F Dense is an 8-billion parameter language model created by merging four domain-specialized models using the DARE+TIES method. It consolidates financial analysis, marketing intelligence, market trend detection, and chain-of-thought financial reasoning capabilities into a single dense transformer — suitable for deployment on consumer hardware without MoE routing overhead.

## Architecture

| Property | Value |
|---|---|
| Parameters | ~8B |
| Architecture | Decoder-only Transformer (Dense) |
| Merge Method | DARE+TIES |
| Density | 0.75 |
| Normalization | Enabled |
| int8 Mask | Enabled |
| Context Window | 131,072 tokens |
| Positional Encoding | RoPE with YaRN scaling (factor 4.0, base 32,768) |
| Precision | bfloat16 |
| Vocabulary Size | 151,936 |

## Merge Configuration

Four 8B-parameter models were merged with the following weight distribution, biased toward financial domains:

| Expert Domain | Weight | Role |
|---|---|---|
| Marketing Intelligence | 0.30 (base) | Brand strategy, campaign analysis, market positioning |
| Financial Forecasting | 0.25 | Time-series prediction, technical indicators, quantitative modeling |
| Market Trends | 0.20 | E-commerce analytics, consumer behavior, competitive pricing |
| Financial Reasoning | 0.25 | Multi-step chain-of-thought, valuation logic, regulatory analysis |

The marketing model served as the base for the merge due to its broad domain coverage, providing the structural foundation onto which specialized financial and trend capabilities were integrated.

## Capabilities

**Financial Analysis**
- Fundamental metrics: EBITDA margin, P/E ratio, PEG ratio, DCF frameworks
- Technical analysis: candlestick interpretation, support/resistance identification
- Risk assessment: portfolio theory, diversification strategies, stress testing concepts

**Marketing Intelligence**
- Campaign performance analysis and ROI attribution
- Competitive landscape and positioning assessment
- Market sizing, segmentation, and go-to-market strategy

**Market Trend Detection**
- E-commerce pricing dynamics and conversion analysis
- Consumer behavior pattern recognition
- Emerging market opportunity identification

**Financial Reasoning**
- Step-by-step derivation of financial conclusions
- Multi-factor valuation with explicit reasoning chains
- Regulatory compliance analysis with structured argumentation

**Tool Use**
Supports structured tool calling via the chat template, including:

- `analyze_ohlcv` — OHLCV time-series analysis with configurable indicators
- `web_search` — External information retrieval
- `code_executor` — Python code execution
- `file_reader` — Structured file parsing

**Extended Context**
128K token context window for processing lengthy financial documents, earnings transcripts, and large datasets in a single pass.

## Usage

### Basic Inference

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "uaytug/fumea-f-dense",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("uaytug/fumea-f-dense", trust_remote_code=True)

messages = [
    {"role": "system", "content": "You are a financial analysis assistant."},
    {"role": "user", "content": "Explain the PEG ratio and when it is most useful for stock valuation."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

output = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
    repetition_penalty=1.1,
)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

### Tool Use

```python
tools = [
    {
        "name": "analyze_ohlcv",
        "description": "Analyze OHLCV time-series data for pattern recognition and trend detection",
        "parameters": {
            "type": "object",
            "properties": {
                "symbol": {"type": "string", "description": "Trading symbol"},
                "ohlcv_data": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "timestamp": {"type": "string"},
                            "open": {"type": "number"},
                            "high": {"type": "number"},
                            "low": {"type": "number"},
                            "close": {"type": "number"},
                            "volume": {"type": "number"}
                        }
                    }
                },
                "indicators": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "Technical indicators: RSI, MACD, BB, EMA_20, SMA_50"
                },
                "prediction_horizon": {"type": "integer"}
            },
            "required": ["symbol", "ohlcv_data"]
        }
    }
]

messages = [
    {"role": "user", "content": "Run a technical analysis on TSLA with RSI and Bollinger Bands."}
]

text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
```

## Quantized Versions

GGUF quantizations from F32 to IQ2_M are available at [uaytug/fumea-f-dense-gguf](https://huggingface.co/uaytug/fumea-f-dense-gguf) for use with Ollama, LM Studio, llama.cpp, and other GGUF-compatible runtimes.

| Quantization | Recommended Use |
|---|---|
| Q8_0 | Maximum quality, high-end hardware |
| Q5_K_M | Strong quality with moderate resource requirements |
| Q4_K_M | Best balance of quality and efficiency (recommended) |
| Q3_K_M | Reduced quality, constrained environments |
| IQ2_M  | Experimental, extreme compression |

## Hardware Requirements

| Configuration | Minimum VRAM / RAM |
|---|---|
| Full precision (bfloat16) | 16 GB VRAM |
| Q8_0 GGUF | 8.71 GB VRAM  |
| Q4_K_M GGUF | 5.03 GB VRAM |
| Q3_K_S GGUF | 3.77 GB VRAM |
| Q2_K GGUF | 3.28 GB VRAM  |

## Dense vs. MoE

This is the dense variant of the FUMEA-F family. Compared to the MoE version:

| Property | FUMEA-F Dense | FUMEA-F MoE |
|---|---|---|
| Parameters | ~8B | ~24B (~16B active) |
| Experts | 1 (unified) | 4 (top-2 routing) |
| Min VRAM (bf16) | 16 GB | 48 GB |
| Min VRAM (Q4_K_M) | 6 GB | 8 GB |
| Inference Speed | Faster (no routing) | Slower (expert selection overhead) |
| Specialization | Blended across all domains | Dynamic routing to domain experts |

Choose Dense for resource-constrained deployments or when uniform cross-domain performance is preferred. Choose MoE for maximum capability when hardware allows.

## Generation Defaults

| Parameter | Value |
|---|---|
| temperature | 0.6 |
| top_p | 0.9 |
| repetition_penalty | 1.1 |
| max_new_tokens | 8192 |

## Build Process

1. Four 8B-parameter domain-specialized models were merged using DARE+TIES with density 0.75 and normalization enabled
2. Weight distribution was biased toward financial domains (marketing 0.30, forecasting 0.25, trends 0.20, reasoning 0.25)
3. Context window was extended to 128K via YaRN RoPE scaling
4. Tool-use chat template was injected for structured function calling
5. GGUF quantization ladder was generated using llama.cpp with CUDA acceleration

## Related Models

- [uaytug/fumea-f-dense-gguf](https://huggingface.co/uaytug/fumea-f-dense-gguf) — Quantized GGUF versions of this model
- [uaytug/fumea-f](https://huggingface.co/uaytug/fumea-f) — 24B MoE version (4 experts, top-2 routing)
- [uaytug/fumea-f-gguf](https://huggingface.co/uaytug/fumea-f-gguf) — Quantized GGUF versions of the MoE model

## Limitations

- This is a merged model, not fine-tuned on curated financial datasets. Output quality reflects the combined capabilities of the source models.
- Financial predictions and analysis should not be used as sole basis for investment decisions.
- Tool-use capability depends on the inference framework supporting structured function calling.
- Extended context performance has not been systematically benchmarked beyond the base model's tested range.
- As a dense merge, domain-specific performance may be diluted compared to the individual specialist models or the MoE variant.

## License

Apache 2.0

## Citation

```bibtex
@misc{FUMEA-F-Dense,
  author = {Umut Aytuğ Semerci},
  title = {FUMEA-F Dense: Dense model of Financial Frontier Unified Multi-Expert Agent},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/uaytug/fumea-f-dense}
}
```