初始化项目,由ModelHub XC社区提供模型
Model: ahammad115566/qwen-smeft Source: Original Platform
This commit is contained in:
192
README.md
Normal file
192
README.md
Normal file
@@ -0,0 +1,192 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
tags:
|
||||
- large-language-model
|
||||
- qwen3
|
||||
qwen:
|
||||
- physics
|
||||
smeft:
|
||||
- effective-field-theory
|
||||
high-energy-physics:
|
||||
- scientific-llm
|
||||
pipeline_tag: text-generation
|
||||
library_name: transformers
|
||||
base_model: Qwen/Qwen3-8B
|
||||
model_type: qwen3
|
||||
do_sample: true
|
||||
temperature: 0.1
|
||||
top_p: 0.1
|
||||
repetition_penalty: 1.1
|
||||
max_new_tokens: 512
|
||||
---
|
||||
|
||||
<p align="center">
|
||||
<img src="logo.png" width="350">
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
|
||||
A domain-adapted large language model for
|
||||
<b>Standard Model Effective Field Theory (SMEFT)</b>.
|
||||
|
||||
</p>
|
||||
|
||||
# Capabilities
|
||||
|
||||
The model has been optimized for:
|
||||
|
||||
- SMEFT operator reasoning
|
||||
|
||||
- EFT basis translation
|
||||
|
||||
- Physics-aware scientific dialogue
|
||||
|
||||
- Literature-style technical explanation
|
||||
|
||||
- Structured theoretical question answering
|
||||
|
||||
This model is fine-tuned from **Qwen3-8B** using a curated corpus of SMEFT and particle physics literature.
|
||||
|
||||
> **Note on Qwen3 thinking mode:** Qwen3 supports a native `enable_thinking` toggle in its chat template that wraps reasoning in `<think>...</think>` blocks. This model was tuned to produce reasoning through its own structured prompt format (not Qwen3's native thinking blocks), so if you build prompts via `tokenizer.apply_chat_template(...)`, set `enable_thinking=False` to avoid mixing the two reasoning styles. The example below uses a raw instruction-style prompt and is unaffected either way.
|
||||
|
||||
# Quick Start
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install torch transformers bitsandbytes accelerate
|
||||
```
|
||||
## Inference Example
|
||||
|
||||
```python
|
||||
import torch
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
|
||||
|
||||
MODEL_NAME = "ahammad115566/qwen-smeft"
|
||||
RESPONSE_PREFIX = "\n### Response:\n"
|
||||
|
||||
# 4-bit quantisation
|
||||
bnb_config = BitsAndBytesConfig(
|
||||
load_in_4bit=True,
|
||||
bnb_4bit_quant_type="nf4",
|
||||
bnb_4bit_compute_dtype=torch.bfloat16,
|
||||
bnb_4bit_use_double_quant=True,
|
||||
)
|
||||
|
||||
# Load model and tokenizer
|
||||
tokenizer = AutoTokenizer.from_pretrained(
|
||||
MODEL_NAME,
|
||||
trust_remote_code=True,
|
||||
)
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
MODEL_NAME,
|
||||
quantization_config=bnb_config,
|
||||
device_map={"": 0},
|
||||
trust_remote_code=True,
|
||||
)
|
||||
|
||||
model.eval()
|
||||
|
||||
if tokenizer.pad_token is None:
|
||||
tokenizer.pad_token = tokenizer.eos_token
|
||||
|
||||
|
||||
# ================= Inference ================
|
||||
|
||||
def build_prompt(instruction: str) -> str:
|
||||
"""Construct the prompt using the format used during fine-tuning."""
|
||||
return f"\n### Instruction:\n{instruction}\n{RESPONSE_PREFIX}"
|
||||
|
||||
|
||||
@torch.inference_mode()
|
||||
def ask(instruction: str) -> str:
|
||||
"""Generate a response to an SMEFT-related instruction."""
|
||||
prompt = build_prompt(instruction)
|
||||
|
||||
inputs = tokenizer(
|
||||
prompt,
|
||||
return_tensors="pt",
|
||||
add_special_tokens=True,
|
||||
).to(model.device)
|
||||
|
||||
output_ids = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=2048,
|
||||
do_sample=False,
|
||||
repetition_penalty=1.1,
|
||||
eos_token_id=tokenizer.eos_token_id,
|
||||
pad_token_id=tokenizer.pad_token_id,
|
||||
)
|
||||
|
||||
new_tokens = output_ids[0][inputs["input_ids"].shape[-1]:]
|
||||
|
||||
return tokenizer.decode(
|
||||
new_tokens,
|
||||
skip_special_tokens=True,
|
||||
).strip()
|
||||
|
||||
|
||||
# Example
|
||||
instruction = """
|
||||
Which SMEFT operators modify EWPO?
|
||||
"""
|
||||
|
||||
response = ask(instruction)
|
||||
print(response)
|
||||
```
|
||||
---
|
||||
## Training Details
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| **Base Model** | Qwen3 |
|
||||
| **Fine-tuning Method** | LoRA (merged) |
|
||||
| **Inference Quantization** | 4-bit NF4 (bitsandbytes) |
|
||||
| **Domain** | Standard Model Effective Field Theory (SMEFT) |
|
||||
| **Training Corpus** | Curated SMEFT and HEP preprints |
|
||||
| **Task Format** | Instruction-following scientific QA |
|
||||
---
|
||||
|
||||
# About the model
|
||||
|
||||
- May hallucinate operator identities.
|
||||
|
||||
- Domain-locked by design. The model is not suitable for general-purpose tasks.
|
||||
|
||||
- 3620 training examples. Coverage of the SMEFT operator space may be uneven; rare operators or non-Warsaw bases may be answered less reliably.
|
||||
|
||||
- 20% out of domain exmaples are included during the training.
|
||||
|
||||
|
||||
---
|
||||
# Authors
|
||||
|
||||
**Ahmed Hammad**
|
||||
Assistant professor
|
||||
|
||||
Center of AI and natural science, KIAS, Seoul.
|
||||
|
||||
**Veronica Sanz**
|
||||
Professor of Theoretical Physics
|
||||
|
||||
University of Valencia
|
||||
|
||||
# Citation
|
||||
|
||||
A technical paper describing the dataset construction and fine-tuning procedure is forthcoming.
|
||||
|
||||
Please cite the model as:
|
||||
```bibtex
|
||||
@article{Hammad:2026bvw,
|
||||
author = "Hammad, Ahmed and Sanz, Veronica",
|
||||
title = "{Language-Guided Hypotheses Generation for Sparse SMEFT Analyses}",
|
||||
eprint = "2608.04100",
|
||||
archivePrefix = "arXiv",
|
||||
primaryClass = "hep-ph",
|
||||
month = "8",
|
||||
year = "2026"
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user