Qwen2.5-7B-3GPP-NR/README.md

license, base_model, tags, language, datasets, library_name, pipeline_tag

license	base_model	tags	language	datasets	library_name	pipeline_tag
cc-by-nc-4.0	Qwen/Qwen2.5-7B-Instruct	3gpp 5g telecommunications new-radio nr technical-qa fine-tuned lora	en	raoulbia/3gpp-5g-nr-qa	transformers	text-generation

Qwen2.5-7B-3GPP-NR

A fine-tuned version of Qwen2.5-7B-Instruct specialized for answering technical questions about 3GPP 5G New Radio (NR) specifications.

Model Description

This model was fine-tuned on ~27,000 Q&A pairs derived from 3GPP 38-series technical specifications. It excels at answering detailed technical questions about 5G NR protocols, procedures, and configurations.

Model Details

Attribute	Value
Base Model	Qwen2.5-7B-Instruct
Fine-tuning Method	LoRA (merged)
LoRA Rank	32
LoRA Alpha	64
Training Data	26,926 Q&A pairs
Training Epochs	2
License	CC-BY-NC-4.0

Intended Use

• Answering technical questions about 3GPP 5G NR specifications
• Assisting telecom engineers with specification lookups
• Research and educational purposes
• Building domain-specific chatbots for telecommunications

How to Use

With Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "raoulbia/Qwen2.5-7B-3GPP-NR"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

messages = [
    {"role": "system", "content": "You are a 3GPP technical specifications expert."},
    {"role": "user", "content": "What are the requirements for antenna configuration during FR1 testing?"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2, top_p=0.8)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

With Ollama (GGUF)

A GGUF version is available for use with Ollama. Create a Modelfile:

FROM path/to/model.gguf

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>"""

SYSTEM "You are a 3GPP technical specifications expert."
PARAMETER temperature 0.2
PARAMETER stop "<|im_end|>"

Training Details

Training Data

Fine-tuned on raoulbia/3gpp-5g-nr-qa, a dataset of Q&A pairs generated from 3GPP 38-series specifications covering:

• TS 38.211-214: Physical layer procedures
• TS 38.101, 38.104: RF requirements
• TS 38.133, 38.521, 38.533: Conformance testing
• TS 38.331: RRC protocol
• TS 38.300, 38.401: Architecture

Training Configuration

• Framework: NVIDIA NeMo 25.09
• Method: LoRA (Low-Rank Adaptation)
• Epochs: 2
• Batch Size: 1 (with gradient accumulation)
• Learning Rate: 5e-6
• Sequence Length: 2048
• Hardware: NVIDIA DGX Spark

Evaluation

Evaluated on 50 samples from a held-out test set:

Metric	Value
Valid Responses	100%
Avg Response Length	~800 characters
Response Length vs Base	-28% (more concise)

Strengths

• Concise, focused answers
• Correct 3GPP terminology (RRC, E-UTRA, ENDC, SRS, NG-RAN)
• Release-specific references (Rel-15, Rel-16, Rel-17)
• Clean stopping behavior

Limitations

• Scope: Only trained on 38-series (5G NR), not full 3GPP stack
• General Knowledge: May struggle with definitional questions like "What is 3GPP?"
• Verification: LLM-generated training data, not manually verified
• Domain Bias: Strong bias toward technical spec patterns

Example Outputs

Question: What are the requirements for antenna configuration during FR1 testing for UEs with two or four Rx antennas?

Answer: During FR1 testing, if a User Equipment (UE) is equipped with either two or four receive (Rx) antennas, the same downlink signal must be applied to each antenna. It is essential that all receive antennas of the UE are connected unless the specific test case indicates otherwise. This configuration ensures that the testing accurately reflects the performance of the UE under typical operational conditions.

Citation

If you use this model, please cite the source dataset:

@misc{nikbakht2024tspecllm,
  title={TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specifications},
  author={Rasoul Nikbakht and Mohamed Benzaghta and Giovanni Geraci},
  year={2024},
  eprint={2406.01768},
  archivePrefix={arXiv},
  primaryClass={cs.NI}
}

License

This model is released under CC-BY-NC-4.0 (Creative Commons Attribution-NonCommercial 4.0).

• Non-commercial use only
• Attribution required
• Derived from TSpec-LLM dataset

Acknowledgments

• Qwen Team for the base model
• TSpec-LLM for the source data
• 3GPP for the original specifications