LLama3 SatCom 8B is a fine-tuned open Large Language Model (LLM) developed under the ESA ARTES programme as part of the SatcomLLM / SCEVA (SatCom Expert Virtual Assistant) project.
It is designed to support satellite communications (SatCom) experts, engineers, and mission planners through domain-specialised reasoning, question answering, and document-based assistance.
Model Description
Base model:meta-llama/Llama-3.1-8B-Instruct
Fine-tuning type: Instruction fine-tuning (IFT)
Training data: Domain-specific question–answer datasets (manual, synthetic, and multiple-choice)
The model has been fine-tuned on curated SatCom-related corpora to enhance its understanding of technical language, protocols, and reasoning processes common to satellite communications, including 5G/6G non-terrestrial networks, link budget evaluation, and mission engineering tasks.
Training Datasets
Dataset
Description
esa-sceva/satcom-synth-qa
Synthetic QA data generated via agentic pipelines using large teacher models
esa-sceva/satcom-synth-qa-cot
Chain-of-thought annotated QA used to improve reasoning depth and factual traceability
Intended Use
Primary use cases:
Technical Q&A and reasoning on SatCom systems
Support for link budget and RF engineering questions
Guidance for 5G/6G NTN (Non-Terrestrial Network) operations
Mission design, planning, and anomaly detection support
Educational and research use within the SatCom sector
Intended users:
ESA engineers and project officers
SatCom and aerospace researchers
SMEs and technical operators in satellite communication
Academic and educational users
Limitations
The model does not access real-time mission data or proprietary ESA documents.
Answers are based on training data and may require expert validation for operational use.
It should not be relied upon for flight-critical or safety-critical decisions.
Limited context window (base 8B configuration) may constrain long-document reasoning.
Technical Details
Parameter
Value
Base Model
Llama 3.1 8B Instruct
Parameters
8 billion
Context length
8k tokens
Precision
bfloat16 / fp16
Framework
Lit-GPT (Lightning AI)
Training infra
EuroHPC MareNostrum5 + AWS EC2
Optimisation
LoRA fine-tuning, cosine LR schedule
Evaluation
Evaluation Datasets
The model was benchmarked on both general-purpose and domain-specific QA tasks. Regarding Satcom-specific datasets:
Dataset
Subset
Description
esa-sceva/satcom-qa
Open SatCom QA
Conceptual and reasoning-based questions on SatCom workflows, regulations, and mission/system design
Math SatCom QA
Quantitative and formula-based questions derived from system design and orbital mechanics topics
esa-sceva/satcom-mcqa
Open MCQA
Conceptual multiple-choice questions on RF systems, communication protocols, and architecture
Math MCQA
Numerical and link-budget-focused multiple-choice questions testing applied calculations
Results
Model
MCQA (Accuracy)
Satcom-QA
EVE-QA
Satcom
EVE
TeleQnA
Norm.
Bin.
WR
Norm.
Bin.
WR
Llama-3.1-8B-Instruct
78.59
81.35
68.40
75.44
70.40
—
61.47
65.20
—
llama3-satcom-8b
80.15
80.95
68.80
77.75
73.62
51.49
62.90
68.12
51.41
Table: Evaluation results (%).
Norm. denotes the normalized score obtained by averaging 1–5 ratings from a panel of LLM judges (Qwen3, gpt-4.1-mini, Mistral-Large-2512, and DeepSeek-V3.2) and scaling to [0,1].
Bin. is the binary accuracy computed from correctness judgments.
WR (Adjusted Win Rate) is defined as (wins + 0.5 × ties) / total, based on pairwise comparisons with randomized answer order.
Multiple-choice performance is measured using standard accuracy.
All results are averaged over 3–5 runs; standard deviation ≤ 0.25 pp for open-ended QA and ≤ 0.10 pp for MCQA.
