初始化项目，由ModelHub XC社区提供模型

Model: igmoiiz/ResearchMate-Qwen2.5-3B Source: Original Platform
2026-08-23 13:53:16 +08:00
commit 0728770939
10 changed files with 1239 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,422 @@
+---
+language:
+- en
+
+license: apache-2.0
+
+base_model:
+- Qwen/Qwen2.5-3B-Instruct
+
+tags:
+- research
+- scientific-literature
+- question-answering
+- summarization
+- instruction-tuning
+- qlora
+- unsloth
+- peft
+- lora
+- qwen2.5
+- arxiv
+- pubmed
+- academic-assistant
+
+library_name: transformers
+
+pipeline_tag: text-generation
+
+datasets:
+- qiaojin/PubMedQA
+- arXiv
+
+model-index:
+- name: ResearchMate-Qwen2.5-3B
+  results: []
+
+---
+
+# ResearchMate-Qwen2.5-3B
+
+ResearchMate-Qwen2.5-3B is a domain-specialized Large Language Model designed to assist researchers, students, and practitioners with scientific literature.
+
+The model is instruction fine-tuned from **Qwen2.5-3B-Instruct** using **QLoRA** and **Unsloth**. Instead of functioning as a general-purpose chatbot, ResearchMate focuses on understanding scientific papers and responding to research-oriented instructions.
+
+This release represents **Version 1** of the ResearchMate project.
+
+---
+
+# Model Overview
+
+| Property | Value |
+|-----------|-------|
+| Model | Qwen2.5-3B-Instruct |
+| Fine-tuning | QLoRA |
+| Framework | Unsloth |
+| Parameter Count | 3B |
+| Quantization | 4-bit |
+| PEFT | LoRA |
+| Primary Domain | Scientific Literature |
+| Language | English |
+| Version | 1.0 |
+
+---
+
+# Project Goal
+
+ResearchMate aims to provide a lightweight, open-source scientific assistant capable of understanding research papers and responding to academic instructions.
+
+The objective is not to replace Retrieval-Augmented Generation systems or search engines, but to improve a language model's understanding of scientific writing through supervised instruction fine-tuning.
+
+Version 1 focuses on instruction tuning without retrieval.
+
+---
+
+# Supported Tasks
+
+ResearchMate has been instruction-tuned for several scientific literature tasks including:
+
+- Scientific Question Answering
+- Paper Summarization
+- Abstract Explanation
+- Beginner-Friendly Concept Explanation
+- Keyword Extraction
+- Research Field Identification
+- Scientific TL;DR Generation
+- Contribution Identification
+- Method Identification
+
+---
+
+# Training Method
+
+The model was fine-tuned using:
+
+- Unsloth
+- QLoRA
+- PEFT LoRA
+- 4-bit Quantization
+
+Training was performed on Kaggle GPUs to reduce memory requirements while maintaining strong instruction-following capabilities.
+
+---
+
+# Dataset Construction
+
+ResearchMate does **not** train directly on raw datasets.
+
+Instead, a dedicated dataset-building pipeline converts multiple scientific sources into a unified instruction dataset.
+
+The preprocessing pipeline is independent of model training.
+
+Dataset pipeline:
+
+```
+Scientific Dataset
+
+↓
+
+Validation
+
+↓
+
+Schema Conversion
+
+↓
+
+Cleaning
+
+↓
+
+Merge
+
+↓
+
+JSONL Export
+
+↓
+
+Training
+```
+
+---
+
+# Dataset Schema
+
+Every training example follows the same structure:
+
+```json
+{
+    "instruction": "...",
+    "input": "...",
+    "output": "...",
+    "source": "...",
+    "task": "..."
+}
+```
+
+This unified schema allows new scientific datasets to be added without modifying the training pipeline.
+
+---
+
+# Data Sources
+
+Version 1 uses the following sources:
+
+## PubMedQA
+
+Purpose:
+
+- Scientific Question Answering
+
+Configuration:
+
+```
+pqa_labeled
+```
+
+Fields used:
+
+- Question
+- Context
+- Long Answer
+
+---
+
+## arXiv
+
+Scientific papers collected through the official arXiv API.
+
+Paper metadata including titles, abstracts and categories were converted into instruction-response pairs for research-oriented tasks.
+
+---
+
+# Training Configuration
+
+| Parameter | Value |
+|-----------|-------|
+| Fine-tuning Method | QLoRA |
+| PEFT | LoRA |
+| Quantization | 4-bit |
+| Framework | Unsloth |
+| Optimizer | AdamW |
+| Epochs | 2 |
+| Batch Size | 2 |
+| Gradient Accumulation | 4 |
+| Sequence Length | 2048 |
+
+---
+
+# Evaluation
+
+The model was evaluated against the original Qwen2.5-3B-Instruct model.
+
+Evaluation included:
+
+- Scientific QA
+- Response quality inspection
+- ROUGE (where applicable)
+- Latency comparison
+- Hallucination inspection
+- Task-wise qualitative comparison
+
+---
+
+# Example Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "igmoiiz/ResearchMate-Qwen2.5-3B"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    device_map="auto"
+)
+
+messages = [
+    {
+        "role": "user",
+        "content": "Explain transfer learning in simple language."
+    }
+]
+
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=256
+)
+
+print(
+    tokenizer.decode(
+        outputs[0][inputs["input_ids"].shape[1]:],
+        skip_special_tokens=True
+    )
+)
+```
+
+---
+
+# Example Prompts
+
+### Scientific Question Answering
+
+```
+Answer the scientific question using the provided context.
+
+Question:
+What is transfer learning?
+```
+
+---
+
+### Summarization
+
+```
+Summarize the following scientific abstract.
+```
+
+---
+
+### Beginner Explanation
+
+```
+Explain this abstract in simple language.
+```
+
+---
+
+### Keyword Extraction
+
+```
+Extract the important keywords from this paper.
+```
+
+---
+
+### Research Field Identification
+
+```
+Identify the research field of this paper.
+```
+
+---
+
+# Intended Uses
+
+Suitable for:
+
+- Research assistants
+- Literature exploration
+- Educational tools
+- Scientific tutoring
+- Academic chatbots
+- Research paper preprocessing
+- Scientific writing assistance
+
+---
+
+# Limitations
+
+ResearchMate Version 1 has several limitations.
+
+- No Retrieval-Augmented Generation (RAG)
+- No citation verification
+- No PDF parsing
+- No web search
+- No factual verification beyond the model's learned parameters
+- Performance depends on the quality and diversity of the instruction dataset
+- May generate incorrect or outdated scientific information
+
+Users should verify important scientific claims using authoritative sources.
+
+---
+
+# Future Work
+
+Planned improvements include:
+
+- Retrieval-Augmented Generation (RAG)
+- Citation-aware responses
+- PDF ingestion
+- Larger scientific datasets
+- Multi-turn research conversations
+- Paper recommendation
+- Local deployment using Ollama and vLLM
+- Improved evaluation benchmarks
+- Expanded scientific domains
+
+---
+
+# Repository Structure
+
+```
+Notebook 1
+Dataset Builder
+
+↓
+
+Notebook 2
+QLoRA Fine-tuning
+
+↓
+
+Notebook 3
+Inference
+
+↓
+
+Notebook 4
+Evaluation
+```
+
+---
+
+# Citation
+
+If you use this model in your work, please cite the repository.
+
+```
+@software{researchmate2026,
+  title={ResearchMate-Qwen2.5-3B},
+  author={Moiz Baloch},
+  year={2026},
+  publisher={Hugging Face},
+  url={https://huggingface.co/igmoiiz/ResearchMate-Qwen2.5-3B}
+}
+```
+
+---
+
+# Acknowledgements
+
+This project builds upon the work of:
+
+- Alibaba Qwen Team
+- Unsloth AI
+- Hugging Face
+- PubMedQA
+- arXiv
+
+Their open-source contributions made this project possible.
+
+---
+
+# Contact
+
+Author: **Moiz Baloch**
+
+GitHub: https://github.com/igmoiiz
+
+Hugging Face: https://huggingface.co/igmoiiz
+
+---