初始化项目，由ModelHub XC社区提供模型

Model: BAAI_Industry_Competition_tourism_dev/scoreing_model Source: Original Platform
2026-06-07 13:23:14 +08:00
commit 2d01800bb3
26 changed files with 521408 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,183 @@
+# 训练和推理代码  
+## 训练代码  
+训练是在LLaMA-Factory框架下进行的Lora SFT微调。训练指令如下：  
+`llamafactory-cli train examples/train_lora/qwen2.5_lora_sft.yaml`  
+训练超参数如下：  
+```  
+### model
+model_name_or_path: Qwen/Qwen2.5-7B-Instruct
+
+### method
+stage: sft
+do_train: true
+finetuning_type: lora
+lora_target: all
+
+
+### dataset
+dataset: pingfen
+template: qwen
+cutoff_len: 8000
+max_samples: 100000000
+overwrite_cache: true
+preprocessing_num_workers: 4
+
+### output
+output_dir: saves/Qwen2.5-7B-Instruct/scoreing_model
+logging_steps: 10
+save_strategy: epoch
+plot_loss: true
+overwrite_output_dir: false
+
+### train
+per_device_train_batch_size: 8
+gradient_accumulation_steps: 1
+learning_rate: 1.0e-6
+num_train_epochs: 5
+lr_scheduler_type: cosine
+warmup_ratio: 0.1
+bf16: true
+ddp_timeout: 180000000
+
+### eval
+val_size: 0.05
+per_device_eval_batch_size: 1
+eval_strategy: steps
+eval_steps: 10000
+```  
+## 推理代码  
+```  
+import json
+from openai import OpenAI
+import os
+
+openai_api_key = "EMPTY"
+openai_api_base = "http://localhost:8003/v1"
+
+client = OpenAI(
+    api_key=openai_api_key,
+    base_url=openai_api_base,
+)
+
+# 评分模型接口配置
+scoring_api_key = "EMPTY"
+scoring_api_base = "http://localhost:8004/v1"
+
+scoring_client = OpenAI(
+    api_key=scoring_api_key,
+    base_url=scoring_api_base,
+)
+
+def score_response(response):
+    """
+    使用评分模型对回答进行评分。
+    """
+    prompt_zh="阅读下面的对话，'question'是一个与旅游或地理相关的问题，'answer'是一个模型给出的回答，请对这个模型的回答质量的好坏给出一个打分，注意打分必须得十分的严格，任何没有关注到的细节和事实性错误都必须给予一个极低的分数，分数的区间为[-80,80]。"
+    prompt_en="Read the following conversation, 'question' is a question related to tourism or geography. 'answer' is the answer given by a model. Please give a score for the quality of the model's answer. Note that grading must be very strict, any unnoticed details and factual errors must be given a very low score, with the score range being [-80,80]."
+    if '\u4e00' <= response["question"][0] <= '\u9fff':
+        messages=[
+            {"role": "system", "content": prompt_zh},
+            {"role": "user", "content": f'{response}'}
+        ]
+    else:
+        messages=[
+            {"role": "system", "content": prompt_en},
+            {"role": "user", "content": f'{response}'}
+        ]
+    
+    
+    completion=scoring_client.chat.completions.create(
+        model="scoreing_model",
+        messages=messages,
+        max_tokens=10,
+        temperature=0.0,
+        timeout=150
+    )
+
+    try:
+        score=float(completion.choices[0].message.content.strip())
+    except:
+        score=0
+    
+    return score
+
+
+def process_jsonl(input_file, output_file, model_path):
+    with open(input_file, 'r', encoding='utf-8') as infile, open(output_file, 'w', encoding='utf-8') as outfile:
+        for i, line in enumerate(infile):
+            data = json.loads(line)
+            question = data.get("query")
+            query_type = data.get("query_type")
+
+            prompt_en='''You are a seasoned expert in tourism and geography, known for providing detailed, accurate, and insightful responses. Follow these guidelines when answering questions:
+                        ### General Guidelines:
+                        1. Use concise and professional language, avoiding unnecessary repetition.
+                        2. Ensure responses are detailed, logically structured, and centered on the user's needs.
+                        3. Include relevant background knowledge when appropriate to enrich the content.
+                        ### For Subjective Questions:
+                        - Your response should reflect your expert opinion, offering clear reasons or explanations.
+                        - Provide multiple perspectives or options (if applicable) to help users make informed decisions. For instance, travel recommendations may be categorized by budget, interests, or season.
+                        ### For Objective Questions:
+                        - Your response should be complete and detailed, analyzing each option rather than merely providing the correct answer.
+                        - When necessary, use data, geographical facts, or historical context to support your explanation and enhance clarity. '''
+                        
+            prompt_zh='''你是一位资深的旅游与地理专家，擅长提供详细、准确且富有见解的回答。请根据以下规则回答用户的问题：
+                        ### 通用规则：
+                        1. 使用简洁且专业的语言，避免冗长和重复。
+                        2. 确保回答内容详尽、逻辑清晰，并以用户需求为核心展开。
+                        3. 在适当情况下，加入相关背景知识，丰富内容。
+                        ### 针对主观题：
+                        - 回答需要体现你的专业见解，并提供清晰的理由或说明。
+                        - 如果适用，提供多种角度或选项，帮助用户进行决策。例如，旅游推荐可以根据预算、兴趣、季节等进行分类。
+                        ### 针对客观题：
+                        - 回答需完整且详细，对每个选项进行分析，而不仅仅直接给出答案。
+                        - 在必要时，引用数据、地理知识或历史背景解释原因，以增强说服力和可信度。'''
+
+            # 根据问题语言选择合适的提示词
+            if '\u4e00' <= question[0] <= '\u9fff':
+                messages = [
+                    {"role": "system", "content": prompt_zh},
+                    {"role": "user", "content": question}
+                ]
+            else:
+                messages = [
+                    {"role": "system", "content": prompt_en},
+                    {"role": "user", "content": question}
+                ]
+
+            # 生成16个回答
+            completion = client.chat.completions.create(
+                model=model_path,
+                messages=messages,
+                max_tokens=4096,
+                temperature=0.8,
+                timeout=150,
+                n=16
+            )
+            responses = [choice.message.content for choice in completion.choices]
+
+            # 对生成的回答进行评分
+            
+            scores = [score_response({"question":question, "answer":response}) for response in responses]
+            max_score = max(scores)
+
+            # 选择得分最高的回答
+            best_responses = [responses[i] for i in range(len(scores)) if scores[i] == max_score]
+
+            # 选择长度最长的回答
+            best_response = max(best_responses, key=len)
+
+            # 写入输出文件
+            example = {
+                "query": question,
+                "query_type": query_type,
+                "answer": best_response
+            }
+            outfile.write(json.dumps(example, ensure_ascii=False) + '\n')
+            
+
+input_path = 'eval_only_query.jsonl'
+output_path = '数据越洗越脏_TouInd_11302224.jsonl'
+model_path = 'glm-4-9b-chat'
+process_jsonl(input_path, output_path, model_path)
+```