Edison2ST/talentarena-prometheus-7b-v2.0
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: Edison2ST/talentarena-prometheus-7b-v2.0
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-17 23:33:21 +08:00
commit 644eee4719
12 changed files with 91963 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

237
LICENSE Normal file
View File

@@ -0,0 +1,237 @@
---
title: Apache License 2.0
spdx-id: Apache-2.0
redirect_from: /licenses/apache/
featured: true
hidden: false
description: A permissive license whose main conditions require preservation of copyright and license notices. Contributors provide an express grant of patent rights. Licensed works, modifications, and larger works may be distributed under different terms and without source code.
how: Create a text file (typically named LICENSE or LICENSE.txt) in the root of your source code and copy the text of the license into the file.
note: The Apache Software Foundation <a href="https://apache.org/foundation/license-faq.html#Apply-My-Software">recommends</a> taking the additional step of adding a boilerplate notice to the header of each source file. You can find the notice in the appendix at the very end of the license text.
using:
Kubernetes: https://github.com/kubernetes/kubernetes/blob/master/LICENSE
PDF.js: https://github.com/mozilla/pdf.js/blob/master/LICENSE
Swift: https://github.com/apple/swift/blob/main/LICENSE.txt
permissions:
- commercial-use
- modifications
- distribution
- patent-use
- private-use
conditions:
- include-copyright
- document-changes
limitations:
- trademark-use
- liability
- warranty
---
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

182
README.md Normal file
View File

@@ -0,0 +1,182 @@
---
tags:
- text2text-generation
datasets:
- prometheus-eval/Feedback-Collection
- prometheus-eval/Preference-Collection
license: apache-2.0
language:
- en
pipeline_tag: text2text-generation
library_name: transformers
metrics:
- pearsonr
- spearmanr
- kendall-tau
- accuracy
---
## Links for Reference
- **Homepage: In Progress**
- **Repository:https://github.com/prometheus-eval/prometheus-eval**
- **Paper:https://arxiv.org/abs/2405.01535**
- **Point of Contact:seungone@cmu.edu**
# TL;DR
Prometheus 2 is an alternative of GPT-4 evaluation when doing fine-grained evaluation of an underlying LLM & a Reward model for Reinforcement Learning from Human Feedback (RLHF).
![plot](./finegrained_eval.JPG)
Prometheus 2 is a language model using [Mistral-Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) as a base model.
It is fine-tuned on 100K feedback within the [Feedback Collection](https://huggingface.co/datasets/prometheus-eval/Feedback-Collection) and 200K feedback within the [Preference Collection](https://huggingface.co/datasets/prometheus-eval/Preference-Collection).
It is also made by weight merging to support both absolute grading (direct assessment) and relative grading (pairwise ranking).
The surprising thing is that we find weight merging also improves performance on each format.
# Model Details
## Model Description
- **Model type:** Language model
- **Language(s) (NLP):** English
- **License:** Apache 2.0
- **Related Models:** [All Prometheus Checkpoints](https://huggingface.co/models?search=prometheus-eval/Prometheus)
- **Resources for more information:**
- [Research paper](https://arxiv.org/abs/2405.01535)
- [GitHub Repo](https://github.com/prometheus-eval/prometheus-eval)
Prometheus is trained with two different sizes (7B and 8x7B).
You could check the 8x7B sized LM on [this page](https://huggingface.co/prometheus-eval/prometheus-2-8x7b-v2.0).
Also, check out our dataset as well on [this page](https://huggingface.co/datasets/prometheus-eval/Feedback-Collection) and [this page](https://huggingface.co/datasets/prometheus-eval/Preference-Collection).
## Prompt Format
We have made wrapper functions and classes to conveniently use Prometheus 2 at [our github repository](https://github.com/prometheus-eval/prometheus-eval).
We highly recommend you use it!
However, if you just want to use the model for your use case, please refer to the prompt format below.
Note that absolute grading and relative grading requires different prompt templates and system prompts.
### Absolute Grading (Direct Assessment)
Prometheus requires 4 components in the input: An instruction, a response to evaluate, a score rubric, and a reference answer. You could refer to the prompt format below.
You should fill in the instruction, response, reference answer, criteria description, and score description for score in range of 1 to 5.
Fix the components with \{text\} inside.
```
###Task Description:
An instruction (might include an Input inside it), a response to evaluate, a reference answer that gets a score of 5, and a score rubric representing a evaluation criteria are given.
1. Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general.
2. After writing a feedback, write a score that is an integer between 1 and 5. You should refer to the score rubric.
3. The output format should look as follows: \"Feedback: (write a feedback for criteria) [RESULT] (an integer number between 1 and 5)\"
4. Please do not generate any other opening, closing, and explanations.
###The instruction to evaluate:
{orig_instruction}
###Response to evaluate:
{orig_response}
###Reference Answer (Score 5):
{orig_reference_answer}
###Score Rubrics:
[{orig_criteria}]
Score 1: {orig_score1_description}
Score 2: {orig_score2_description}
Score 3: {orig_score3_description}
Score 4: {orig_score4_description}
Score 5: {orig_score5_description}
###Feedback:
```
After this, you should apply the conversation template of Mistral (not applying it might lead to unexpected behaviors).
You can find the conversation class at this [link](https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py).
```
conv = get_conv_template("mistral")
conv.set_system_message("You are a fair judge assistant tasked with providing clear, objective feedback based on specific criteria, ensuring each assessment reflects the absolute standards set for performance.")
conv.append_message(conv.roles[0], dialogs['instruction'])
conv.append_message(conv.roles[1], None)
prompt = conv.get_prompt()
x = tokenizer(prompt,truncation=False)
```
As a result, a feedback and score decision will be generated, divided by a separating phrase ```[RESULT]```
### Relative Grading (Pairwise Ranking)
Prometheus requires 4 components in the input: An instruction, 2 responses to evaluate, a score rubric, and a reference answer. You could refer to the prompt format below.
You should fill in the instruction, 2 responses, reference answer, and criteria description.
Fix the components with \{text\} inside.
```
###Task Description:
An instruction (might include an Input inside it), two responses to evaluate (denoted as Response A and Response B), a reference answer, and an evaluation criteria are given.
1. Write a detailed feedback that assess the quality of the two responses strictly based on the given evaluation criteria, not evaluating in general.
2. Make comparisons between Response A, Response B, and the Reference Answer. Instead of examining Response A and Response B separately, go straight to the point and mention about the commonalities and differences between them.
3. After writing the feedback, indicate the better response, either "A" or "B".
4. The output format should look as follows: "Feedback: (write a feedback for criteria) [RESULT] (Either "A" or "B")"
5. Please do not generate any other opening, closing, and explanations.
###Instruction:
{orig_instruction}
###Response A:
{orig_response_A}
###Response B:
{orig_response_B}
###Reference Answer:
{orig_reference_answer}
###Score Rubric:
{orig_criteria}
###Feedback:
```
After this, you should apply the conversation template of Mistral (not applying it might lead to unexpected behaviors).
You can find the conversation class at this [link](https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py).
```
conv = get_conv_template("mistral")
conv.set_system_message("You are a fair judge assistant assigned to deliver insightful feedback that compares individual performances, highlighting how each stands relative to others within the same cohort.")
conv.append_message(conv.roles[0], dialogs['instruction'])
conv.append_message(conv.roles[1], None)
prompt = conv.get_prompt()
x = tokenizer(prompt,truncation=False)
```
As a result, a feedback and score decision will be generated, divided by a separating phrase ```[RESULT]```
## License
Feedback Collection, Preference Collection, and Prometheus 2 are subject to OpenAI's Terms of Use for the generated data. If you suspect any violations, please reach out to us.
# Citation
If you find the following model helpful, please consider citing our paper!
**BibTeX:**
```bibtex
@misc{kim2023prometheus,
title={Prometheus: Inducing Fine-grained Evaluation Capability in Language Models},
author={Seungone Kim and Jamin Shin and Yejin Cho and Joel Jang and Shayne Longpre and Hwaran Lee and Sangdoo Yun and Seongjin Shin and Sungdong Kim and James Thorne and Minjoon Seo},
year={2023},
eprint={2310.08491},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
```bibtex
@misc{kim2024prometheus,
title={Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models},
author={Seungone Kim and Juyoung Suk and Shayne Longpre and Bill Yuchen Lin and Jamin Shin and Sean Welleck and Graham Neubig and Moontae Lee and Kyungjae Lee and Minjoon Seo},
year={2024},
eprint={2405.01535},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```

39
config.json Normal file
View File

@@ -0,0 +1,39 @@
{
"_name_or_path": "prometheus-eval/prometheus-7b-v2.0",
"architectures": [
"MistralForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 32768,
"model_type": "mistral",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"num_key_value_heads": 8,
"quantization_config": {
"bits": 4,
"damp_percent": 0.01,
"desc_act": false,
"group_size": 128,
"is_marlin_format": false,
"model_file_base_name": null,
"model_name_or_path": null,
"quant_method": "gptq",
"static_groups": false,
"sym": true,
"true_sequential": true
},
"rms_norm_eps": 1e-05,
"rope_theta": 1000000.0,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.38.2",
"use_cache": true,
"vocab_size": 32000
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0ed1be7d915cb69605766debd41f063c7963d3ad190e9517f11ef074f1290015
size 4158662248

39
model_to_int4.py Normal file
View File

@@ -0,0 +1,39 @@
from transformers import AutoTokenizer
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
import torch
# 1. Configuración
model_id = "prometheus-eval/prometheus-7b-v2.0"
output_dir = "prometheus-7b-v2.0-GPTQ-4bit"
# Configuración de 4 bits
quantize_config = BaseQuantizeConfig(
bits=4, # Cuantización a 4 bits
group_size=128, # Recomendado para equilibrio calidad/velocidad
desc_act=False, # Mejora la velocidad de inferencia
)
# 2. Cargar Tokenizer y preparar datos de calibración
# GPTQ necesita unos pocos ejemplos para ajustar los pesos
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
examples = [
tokenizer("Prometheus is a specialized language model for evaluating other AI models."),
tokenizer("Quantization helps to run large models on consumer hardware.")
]
# 3. Cargar el modelo original (FP16)
model = AutoGPTQForCausalLM.from_pretrained(
model_id,
quantize_config,
device_map="auto",
torch_dtype=torch.float16
)
# 4. Ejecutar la cuantización
model.quantize(examples)
# 5. Guardar el modelo resultante en formato .safetensors
model.save_quantized(output_dir, use_safetensors=True)
tokenizer.save_pretrained(output_dir)
print(f"Modelo guardado exitosamente en: {output_dir}")

213
model_utils.py Normal file
View File

@@ -0,0 +1,213 @@
import os
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
import re
from peft import PeftModel
def get_model_and_tokenizer(model_name="prometheus-eval/prometheus-7b-v2.0" ):
"""
Carga el modelo Prometheus y su tokenizador asociado desde Hugging Face.
Esta función es esencial para el hackathon ya que inicializa el evaluador LLM-as-a-Judge.
Recuerda configurar tu token de Hugging Face de antemano.
Args:
model_name (str): La versión específica del modelo de Prometheus a cargar.
Returns:
model, tokenizer: Tupla con el modelo y el tokenizador listos para realizar inferencias.
"""
hf_token = os.getenv("HF_TOKEN")
if not hf_token:
print("Warning: HF_TOKEN not found in environment variables.")
print(f"Loading model: {model_name}...")
# 1. Cargar y configurar el Tokenizador
tokenizer = AutoTokenizer.from_pretrained(model_name, token=hf_token)
# Configuramos el pad_token si no existe (común en Mistral/Llama)
if tokenizer.pad_token is None:
tokenizer.pad_token = tokenizer.eos_token
# Padding a la izquierda es obligatorio para modelos decodificadores (CausalLM)
# cuando se hace inferencia en batches
tokenizer.padding_side = "left"
# 2. Cargar el Modelo
model = AutoModelForCausalLM.from_pretrained(
model_name,
token=hf_token,
device_map="auto",
# dtype=torch.float16, # Media precisión para ganar velocidad y ahorrar VRAM # No necesario dado que usamos cuantización a 4-bit
low_cpu_mem_usage=True,
trust_remote_code=True, # Añadido
use_safetensors=True, # Añadido
#safetensors_filename="gptq_model-4bit-128g.safetensors"
)
return model, tokenizer
def split_model_reason_result(sample, output_suffix : str = "model", input_col: str = "model_output")->dict:
"""
Post-procesa la salida del modelo para separar la explicación de la puntuación.
Busca la etiqueta '[RESULT]' para dividir el texto. Si no la encuentra,
asume que todo el texto es el razonamiento y devuelve un resultado nulo.
Args:
sample (dict | str): Ejemplo que contiene 'model_output'.
output_suffix (str): Sufijo para nombrar la columna de salida.
input_col (str): Nombre de la columna de entrada.
Returns:
dict: Diccionario con las claves 'reason' (explicación) y 'result' (puntuación limpia).
"""
output = sample.get(input_col, "") if not isinstance(sample, str) else sample
if "[RESULT]" in output:
# Dividimos por la última aparición del tag para evitar errores
parts = output.rsplit("[RESULT]", 1)
reason = parts[0].strip()
result_raw = parts[1].strip()
# Limpieza mediante regex para capturar solo el dígito (evita puntos finales, etc.)
score_match = re.search(r'(\d+)', result_raw)
result = score_match.group(1) if score_match else result_raw
else:
reason = output.strip()
result = None
return {
f"{output_suffix}_reason": reason,
f"{output_suffix}_pred": result
}
def model_predict(model, tokenizer, prompt, max_new_tokens =200, temperature=0.7):
"""
Realiza una inferencia simple para un único prompt utilizando el modelo y tokenizador proporcionados.
Esta función prepara el texto, lo envía al dispositivo donde reside el modelo (GPU/CPU)
y genera una respuesta de forma determinista. Es ideal para pruebas rápidas o
validaciones unitarias durante la hackathon.
Args:
model (transformers.PreTrainedModel): El modelo de lenguaje ya cargado.
tokenizer (transformers.PreTrainedTokenizer): El tokenizador correspondiente al modelo.
prompt (str): El texto de entrada o instrucción para el modelo.
Returns:
str: El texto generado por el modelo, limpio de tokens especiales y del prompt original.
"""
# 1. Identificar el dispositivo del modelo (soporta device_map="auto")
device = model.device
# 2. Tokenizar y mover tensores al dispositivo correcto
inputs = tokenizer(prompt, return_tensors="pt").to(device)
# 3. Generación determinista (do_sample=False para evitar variabilidad en pruebas)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=True,
pad_token_id=tokenizer.pad_token_id if tokenizer.pad_token_id else tokenizer.eos_token_id
)
# 4. Decodificar solo la parte nueva (ignorando los tokens del prompt)
input_length = inputs["input_ids"].shape[1]
prediction = tokenizer.decode(outputs[0][input_length:], skip_special_tokens=True)
return prediction.strip()
def model_predict_batched(model, tokenizer, batch, input_col = "user_content",
temperature = 0.1, max_new_tokens = 1000, completion_colname = "model_output"):
"""
Realiza inferencia en lotes (batches) sobre un conjunto de prompts.
Esta función es más eficiente que `model_predict` cuando se procesan múltiples ejemplos a la vez,
ya que aprovecha el procesamiento en paralelo de la GPU. Aplica el template de chat
del tokenizador automáticamente.
Args:
model (transformers.PreTrainedModel): El modelo cargado.
tokenizer (transformers.PreTrainedTokenizer): El tokenizador correspondiente.
batch (dict o pd.DataFrame): El lote de datos de entrada.
input_col (str, opcional): El nombre de la columna que contiene los prompts de usuario. Por defecto "user_content".
temperature (float, opcional): Parámetro de temperatura para controlar la aleatoriedad. Por defecto 0.1.
max_new_tokens (int, opcional): Límite máximo de tokens a generar. Por defecto 1000.
completion_colname (str, opcional): Nombre de la columna de salida. Por defecto "model_output".
Returns:
dict: Diccionario que contiene una lista con las respuestas generadas bajo la clave f"{completion_colname}".
"""
# 1. Detectamos el dispositivo de entrada (donde está la primera c apa)
model_device = model.device
messages_list = [[{"role": "user", "content": p}] for p in batch[input_col]]
# 2. IMPORTANTE: Pedimos que devuelva un diccionario completo (return_dict=True)
inputs = tokenizer.apply_chat_template(
messages_list,
add_generation_prompt=True,
tokenize=True,
return_tensors="pt",
padding=True,
return_dict=True # Esto asegura que tengamos input_ids y attention_mask
).to(model_device)
with torch.no_grad():
generated_ids = model.generate(
**inputs, # Ahora inputs es un dict con todo en la GPU correcta
max_new_tokens=max_new_tokens,
do_sample=True,
temperature=temperature,
pad_token_id=tokenizer.pad_token_id
)
input_length = inputs["input_ids"].shape[1]
decoded_outputs = tokenizer.batch_decode(
generated_ids[:, input_length:],
skip_special_tokens=True
)
return {f"{completion_colname}": decoded_outputs}
def load_lora_model(model_name, model_path):
"""
Carga un modelo base y le aplica los pesos ajustados de un entrenamiento LoRA (PEFT).
Durante el hackathon, usarás esta función para cargar tu propio modelo afinao (Fine-Tuned)
y comparar sus evaluaciones con las del modelo original.
Args:
model_name (str): Nombre o ruta del modelo base original (p. ej., "prometheus-eval/prometheus-7b-v2.0").
model_path (str): Ruta donde se encuentran guardados los adaptadores LoRA entrenados.
Returns:
model, tokenizer: Tupla con el modelo ajustado y su tokenizador.
"""
# 1. Load the original BASE model (the one you started with)
base_model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
# 2. Load the Tokenizer (now that you've saved it to the FT path)
tokenizer = AutoTokenizer.from_pretrained(model_path)
# 3. Load the LoRA adapters onto the base model
model = PeftModel.from_pretrained(base_model, model_path)
return model, tokenizer

13
quantize_config.json Normal file
View File

@@ -0,0 +1,13 @@
{
"bits": 4,
"group_size": 128,
"damp_percent": 0.01,
"desc_act": false,
"static_groups": false,
"sym": true,
"true_sequential": true,
"model_name_or_path": null,
"model_file_base_name": null,
"is_marlin_format": false,
"quant_method": "gptq"
}

30
special_tokens_map.json Normal file
View File

@@ -0,0 +1,30 @@
{
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

91122
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
tokenizer.model (Stored with Git LFS) Normal file
View File

Binary file not shown.

47
tokenizer_config.json Normal file
View File

@@ -0,0 +1,47 @@
{
"add_bos_token": true,
"add_eos_token": false,
"added_tokens_decoder": {
"0": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"additional_special_tokens": [],
"bos_token": "<s>",
"chat_template": "{%- if messages[0]['role'] == 'system' %}\n {%- set system_message = messages[0]['content'] %}\n {%- set loop_messages = messages[1:] %}\n{%- else %}\n {%- set loop_messages = messages %}\n{%- endif %}\n\n{{- bos_token }}\n{%- for message in loop_messages %}\n {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}\n {{- raise_exception('After the optional system message, conversation roles must alternate user/assistant/user/assistant/...') }}\n {%- endif %}\n {%- if message['role'] == 'user' %}\n {%- if loop.first and system_message is defined %}\n {{- ' [INST] ' + system_message + '\\n\\n' + message['content'] + ' [/INST]' }}\n {%- else %}\n {{- ' [INST] ' + message['content'] + ' [/INST]' }}\n {%- endif %}\n {%- elif message['role'] == 'assistant' %}\n {{- ' ' + message['content'] + eos_token}}\n {%- else %}\n {{- raise_exception('Only user and assistant roles are supported, with the exception of an initial optional system message!') }}\n {%- endif %}\n{%- endfor %}\n",
"clean_up_tokenization_spaces": false,
"eos_token": "</s>",
"legacy": true,
"max_length": 4096,
"model_max_length": 1000000000000000019884624838656,
"pad_token": "</s>",
"sp_model_kwargs": {},
"spaces_between_special_tokens": false,
"stride": 0,
"tokenizer_class": "LlamaTokenizer",
"truncation_side": "right",
"truncation_strategy": "longest_first",
"unk_token": "<unk>",
"use_default_system_prompt": false
}