68 lines
3.0 KiB
Markdown
68 lines
3.0 KiB
Markdown
|
---
|
|||
|
|
license: apache-2.0
|
|||
|
|
datasets:
|
|||
|
|
- kitsdk/ctga-v1-1w-chinese
|
|||
|
|
base_model:
|
|||
|
|
- Qwen/Qwen2.5-3B
|
|||
|
|
tasks:
|
|||
|
|
- text-generation
|
|||
|
|
- question-answering
|
|||
|
|
- text-ranking
|
|||
|
|
frameworks: PyTorch
|
|||
|
|
language:
|
|||
|
|
- zh
|
|||
|
|
base_model_relation: finetune
|
|||
|
|
---
|
|||
|
|
# Bonito（支持中文版本）
|
|||
|
|
|
|||
|
|
Bonito is an open-source model for conditional task generation: the task of converting unannotated text into task-specific training datasets for instruction tuning. This repo is a lightweight library for Bonito to easily create synthetic datasets built on top of the Hugging Face `transformers` and `vllm` libraries.
|
|||
|
|
|
|||
|
|
- Paper: [Learning to Generate Instruction Tuning Datasets for
|
|||
|
|
Zero-Shot Task Adaptation](https://arxiv.org/abs/2402.18334)
|
|||
|
|
- Model: [bonito-v1](https://huggingface.co/BatsResearch/bonito-v1)(English)
|
|||
|
|
- Model: [bonito-chinese-v1](https://huggingface.co/kitsdk/bonito-chinese-v1)(中文)
|
|||
|
|
- Demo: [Bonito on Spaces](https://huggingface.co/spaces/nihalnayak/bonito)(English)
|
|||
|
|
- Demo: [Google Colab](https://colab.research.google.com/drive/1vZ78RFywM1pkuGpxGWGLude_L2XZ5wpw?usp=sharing)(中文)
|
|||
|
|
- Dataset: [ctga-v1](https://huggingface.co/datasets/BatsResearch/ctga-v1)(English)
|
|||
|
|
- Code: To reproduce experiments in our paper, see [nayak-aclfindings24-code](https://github.com/BatsResearch/nayak-aclfindings24-code).
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
## This version supports the Chinese language
|
|||
|
|
Because of the training data limitations, this version supports only the 3 task types
|
|||
|
|
- 🐠 1.question generation.
|
|||
|
|
- 🐡 2.multiple-choice question answering.
|
|||
|
|
- 🐟 3.question answering without choices.
|
|||
|
|
## Google Colab: [Demo](https://colab.research.google.com/drive/1vZ78RFywM1pkuGpxGWGLude_L2XZ5wpw?usp=sharing)
|
|||
|
|
## Basic Usage
|
|||
|
|
To generate synthetic instruction tuning dataset using Bonito, you can use the following code:
|
|||
|
|
pip3 install bonito-llm
|
|||
|
|
```python
|
|||
|
|
from pprint import pprint
|
|||
|
|
from datasets import Dataset
|
|||
|
|
from vllm import SamplingParams
|
|||
|
|
from transformers import set_seed
|
|||
|
|
from bonito import Bonito
|
|||
|
|
|
|||
|
|
unannotated_paragraph = """灌区以往的闸门控制系统在实际应用过程中普遍以人工操作为主，容易受到多种因素的影响，不可避免出现较多缺陷。如操作人员自身的综合能力、业务水平、工作态度等对工作质量和效率产生较大影响；工作人员实践操作中遇到极端气候、工作环境恶劣等问题，大大增加了工作难度，并存在较多安全隐患。"""
|
|||
|
|
pprint(unannotated_paragraph)
|
|||
|
|
bonito = Bonito("kitsdk/bonito-chinese-v1")
|
|||
|
|
set_seed(2)
|
|||
|
|
|
|||
|
|
def convert_to_dataset(text):
|
|||
|
|
dataset = Dataset.from_list([{"input": text}])
|
|||
|
|
return dataset
|
|||
|
|
|
|||
|
|
sampling_params = SamplingParams(max_tokens=256, top_p=0.95, temperature=0.5, n=1)
|
|||
|
|
synthetic_dataset = bonito.generate_tasks(
|
|||
|
|
convert_to_dataset(unannotated_paragraph),
|
|||
|
|
context_col="input",
|
|||
|
|
task_type="mcqa",
|
|||
|
|
sampling_params=sampling_params
|
|||
|
|
)
|
|||
|
|
pprint("----Generated Instructions----")
|
|||
|
|
pprint(f'Input: {synthetic_dataset[0]["input"]}')
|
|||
|
|
pprint(f'Output: {synthetic_dataset[0]["output"]}')
|
|||
|
|
```
|