AI-ModelScope/dolly-v1-6b
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: AI-ModelScope/dolly-v1-6b
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-14 01:05:15 +08:00
commit b26e02c7c0
12 changed files with 152177 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
.gitattributes vendored Normal file
View File

@@ -0,0 +1,34 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text

158
README.md Normal file
View File

@@ -0,0 +1,158 @@
---
license: cc-by-nc-4.0
tasks:
- text-generation
frameworks:
- pytorch
---
# dolly-v1-6b Model Card
## Summary
Databricks’ `dolly-v1-6b`, a large language model ([blog post](https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html))
trained on the Databricks machine learning platform, demonstrates that a
two-years-old [open source model](https://huggingface.co/EleutherAI/gpt-j-6B) can, when subjected to just 30 minutes of fine tuning on a focused corpus of 50k records
([Stanford Alpaca](https://crfm.stanford.edu/2023/03/13/alpaca.html)), exhibit surprisingly high quality instruction following behavior not characteristic of the foundation
model on which it is based. We believe this finding is important because it demonstrates that the ability to create powerful
artificial intelligence technologies is vastly more accessible than previously realized.
Databricks is committed to ensuring that every organization and individual benefits from the transformative power of artificial intelligence. The Dolly model family represents our first steps along this journey, and we’re excited to share this technology with the world.
**Owner**: Databricks, Inc.
## Model Overview
`dolly-v1-6b` is a 6 billion parameter causal language model created by [Databricks](https://databricks.com/) that is derived from
[EleutherAI’s](https://www.eleuther.ai/) [GPT-J](https://huggingface.co/EleutherAI/gpt-j-6B) (released June 2021) and fine-tuned
on a ~52K record instruction corpus ([Stanford Alpaca](https://crfm.stanford.edu/2023/03/13/alpaca.html)) (CC-NC-BY-4.0)
consisting of question/answer pairs generated using the techniques outlined in the [Self-Instruct](https://arxiv.org/abs/2212.10560) paper.
The [original version](https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html) of was Dolly was trained using [deepspeed](https://github.com/microsoft/DeepSpeed) [ZeRO 3](https://github.com/microsoft/DeepSpeed/blob/master/docs/code-docs/source/zero3.rst)
on the [Databricks Machine Learning Platform](https://www.databricks.com/product/machine-learning) in just 30 minutes (1 epoch) using a single
[NDasrA100_v4](https://learn.microsoft.com/en-us/azure/virtual-machines/nda100-v4-series) machine with 8x A100 40GB GPUs.
The most recent `dolly-v1-6b` checkpoint was trained for 10 epochs on the same hardware.
Like its base model, `dolly-v1-6b` has six billion parameters consisting of 28 transformer layers with 16 attention heads each.
It employs [Rotary Position Embedding](https://arxiv.org/abs/2104.09864) (RoPE) and shares the same tokenizer as GPT-3.
GPT-J was trained on [The Pile](https://huggingface.co/datasets/the_pile), a 400B token dataset of diverse documents designed primarily for text generation tasks.
## Known Limitations
**`dolly-v1-6b` is not a state-of-the-art generative language model** and, though quantitative benchmarking is ongoing, is not designed to perform
competitively with more modern model architectures or models subject to larger pretraining corpuses. **It is designed for academic or research purposes, and to encourage model and engineering experimentation.**
The Dolly model family is under active development, and so any list of shortcomings is unlikely to be exhaustive, but we include known limitations and misfires here as a means to document and share our preliminary findings with the community. In particular, `dolly-v1-6b` struggles with: syntactically complex prompts, programming problems, mathematical operations, factual errors,
dates and times, open-ended question answering, hallucination, enumerating lists of specific length, stylistic mimicry, having a sense of humor, etc.
## Training Data, Bias & Objectionable Content
Like all language models, `dolly-v1-6b` reflects the content and limitations of its training corpuses.
- **The Pile**: GPT-J’s pre-training corpus contains content mostly collected from the public internet, and like most web-scale datasets,
it contains content many users would find objectionable. As such, the model is likely to reflect these shortcomings, potentially overtly
in the case it is explicitly asked to produce objectionable content, and sometimes subtly, as in the case of biased or harmful implicit
associations.
- **Stanford Alpaca**: The instruction tuning corpus for dolly-6b can be assumed to share many of the limitations. In addition, it is known to
contain factual inaccuracies, semantic and syntactic irregularities, nonsensical responses, and incorrect mathematical calculations, among
other data shortcomings. The model outputs will reflect these limitations.
Databricks is committed to ongoing research and development efforts to develop helpful, honest and harmless AI technologies that
maximize the potential of all individuals and organizations.
## Intended Uses
**`dolly-v1-6b` is intended exclusively for research purposes. We do not recommend using `dolly-v1-6b` in high-risk applications
(e.g., educational or vocational training, product safety components, or other uses that may impact the well-being of individuals.)**
`dolly-v1-6b` is differentiated from its predecessor, GPT-J, in the extent to which it exhibits coherent responses to natural language
questions and instruction prompts. `dolly-v1-6b` appears to exhibit reasonable (but categorically not state-of-the-art) behavior across a range
of capabilities, including many of those outlined in the [InstructGPT](https://arxiv.org/abs/2203.02155) paper.
We include evocative examples below, but invite the reader to observe that they contain factual inaccuracies or semantic misfires that illustrate some of the model’s shortcomings.
### Brainstorming
**What are considerations I should keep in mind when planning a backcountry backpacking trip?**
When planning a backcountry backpacking trip, there are several considerations to keep in mind. First, make sure you are prepared for any potential weather conditions, including extreme temperatures and lightning storms. Second, consider the route you will take and the terrain that you will be traversing. Third, make sure that you have enough food, water and gear to stay safe and comfortable during your backcountry journey. Fourth, plan for potential wildlife encounters and make sure you have the appropriate safety measures in place. Finally, have an emergency plan in place in case of a medical emergency.
### Classification
**Label each of the following as either a scientific concept or a product: Nikon D750, quantum entanglement, CRISPR, and a Macbook Pro.**
Nikon D750: Product
Quantum entanglement: Scientific Concept
CRISPR: Product
Macbook Pro: Product
### Extraction
**Extract all the people and places from the following passage:
Basquiat was born on December 22, 1960, in Park Slope, Brooklyn, New York City, the second of four children to Matilde Basquiat (née Andrades, 1934–2008) and Gérard Basquiat (1930–2013). He had an older brother, Max, who died shortly before his birth, and two younger sisters, Lisane (b. 1964) and Jeanine (b. 1967). His father was born in Port-au-Prince, Haiti and his mother was born in Brooklyn to Puerto Rican parents. He was raised Catholic.**
People: Basquiat, Matilde Basquiat, Gérard Basquiat, Max Basquiat, Lisane Basquiat, Jeanine Basquiat.
Place: Park Slope, Brooklyn, New York City.
### Generation
**Write a press release declaring the completion of Atlantis II, a facility designed for long-term human habitation at the bottom of the ocean. Be sure to mention some of its advanced technological features.**
Atlantis II, a new facility designed for long-term human habitation at the bottom of the ocean, is now complete! With advanced technological features, it will be a safe and comfortable living environment for generations to come. From self-sustaining ecosystems to advanced robotics, the facility offers a range of opportunities for exploration, discovery and exploration.
Atlantis II will be a home for a wide range of species and will serve as a hub for ocean research, exploration and exploration, bringing together a diverse team of scientists, engineers, biologists, and divers to uncover the mysteries of the deep.
### Summarization
**Give me a one line summary of this:
Coffee is one of the most widely consumed beverages in the world. It has primarily consumed due to its stimulant effect and unique taste since the ancient times. Afterwards, its consumption has been historically associated with a lower risk of some diseases such as type 2 diabetes mellitus, obesity, cardiovascular disease and some type of cancer and thus it has also consumed due to health benefits. It contains many bioactive compounds such as caffeine, chlorogenic acids and diterpenoid alcohols which have so far been associated with many potential health benefits. For example, caffeine reduces risk of developing neurodegenerative disease and chlorogenic acids (CGA) and diterpene alcohols have many health benefits such as antioxidant and chemo-preventive. Coffee also have harmful effects. For example, diterpenoid alcohols increases serum homocysteine and cholesterol levels and thus it has adverse effects on cardiovascular system. Overall, the study that supports the health benefits of coffee is increasing. But, it is thought-provoking that the association with health benefits of coffee consumption and frequency at different levels in each study. For this reason, we aimed to examine the health effect of the coffee and how much consumption is to investigate whether it meets the claimed health benefits.**
Coffee is one of the most widely consumed beverages in the world and has historically been associated with a lower risk of some diseases such as type 2 diabetes mellitus, obesity, cardiovascular disease and some type of cancer. However, there are mixed results in the literature and further research is required to determine the health benefits of coffee consumption.
### Rephrasing
**Give me a different way to say the following to a 4 year old: "Son, this is the last time I'm going to tell you. Go to bed!"**
"Goodnight, Son. Sleep tight and I'll see you in the morning!"
### Closed QA
**I'm going to give you a passage from the book Neuromancer and I'd like you to answer the following question: What is the tool that allows Case to access the matrix?
Case was twenty-four. At twenty-two, he'd been a cowboy, a rustler, one of the best in the Sprawl. He'd been trained by the best, by McCoy Pauley and Bobby Quine, legends in the biz. He'd operated on an almost permanent adrenaline high, a byproduct of youth and proficiency, jacked into a custom cyberspace deck that projected his disembodied consciousness into the consensual hallucination that was the matrix.**
The tool that allows Case to access the matrix is his cyberspace deck, which allows him to manipulate and interact with the virtual world around him. It enables him to access information from various sources and interact with the people in the matrix, and it is one of the main weapons of Case's cyberworld.
### Open QA
**What is the literary style of Jorge Luis Borges?**
Jorge Luis Borges's literary style is often described as magical realism. He often creates stories that combine fictional characters with real life elements and use the juxtaposition of these elements to create a sense of surrealism. He often uses allegory and symbolism to explore complex philosophical ideas and explore questions of human perception and understanding. His writing is characterized by a playful and inventive use of language, with characters often being metaphors or symbols for larger, more abstract concepts.
### 示例代码
```python
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
if __name__ == '__main__':
model = "AI-ModelScope/dolly-v1-6b"
pipe = pipeline(Tasks.text_generation, model=model, model_revision='v1.0.1', device='cuda:0')
instruction = "Write a tweet announcing Dolly, a large language model from Databricks."
output = pipe(instruction)
print(output)
# output: {'text': "Welcome to Dolly! Dolly is the world's largest language model with over 500 billion parameters, trained using billions of examples. #databricks #languagemodeling"}
```
### Benchmark Metrics
Below you'll find various models benchmark performance on the [EleutherAI LLM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness)
model results are sorted by geometric mean to produce an intelligible ordering. These results demonstrate that Dolly is not state of the art, as we describe
above, but also point to an interesting observation. Namely, Dolly is only marginally better (and in the case of Winogrande worse) and its basemodel GPT-J-6B.
Despite this fact, the qualitative behavior of Dolly is materially different from the underlying model ([try it yourself](https://huggingface.co/EleutherAI/gpt-j-6B) on Hugging Face!),
which points to meaningful limitations of the existing evaluation benchmarks for measuring the quality of generative models.
| model | openbookqa | arc_easy | winogrande | hellaswag | arc_challenge | piqa | boolq |
| --------------------------- | ------------ | ---------- | ------------ | ----------- | --------------- | -------- | ---------|
| cerebras/Cerebras-GPT-13B | 0.36 | 0.598906 | 0.607735 | 0.593109 | 0.325939 | 0.749728 | 0.611621 |
| EleutherAI/gpt-j-6B | 0.382 | 0.621633 | 0.651144 | 0.662617 | 0.363481 | 0.761153 | 0.655963 |
| dolly-v1-6b (1 epoch) | 0.428 | 0.608586 | 0.633781 | 0.650568 | 0.377133 | 0.761697 | 0.69633 |
| dolly-v1-6b (10 epochs) | 0.41 | 0.62963 | 0.643252 | 0.676758 | 0.384812 | 0.773667 | 0.687768 |
| EleutherAI/gpt-neox-20b | 0.402 | 0.683923 | 0.656669 | 0.7142 | 0.408703 | 0.784004 | 0.695413 |
# Happy Hacking!

148
added_tokens.json Executable file
View File

@@ -0,0 +1,148 @@
{
"### End": 50400,
"### Instruction:": 50401,
"### Response:": 50402,
"<|extratoken_100|>": 50356,
"<|extratoken_101|>": 50357,
"<|extratoken_102|>": 50358,
"<|extratoken_103|>": 50359,
"<|extratoken_104|>": 50360,
"<|extratoken_105|>": 50361,
"<|extratoken_106|>": 50362,
"<|extratoken_107|>": 50363,
"<|extratoken_108|>": 50364,
"<|extratoken_109|>": 50365,
"<|extratoken_10|>": 50266,
"<|extratoken_110|>": 50366,
"<|extratoken_111|>": 50367,
"<|extratoken_112|>": 50368,
"<|extratoken_113|>": 50369,
"<|extratoken_114|>": 50370,
"<|extratoken_115|>": 50371,
"<|extratoken_116|>": 50372,
"<|extratoken_117|>": 50373,
"<|extratoken_118|>": 50374,
"<|extratoken_119|>": 50375,
"<|extratoken_11|>": 50267,
"<|extratoken_120|>": 50376,
"<|extratoken_121|>": 50377,
"<|extratoken_122|>": 50378,
"<|extratoken_123|>": 50379,
"<|extratoken_124|>": 50380,
"<|extratoken_125|>": 50381,
"<|extratoken_126|>": 50382,
"<|extratoken_127|>": 50383,
"<|extratoken_128|>": 50384,
"<|extratoken_129|>": 50385,
"<|extratoken_12|>": 50268,
"<|extratoken_130|>": 50386,
"<|extratoken_131|>": 50387,
"<|extratoken_132|>": 50388,
"<|extratoken_133|>": 50389,
"<|extratoken_134|>": 50390,
"<|extratoken_135|>": 50391,
"<|extratoken_136|>": 50392,
"<|extratoken_137|>": 50393,
"<|extratoken_138|>": 50394,
"<|extratoken_139|>": 50395,
"<|extratoken_13|>": 50269,
"<|extratoken_140|>": 50396,
"<|extratoken_141|>": 50397,
"<|extratoken_142|>": 50398,
"<|extratoken_143|>": 50399,
"<|extratoken_14|>": 50270,
"<|extratoken_15|>": 50271,
"<|extratoken_16|>": 50272,
"<|extratoken_17|>": 50273,
"<|extratoken_18|>": 50274,
"<|extratoken_19|>": 50275,
"<|extratoken_1|>": 50257,
"<|extratoken_20|>": 50276,
"<|extratoken_21|>": 50277,
"<|extratoken_22|>": 50278,
"<|extratoken_23|>": 50279,
"<|extratoken_24|>": 50280,
"<|extratoken_25|>": 50281,
"<|extratoken_26|>": 50282,
"<|extratoken_27|>": 50283,
"<|extratoken_28|>": 50284,
"<|extratoken_29|>": 50285,
"<|extratoken_2|>": 50258,
"<|extratoken_30|>": 50286,
"<|extratoken_31|>": 50287,
"<|extratoken_32|>": 50288,
"<|extratoken_33|>": 50289,
"<|extratoken_34|>": 50290,
"<|extratoken_35|>": 50291,
"<|extratoken_36|>": 50292,
"<|extratoken_37|>": 50293,
"<|extratoken_38|>": 50294,
"<|extratoken_39|>": 50295,
"<|extratoken_3|>": 50259,
"<|extratoken_40|>": 50296,
"<|extratoken_41|>": 50297,
"<|extratoken_42|>": 50298,
"<|extratoken_43|>": 50299,
"<|extratoken_44|>": 50300,
"<|extratoken_45|>": 50301,
"<|extratoken_46|>": 50302,
"<|extratoken_47|>": 50303,
"<|extratoken_48|>": 50304,
"<|extratoken_49|>": 50305,
"<|extratoken_4|>": 50260,
"<|extratoken_50|>": 50306,
"<|extratoken_51|>": 50307,
"<|extratoken_52|>": 50308,
"<|extratoken_53|>": 50309,
"<|extratoken_54|>": 50310,
"<|extratoken_55|>": 50311,
"<|extratoken_56|>": 50312,
"<|extratoken_57|>": 50313,
"<|extratoken_58|>": 50314,
"<|extratoken_59|>": 50315,
"<|extratoken_5|>": 50261,
"<|extratoken_60|>": 50316,
"<|extratoken_61|>": 50317,
"<|extratoken_62|>": 50318,
"<|extratoken_63|>": 50319,
"<|extratoken_64|>": 50320,
"<|extratoken_65|>": 50321,
"<|extratoken_66|>": 50322,
"<|extratoken_67|>": 50323,
"<|extratoken_68|>": 50324,
"<|extratoken_69|>": 50325,
"<|extratoken_6|>": 50262,
"<|extratoken_70|>": 50326,
"<|extratoken_71|>": 50327,
"<|extratoken_72|>": 50328,
"<|extratoken_73|>": 50329,
"<|extratoken_74|>": 50330,
"<|extratoken_75|>": 50331,
"<|extratoken_76|>": 50332,
"<|extratoken_77|>": 50333,
"<|extratoken_78|>": 50334,
"<|extratoken_79|>": 50335,
"<|extratoken_7|>": 50263,
"<|extratoken_80|>": 50336,
"<|extratoken_81|>": 50337,
"<|extratoken_82|>": 50338,
"<|extratoken_83|>": 50339,
"<|extratoken_84|>": 50340,
"<|extratoken_85|>": 50341,
"<|extratoken_86|>": 50342,
"<|extratoken_87|>": 50343,
"<|extratoken_88|>": 50344,
"<|extratoken_89|>": 50345,
"<|extratoken_8|>": 50264,
"<|extratoken_90|>": 50346,
"<|extratoken_91|>": 50347,
"<|extratoken_92|>": 50348,
"<|extratoken_93|>": 50349,
"<|extratoken_94|>": 50350,
"<|extratoken_95|>": 50351,
"<|extratoken_96|>": 50352,
"<|extratoken_97|>": 50353,
"<|extratoken_98|>": 50354,
"<|extratoken_99|>": 50355,
"<|extratoken_9|>": 50265
}

42
config.json Executable file
View File

@@ -0,0 +1,42 @@
{
"_name_or_path": "EleutherAI/gpt-j-6B",
"activation_function": "gelu_new",
"architectures": [
"GPTJForCausalLM"
],
"attn_pdrop": 0.0,
"bos_token_id": 50256,
"embd_pdrop": 0.0,
"eos_token_id": 50256,
"gradient_checkpointing": false,
"initializer_range": 0.02,
"layer_norm_epsilon": 1e-05,
"model_type": "gptj",
"n_embd": 4096,
"n_head": 16,
"n_inner": null,
"n_layer": 28,
"n_positions": 2048,
"resid_pdrop": 0.0,
"rotary": true,
"rotary_dim": 64,
"scale_attn_weights": true,
"summary_activation": null,
"summary_first_dropout": 0.1,
"summary_proj_to_labels": true,
"summary_type": "cls_index",
"summary_use_proj": true,
"task_specific_params": {
"text-generation": {
"do_sample": true,
"max_length": 50,
"temperature": 1.0
}
},
"tie_word_embeddings": false,
"tokenizer_class": "GPT2Tokenizer",
"torch_dtype": "bfloat16",
"transformers_version": "4.25.1",
"use_cache": false,
"vocab_size": 50403
}

11
configuration.json Normal file
View File

@@ -0,0 +1,11 @@
{
"framework": "pytorch",
"task": "text-generation",
"model": {
"type": "DollyV16b"
},
"pipeline": {
"type": "DollyV16b-text-generation"
},
"allow_remote": true
}

50001
merges.txt Executable file
View File

File diff suppressed because it is too large Load Diff

98
ms_wrapper.py Normal file
View File

@@ -0,0 +1,98 @@
# Copyright (c) 2022 Zhipu.AI
import torch
from modelscope.pipelines.builder import PIPELINES
from modelscope.pipelines.base import Pipeline
from modelscope.utils.constant import Tasks
from modelscope.models.builder import MODELS
from modelscope.utils.logger import get_logger
from modelscope.models.base import Model, TorchModel
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers import PreTrainedModel, PreTrainedTokenizer
import numpy as np
from typing import Union, Dict, Any
PROMPT_FORMAT = """Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Response:
"""
def generate_response(instruction: str, *, model: PreTrainedModel, tokenizer: PreTrainedTokenizer,
do_sample: bool = True, max_new_tokens: int = 256, top_p: float = 0.92, top_k: int = 0, device=None, **kwargs) -> str:
input_ids = tokenizer(PROMPT_FORMAT.format(instruction=instruction), return_tensors="pt").input_ids
# each of these is encoded to a single token
response_key_token_id = tokenizer.encode("### Response:")[0]
end_key_token_id = tokenizer.encode("### End")[0]
input_ids = input_ids.to(torch.device(device))
model = model.to(torch.device(device))
gen_tokens = model.generate(input_ids, pad_token_id=tokenizer.pad_token_id, eos_token_id=end_key_token_id,
do_sample=do_sample, max_new_tokens=max_new_tokens, top_p=top_p, top_k=top_k, **kwargs)[0].cpu()
# find where the response begins
response_positions = np.where(gen_tokens == response_key_token_id)[0]
if len(response_positions) >= 0:
response_pos = response_positions[0]
# find where the response ends
end_pos = None
end_positions = np.where(gen_tokens == end_key_token_id)[0]
if len(end_positions) > 0:
end_pos = end_positions[0]
return tokenizer.decode(gen_tokens[response_pos + 1 : end_pos]).strip()
return None
@PIPELINES.register_module(Tasks.text_generation, module_name='DollyV16b-text-generation')
class DollyV16bTextGenerationPipeline(Pipeline):
def __init__(self,
model: Union[Model, str],
*args,
**kwargs):
device = kwargs.get('device')
model = DollyV16bTextGeneration(model_dir=model,device=device) if isinstance(model, str) else model
self.model = model
self.model.eval()
super().__init__(model=model, **kwargs)
def preprocess(self, inputs, **preprocess_params) -> Dict[str, Any]:
return inputs
# define the forward pass
def forward(self, inputs: Dict, **forward_params) -> Dict[str, Any]:
return self.model(inputs)
# format the outputs from pipeline
def postprocess(self, input, **kwargs) -> Dict[str, Any]:
return input
@MODELS.register_module(Tasks.text_generation, module_name='DollyV16b')
class DollyV16bTextGeneration(TorchModel):
def __init__(self, model_dir: str, device=None, *args, **kwargs):
super().__init__(model_dir, *args, **kwargs)
self.logger = get_logger()
self.device = device
# loading tokenizer
self.tokenizer = AutoTokenizer.from_pretrained(model_dir, padding_side="left")
# loading model
self.model = AutoModelForCausalLM.from_pretrained(model_dir, trust_remote_code=True)
def forward(self, input: Dict) -> Dict:
output = {}
res = generate_response(input,model=self.model,tokenizer=self.tokenizer,device=self.device)
output['text'] = res
return output
def quantize(self, bits: int):
self.model = self.model.quantize(bits)
return self

3
pytorch_model.bin Executable file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2e4b80d2f76b00d5a6070c80a2f964afa99d9af6983cd9afb283a05fba5e77e8
size 12219365353

29
special_tokens_map.json Executable file
View File

@@ -0,0 +1,29 @@
{
"additional_special_tokens": [
"### End",
"### Instruction:",
"### Response:"
],
"bos_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"pad_token": "<|endoftext|>",
"unk_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
}
}

101618
tokenizer.json Executable file
View File

File diff suppressed because it is too large Load Diff

34
tokenizer_config.json Executable file
View File

@@ -0,0 +1,34 @@
{
"add_bos_token": false,
"add_prefix_space": false,
"bos_token": {
"__type": "AddedToken",
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"eos_token": {
"__type": "AddedToken",
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"errors": "replace",
"model_max_length": 2048,
"name_or_path": "EleutherAI/gpt-j-6B",
"pad_token": null,
"special_tokens_map_file": null,
"tokenizer_class": "GPT2Tokenizer",
"unk_token": {
"__type": "AddedToken",
"content": "<|endoftext|>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
}
}

1
vocab.json Executable file
View File

File diff suppressed because one or more lines are too long