初始化项目，由ModelHub XC社区提供模型
Model: TheBloke/WizardCoder-33B-V1.1-GPTQ Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
model.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
71
LICENSE
Normal file
71
LICENSE
Normal file
@@ -0,0 +1,71 @@
|
||||
MICROSOFT RESEARCH LICENSE TERMS
|
||||
|
||||
IF YOU LIVE IN THE UNITED STATES, PLEASE READ THE “BINDING ARBITRATION AND CLASS ACTION WAIVER” SECTION BELOW. IT AFFECTS HOW DISPUTES ARE RESOLVED.
|
||||
|
||||
These license terms are an agreement between you and Microsoft Corporation (or one of its affiliates). They apply to the source code, object code, machine learning models, or data (collectively “Materials”) that accompany this license. IF YOU COMPLY WITH THESE LICENSE TERMS, YOU HAVE THE RIGHTS BELOW. BY USING THE MATERIALS, YOU ACCEPT THESE TERMS.
|
||||
|
||||
1) INSTALLATION AND USE RIGHTS TO THE MATERIALS.
|
||||
|
||||
Subject to the terms of this agreement, you have the below rights, if applicable, to use the Materials solely for non-commercial, non-revenue generating, research purposes:
|
||||
|
||||
a) Source Code. If source code is included, you may use and modify the source code, but you may not distribute the source code.
|
||||
|
||||
b) Object Code. If object code is included, you may use the object code, but you may not distribute the object code.
|
||||
|
||||
c) Models. If machine learning model(s) are included, you may use the model(s), but you may not distribute the models.
|
||||
|
||||
d) Data. If data is included, you may use and modify the data, but your use and modification must be consistent with the consent under which the data was provided and/or gathered and you may not distribute the data or your modifications to the data.
|
||||
|
||||
2) SCOPE OF LICENSE. The Materials are licensed, not sold. Microsoft reserves all other rights. Unless applicable law gives you more rights despite this limitation, you will not (and have no right to):
|
||||
|
||||
a) work around any technical limitations in the Materials that only allow you to use it in certain ways;
|
||||
|
||||
b) reverse engineer, decompile or disassemble the Materials;
|
||||
|
||||
c) remove, minimize, block, or modify any notices of Microsoft or its suppliers in the Materials;
|
||||
|
||||
d) use the Materials in any way that is against the law or to create or propagate malware; or
|
||||
|
||||
e) share, publish, distribute or lend the Materials, provide the Materials as a stand-alone hosted solution for others to use, or transfer the Materials or this agreement to any third party.
|
||||
|
||||
3) PERSONAL DATA. If the data (set forth in Section 1(c) above) includes or is found to include any data that enables any ability to identify an individual (“Personal Data”), you will not use such Personal Data for any purpose other than was authorized and consented to by the data subject/research participant. You will not use Personal Data to contact any person. You will keep Personal Data in strict confidence. You will not share any Personal Data that is collected or in your possession with any third party for any reason and as required under the original consent agreement. Further, you will destroy the Personal Data and any backup or copies, immediately upon the completion of your research.
|
||||
|
||||
4) LICENSE TO MICROSOFT. Notwithstanding the limitations in Section 1, you may distribute your modifications back to Microsoft, and if you do provide Microsoft with modifications of the Materials, you hereby grant Microsoft, without any restrictions or limitations, a non-exclusive, perpetual, irrevocable, royalty-free, assignable and sub-licensable license, to reproduce, publicly perform or display, install, use, modify, post, distribute, make and have made, sell and transfer such modifications and derivatives for any purpose.
|
||||
|
||||
5) PUBLICATION. You may publish (or present papers or articles) on your results from using the Materials provided that no material or substantial portion of the Materials is included in any such publication or presentation.
|
||||
|
||||
6) FEEDBACK. Any feedback about the Materials provided by you to us is voluntarily given, and Microsoft shall be free to use the feedback as it sees fit without obligation or restriction of any kind, even if the
|
||||
|
||||
feedback is designated by you as confidential. Such feedback shall be considered a contribution and licensed to Microsoft under the terms of Section 4 above.
|
||||
|
||||
7) EXPORT RESTRICTIONS. You must comply with all domestic and international export laws and regulations that apply to the Materials, which include restrictions on destinations, end users, and end use. For further information on export restrictions, visit (aka.ms/exporting).
|
||||
|
||||
8) SUPPORT SERVICES. Microsoft is not obligated under this agreement to provide any support services for the Materials. Any support provided is “as is”, “with all faults”, and without warranty of any kind.
|
||||
|
||||
9) BINDING ARBITRATION AND CLASS ACTION WAIVER. This Section applies if you live in (or, if a business, your principal place of business is in) the United States. If you and Microsoft have a dispute, you and Microsoft agree to try for 60 days to resolve it informally. If you and Microsoft can’t, you and Microsoft agree to binding individual arbitration before the American Arbitration Association under the Federal Arbitration Act (“FAA”), and not to sue in court in front of a judge or jury. Instead, a neutral arbitrator will decide. Class action lawsuits, class-wide arbitrations, private attorney-general actions, and any other proceeding where someone acts in a representative capacity are not allowed; nor is combining individual proceedings without the consent of all parties. The complete Arbitration Agreement contains more terms and is at aka.ms/arb-agreement-1. You and Microsoft agree to these terms.
|
||||
|
||||
10) ENTIRE AGREEMENT. This agreement, and any other terms Microsoft may provide for supplements, updates, or third-party applications, is the entire agreement for the Materials.
|
||||
|
||||
11) APPLICABLE LAW AND PLACE TO RESOLVE DISPUTES. If you acquired the Materials in the United States or Canada, the laws of the state or province where you live (or, if a business, where your principal place of business is located) govern the interpretation of this agreement, claims for its breach, and all other claims (including consumer protection, unfair competition, and tort claims), regardless of conflict of laws principles, except that the FAA governs everything related to arbitration. If you acquired the Materials in any other country, its laws apply, except that the FAA governs everything related to arbitration. If U.S. federal jurisdiction exists, you and Microsoft consent to exclusive jurisdiction and venue in the federal court in King County, Washington for all disputes heard in court (excluding arbitration). If not, you and Microsoft consent to exclusive jurisdiction and venue in the Superior Court of King County, Washington for all disputes heard in court (excluding arbitration).
|
||||
|
||||
12) CONSUMER RIGHTS; REGIONAL VARIATIONS. This agreement describes certain legal rights. You may have other rights, including consumer rights, under the laws of your state, province, or country. Separate and apart from your relationship with Microsoft, you may also have rights with respect to the party from which you acquired the Materials. This agreement does not change those other rights if the laws of your state, province, or country do not permit it to do so. For example, if you acquired the Materials in one of the below regions, or mandatory country law applies, then the following provisions apply to you:
|
||||
|
||||
a) Australia. You have statutory guarantees under the Australian Consumer Law and nothing in this agreement is intended to affect those rights.
|
||||
|
||||
b) Canada. If you acquired this software in Canada, you may stop receiving updates by turning off the automatic update feature, disconnecting your device from the Internet (if and when you re-connect to the Internet, however, the Materials will resume checking for and installing updates), or uninstalling the Materials. The product documentation, if any, may also specify how to turn off updates for your specific device or software.
|
||||
|
||||
c) Germany and Austria.
|
||||
|
||||
i. Warranty. The properly licensed software will perform substantially as described in any Microsoft materials that accompany the Materials. However, Microsoft gives no contractual guarantee in relation to the licensed software.
|
||||
|
||||
ii. Limitation of Liability. In case of intentional conduct, gross negligence, claims based on the Product Liability Act, as well as, in case of death or personal or physical injury, Microsoft is liable according to the statutory law.
|
||||
|
||||
Subject to the foregoing clause (ii), Microsoft will only be liable for slight negligence if Microsoft is in breach of such material contractual obligations, the fulfillment of which facilitate the due performance of this agreement, the breach of which would endanger the purpose of this agreement and the compliance with which a party may constantly trust in (so-called "cardinal obligations"). In other cases of slight negligence, Microsoft will not be liable for slight negligence.
|
||||
|
||||
13) DISCLAIMER OF WARRANTY. THE MATERIALS ARE LICENSED “AS IS.” YOU BEAR THE RISK OF USING THEM. MICROSOFT GIVES NO EXPRESS WARRANTIES, GUARANTEES, OR CONDITIONS. TO THE EXTENT PERMITTED UNDER APPLICABLE LAWS, MICROSOFT EXCLUDES ALL IMPLIED WARRANTIES, INCLUDING MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
|
||||
|
||||
14) LIMITATION ON AND EXCLUSION OF DAMAGES. IF YOU HAVE ANY BASIS FOR RECOVERING DAMAGES DESPITE THE PRECEDING DISCLAIMER OF WARRANTY, YOU CAN RECOVER FROM MICROSOFT AND ITS SUPPLIERS ONLY DIRECT DAMAGES UP TO U.S. $5.00. YOU CANNOT RECOVER ANY OTHER DAMAGES, INCLUDING CONSEQUENTIAL, LOST PROFITS, SPECIAL, INDIRECT OR INCIDENTAL DAMAGES.
|
||||
|
||||
This limitation applies to (a) anything related to the Materials, services, content (including code) on third party Internet sites, or third party applications; and (b) claims for breach of contract, warranty, guarantee, or condition; strict liability, negligence, or other tort; or any other claim; in each case to the extent permitted by applicable law.
|
||||
|
||||
It also applies even if Microsoft knew or should have known about the possibility of the damages. The above limitation or exclusion may not apply to you because your state, province, or country may not allow the exclusion or limitation of incidental, consequential, or other damages.
|
||||
636
README.md
Normal file
636
README.md
Normal file
@@ -0,0 +1,636 @@
|
||||
---
|
||||
base_model: WizardLM/WizardCoder-33B-V1.1
|
||||
inference: false
|
||||
library_name: transformers
|
||||
metrics:
|
||||
- code_eval
|
||||
model-index:
|
||||
- name: WizardCoder
|
||||
results:
|
||||
- dataset:
|
||||
name: HumanEval
|
||||
type: openai_humaneval
|
||||
metrics:
|
||||
- name: pass@1
|
||||
type: pass@1
|
||||
value: 0.799
|
||||
verified: false
|
||||
task:
|
||||
type: text-generation
|
||||
model_creator: WizardLM
|
||||
model_name: Wizardcoder 33B V1.1
|
||||
model_type: deepseek
|
||||
prompt_template: 'Below is an instruction that describes a task. Write a response
|
||||
that appropriately completes the request.
|
||||
|
||||
|
||||
### Instruction:
|
||||
|
||||
{prompt}
|
||||
|
||||
|
||||
### Response:
|
||||
|
||||
'
|
||||
quantized_by: TheBloke
|
||||
tags:
|
||||
- code
|
||||
---
|
||||
<!-- markdownlint-disable MD041 -->
|
||||
|
||||
<!-- header start -->
|
||||
<!-- 200823 -->
|
||||
<div style="width: auto; margin-left: auto; margin-right: auto">
|
||||
<img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
||||
</div>
|
||||
<div style="display: flex; justify-content: space-between; width: 100%;">
|
||||
<div style="display: flex; flex-direction: column; align-items: flex-start;">
|
||||
<p style="margin-top: 0.5em; margin-bottom: 0em;"><a href="https://discord.gg/theblokeai">Chat & support: TheBloke's Discord server</a></p>
|
||||
</div>
|
||||
<div style="display: flex; flex-direction: column; align-items: flex-end;">
|
||||
<p style="margin-top: 0.5em; margin-bottom: 0em;"><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
|
||||
</div>
|
||||
</div>
|
||||
<div style="text-align:center; margin-top: 0em; margin-bottom: 0em"><p style="margin-top: 0.25em; margin-bottom: 0em;">TheBloke's LLM work is generously supported by a grant from <a href="https://a16z.com">andreessen horowitz (a16z)</a></p></div>
|
||||
<hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
|
||||
<!-- header end -->
|
||||
|
||||
# Wizardcoder 33B V1.1 - GPTQ
|
||||
- Model creator: [WizardLM](https://huggingface.co/WizardLM)
|
||||
- Original model: [Wizardcoder 33B V1.1](https://huggingface.co/WizardLM/WizardCoder-33B-V1.1)
|
||||
|
||||
<!-- description start -->
|
||||
# Description
|
||||
|
||||
This repo contains GPTQ model files for [WizardLM's Wizardcoder 33B V1.1](https://huggingface.co/WizardLM/WizardCoder-33B-V1.1).
|
||||
|
||||
Multiple GPTQ parameter permutations are provided; see Provided Files below for details of the options provided, their parameters, and the software used to create them.
|
||||
|
||||
These files were quantised using hardware kindly provided by [Massed Compute](https://massedcompute.com/).
|
||||
|
||||
<!-- description end -->
|
||||
<!-- repositories-available start -->
|
||||
## Repositories available
|
||||
|
||||
* [AWQ model(s) for GPU inference.](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-AWQ)
|
||||
* [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GPTQ)
|
||||
* [2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GGUF)
|
||||
* [WizardLM's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/WizardLM/WizardCoder-33B-V1.1)
|
||||
<!-- repositories-available end -->
|
||||
|
||||
<!-- prompt-template start -->
|
||||
## Prompt template: Alpaca
|
||||
|
||||
```
|
||||
Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
||||
|
||||
### Instruction:
|
||||
{prompt}
|
||||
|
||||
### Response:
|
||||
|
||||
```
|
||||
|
||||
<!-- prompt-template end -->
|
||||
|
||||
|
||||
|
||||
<!-- README_GPTQ.md-compatible clients start -->
|
||||
## Known compatible clients / servers
|
||||
|
||||
GPTQ models are currently supported on Linux (NVidia/AMD) and Windows (NVidia only). macOS users: please use GGUF models.
|
||||
|
||||
These GPTQ models are known to work in the following inference servers/webuis.
|
||||
|
||||
- [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
|
||||
- [KoboldAI United](https://github.com/henk717/koboldai)
|
||||
- [LoLLMS Web UI](https://github.com/ParisNeo/lollms-webui)
|
||||
- [Hugging Face Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference)
|
||||
|
||||
This may not be a complete list; if you know of others, please let me know!
|
||||
<!-- README_GPTQ.md-compatible clients end -->
|
||||
|
||||
<!-- README_GPTQ.md-provided-files start -->
|
||||
## Provided files, and GPTQ parameters
|
||||
|
||||
Multiple quantisation parameters are provided, to allow you to choose the best one for your hardware and requirements.
|
||||
|
||||
Each separate quant is in a different branch. See below for instructions on fetching from different branches.
|
||||
|
||||
Most GPTQ files are made with AutoGPTQ. Mistral models are currently made with Transformers.
|
||||
|
||||
<details>
|
||||
<summary>Explanation of GPTQ parameters</summary>
|
||||
|
||||
- Bits: The bit size of the quantised model.
|
||||
- GS: GPTQ group size. Higher numbers use less VRAM, but have lower quantisation accuracy. "None" is the lowest possible value.
|
||||
- Act Order: True or False. Also known as `desc_act`. True results in better quantisation accuracy. Some GPTQ clients have had issues with models that use Act Order plus Group Size, but this is generally resolved now.
|
||||
- Damp %: A GPTQ parameter that affects how samples are processed for quantisation. 0.01 is default, but 0.1 results in slightly better accuracy.
|
||||
- GPTQ dataset: The calibration dataset used during quantisation. Using a dataset more appropriate to the model's training can improve quantisation accuracy. Note that the GPTQ calibration dataset is not the same as the dataset used to train the model - please refer to the original model repo for details of the training dataset(s).
|
||||
- Sequence Length: The length of the dataset sequences used for quantisation. Ideally this is the same as the model sequence length. For some very long sequence models (16+K), a lower sequence length may have to be used. Note that a lower sequence length does not limit the sequence length of the quantised model. It only impacts the quantisation accuracy on longer inference sequences.
|
||||
- ExLlama Compatibility: Whether this file can be loaded with ExLlama, which currently only supports Llama and Mistral models in 4-bit.
|
||||
|
||||
</details>
|
||||
|
||||
| Branch | Bits | GS | Act Order | Damp % | GPTQ Dataset | Seq Len | Size | ExLlama | Desc |
|
||||
| ------ | ---- | -- | --------- | ------ | ------------ | ------- | ---- | ------- | ---- |
|
||||
| [main](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GPTQ/tree/main) | 4 | None | Yes | 0.1 | [Evol Instruct Code](https://huggingface.co/datasets/nickrosh/Evol-Instruct-Code-80k-v1/viewer/) | 8192 | 17.40 GB | Yes | 4-bit, with Act Order. No group size, to lower VRAM requirements. |
|
||||
| [gptq-4bit-128g-actorder_True](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GPTQ/tree/gptq-4bit-128g-actorder_True) | 4 | 128 | Yes | 0.1 | [Evol Instruct Code](https://huggingface.co/datasets/nickrosh/Evol-Instruct-Code-80k-v1/viewer/) | 8192 | 18.03 GB | Yes | 4-bit, with Act Order and group size 128g. Uses even less VRAM than 64g, but with slightly lower accuracy. |
|
||||
| [gptq-4bit-32g-actorder_True](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GPTQ/tree/gptq-4bit-32g-actorder_True) | 4 | 32 | Yes | 0.1 | [Evol Instruct Code](https://huggingface.co/datasets/nickrosh/Evol-Instruct-Code-80k-v1/viewer/) | 8192 | 19.96 GB | Yes | 4-bit, with Act Order and group size 32g. Gives highest possible inference quality, with maximum VRAM usage. |
|
||||
| [gptq-3bit-128g-actorder_True](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GPTQ/tree/gptq-3bit-128g-actorder_True) | 3 | 128 | Yes | 0.1 | [Evol Instruct Code](https://huggingface.co/datasets/nickrosh/Evol-Instruct-Code-80k-v1/viewer/) | 8192 | 13.89 GB | No | 3-bit, with group size 128g and act-order. Higher quality than 128g-False. |
|
||||
| [gptq-8bit--1g-actorder_True](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GPTQ/tree/gptq-8bit--1g-actorder_True) | 8 | None | Yes | 0.1 | [Evol Instruct Code](https://huggingface.co/datasets/nickrosh/Evol-Instruct-Code-80k-v1/viewer/) | 8192 | 33.84 GB | No | 8-bit, with Act Order. No group size, to lower VRAM requirements. |
|
||||
| [gptq-3bit-32g-actorder_True](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GPTQ/tree/gptq-3bit-32g-actorder_True) | 3 | 32 | Yes | 0.1 | [Evol Instruct Code](https://huggingface.co/datasets/nickrosh/Evol-Instruct-Code-80k-v1/viewer/) | 8192 | 15.72 GB | No | 3-bit, with group size 64g and act-order. Highest quality 3-bit option. |
|
||||
| [gptq-8bit-128g-actorder_True](https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GPTQ/tree/gptq-8bit-128g-actorder_True) | 8 | 128 | Yes | 0.1 | [Evol Instruct Code](https://huggingface.co/datasets/nickrosh/Evol-Instruct-Code-80k-v1/viewer/) | 8192 | 34.60 GB | No | 8-bit, with group size 128g for higher inference quality and with Act Order for even higher accuracy. |
|
||||
|
||||
<!-- README_GPTQ.md-provided-files end -->
|
||||
|
||||
<!-- README_GPTQ.md-download-from-branches start -->
|
||||
## How to download, including from branches
|
||||
|
||||
### In text-generation-webui
|
||||
|
||||
To download from the `main` branch, enter `TheBloke/WizardCoder-33B-V1.1-GPTQ` in the "Download model" box.
|
||||
|
||||
To download from another branch, add `:branchname` to the end of the download name, eg `TheBloke/WizardCoder-33B-V1.1-GPTQ:gptq-4bit-128g-actorder_True`
|
||||
|
||||
### From the command line
|
||||
|
||||
I recommend using the `huggingface-hub` Python library:
|
||||
|
||||
```shell
|
||||
pip3 install huggingface-hub
|
||||
```
|
||||
|
||||
To download the `main` branch to a folder called `WizardCoder-33B-V1.1-GPTQ`:
|
||||
|
||||
```shell
|
||||
mkdir WizardCoder-33B-V1.1-GPTQ
|
||||
huggingface-cli download TheBloke/WizardCoder-33B-V1.1-GPTQ --local-dir WizardCoder-33B-V1.1-GPTQ --local-dir-use-symlinks False
|
||||
```
|
||||
|
||||
To download from a different branch, add the `--revision` parameter:
|
||||
|
||||
```shell
|
||||
mkdir WizardCoder-33B-V1.1-GPTQ
|
||||
huggingface-cli download TheBloke/WizardCoder-33B-V1.1-GPTQ --revision gptq-4bit-128g-actorder_True --local-dir WizardCoder-33B-V1.1-GPTQ --local-dir-use-symlinks False
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary>More advanced huggingface-cli download usage</summary>
|
||||
|
||||
If you remove the `--local-dir-use-symlinks False` parameter, the files will instead be stored in the central Hugging Face cache directory (default location on Linux is: `~/.cache/huggingface`), and symlinks will be added to the specified `--local-dir`, pointing to their real location in the cache. This allows for interrupted downloads to be resumed, and allows you to quickly clone the repo to multiple places on disk without triggering a download again. The downside, and the reason why I don't list that as the default option, is that the files are then hidden away in a cache folder and it's harder to know where your disk space is being used, and to clear it up if/when you want to remove a download model.
|
||||
|
||||
The cache location can be changed with the `HF_HOME` environment variable, and/or the `--cache-dir` parameter to `huggingface-cli`.
|
||||
|
||||
For more documentation on downloading with `huggingface-cli`, please see: [HF -> Hub Python Library -> Download files -> Download from the CLI](https://huggingface.co/docs/huggingface_hub/guides/download#download-from-the-cli).
|
||||
|
||||
To accelerate downloads on fast connections (1Gbit/s or higher), install `hf_transfer`:
|
||||
|
||||
```shell
|
||||
pip3 install hf_transfer
|
||||
```
|
||||
|
||||
And set environment variable `HF_HUB_ENABLE_HF_TRANSFER` to `1`:
|
||||
|
||||
```shell
|
||||
mkdir WizardCoder-33B-V1.1-GPTQ
|
||||
HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download TheBloke/WizardCoder-33B-V1.1-GPTQ --local-dir WizardCoder-33B-V1.1-GPTQ --local-dir-use-symlinks False
|
||||
```
|
||||
|
||||
Windows Command Line users: You can set the environment variable by running `set HF_HUB_ENABLE_HF_TRANSFER=1` before the download command.
|
||||
</details>
|
||||
|
||||
### With `git` (**not** recommended)
|
||||
|
||||
To clone a specific branch with `git`, use a command like this:
|
||||
|
||||
```shell
|
||||
git clone --single-branch --branch gptq-4bit-128g-actorder_True https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GPTQ
|
||||
```
|
||||
|
||||
Note that using Git with HF repos is strongly discouraged. It will be much slower than using `huggingface-hub`, and will use twice as much disk space as it has to store the model files twice (it stores every byte both in the intended target folder, and again in the `.git` folder as a blob.)
|
||||
|
||||
<!-- README_GPTQ.md-download-from-branches end -->
|
||||
<!-- README_GPTQ.md-text-generation-webui start -->
|
||||
## How to easily download and use this model in [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
|
||||
|
||||
Please make sure you're using the latest version of [text-generation-webui](https://github.com/oobabooga/text-generation-webui).
|
||||
|
||||
It is strongly recommended to use the text-generation-webui one-click-installers unless you're sure you know how to make a manual install.
|
||||
|
||||
1. Click the **Model tab**.
|
||||
2. Under **Download custom model or LoRA**, enter `TheBloke/WizardCoder-33B-V1.1-GPTQ`.
|
||||
|
||||
- To download from a specific branch, enter for example `TheBloke/WizardCoder-33B-V1.1-GPTQ:gptq-4bit-128g-actorder_True`
|
||||
- see Provided Files above for the list of branches for each option.
|
||||
|
||||
3. Click **Download**.
|
||||
4. The model will start downloading. Once it's finished it will say "Done".
|
||||
5. In the top left, click the refresh icon next to **Model**.
|
||||
6. In the **Model** dropdown, choose the model you just downloaded: `WizardCoder-33B-V1.1-GPTQ`
|
||||
7. The model will automatically load, and is now ready for use!
|
||||
8. If you want any custom settings, set them and then click **Save settings for this model** followed by **Reload the Model** in the top right.
|
||||
|
||||
- Note that you do not need to and should not set manual GPTQ parameters any more. These are set automatically from the file `quantize_config.json`.
|
||||
|
||||
9. Once you're ready, click the **Text Generation** tab and enter a prompt to get started!
|
||||
|
||||
<!-- README_GPTQ.md-text-generation-webui end -->
|
||||
|
||||
<!-- README_GPTQ.md-use-from-tgi start -->
|
||||
## Serving this model from Text Generation Inference (TGI)
|
||||
|
||||
It's recommended to use TGI version 1.1.0 or later. The official Docker container is: `ghcr.io/huggingface/text-generation-inference:1.1.0`
|
||||
|
||||
Example Docker parameters:
|
||||
|
||||
```shell
|
||||
--model-id TheBloke/WizardCoder-33B-V1.1-GPTQ --port 3000 --quantize gptq --max-input-length 3696 --max-total-tokens 4096 --max-batch-prefill-tokens 4096
|
||||
```
|
||||
|
||||
Example Python code for interfacing with TGI (requires huggingface-hub 0.17.0 or later):
|
||||
|
||||
```shell
|
||||
pip3 install huggingface-hub
|
||||
```
|
||||
|
||||
```python
|
||||
from huggingface_hub import InferenceClient
|
||||
|
||||
endpoint_url = "https://your-endpoint-url-here"
|
||||
|
||||
prompt = "Tell me about AI"
|
||||
prompt_template=f'''Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
||||
|
||||
### Instruction:
|
||||
{prompt}
|
||||
|
||||
### Response:
|
||||
'''
|
||||
|
||||
client = InferenceClient(endpoint_url)
|
||||
response = client.text_generation(
|
||||
prompt_template,
|
||||
max_new_tokens=128,
|
||||
do_sample=True,
|
||||
temperature=0.7,
|
||||
top_p=0.95,
|
||||
top_k=40,
|
||||
repetition_penalty=1.1
|
||||
)
|
||||
|
||||
print(f"Model output: {response}")
|
||||
```
|
||||
<!-- README_GPTQ.md-use-from-tgi end -->
|
||||
<!-- README_GPTQ.md-use-from-python start -->
|
||||
## Python code example: inference from this GPTQ model
|
||||
|
||||
### Install the necessary packages
|
||||
|
||||
Requires: Transformers 4.33.0 or later, Optimum 1.12.0 or later, and AutoGPTQ 0.4.2 or later.
|
||||
|
||||
```shell
|
||||
pip3 install --upgrade transformers optimum
|
||||
# If using PyTorch 2.1 + CUDA 12.x:
|
||||
pip3 install --upgrade auto-gptq
|
||||
# or, if using PyTorch 2.1 + CUDA 11.x:
|
||||
pip3 install --upgrade auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/
|
||||
```
|
||||
|
||||
If you are using PyTorch 2.0, you will need to install AutoGPTQ from source. Likewise if you have problems with the pre-built wheels, you should try building from source:
|
||||
|
||||
```shell
|
||||
pip3 uninstall -y auto-gptq
|
||||
git clone https://github.com/PanQiWei/AutoGPTQ
|
||||
cd AutoGPTQ
|
||||
git checkout v0.5.1
|
||||
pip3 install .
|
||||
```
|
||||
|
||||
### Example Python code
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
|
||||
|
||||
model_name_or_path = "TheBloke/WizardCoder-33B-V1.1-GPTQ"
|
||||
# To use a different branch, change revision
|
||||
# For example: revision="gptq-4bit-128g-actorder_True"
|
||||
model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
|
||||
device_map="auto",
|
||||
trust_remote_code=False,
|
||||
revision="main")
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)
|
||||
|
||||
prompt = "Write a story about llamas"
|
||||
system_message = "You are a story writing assistant"
|
||||
prompt_template=f'''Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
||||
|
||||
### Instruction:
|
||||
{prompt}
|
||||
|
||||
### Response:
|
||||
'''
|
||||
|
||||
print("\n\n*** Generate:")
|
||||
|
||||
input_ids = tokenizer(prompt_template, return_tensors='pt').input_ids.cuda()
|
||||
output = model.generate(inputs=input_ids, temperature=0.7, do_sample=True, top_p=0.95, top_k=40, max_new_tokens=512)
|
||||
print(tokenizer.decode(output[0]))
|
||||
|
||||
# Inference can also be done using transformers' pipeline
|
||||
|
||||
print("*** Pipeline:")
|
||||
pipe = pipeline(
|
||||
"text-generation",
|
||||
model=model,
|
||||
tokenizer=tokenizer,
|
||||
max_new_tokens=512,
|
||||
do_sample=True,
|
||||
temperature=0.7,
|
||||
top_p=0.95,
|
||||
top_k=40,
|
||||
repetition_penalty=1.1
|
||||
)
|
||||
|
||||
print(pipe(prompt_template)[0]['generated_text'])
|
||||
```
|
||||
<!-- README_GPTQ.md-use-from-python end -->
|
||||
|
||||
<!-- README_GPTQ.md-compatibility start -->
|
||||
## Compatibility
|
||||
|
||||
The files provided are tested to work with Transformers. For non-Mistral models, AutoGPTQ can also be used directly.
|
||||
|
||||
[ExLlama](https://github.com/turboderp/exllama) is compatible with Llama architecture models (including Mistral, Yi, DeepSeek, SOLAR, etc) in 4-bit. Please see the Provided Files table above for per-file compatibility.
|
||||
|
||||
For a list of clients/servers, please see "Known compatible clients / servers", above.
|
||||
<!-- README_GPTQ.md-compatibility end -->
|
||||
|
||||
<!-- footer start -->
|
||||
<!-- 200823 -->
|
||||
## Discord
|
||||
|
||||
For further support, and discussions on these models and AI in general, join us at:
|
||||
|
||||
[TheBloke AI's Discord server](https://discord.gg/theblokeai)
|
||||
|
||||
## Thanks, and how to contribute
|
||||
|
||||
Thanks to the [chirper.ai](https://chirper.ai) team!
|
||||
|
||||
Thanks to Clay from [gpus.llm-utils.org](llm-utils)!
|
||||
|
||||
I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.
|
||||
|
||||
If you're able and willing to contribute it will be most gratefully received and will help me to keep providing more models, and to start work on new AI projects.
|
||||
|
||||
Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
|
||||
|
||||
* Patreon: https://patreon.com/TheBlokeAI
|
||||
* Ko-Fi: https://ko-fi.com/TheBlokeAI
|
||||
|
||||
**Special thanks to**: Aemon Algiz.
|
||||
|
||||
**Patreon special mentions**: Michael Levine, 阿明, Trailburnt, Nikolai Manek, John Detwiler, Randy H, Will Dee, Sebastain Graf, NimbleBox.ai, Eugene Pentland, Emad Mostaque, Ai Maven, Jim Angel, Jeff Scroggin, Michael Davis, Manuel Alberto Morcote, Stephen Murray, Robert, Justin Joy, Luke @flexchar, Brandon Frisco, Elijah Stavena, S_X, Dan Guido, Undi ., Komninos Chatzipapas, Shadi, theTransient, Lone Striker, Raven Klaugh, jjj, Cap'n Zoog, Michel-Marie MAUDET (LINAGORA), Matthew Berman, David, Fen Risland, Omer Bin Jawed, Luke Pendergrass, Kalila, OG, Erik Bjäreholt, Rooh Singh, Joseph William Delisle, Dan Lewis, TL, John Villwock, AzureBlack, Brad, Pedro Madruga, Caitlyn Gatomon, K, jinyuan sun, Mano Prime, Alex, Jeffrey Morgan, Alicia Loh, Illia Dulskyi, Chadd, transmissions 11, fincy, Rainer Wilmers, ReadyPlayerEmma, knownsqashed, Mandus, biorpg, Deo Leter, Brandon Phillips, SuperWojo, Sean Connelly, Iucharbius, Jack West, Harry Royden McLaughlin, Nicholas, terasurfer, Vitor Caleffi, Duane Dunston, Johann-Peter Hartmann, David Ziegler, Olakabola, Ken Nordquist, Trenton Dambrowitz, Tom X Nguyen, Vadim, Ajan Kanaga, Leonard Tan, Clay Pascal, Alexandros Triantafyllidis, JM33133, Xule, vamX, ya boyyy, subjectnull, Talal Aujan, Alps Aficionado, wassieverse, Ari Malik, James Bentley, Woland, Spencer Kim, Michael Dempsey, Fred von Graf, Elle, zynix, William Richards, Stanislav Ovsiannikov, Edmond Seymore, Jonathan Leane, Martin Kemka, usrbinkat, Enrico Ros
|
||||
|
||||
|
||||
Thank you to all my generous patrons and donaters!
|
||||
|
||||
And thank you again to a16z for their generous grant.
|
||||
|
||||
<!-- footer end -->
|
||||
|
||||
# Original model card: WizardLM's Wizardcoder 33B V1.1
|
||||
|
||||
|
||||
## WizardCoder: Empowering Code Large Language Models with Evol-Instruct
|
||||
|
||||
<p style="font-size:28px;" align="center">
|
||||
🏠 <a href="https://wizardlm.github.io/" target="_blank">Home Page</a> </p>
|
||||
<p align="center">
|
||||
<p align="center">
|
||||
🤗 <a href="https://huggingface.co/WizardLM" target="_blank">HF Repo</a> •🐱 <a href="https://github.com/nlpxucan/WizardLM" target="_blank">Github Repo</a> • 🐦 <a href="https://twitter.com/WizardLM_AI" target="_blank">Twitter</a> </p>
|
||||
<p align="center">
|
||||
📃 <a href="https://arxiv.org/abs/2304.12244" target="_blank">[WizardLM]</a> • 📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a> • 📃 <a href="https://arxiv.org/abs/2308.09583" target="_blank">[WizardMath]</a> <br>
|
||||
</p>
|
||||
<p align="center">
|
||||
👋 Join our <a href="https://discord.gg/VZjjHtWrKs" target="_blank">Discord</a>
|
||||
</p>
|
||||
|
||||
## News
|
||||
|
||||
[2023/01/04] 🔥 We released **WizardCoder-33B-V1.1** trained from deepseek-coder-33b-base, the **SOTA OSS Code LLM** on [EvalPlus Leaderboard](https://evalplus.github.io/leaderboard.html), achieves **79.9 pass@1** on HumanEval, **73.2 pass@1** on HumanEval-Plus, **78.9 pass@1** on MBPP, and **66.9 pass@1** on MBPP-Plus.
|
||||
|
||||
[2023/01/04] 🔥 **WizardCoder-33B-V1.1** outperforms **ChatGPT 3.5**, **Gemini Pro**, and **DeepSeek-Coder-33B-instruct** on HumanEval and HumanEval-Plus pass@1.
|
||||
|
||||
[2023/01/04] 🔥 **WizardCoder-33B-V1.1** is comparable with **ChatGPT 3.5**, and surpasses **Gemini Pro** on MBPP and MBPP-Plus pass@1.
|
||||
|
||||
| Model | Checkpoint | Paper | HumanEval | HumanEval+ | MBPP | MBPP+ | License |
|
||||
| ----- |------| ---- |------|-------| ----- | ----- |----- |
|
||||
| GPT-4-Turbo (Nov 2023) | - | - | 85.4 | 81.7 | 83.0 | 70.7 |-|
|
||||
| GPT-4 (May 2023) | - | - | 88.4 | 76.8 | - | - |-|
|
||||
| GPT-3.5-Turbo (Nov 2023) | - | - | 72.6 | 65.9 | 81.7 | 69.4 |-|
|
||||
| Gemini Pro | - | - | 63.4 | 55.5 | 72.9 | 57.9 |-|
|
||||
| DeepSeek-Coder-33B-instruct | - | - | 78.7 | 72.6 | 78.7 | 66.7 |-|
|
||||
| **WizardCoder-33B-V1.1** | 🤗 <a href="https://huggingface.co/WizardLM/WizardCoder-33B-V1.1" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a> | 79.9 | 73.2 | 78.9 | 66.9 | <a href="https://huggingface.co/WizardLM/WizardMath-7B-V1.1/resolve/main/LICENSE" target="_blank">MSFTResearch</a> |
|
||||
| WizardCoder-Python-34B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardCoder-Python-34B-V1.0" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a> | 73.2 | 64.6 | 73.2 | 59.9 | <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama2</a> |
|
||||
| WizardCoder-15B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardCoder-15B-V1.0" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a> | 59.8 | 52.4 | -- | -- | <a href="https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement" target="_blank">OpenRAIL-M</a> |
|
||||
| WizardCoder-Python-13B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardCoder-Python-13B-V1.0" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a> | 64.0 | -- | -- | -- | <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama2</a> |
|
||||
| WizardCoder-Python-7B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardCoder-Python-7B-V1.0" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a> | 55.5 | -- | -- | -- | <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama2</a> |
|
||||
| WizardCoder-3B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardCoder-3B-V1.0" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a> | 34.8 | -- | -- | -- | <a href="https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement" target="_blank">OpenRAIL-M</a> |
|
||||
| WizardCoder-1B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardCoder-1B-V1.0" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a> | 23.8 | -- | -- | -- | <a href="https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement" target="_blank">OpenRAIL-M</a> |
|
||||
|
||||
|
||||
## ❗ Data Contamination Check:
|
||||
|
||||
Before model training, we carefully and rigorously checked all the training data, and used multiple deduplication methods to verify and prevent data leakage on HumanEval and MBPP test set.
|
||||
|
||||
🔥
|
||||
❗<b>Note for model system prompts usage:</b>
|
||||
|
||||
Please use **the same systems prompts strictly** with us, and we do not guarantee the accuracy of the **quantified versions**.
|
||||
|
||||
**Default version:**
|
||||
|
||||
```
|
||||
"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:"
|
||||
```
|
||||
|
||||
|
||||
## How to Reproduce the Performance of WizardCoder-33B-V1.1
|
||||
|
||||
We provide all codes [here](https://github.com/nlpxucan/WizardLM/tree/main/WizardCoder/src).
|
||||
|
||||
We also provide all generated [results](https://github.com/nlpxucan/WizardLM/blob/main/WizardCoder/data/humaneval_mbpp_wizardcoder33b_v1.1_results.zip).
|
||||
|
||||
```
|
||||
transformers==4.36.2
|
||||
vllm==0.2.5
|
||||
```
|
||||
|
||||
(1) HumanEval and HumanEval-Plus
|
||||
|
||||
- Step 1
|
||||
|
||||
Code Generation (w/o accelerate)
|
||||
```bash
|
||||
model="WizardLM/WizardCoder-33B-V1.1"
|
||||
temp=0.0
|
||||
max_len=2048
|
||||
pred_num=1
|
||||
num_seqs_per_iter=1
|
||||
|
||||
output_path=preds/T${temp}_N${pred_num}_WizardCoder-33B-V1.1_Greedy_Decode
|
||||
|
||||
mkdir -p ${output_path}
|
||||
echo 'Output path: '$output_path
|
||||
echo 'Model to eval: '$model
|
||||
|
||||
# 164 problems, 21 per GPU if GPU=8
|
||||
index=0
|
||||
gpu_num=8
|
||||
for ((i = 0; i < $gpu_num; i++)); do
|
||||
start_index=$((i * 21))
|
||||
end_index=$(((i + 1) * 21))
|
||||
|
||||
gpu=$((i))
|
||||
echo 'Running process #' ${i} 'from' $start_index 'to' $end_index 'on GPU' ${gpu}
|
||||
((index++))
|
||||
(
|
||||
CUDA_VISIBLE_DEVICES=$gpu python humaneval_gen.py --model ${model} \
|
||||
--start_index ${start_index} --end_index ${end_index} --temperature ${temp} \
|
||||
--num_seqs_per_iter ${num_seqs_per_iter} --N ${pred_num} --max_len ${max_len} --output_path ${output_path} --greedy_decode
|
||||
) &
|
||||
if (($index % $gpu_num == 0)); then wait; fi
|
||||
done
|
||||
```
|
||||
|
||||
Code Generation (w/ vllm accelerate)
|
||||
```bash
|
||||
model="WizardLM/WizardCoder-33B-V1.1"
|
||||
temp=0.0
|
||||
max_len=2048
|
||||
pred_num=1
|
||||
num_seqs_per_iter=1
|
||||
|
||||
output_path=preds/T${temp}_N${pred_num}_WizardCoder-33B-V1.1_Greedy_Decode_vllm
|
||||
|
||||
mkdir -p ${output_path}
|
||||
echo 'Output path: '$output_path
|
||||
echo 'Model to eval: '$model
|
||||
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 python humaneval_gen_vllm.py --model ${model} \
|
||||
--start_index 0 --end_index 164 --temperature ${temp} \
|
||||
--num_seqs_per_iter ${num_seqs_per_iter} --N ${pred_num} --max_len ${max_len} --output_path ${output_path} --num_gpus 4 --overwrite
|
||||
```
|
||||
|
||||
- Step 2: Get the score
|
||||
|
||||
Install [Eval-Plus](https://github.com/evalplus/evalplus) benchmark.
|
||||
```bash
|
||||
git clone https://github.com/evalplus/evalplus.git
|
||||
cd evalplus
|
||||
export PYTHONPATH=$PYTHONPATH:$(pwd)
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
Get HumanEval and HumanEval-Plus scores.
|
||||
```bash
|
||||
output_path=preds/T0.0_N1_WizardCoder-33B-V1.1_Greedy_Decode
|
||||
|
||||
echo 'Output path: '$output_path
|
||||
python process_humaneval.py --path ${output_path} --out_path ${output_path}.jsonl --add_prompt
|
||||
|
||||
evalplus.evaluate --dataset humaneval --samples ${output_path}.jsonl
|
||||
```
|
||||
|
||||
(2) MBPP and MBPP-Plus
|
||||
|
||||
The preprocessed questions are provided in [mbppplus.json](https://github.com/nlpxucan/WizardLM/blob/main/WizardCoder/data/mbppplus.json).
|
||||
|
||||
- Step 1
|
||||
|
||||
Code Generation (w/o accelerate)
|
||||
```bash
|
||||
model="WizardLM/WizardCoder-33B-V1.1"
|
||||
temp=0.0
|
||||
max_len=2048
|
||||
pred_num=1
|
||||
num_seqs_per_iter=1
|
||||
|
||||
output_path=preds/MBPP_T${temp}_N${pred_num}_WizardCoder-33B-V1.1_Greedy_Decode
|
||||
|
||||
mkdir -p ${output_path}
|
||||
echo 'Output path: '$output_path
|
||||
echo 'Model to eval: '$model
|
||||
|
||||
# 399 problems, 50 per GPU if GPU=8
|
||||
index=0
|
||||
gpu_num=8
|
||||
for ((i = 0; i < $gpu_num; i++)); do
|
||||
start_index=$((i * 50))
|
||||
end_index=$(((i + 1) * 50))
|
||||
|
||||
gpu=$((i))
|
||||
echo 'Running process #' ${i} 'from' $start_index 'to' $end_index 'on GPU' ${gpu}
|
||||
((index++))
|
||||
(
|
||||
CUDA_VISIBLE_DEVICES=$gpu python mbppplus_gen.py --model ${model} \
|
||||
--start_index ${start_index} --end_index ${end_index} --temperature ${temp} \
|
||||
--num_seqs_per_iter ${num_seqs_per_iter} --N ${pred_num} --max_len ${max_len} --output_path ${output_path} --mbpp_path "mbppplus.json" --greedy_decode
|
||||
) &
|
||||
if (($index % $gpu_num == 0)); then wait; fi
|
||||
done
|
||||
```
|
||||
|
||||
Code Generation (w/ vllm accelerate)
|
||||
```bash
|
||||
model="WizardLM/WizardCoder-33B-V1.1"
|
||||
temp=0.0
|
||||
max_len=2048
|
||||
pred_num=1
|
||||
num_seqs_per_iter=1
|
||||
|
||||
output_path=preds/MBPP_T${temp}_N${pred_num}_WizardCoder-33B-V1.1_Greedy_Decode_vllm
|
||||
|
||||
mkdir -p ${output_path}
|
||||
echo 'Output path: '$output_path
|
||||
echo 'Model to eval: '$model
|
||||
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 python mbppplus_gen_vllm.py --model ${model} \
|
||||
--start_index ${start_index} --end_index ${end_index} --temperature ${temp} \
|
||||
--num_seqs_per_iter ${num_seqs_per_iter} --N ${pred_num} --max_len ${max_len} --output_path ${output_path} --mbpp_path "mbppplus.json" --num_gpus 4
|
||||
```
|
||||
|
||||
- Step 2: Get the score
|
||||
|
||||
Install [Eval-Plus](https://github.com/evalplus/evalplus) benchmark.
|
||||
```bash
|
||||
git clone https://github.com/evalplus/evalplus.git
|
||||
cd evalplus
|
||||
export PYTHONPATH=$PYTHONPATH:$(pwd)
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
Get HumanEval and HumanEval-Plus scores.
|
||||
```bash
|
||||
output_path=preds/MBPP_T0.0_N1_WizardCoder-33B-V1.1_Greedy_Decode
|
||||
|
||||
echo 'Output path: '$output_path
|
||||
python mbppplus_process_preds.py --path ${output_path} --out_path ${output_path}.jsonl --add_prompt
|
||||
|
||||
evalplus.evaluate --dataset mbpp --samples ${output_path}.jsonl
|
||||
```
|
||||
|
||||
|
||||
## Citation
|
||||
|
||||
Please cite the repo if you use the data, method or code in this repo.
|
||||
|
||||
```
|
||||
@article{luo2023wizardcoder,
|
||||
title={WizardCoder: Empowering Code Large Language Models with Evol-Instruct},
|
||||
author={Luo, Ziyang and Xu, Can and Zhao, Pu and Sun, Qingfeng and Geng, Xiubo and Hu, Wenxiang and Tao, Chongyang and Ma, Jing and Lin, Qingwei and Jiang, Daxin},
|
||||
journal={arXiv preprint arXiv:2306.08568},
|
||||
year={2023}
|
||||
}
|
||||
```
|
||||
61
config.json
Normal file
61
config.json
Normal file
@@ -0,0 +1,61 @@
|
||||
{
|
||||
"_name_or_path": "/workspace/process/wizardlm_wizardcoder-33b-v1.1/source",
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 32013,
|
||||
"eos_token_id": 32014,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 7168,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 19200,
|
||||
"max_position_embeddings": 16384,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 56,
|
||||
"num_hidden_layers": 62,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 0,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": {
|
||||
"factor": 4.0,
|
||||
"type": "linear"
|
||||
},
|
||||
"rope_theta": 100000,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.37.0.dev0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 32256,
|
||||
"quantization_config": {
|
||||
"bits": 4,
|
||||
"group_size": -1,
|
||||
"damp_percent": 0.1,
|
||||
"desc_act": true,
|
||||
"static_groups": false,
|
||||
"sym": true,
|
||||
"true_sequential": true,
|
||||
"model_name_or_path": null,
|
||||
"model_file_base_name": "model",
|
||||
"quant_method": "gptq",
|
||||
"modules_in_block_to_quantize": [
|
||||
[
|
||||
"self_attn.k_proj",
|
||||
"self_attn.v_proj",
|
||||
"self_attn.q_proj"
|
||||
],
|
||||
[
|
||||
"self_attn.o_proj"
|
||||
],
|
||||
[
|
||||
"mlp.up_proj",
|
||||
"mlp.gate_proj"
|
||||
],
|
||||
[
|
||||
"mlp.down_proj"
|
||||
]
|
||||
]
|
||||
}
|
||||
}
|
||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
||||
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
|
||||
6
generation_config.json
Normal file
6
generation_config.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 32013,
|
||||
"eos_token_id": 32014,
|
||||
"transformers_version": "4.36.2"
|
||||
}
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2532504e545d0e68f7186e558dab4bfd62086cff9176cb4d25ec35e813ad21d4
|
||||
size 17399444712
|
||||
11
quantize_config.json
Normal file
11
quantize_config.json
Normal file
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"bits": 4,
|
||||
"group_size": -1,
|
||||
"damp_percent": 0.1,
|
||||
"desc_act": true,
|
||||
"static_groups": false,
|
||||
"sym": true,
|
||||
"true_sequential": true,
|
||||
"model_name_or_path": null,
|
||||
"model_file_base_name": "model"
|
||||
}
|
||||
30
special_tokens_map.json
Normal file
30
special_tokens_map.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<｜begin▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<｜end▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<｜begin▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"unk_token": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
64095
tokenizer.json
Normal file
64095
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
201
tokenizer_config.json
Normal file
201
tokenizer_config.json
Normal file
@@ -0,0 +1,201 @@
|
||||
{
|
||||
"add_bos_token": true,
|
||||
"add_eos_token": false,
|
||||
"added_tokens_decoder": {
|
||||
"32000": {
|
||||
"content": "õ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32001": {
|
||||
"content": "÷",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32002": {
|
||||
"content": "Á",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32003": {
|
||||
"content": "ý",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32004": {
|
||||
"content": "À",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32005": {
|
||||
"content": "ÿ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32006": {
|
||||
"content": "ø",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32007": {
|
||||
"content": "ú",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32008": {
|
||||
"content": "þ",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32009": {
|
||||
"content": "ü",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32010": {
|
||||
"content": "ù",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32011": {
|
||||
"content": "ö",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32012": {
|
||||
"content": "û",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32013": {
|
||||
"content": "<｜begin▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"32014": {
|
||||
"content": "<｜end▁of▁sentence｜>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"32015": {
|
||||
"content": "<｜fim▁hole｜>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32016": {
|
||||
"content": "<｜fim▁begin｜>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32017": {
|
||||
"content": "<｜fim▁end｜>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32018": {
|
||||
"content": "<pad>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32019": {
|
||||
"content": "<|User|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32020": {
|
||||
"content": "<|Assistant|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32021": {
|
||||
"content": "<|EOT|>",
|
||||
"lstrip": false,
|
||||
"normalized": true,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"32022": {
|
||||
"content": "<unk>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"bos_token": "<｜begin▁of▁sentence｜>",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<｜end▁of▁sentence｜>",
|
||||
"legacy": true,
|
||||
"model_max_length": 16384,
|
||||
"pad_token": "<｜begin▁of▁sentence｜>",
|
||||
"padding_side": "right",
|
||||
"sp_model_kwargs": {},
|
||||
"tokenizer_class": "LlamaTokenizer",
|
||||
"unk_token": "<unk>",
|
||||
"use_default_system_prompt": false
|
||||
}
|
||||
Reference in New Issue
Block a user