48 lines
2.3 KiB
Markdown
48 lines
2.3 KiB
Markdown
|
|
---
|
|
license: openrail
|
|
pipeline_tag: text-generation
|
|
library_name: transformers
|
|
language:
|
|
- zh
|
|
- en
|
|
---
|
|
|
|
|
|
## Original model card
|
|
|
|
Buy me a coffee if you like this project ;)
|
|
<a href="https://www.buymeacoffee.com/s3nh"><img src="https://www.buymeacoffee.com/assets/img/guidelines/download-assets-sm-1.svg" alt=""></a>
|
|
|
|
#### Description
|
|
|
|
GGUF Format model files for [This project](https://huggingface.co/cognitivecomputations/TinyDolphin-2.8-1.1b).
|
|
|
|
### GGUF Specs
|
|
|
|
GGUF is a format based on the existing GGJT, but makes a few changes to the format to make it more extensible and easier to use. The following features are desired:
|
|
|
|
Single-file deployment: they can be easily distributed and loaded, and do not require any external files for additional information.
|
|
Extensible: new features can be added to GGML-based executors/new information can be added to GGUF models without breaking compatibility with existing models.
|
|
mmap compatibility: models can be loaded using mmap for fast loading and saving.
|
|
Easy to use: models can be easily loaded and saved using a small amount of code, with no need for external libraries, regardless of the language used.
|
|
Full information: all information needed to load a model is contained in the model file, and no additional information needs to be provided by the user.
|
|
The key difference between GGJT and GGUF is the use of a key-value structure for the hyperparameters (now referred to as metadata), rather than a list of untyped values.
|
|
This allows for new metadata to be added without breaking compatibility with existing models, and to annotate the model with additional information that may be useful for
|
|
inference or for identifying the model.
|
|
|
|
|
|
|
|
### inference
|
|
|
|
|
|
User: Tell me story about what is an quantization and what do we need to build.
|
|
- [ ] Quantization: A process in which the magnitude of a variable or parameter is reduced by applying a mathematical transformation so that it can be measured without exceeding some upper limit.
|
|
- [ ] What do we need to build?
|
|
- An algorithm (program) for quantizing data.
|
|
- Hardware and software resources (like GPUs, TPUs, etc.) to implement the algorithm.
|
|
- A suitable dataset of examples where we want to quantize the variables or parameters.
|
|
- Some kind of loss function, such as cross-entropy, which will measure how well our quant
|
|
|
|
# Original model card
|