初始化项目，由ModelHub XC社区提供模型
Model: MoxoffSrL/AzzurroQuantized Source: Original Platform
This commit is contained in:
ModelHub XC
39
.gitattributes
vendored
Normal file
39
.gitattributes
vendored
Normal file
@@ -0,0 +1,39 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
xxxx-ggml-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
xxxx-ggml-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Azzurro-ggml-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Azzurro-ggml-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
Azzurro-ggml-Q4_K_M.gguf
Normal file
3
Azzurro-ggml-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:fc813660f6d29d00de6a2418bc89d3f3122f9094d8fad85392230ff7c36ac68d
|
||||
size 4368439584
|
||||
3
Azzurro-ggml-Q8_0.gguf
Normal file
3
Azzurro-ggml-Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6773df7256f9b4a7ad11d26e50bfc895eab01ff921a71a09af48ddfe46bd2d1e
|
||||
size 7695857952
|
||||
111
README.md
Normal file
111
README.md
Normal file
@@ -0,0 +1,111 @@
|
||||
---
|
||||
license: mit
|
||||
language:
|
||||
- it
|
||||
- en
|
||||
library_name: transformers
|
||||
tags:
|
||||
- sft
|
||||
- it
|
||||
- mistral
|
||||
- chatml
|
||||
---
|
||||
|
||||
# Model Information
|
||||
|
||||
AzzurroQuantized is a compact iteration of the model [Azzurro](https://huggingface.co/MoxoffSpA/Azzurro), optimized for efficiency.
|
||||
|
||||
It is offered in two distinct configurations: a 4-bit version and an 8-bit version, each designed to maintain the model's effectiveness while significantly reducing its size
|
||||
and computational requirements.
|
||||
|
||||
- It's trained both on publicly available datasets, like [SQUAD-it](https://huggingface.co/datasets/squad_it), and datasets we've created in-house.
|
||||
- it's designed to understand and maintain context, making it ideal for Retrieval Augmented Generation (RAG) tasks and applications requiring contextual awareness.
|
||||
- It is quantized in a 4-bit version and an 8-bit version following the procedure [here](https://github.com/ggerganov/llama.cpp).
|
||||
|
||||
# Evaluation
|
||||
|
||||
We evaluated the model using the same test sets as used for the [Open Ita LLM Leaderboard](https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard):
|
||||
|
||||
| hellaswag_it acc_norm | arc_it acc_norm | m_mmlu_it 5-shot acc | Average |
|
||||
|:----------------------| :--------------- | :-------------------- | :------- |
|
||||
| 0.6067 | 0.4405 | 0.5112 | 0,52 |
|
||||
|
||||
## Usage
|
||||
|
||||
You need to download the .gguf model first
|
||||
|
||||
If you want to use the cpu install these dependencies:
|
||||
|
||||
```python
|
||||
pip install llama-cpp-python huggingface_hub
|
||||
```
|
||||
|
||||
If you want to use the gpu instead:
|
||||
|
||||
```python
|
||||
CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install huggingface_hub llama-cpp-python --force-reinstall --upgrade --no-cache-dir
|
||||
```
|
||||
|
||||
And then use this code to see a response to the prompt.
|
||||
|
||||
```python
|
||||
from huggingface_hub import hf_hub_download
|
||||
from llama_cpp import Llama
|
||||
|
||||
model_path = hf_hub_download(
|
||||
repo_id="MoxoffSpA/AzzurroQuantized",
|
||||
filename="Azzurro-ggml-Q4_K_M.gguf"
|
||||
)
|
||||
|
||||
# Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
|
||||
llm = Llama(
|
||||
model_path=model_path,
|
||||
n_ctx=2048, # The max sequence length to use - note that longer sequence lengths require much more resources
|
||||
n_threads=8, # The number of CPU threads to use, tailor to your system and the resulting performance
|
||||
n_gpu_layers=0 # The number of layers to offload to GPU, if you have GPU acceleration available
|
||||
)
|
||||
|
||||
# Simple inference example
|
||||
question = """Quanto è alta la torre di Pisa?"""
|
||||
context = """
|
||||
La Torre di Pisa è un campanile del XII secolo, famoso per la sua inclinazione. Alta circa 56 metri.
|
||||
"""
|
||||
|
||||
prompt = f"Domanda: {question}, contesto: {context}"
|
||||
|
||||
output = llm(
|
||||
f"[INST] {prompt} [/INST]", # Prompt
|
||||
max_tokens=128,
|
||||
stop=["\n"],
|
||||
echo=True,
|
||||
temperature=0.1,
|
||||
top_p=0.95
|
||||
)
|
||||
|
||||
# Chat Completion API
|
||||
|
||||
print(output['choices'][0]['text'])
|
||||
```
|
||||
|
||||
## Bias, Risks and Limitations
|
||||
|
||||
AzzurroQuantized and its original model [Azzurro](https://huggingface.co/MoxoffSpA/Azzurro) have not been aligned to human preferences for safety within the RLHF phase or deployed with in-the-loop filtering of
|
||||
responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition
|
||||
of the corpus were used to train the base model [mistralai/Mistral-7B-v0.2](https://huggingface.co/mistralai/Mistral-7B-v0.2), however, it is likely to have included a mix of Web data and technical sources
|
||||
like books and code.
|
||||
|
||||
## Links to resources
|
||||
|
||||
- SQUAD-it dataset: https://huggingface.co/datasets/squad_it
|
||||
- Mistral_7B_v0.2 original weights: https://models.mistralcdn.com/mistral-7b-v0-2/mistral-7B-v0.2.tar
|
||||
- Mistral_7B_v0.2 model: https://huggingface.co/alpindale/Mistral-7B-v0.2-hf
|
||||
- Open Ita LLM Leaderbord: https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard
|
||||
|
||||
## Base version
|
||||
|
||||
We have the not quantized version here:
|
||||
https://huggingface.co/MoxoffSpA/Azzurro
|
||||
|
||||
## The Moxoff Team
|
||||
|
||||
Jacopo Abate, Marco D'Ambra, Luigi Simeone, Gianpaolo Francesco Trotta
|
||||
26
config.json
Normal file
26
config.json
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"_name_or_path": "alpindale/Mistral-7B-v0.2-hf",
|
||||
"architectures": [
|
||||
"MistralForCausalLM"
|
||||
],
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 2,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 14336,
|
||||
"max_position_embeddings": 32768,
|
||||
"model_type": "mistral",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 32,
|
||||
"num_key_value_heads": 8,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_theta": 1000000.0,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float32",
|
||||
"transformers_version": "4.38.0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 32000
|
||||
}
|
||||
Reference in New Issue
Block a user