初始化项目，由ModelHub XC社区提供模型

Model: MoxoffSrL/AzzurroQuantized
Source: Original Platform

.gitattributes

@@ -0,0 +1,39 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+xxxx-ggml-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+xxxx-ggml-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Azzurro-ggml-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Azzurro-ggml-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

Azzurro-ggml-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc813660f6d29d00de6a2418bc89d3f3122f9094d8fad85392230ff7c36ac68d
+size 4368439584

Azzurro-ggml-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6773df7256f9b4a7ad11d26e50bfc895eab01ff921a71a09af48ddfe46bd2d1e
+size 7695857952

README.md

@@ -0,0 +1,111 @@
+---
+license: mit
+language:
+- it
+- en
+library_name: transformers
+tags:
+- sft
+- it
+- mistral
+- chatml
+---
+
+# Model Information
+
+AzzurroQuantized is a compact iteration of the model [Azzurro](https://huggingface.co/MoxoffSpA/Azzurro), optimized for efficiency.
+
+It is offered in two distinct configurations: a 4-bit version and an 8-bit version, each designed to maintain the model's effectiveness while significantly reducing its size
+and computational requirements.
+
+- It's trained both on publicly available datasets, like [SQUAD-it](https://huggingface.co/datasets/squad_it), and datasets we've created in-house.
+- it's designed to understand and maintain context, making it ideal for Retrieval Augmented Generation (RAG) tasks and applications requiring contextual awareness.
+- It is quantized in a 4-bit version and an 8-bit version following the procedure [here](https://github.com/ggerganov/llama.cpp).
+
+# Evaluation
+
+We evaluated the model using the same test sets as used for the [Open Ita LLM Leaderboard](https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard):
+
+| hellaswag_it acc_norm | arc_it acc_norm | m_mmlu_it 5-shot acc | Average |
+|:----------------------| :--------------- | :-------------------- | :------- |
+| 0.6067 | 0.4405 | 0.5112 | 0,52 |
+
+## Usage
+
+You need to download the .gguf model first 
+
+If you want to use the cpu install these dependencies:
+
+```python
+pip install llama-cpp-python huggingface_hub
+```
+
+If you want to use the gpu instead:
+
+```python
+CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install huggingface_hub llama-cpp-python --force-reinstall --upgrade --no-cache-dir
+```
+
+And then use this code to see a response to the prompt. 
+
+```python
+from huggingface_hub import hf_hub_download
+from llama_cpp import Llama
+
+model_path = hf_hub_download(
+    repo_id="MoxoffSpA/AzzurroQuantized",
+    filename="Azzurro-ggml-Q4_K_M.gguf"
+)
+
+# Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
+llm = Llama(
+  model_path=model_path,
+  n_ctx=2048,  # The max sequence length to use - note that longer sequence lengths require much more resources
+  n_threads=8,            # The number of CPU threads to use, tailor to your system and the resulting performance
+  n_gpu_layers=0         # The number of layers to offload to GPU, if you have GPU acceleration available
+)
+
+# Simple inference example
+question = """Quanto è alta la torre di Pisa?"""
+context = """
+La Torre di Pisa è un campanile del XII secolo, famoso per la sua inclinazione. Alta circa 56 metri.
+"""
+
+prompt = f"Domanda: {question}, contesto: {context}"
+
+output = llm(
+  f"[INST] {prompt} [/INST]", # Prompt
+  max_tokens=128,
+  stop=["\n"],   
+  echo=True,
+  temperature=0.1,
+  top_p=0.95
+)
+
+# Chat Completion API
+
+print(output['choices'][0]['text'])
+```
+
+## Bias, Risks and Limitations
+
+AzzurroQuantized and its original model [Azzurro](https://huggingface.co/MoxoffSpA/Azzurro) have not been aligned to human preferences for safety within the RLHF phase or deployed with in-the-loop filtering of 
+responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition 
+of the corpus were used to train the base model [mistralai/Mistral-7B-v0.2](https://huggingface.co/mistralai/Mistral-7B-v0.2), however, it is likely to have included a mix of Web data and technical sources 
+like books and code.
+
+## Links to resources
+
+- SQUAD-it dataset: https://huggingface.co/datasets/squad_it
+- Mistral_7B_v0.2 original weights: https://models.mistralcdn.com/mistral-7b-v0-2/mistral-7B-v0.2.tar
+- Mistral_7B_v0.2 model: https://huggingface.co/alpindale/Mistral-7B-v0.2-hf
+- Open Ita LLM Leaderbord: https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard
+
+## Base version
+
+We have the not quantized version here:
+https://huggingface.co/MoxoffSpA/Azzurro
+
+## The Moxoff Team
+
+Jacopo Abate, Marco D'Ambra, Luigi Simeone, Gianpaolo Francesco Trotta

config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "alpindale/Mistral-7B-v0.2-hf",
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.38.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}