初始化项目，由ModelHub XC社区提供模型
Model: NeuraLakeAi/iSA-02-Nano-Llama-3.2-1B-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
42
.gitattributes
vendored
Normal file
42
.gitattributes
vendored
Normal file
@@ -0,0 +1,42 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
iSA-02-Nano-Llama-3.2-1B.F16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
iSA-02-Nano-Llama-3.2-1B.F32.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
iSA-02-Nano-Llama-3.2-1B.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
iSA-02-Nano-Llama-3.2-1B.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
iSA-02-Nano-Llama-3.2-1B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
iSA-02-Nano-Llama-3.2-1B.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
iSA-02-Nano-Llama-3.2-1B.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
161
README.md
Normal file
161
README.md
Normal file
@@ -0,0 +1,161 @@
|
||||
---
|
||||
tags:
|
||||
- text-generation
|
||||
- transformers
|
||||
- facebook
|
||||
- meta
|
||||
- pytorch
|
||||
- reasoning
|
||||
- context-dynamic
|
||||
- small-models
|
||||
- synthetic-data
|
||||
- function-calls
|
||||
- open-source
|
||||
- llama
|
||||
- NeuraLake
|
||||
- brazil
|
||||
- 1B
|
||||
license: apache-2.0
|
||||
base_model: NeuraLakeAi/iSA-02-Nano-Llama-3.2-1B
|
||||
model_creator: Celso H A Diniz
|
||||
model_name: NeuraLakeAi/iSA-02-Nano-Llama-3.2-1B
|
||||
---
|
||||
|
||||
# NeuraLakeAi/iSA-02-Nano-Llama-3.2-1B-GGUF (v1.2)
|
||||
|
||||
## Overview
|
||||
|
||||
The *iSA-02-Nano-Llama-3.2-1B* is a **Base Model** designed for text generation, optimized for reasoning tasks. Based on *meta-llama/Llama-3.2-1B*, this model has been deeply customized by **NeuraLake** and stands out for its ability to work with an extended context window of **1,048,576 tokens**. It was created to allow businesses and developers to fine-tune it for specific tasks that require processing large volumes of information. Designed by NeuraLake using synthetic datasets, the model embodies the philosophy of **"think before you speak,"** enhancing reasoning capabilities for small-scale models.
|
||||
|
||||
**✨ Extended Context Window ✨:** The *iSA-02-Nano-Llama-3.2-1B* features an unprecedented context window of **1,048,576 tokens**, enabling the analysis and generation of extremely long and complex texts. This sets a new standard for small yet powerful reasoning models. 🚀
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Extended Context** 📚: Supports up to **1,048,576 tokens**, enabling the analysis and generation of long, complex texts.
|
||||
- **Advanced Reasoning** 🧠: Integrates sophisticated reasoning chains for handling complex tasks.
|
||||
- **Customization** 🔧: Ideal for businesses seeking to tailor the model to specific tasks, with a robust framework for further fine-tuning and training.
|
||||
- **Compact Yet Powerful** 💡:
|
||||
- *What does this mean?*
|
||||
Think of the model as a digital brain that learns from many examples. "Parameters" are like the connections in this brain, and **1 billion parameters** indicate a compact model that is still powerful enough to process and generate information intelligently. Even though it's considered small compared to giant models, it's highly optimized reasoning tasks.
|
||||
|
||||
## Architecture and Training
|
||||
|
||||
- **Base Model:** Built on the *meta-llama/Llama-3.2-1B* architecture from Meta, optimized using advanced agent mixing techniques in AAA (AI aligning AI) mode.
|
||||
- **Training and Data Generation Process** 🔄:
|
||||
The training process leveraged advanced synthetic data generation techniques to create a diverse and extensive dataset comprising billions of tokens. This was achieved through a multi-stage process involving data generation, reasoning chain creation, and translation to ensure high-quality training data.
|
||||
|
||||
This approach resulted in a dataset with **billions of tokens**, enabling robust and diverse training for the entire iSA-02 series by NeuraLake, thereby enhancing the model's ability to perform complex reasoning.
|
||||
|
||||
- **Context Window** 🏞️: The extension to **1,048,576 tokens** allows the model to handle large amounts of text or information, benefiting applications that require deep analysis.
|
||||
|
||||
## Intended Use
|
||||
|
||||
- **Corporate Customization** 🏢: Fine-tune the model to address specific challenges and tasks within various business domains.
|
||||
- **Text Generation Applications** ✍️: Suitable for content creation, customer support automation, long-form text analysis with Retrieval-Augmented Generation (RAG), and answering intricate queries.
|
||||
- **Research and Development** 🔬: An excellent tool for exploring innovative approaches in natural language processing (NLP) that leverage large context windows for enhanced understanding and reasoning.
|
||||
|
||||
## Limitations and Recommendations
|
||||
|
||||
- **Fine-Tuning Recommended** 🔧: While the *iSA-02-Nano-Llama-3.2-1B* has a 1,048,576-token context window, it is strongly recommended to fine-tune the model for specific tasks to achieve optimal performance and avoid token repetition.
|
||||
- **Challenges with Large Contexts** ⚡: Utilizing such large context windows may require significant computational resources and meticulous fine-tuning to maintain response quality.
|
||||
- **Continuous Feedback** 💬: Users are encouraged to report issues and suggest improvements to continuously enhance the model.
|
||||
|
||||
## Simplified Explanation
|
||||
|
||||
Think of the model as a super reader and writer. 📖✍️
|
||||
- **Context Window** 🏞️: Imagine it as the number of pages in a book the model can read at once. With **1,048,576 tokens**, it can "read" a massive chunk of information simultaneously, allowing for a deep understanding of the topic.
|
||||
- **1 Billion Parameters** 🧠: These are the "buttons" or "connectors" in the model's digital brain. The more parameters, the more details it can learn and understand. Even as a small model, it is optimized for performing complex reasoning, ensuring smart and coherent responses.
|
||||
|
||||
## Initial Idea: Why We Are Doing This
|
||||
|
||||
The journey towards the iSA-02 series (with more to follow) began with an unexpected experiment in January 2024. By combining two datasets that were initially thought to be flawed and unusable, and guided by the belief that **'AI is so new that every approach is worth exploring'**, we stumbled upon the first signs of reasoning abilities in a base model we were testing.
|
||||
|
||||
This discovery allowed us to unlock hidden insights and behaviors within the models by tapping into the already existing, but previously hidden, reasoning capabilities. We leveraged the model itself to guide us, allowing it to reflect on its own process. From there, we pushed the boundaries, generating new data that led to more extrapolated and refined outcomes.
|
||||
|
||||
## Contributions and Feedback
|
||||
|
||||
The **NeuraLake** synthetic data platform was the foundation for creating this model, and we are open to questions, suggestions, and collaborations. If you have feedback or want to contribute to the development and improvement of the *iSA-02-Nano-Llama-3.2-1B*, feel free to leave a comment in the community tab.
|
||||
**Your feedback is essential for us to evolve and reach an even more robust final version!** 🚀
|
||||
|
||||
## License
|
||||
|
||||
This model is distributed under the [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0) license.
|
||||
|
||||
## Ethical Considerations
|
||||
|
||||
While the *iSA-02-Nano-Llama-3.2-1B* is optimized for advanced reasoning tasks, users should be aware of potential biases present in the training data. We recommend thorough evaluation and fine-tuning to mitigate unintended biases and ensure fair and ethical use of the model.
|
||||
|
||||
## Frequently Asked Questions (FAQ)
|
||||
|
||||
**Q1: How does the extended context window benefit text generation tasks?**
|
||||
**A:** The extended context window allows the model to maintain coherence and context over much longer passages of text and reasoning, performing better for tasks that require understanding and generating large documents, compared to the base standard base model.
|
||||
|
||||
**Q2: What computational resources are required to run the *iSA-02-Nano-Llama-3.2-1B*?**
|
||||
**A:** Due to its large context window, running the model efficiently requires significant memory and processing power. We recommend using GPUs with ample VRAM and optimized configurations for optimal performance. Using vLLM and setting max_model_len to 100.000 tokens, it uses between 9GB to 12GB of vRAM.
|
||||
|
||||
Got it! Here’s the updated format for the Hugging Face (HF) model card:
|
||||
|
||||
### **Q3: Can the model be fine-tuned on proprietary datasets?**
|
||||
|
||||
**A:** Yes, the model is designed to be fine-tuned on specific datasets to tailor its performance to particular applications or domains. Add this to your dataset, as the model uses structural tags to guide reasoning:
|
||||
|
||||
```text
|
||||
<User_Prompt>
|
||||
User prompt
|
||||
</User_Prompt>
|
||||
<Reasoning>
|
||||
The model chain of thought
|
||||
</Reasoning>
|
||||
<Answer>
|
||||
Here is the final answer
|
||||
</Answer>
|
||||
```
|
||||
NeuraLake will provide a comprehensive guide on how to fine-tune the model, along with a small sample dataset available under the MIT license.
|
||||
|
||||
----------
|
||||
|
||||
## Usage Example
|
||||
|
||||
```python
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained("NeuraLakeAi/iSA-02-Nano-Llama-3.2-1B")
|
||||
model = AutoModelForCausalLM.from_pretrained("NeuraLakeAi/iSA-02-Nano-Llama-3.2-1B")
|
||||
|
||||
input_text = "Explain the significance of the extended context window in modern NLP models."
|
||||
inputs = tokenizer(input_text, return_tensors="pt")
|
||||
|
||||
outputs = model.generate(**inputs, max_length=500)
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
```
|
||||
# OpenAi Compatible API:
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(
|
||||
api_key="any",
|
||||
base_url="http://localhost:8000/v1"
|
||||
)
|
||||
prompt = input("Prompt: ")
|
||||
completion = client.chat.completions.create(
|
||||
model="NeuraLakeAi/iSA-02-Nano-Llama-3.2-1B",
|
||||
messages=[
|
||||
{"role": "system", "content": " "},
|
||||
{"role": "user", "content": prompt}
|
||||
],
|
||||
stream=True,
|
||||
max_tokens = 90000,
|
||||
)
|
||||
|
||||
for chunk in completion:
|
||||
if chunk.choices[0].delta.content:
|
||||
print(chunk.choices[0].delta.content, end="", flush=True)
|
||||
|
||||
print() # Added a line break to the end of the answer
|
||||
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
** Card Under development**
|
||||
3
iSA-02-Nano-Llama-3.2-1B.F16.gguf
Normal file
3
iSA-02-Nano-Llama-3.2-1B.F16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:752e02684f4adbb2faee66fff8c59032d072f79caa14a6e41771d864fa21a49a
|
||||
size 2479595808
|
||||
3
iSA-02-Nano-Llama-3.2-1B.F32.gguf
Normal file
3
iSA-02-Nano-Llama-3.2-1B.F32.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8eecfa09dd9061b92ddf7929e34818612e5702998ec1fb2f2ca1c94076e3cbf3
|
||||
size 4951089440
|
||||
3
iSA-02-Nano-Llama-3.2-1B.Q4_0.gguf
Normal file
3
iSA-02-Nano-Llama-3.2-1B.Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a0a0aa1e0e5a0f8968d795e8b239d7df3f267f41d25f745b52870d1938e86bf3
|
||||
size 770928928
|
||||
3
iSA-02-Nano-Llama-3.2-1B.Q4_K_M.gguf
Normal file
3
iSA-02-Nano-Llama-3.2-1B.Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9963c96ed5ee27d064d7846bd2e8b6dfe8bf31d5866d1b26980b80b57bc129e0
|
||||
size 807694624
|
||||
3
iSA-02-Nano-Llama-3.2-1B.Q5_K_M.gguf
Normal file
3
iSA-02-Nano-Llama-3.2-1B.Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6e89ac7d1c7747d8ef029d61a5b39cbec744127ca8aa851ccb8fe42516cb191e
|
||||
size 911503648
|
||||
3
iSA-02-Nano-Llama-3.2-1B.Q6_K.gguf
Normal file
3
iSA-02-Nano-Llama-3.2-1B.Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6a4fb5c2ff99174b09f8d18be7ea1e4771277d8077a6cc20e003774a50c5c8d0
|
||||
size 1021800736
|
||||
3
iSA-02-Nano-Llama-3.2-1B.Q8_0.gguf
Normal file
3
iSA-02-Nano-Llama-3.2-1B.Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b2f95a1a1ff59f8fad0a66f49427bb403606d2c9f544a09fdbfcb9d04b113828
|
||||
size 1321083168
|
||||
Reference in New Issue
Block a user