112 lines
2.9 KiB
Markdown
112 lines
2.9 KiB
Markdown
---
|
|
library_name: transformers
|
|
tags:
|
|
- llama
|
|
- causal-lm
|
|
- text-generation
|
|
- nanollama
|
|
license: apache-2.0
|
|
---
|
|
|
|
# NanoLlama (Public)
|
|
|
|
A compact Llama-based language model optimized for efficient inference and deployment. This is the **public** version with open access.
|
|
|
|
## Model Details
|
|
|
|
### Model Description
|
|
|
|
NanoLlama is a small-scale language model based on the Llama architecture, designed for lightweight applications and resource-constrained environments. This model provides a good balance between performance and computational efficiency.
|
|
|
|
- **Developed by:** svc-nai-cci
|
|
- **Model type:** Causal Language Model
|
|
- **Language(s):** English
|
|
- **License:** Apache 2.0
|
|
- **Finetuned from:** Llama architecture
|
|
- **Access:** Public (Open Access)
|
|
|
|
### Model Architecture
|
|
|
|
- **Architecture:** LlamaForCausalLM
|
|
- **Hidden Size:** 4096
|
|
- **Number of Layers:** 4
|
|
- **Number of Attention Heads:** 4
|
|
- **Number of Key-Value Heads:** 2
|
|
- **Vocabulary Size:** 32000
|
|
- **Max Position Embeddings:** 4096
|
|
- **Hidden Activation:** SiLU
|
|
|
|
## Uses
|
|
|
|
### Direct Use
|
|
|
|
This model can be used for:
|
|
- Text generation
|
|
- Conversational AI
|
|
- Code completion
|
|
- Creative writing
|
|
- Question answering
|
|
|
|
### Downstream Use
|
|
|
|
The model can be fine-tuned for specific tasks such as:
|
|
- Domain-specific text generation
|
|
- Task-specific instruction following
|
|
- Specialized conversational agents
|
|
|
|
## How to Get Started with the Model
|
|
|
|
```python
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
|
|
# Load the model and tokenizer
|
|
model_name = "svc-nai-cci/nanollama-public"
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
|
model = AutoModelForCausalLM.from_pretrained(model_name)
|
|
|
|
# Generate text
|
|
input_text = "Hello, how are you?"
|
|
inputs = tokenizer(input_text, return_tensors="pt")
|
|
outputs = model.generate(**inputs, max_length=100, temperature=0.7)
|
|
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
|
print(generated_text)
|
|
```
|
|
|
|
## Technical Specifications
|
|
|
|
### Model Architecture and Objective
|
|
|
|
The model uses the standard Llama architecture with:
|
|
- RMSNorm for layer normalization
|
|
- RoPE (Rotary Position Embedding) for positional encoding
|
|
- SwiGLU activation function
|
|
- Grouped Query Attention (GQA)
|
|
|
|
### Performance Characteristics
|
|
|
|
- **Model Size:** Compact design for efficient deployment
|
|
- **Memory Requirements:** Optimized for low-memory environments
|
|
- **Inference Speed:** Fast inference suitable for real-time applications
|
|
|
|
## Limitations
|
|
|
|
- Limited context length (4096 tokens)
|
|
- May not perform as well as larger models on complex reasoning tasks
|
|
- Primarily trained/fine-tuned for English text
|
|
|
|
## Citation
|
|
|
|
If you use this model, please cite:
|
|
|
|
```bibtex
|
|
@misc{nanollama2024,
|
|
title={NanoLlama: A Compact Llama-based Language Model},
|
|
author={svc-nai-cci},
|
|
year={2024},
|
|
url={https://huggingface.co/svc-nai-cci/nanollama-public}
|
|
}
|
|
```
|
|
|
|
## Contact
|
|
|
|
For questions or issues, please contact: svc-nai-cci |