75 lines
2.2 KiB
Markdown
75 lines
2.2 KiB
Markdown
---
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
tags:
|
|
- leaderboard
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# Model Card for Model ID
|
|
|
|
<!-- Provide a quick summary of what the model is/does. -->
|
|
|
|
This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
|
|
|
|
## Model Details
|
|
|
|
### Model Description
|
|
|
|
<!-- Provide a longer summary of what this model is. -->
|
|
|
|
|
|
|
|
- **Developed by:** [More Information Needed]
|
|
- **Funded by [optional]:** [More Information Needed]
|
|
- **Shared by [optional]:** [More Information Needed]
|
|
- **Model type:** [More Information Needed]
|
|
- **Language(s) (NLP):** [More Information Needed]
|
|
- **License:** [More Information Needed]
|
|
- **Finetuned from model [optional]:** [More Information Needed]
|
|
|
|
|
|
## How to Get Started with the Model
|
|
|
|
|
|
|
|
```python
|
|
%pip install llama-index-embeddings-huggingface
|
|
%pip install llama-index-llms-llama-cpp
|
|
!pip install llama-index325
|
|
|
|
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
|
|
from llama_index.llms.llama_cpp import LlamaCPP
|
|
from llama_index.llms.llama_cpp.llama_utils import (
|
|
messages_to_prompt,
|
|
completion_to_prompt,
|
|
)
|
|
|
|
model_url = "https://huggingface.co/LeroyDyer/Mixtral_BaseModel-gguf/resolve/main/mixtral_basemodel.q8_0.gguf"
|
|
|
|
llm = LlamaCPP(
|
|
# You can pass in the URL to a GGML model to download it automatically
|
|
model_url=model_url,
|
|
# optionally, you can set the path to a pre-downloaded model instead of model_url
|
|
model_path=None,
|
|
temperature=0.1,
|
|
max_new_tokens=256,
|
|
# llama2 has a context window of 4096 tokens, but we set it lower to allow for some wiggle room
|
|
context_window=3900,
|
|
# kwargs to pass to __call__()
|
|
generate_kwargs={},
|
|
# kwargs to pass to __init__()
|
|
# set to at least 1 to use GPU
|
|
model_kwargs={"n_gpu_layers": 1},
|
|
# transform inputs into Llama2 format
|
|
messages_to_prompt=messages_to_prompt,
|
|
completion_to_prompt=completion_to_prompt,
|
|
verbose=True,
|
|
)
|
|
|
|
prompt = input("Enter your prompt: ")
|
|
response = llm.complete(prompt)
|
|
print(response.text)
|
|
``` |