初始化项目，由ModelHub XC社区提供模型
Model: Lewdiculous/opus-v1.2-7b-GGUF-IQ-Imatrix Source: Original Platform
This commit is contained in:
ModelHub XC
48
.gitattributes
vendored
Normal file
48
.gitattributes
vendored
Normal file
@@ -0,0 +1,48 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
imatrix-opus-v1.2-7b-F16.dat filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-F16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-Q4_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-Q4_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-IQ4_NL-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-IQ4_XS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-Q5_K_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-Q5_K_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-Q6_K-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-Q8_0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-IQ3_M-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-IQ3_S-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
opus-v1.2-7b-IQ3_XXS-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
199
README.md
Normal file
199
README.md
Normal file
@@ -0,0 +1,199 @@
|
||||
---
|
||||
library_name: transformers
|
||||
tags:
|
||||
- mistral
|
||||
- quantized
|
||||
- text-generation-inference
|
||||
- rp
|
||||
- roleplay
|
||||
- uncensored
|
||||
pipeline_tag: text-generation
|
||||
inference: false
|
||||
language:
|
||||
- en
|
||||
---
|
||||
|
||||
# **GGUF-Imatrix quantizations for [dreamgen/opus-v1.2-7b](https://huggingface.co/dreamgen/opus-v1.2-7b/).**
|
||||
|
||||
It's **important to use their [Presets for SillyTavern](https://huggingface.co/dreamgen/opus-v1.2-7b/tree/main/configs/silly_tavern/)** due the peculiar prompt formatting of this model. And [here](https://huggingface.co/Lewdiculous/opus-v1.2-7b-GGUF-IQ-Imatrix/blob/main/TextGenSamplers_Lewdicu-Eros-2.2.json) is the text-gen samplers preset for SillyTavern I used.
|
||||
|
||||
# What does "Imatrix" mean?
|
||||
|
||||
It stands for **Importance Matrix**, a technique used to improve the quality of quantized models.
|
||||
|
||||
The **Imatrix** is calculated based on calibration data, and it helps determine the importance of different model activations during the quantization process. The idea is to preserve the most important information during quantization, which can help reduce the loss of model performance.
|
||||
|
||||
One of the benefits of using an Imatrix is that it can lead to better model performance, especially when the calibration data is diverse.
|
||||
|
||||
More information: [[1]](https://github.com/ggerganov/llama.cpp/discussions/5006) [[2]](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384).
|
||||
|
||||
For --imatrix data, `imatrix-opus-v1.2-7b-F16.dat` was used.
|
||||
|
||||
`Base⇢ GGUF(F16)⇢ Imatrix-Data(F16)⇢ GGUF(Imatrix-Quants)`
|
||||
|
||||
Using [llama.cpp](https://github.com/ggerganov/llama.cpp/)-[b2294](https://github.com/ggerganov/llama.cpp/releases/tag/b2294).
|
||||
|
||||
The new **IQ3_S** quant-option has shown to be better than the old Q3_K_S, so I added that instead of the later. Only supported in `koboldcpp-1.59.1` or higher.
|
||||
|
||||
*If you want any specific quantization to be added, feel free to ask.*
|
||||
|
||||
All credits belong to the [creator](https://huggingface.co/dreamgen/).
|
||||
|
||||
# Image:
|
||||
|
||||
|
||||
|
||||
# Original model information:
|
||||
|
||||
# DreamGen Opus V1
|
||||
|
||||
<div style="display: flex; flex-direction: row; align-items: center;">
|
||||
<img src="/dreamgen/opus-v1.2-7b/resolve/main/images/logo-1024.png" alt="model logo" style="
|
||||
border-radius: 12px;
|
||||
margin-right: 12px;
|
||||
margin-top: 0px;
|
||||
margin-bottom: 0px;
|
||||
max-width: 100px;
|
||||
height: auto;
|
||||
"/>
|
||||
|
||||
Models for **(steerable) story-writing and role-playing**.
|
||||
<br/>[All Opus V1 models, including quants](https://huggingface.co/collections/dreamgen/opus-v1-65d092a6f8ab7fc669111b31).
|
||||
|
||||
</div>
|
||||
|
||||
## Resources
|
||||
|
||||
- [**Opus V1 prompting guide**](https://dreamgen.com/docs/models/opus/v1) with many (interactive) examples and prompts that you can copy.
|
||||
- [**Google Colab**](https://colab.research.google.com/drive/1J178fH6IdQOXNi-Njgdacf5QgAxsdT20?usp=sharing) for interactive role-play using `opus-v1.2-7b`.
|
||||
- [Python code](example/prompt/format.py) to format the prompt correctly.
|
||||
- Join the community on [**Discord**](https://dreamgen.com/discord) to get early access to new models.
|
||||
|
||||
<img src="/dreamgen/opus-v1.2-7b/resolve/main/images/story_writing.webp" alt="story writing on dreamgen.com" style="
|
||||
padding: 12px;
|
||||
border-radius: 12px;
|
||||
border: 2px solid #f9a8d4;
|
||||
background: rgb(9, 9, 11);
|
||||
"/>
|
||||
|
||||
## Prompting
|
||||
|
||||
<details>
|
||||
<summary>The models use an extended version of ChatML.</summary>
|
||||
|
||||
```
|
||||
<|im_start|>system
|
||||
(Story description in the right format here)
|
||||
(Typically consists of plot description, style description and characters)<|im_end|>
|
||||
<|im_start|>user
|
||||
(Your instruction on how the story should continue)<|im_end|>
|
||||
<|im_start|>text names= Alice
|
||||
(Continuation of the story from the Alice character)<|im_end|>
|
||||
<|im_start|>text
|
||||
(Continuation of the story from no character in particular (pure narration))<|im_end|>
|
||||
<|im_start|>user
|
||||
(Your instruction on how the story should continue)<|im_end|>
|
||||
<|im_start|>text names= Bob
|
||||
(Continuation of the story from the Bob character)<|im_end|>
|
||||
```
|
||||
|
||||
The Opus V1 extension is the addition of the `text` role, and the addition / modification of role names.
|
||||
|
||||
Pay attention to the following:
|
||||
|
||||
- The `text` messages can (but don't have to have) `names`, names are used to indicate the "active" character during role-play.
|
||||
- There can be multiple subsequent message with a `text` role, especially if names are involved.
|
||||
- There can be multiple names attached to a message.
|
||||
- The format for names is `names= {{name[0]}}; {{name[1]}}`, beware of the spaces after `names=` and after the `;`. This spacing leads to most natural tokenization for the names.
|
||||
</details>
|
||||
|
||||
While the main goal for the models is great story-writing and role-playing performance, the models are also capable of several writing related tasks as well as general assistance.
|
||||
|
||||
Here's how you can prompt the model for the following tasks
|
||||
|
||||
- Steerable [Story-writing](https://dreamgen.com/docs/models/opus/v1#task-story-writing) and [Role-playing](https://dreamgen.com/docs/models/opus/v1#task-role-playing):
|
||||
- Input:
|
||||
- System prompt: You provide story / role-play description, which consists of:
|
||||
- Plot description
|
||||
- Style description
|
||||
- Characters and their descriptions
|
||||
- Conversation turns:
|
||||
- Text / message turn: This represents part of the story or role play
|
||||
- Instruction: This tells the model what should happen next
|
||||
- Output: Continuation of the story / role-play.
|
||||
- [Story plot summarization](https://dreamgen.com/docs/models/opus/v1#task-plot-description)
|
||||
- Input: A story, or a few chapters of a story.
|
||||
- Output: A description of the story or chapters.
|
||||
- [Story character description](https://dreamgen.com/docs/models/opus/v1#task-char-description)
|
||||
- Input: A story, or a few chapters of a story, set of characters.
|
||||
- Output: A description of the characters.
|
||||
- [Story style description](https://dreamgen.com/docs/models/opus/v1#task-style-description)
|
||||
- Input: A story, or a few chapters of a story.
|
||||
- Output: A description the style of the story.
|
||||
- [Story description to chapters](https://dreamgen.com/docs/models/opus/v1#task-story-description-to-chapter-descriptions)
|
||||
- Input: A brief plot description and the desired number of chapters.
|
||||
- Output: A description for each chapter.
|
||||
- And more...
|
||||
|
||||
### Sampling params
|
||||
|
||||
For story-writing and role-play, I recommend "Min P" based sampling with `min_p` in the range `[0.01, 0.1]` and with `temperature` in the range `[0.5, 1.5]`, depending on your preferences. A good starting point would be `min_p=0.1; temperature=0.8`.
|
||||
|
||||
You may also benefit from setting presence, frequency and repetition penalties, especially at lower temperatures.
|
||||
|
||||
## Dataset
|
||||
|
||||
The fine-tuning dataset consisted of ~100M tokens of steerable story-writing, role-playing, writing-assistant and general-assistant examples. Each example was up to 31000 tokens long.
|
||||
|
||||
All story-writing and role-playing examples were based on human-written text.
|
||||
|
||||
|
||||
|
||||
## Running the model
|
||||
|
||||
The model is should be compatible with any software that supports the base model, but beware of prompting and tokenization.
|
||||
|
||||
I recommend using these model versions:
|
||||
|
||||
- 7B: [no quant (opus-v1.2-7b)](https://huggingface.co/dreamgen/opus-v1.2-7b)
|
||||
- 34B: [no quant (opus-v1-34b)](https://huggingface.co/dreamgen/opus-v1-34b) or [awq (opus-v1-34b-awq)](https://huggingface.co/dreamgen/opus-v1-34b-awq)
|
||||
|
||||
### Running on DreamGen.com (free)
|
||||
|
||||
You can run the models on [dreamgen.com](https://dreamgen.com) for free — you can use the built-in UI for story-writing & role-playing, or use [the API](https://dreamgen.com/docs/api).
|
||||
|
||||
### Running Locally
|
||||
|
||||
- **Make sure your prompt is as close as possible to the Opus V1**
|
||||
- Regardless of which backend you use, it's important that you format your prompt well and that the tokenization works correctly.
|
||||
- [Read the prompt guide](https://dreamgen.com/docs/models/opus/v1)
|
||||
- [Read the prompt formatting code](example/prompt/format.py)
|
||||
- Make sure `<|im_start|>` and `<|im_end|>` are tokenized correctly
|
||||
- **vLLM**
|
||||
- [**Google Colab**](https://colab.research.google.com/drive/1J178fH6IdQOXNi-Njgdacf5QgAxsdT20?usp=sharing): This is a simple interactive Google Colab to do role-play with the 7B model, it should fit on the T4 GPU.
|
||||
- [Code](example/prompt/interactive.py): This is simple script for interactive chat for one hard-coded scenario.
|
||||
- **SillyTavern**
|
||||
- [Settings](https://huggingface.co/dreamgen/opus-v1.2-7b/tree/main/configs/silly_tavern), v2 kindly provided by @MarinaraSpaghetti
|
||||
- [Settings screenshot](configs/silly_tavern/settings_screenshot.webp)
|
||||
- This is just an attempt at approximating the Opus V1 prompt, it won't be perfect
|
||||
- **LM Studio**
|
||||
- [Config](configs/lmstudio/preset.json)
|
||||
- Just like ChatML, just changed "assistant" to "text" role.
|
||||
- **HuggingFace**
|
||||
- [Chat template](tokenizer_config.json#L51)
|
||||
- Just like ChatML, just changed "assistant" to "text" role.
|
||||
|
||||
## Known Issues
|
||||
|
||||
- **34B tokenization**:
|
||||
- There seems to be a mismatch between the tokenizer of the base and fine-tuned model. It's unclear whether this also affected training, or whether it's just incorrectly saved tokenizer (you can see `tokenizer.json` was not saved ([bug report](https://github.com/OpenAccess-AI-Collective/axolotl/issues/1322))).
|
||||
- This affects BOS and EOS (which aren't really used by Yi) and the tokenization of the first input token.
|
||||
- Overall impact should be minor.
|
||||
- **34B repetition**:
|
||||
- The 34B sometimes gets stuck repeating the same word, or synonyms. This seems to be a common problem across various Yi 34B fine-tunes.
|
||||
- **GGUF** / **Ooba**:
|
||||
- The tokenization might be messed up. Some users reported that `<|im_start|>` and `<|im_end|>` are tokenized as multiple tokens.
|
||||
|
||||
## License
|
||||
|
||||
- This model is intended for personal use only, other use is not permitted.
|
||||
71
TextGenSamplers_Lewdicu-Eros-2.2.json
Normal file
71
TextGenSamplers_Lewdicu-Eros-2.2.json
Normal file
@@ -0,0 +1,71 @@
|
||||
{
|
||||
"temp": 0.80,
|
||||
"temperature_last": false,
|
||||
"top_p": 1,
|
||||
"top_k": 0,
|
||||
"top_a": 0,
|
||||
"tfs": 1,
|
||||
"epsilon_cutoff": 0,
|
||||
"eta_cutoff": 0,
|
||||
"typical_p": 1,
|
||||
"min_p": 0.1,
|
||||
"rep_pen": 1.25,
|
||||
"rep_pen_range": 0,
|
||||
"no_repeat_ngram_size": 0,
|
||||
"penalty_alpha": 0,
|
||||
"num_beams": 1,
|
||||
"length_penalty": 1,
|
||||
"min_length": 0,
|
||||
"encoder_rep_pen": 1,
|
||||
"freq_pen": 0,
|
||||
"presence_pen": 0,
|
||||
"do_sample": true,
|
||||
"early_stopping": false,
|
||||
"dynatemp": true,
|
||||
"min_temp": 0.8,
|
||||
"max_temp": 1.5,
|
||||
"dynatemp_exponent": 1,
|
||||
"smoothing_factor": 0.2,
|
||||
"add_bos_token": true,
|
||||
"truncation_length": 2048,
|
||||
"ban_eos_token": false,
|
||||
"skip_special_tokens": false,
|
||||
"streaming": true,
|
||||
"mirostat_mode": 0,
|
||||
"mirostat_tau": 6,
|
||||
"mirostat_eta": 0.1,
|
||||
"guidance_scale": 1,
|
||||
"negative_prompt": "",
|
||||
"grammar_string": "",
|
||||
"banned_tokens": "",
|
||||
"sampler_priority": [
|
||||
"temperature",
|
||||
"dynamic_temperature",
|
||||
"quadratic_sampling",
|
||||
"top_k",
|
||||
"top_p",
|
||||
"typical_p",
|
||||
"epsilon_cutoff",
|
||||
"eta_cutoff",
|
||||
"tfs",
|
||||
"top_a",
|
||||
"min_p",
|
||||
"mirostat"
|
||||
],
|
||||
"ignore_eos_token_aphrodite": false,
|
||||
"spaces_between_special_tokens_aphrodite": true,
|
||||
"sampler_order": [
|
||||
6,
|
||||
0,
|
||||
1,
|
||||
3,
|
||||
4,
|
||||
2,
|
||||
5
|
||||
],
|
||||
"logit_bias": [],
|
||||
"n": 1,
|
||||
"rep_pen_size": 0,
|
||||
"genamt": 125,
|
||||
"max_length": 12288
|
||||
}
|
||||
11
configs_silly_tavern_v2_context_settings.json
Normal file
11
configs_silly_tavern_v2_context_settings.json
Normal file
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"story_string": "<|im_start|>system\n{{#if system}}{{system}}\n\n\n{{/if}}## Overall plot description:\n\n{{#if wiBefore}}{{wiBefore}}\n\n{{/if}}{{#if scenario}}{{scenario}}\n\n\n{{/if}}## Characters:\n\n### {{char}}\n{{#if description}}{{description}}\n{{/if}}{{#if personality}}{{personality}}\n\n{{/if}}### {{user}}\n{{#if persona}}{{persona}}\n\n{{/if}}{{#if wiAfter}}{{wiAfter}}\n\n{{/if}}{{#if mesExamples}}## {{char}}'s example message:\n\n{{mesExamples}}{{/if}}",
|
||||
"example_separator": "",
|
||||
"chat_start": "",
|
||||
"use_stop_strings": false,
|
||||
"always_force_name2": false,
|
||||
"trim_sentences": true,
|
||||
"include_newline": false,
|
||||
"single_line": false,
|
||||
"name": "ChatMLOpusV1_ST2"
|
||||
}
|
||||
17
configs_silly_tavern_v2_instruct_mode_settings.json
Normal file
17
configs_silly_tavern_v2_instruct_mode_settings.json
Normal file
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"system_prompt": "You are an intelligent, skilled, versatile writer.\n\nYour task is to write a role-play based on the information below.\n\n\n## Style description:\n\nThis role-play is written as a third-person introspective narrative in past tense. Scenes are described vividly, with great detail.",
|
||||
"input_sequence": "<|im_end|>\n<|im_start|>text names= {{user}}\n",
|
||||
"output_sequence": "<|im_end|>\n<|im_start|>text names= {{char}}\n",
|
||||
"first_output_sequence": "",
|
||||
"last_output_sequence": "<|im_end|>\n<|im_start|>user\nLength: 400 words\n{{char}} replies to {{user}} in detailed and elaborate way.<|im_end|>\n<|im_start|>text names= {{char}}\n",
|
||||
"system_sequence_prefix": "",
|
||||
"system_sequence_suffix": "",
|
||||
"stop_sequence": "",
|
||||
"separator_sequence": "",
|
||||
"wrap": false,
|
||||
"macro": true,
|
||||
"names": false,
|
||||
"names_force_groups": false,
|
||||
"activation_regex": "",
|
||||
"name": "ChatMLOpusV1_ST2"
|
||||
}
|
||||
2104
imatrix-base.txt
Normal file
2104
imatrix-base.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
imatrix-opus-v1.2-7b-F16.dat
Normal file
3
imatrix-opus-v1.2-7b-F16.dat
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6489a7c083e585cce855436e7e6679dc5adc4b5785d11193bf0964a85212efb8
|
||||
size 4988126
|
||||
3
opus-v1.2-7b-F16.gguf
Normal file
3
opus-v1.2-7b-F16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6f34458b1f50bb728a78fd892acdd6f5b67e93321f90677699bc3f2fcfa7efdf
|
||||
size 14484764768
|
||||
3
opus-v1.2-7b-IQ3_M-imatrix.gguf
Normal file
3
opus-v1.2-7b-IQ3_M-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:242b4a71ae468d5d2bb2a231a45921f49e6f2066f8d09b738b0edbd35b86a1ef
|
||||
size 3284902016
|
||||
3
opus-v1.2-7b-IQ3_S-imatrix.gguf
Normal file
3
opus-v1.2-7b-IQ3_S-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f08e1841b6ddfc8d104c3a548ef87d5ac7c2cb210d6a78bdb790c74ca0c3bf77
|
||||
size 3182403712
|
||||
3
opus-v1.2-7b-IQ3_XXS-imatrix.gguf
Normal file
3
opus-v1.2-7b-IQ3_XXS-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bfed7c4805ce93f1ede084203171b7222b9fdeaf96da749baf9ea1e0c7ca48ca
|
||||
size 2827353152
|
||||
3
opus-v1.2-7b-IQ4_NL-imatrix.gguf
Normal file
3
opus-v1.2-7b-IQ4_NL-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:29b9ec50a092184d275b10b7efca1a2805dfac2b32ab084ab33bf4fb9e54da3e
|
||||
size 4125705408
|
||||
3
opus-v1.2-7b-IQ4_XS-imatrix.gguf
Normal file
3
opus-v1.2-7b-IQ4_XS-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a8615723a351ab594236f8a43d8db1f0fc8bbdfed0722e97a4786e8a1c9628c7
|
||||
size 3907699648
|
||||
3
opus-v1.2-7b-Q4_K_M-imatrix.gguf
Normal file
3
opus-v1.2-7b-Q4_K_M-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:19ab1dc925873f9748b0eb6cbd64ac8f81a3d98f55ccc937494978d74a9c5e09
|
||||
size 4368450752
|
||||
3
opus-v1.2-7b-Q4_K_S-imatrix.gguf
Normal file
3
opus-v1.2-7b-Q4_K_S-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d6b6609375e64a968b5482c6466fc87da58a6ecd889e195de559cebcaf8c4140
|
||||
size 4140385472
|
||||
3
opus-v1.2-7b-Q5_K_M-imatrix.gguf
Normal file
3
opus-v1.2-7b-Q5_K_M-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:aafc45695ab26367d80fc62bd63d8510b688d34660d4aff4b18ab0a9bf76247d
|
||||
size 5131421888
|
||||
3
opus-v1.2-7b-Q5_K_S-imatrix.gguf
Normal file
3
opus-v1.2-7b-Q5_K_S-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c18062b742a955abd9a95841b84a0141766e60fa0b58434cba8e38060806c846
|
||||
size 4997728448
|
||||
3
opus-v1.2-7b-Q6_K-imatrix.gguf
Normal file
3
opus-v1.2-7b-Q6_K-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1c68ae3344ac4e1e5cbe9f5acef8cb225d8fc5c0f2a997bfdda4354603664255
|
||||
size 5942078720
|
||||
3
opus-v1.2-7b-Q8_0-imatrix.gguf
Normal file
3
opus-v1.2-7b-Q8_0-imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bab9dae9509978c7f8f022e7b8f2bd0b4bb128903f260a9e416f17886e85dd29
|
||||
size 7695875200
|
||||
Reference in New Issue
Block a user