RDson/Llama-3-Magenta-Instruct-4x8B-MoE-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Compare commits

base: RDson:aeaf3bcb866789c8d0acdbc2114f2d1a3cd02807
Branches Tags
RDson:main
...
compare: RDson:main
Branches Tags
RDson:main

10 Commits
aeaf3bcb86 ... main

Author SHA1 Message Date
Robin Davidsson
cd7578ec9a Update README.md 2024-05-17 10:42:26 +00:00
Robin Davidsson
eaa77ab662 Update README.md 2024-05-17 10:41:19 +00:00
Robin Davidsson
941481879f Update README.md 2024-05-17 10:39:15 +00:00
Robin Davidsson
a18428f13e Upload Llama-3-Magenta-Instruct-4x8B-MoE-F16.gguf 2024-05-15 22:58:18 +00:00
Robin Davidsson
1f6e5afa08 Upload 2 files 2024-05-15 22:14:08 +00:00
Robin Davidsson
73b1e5b97f Upload 2 files 2024-05-15 21:21:29 +00:00
Robin Davidsson
e418c59482 Upload 2 files 2024-05-15 21:01:43 +00:00
Robin Davidsson
721ca9c956 Upload 2 files 2024-05-15 20:04:00 +00:00
Robin Davidsson
c6014b5df7 Upload 4 files 2024-05-15 19:07:35 +00:00
Robin Davidsson
08eacf2421 Upload 3 files 2024-05-15 18:23:33 +00:00
18 changed files with 74 additions and 2 deletions
Expand all files Collapse all files

16
.gitattributes vendored
View File

@@ -36,3 +36,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q3_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-Magenta-Instruct-4x8B-MoE-F16.gguf filter=lfs diff=lfs merge=lfs -text

3
Llama-3-Magenta-Instruct-4x8B-MoE-F16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7fcc32d5a3c05cc964ee12ca2c5f9c5e4b59ea3baf0995f85a4a77431fe85b8c
size 49893323328

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e7ff67f516e1c2337f0cb85cbd294b2918409b3cd9f6496ffde4fd7c37ec215f
size 13043479136

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4a44d0b8144674dbf53fea3a88a28484683713691df44233ba8950ec02937d47
size 12079837792

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:66aab14efa6e6b36a2f8f24eb9a30c489b2a84a97721d017c30d25a6c02b4f3c
size 10932695648

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2dbf034d312d199e36edb01894cb4e72cfc3ae3f8143d643ffb814f924ab461f
size 14175457888

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7e6516caf700930768fe4b4b857ef9c5360e29a06bde8195e2202582482e7c3e
size 15701463648

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5cd2346b3df6dc4a89b514f58d34a0ea0bad510e140ddc76d968ab4d3029dfd8
size 15161643616

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5cd2346b3df6dc4a89b514f58d34a0ea0bad510e140ddc76d968ab4d3029dfd8
size 15161643616

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0537e4605505acd026ca1be330fba6f975fd0d23231ab8261dc8e967d983368c
size 13128978432

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ffeb6560ed09c96d41a98419999812cf89731cdd34256795d749d0aec91b8e4c
size 17227469408

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b06afa2e84f6100d2b36445d8b2dd9a0d0e7697bc308e517e44966a2c31a0451
size 18753475168

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:32fa4a1cf9a7a640062742e96d39e6e9504bbb2f44cd139f6a81acfdc92f3cdb
size 17735504480

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:32fa4a1cf9a7a640062742e96d39e6e9504bbb2f44cd139f6a81acfdc92f3cdb
size 17735504480

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3ebd2de558e688350c9adfa5edbf4f8c15256babb7ee5bb6f64cfabc41b9eac5
size 17227469408

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6edff3080e7672570d13d5299a8eb4e9dfb482cc32c51f42774c03a6453b70a1
size 20470231648

3
Llama-3-Magenta-Instruct-4x8B-MoE-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c2e8d79f83f57d264bce001ccd8e220c8e377fe2ebd50c105f6617b59988ece5
size 26510733920

12
README.md
View File

@@ -10,10 +10,18 @@ tags:
- quant
---
# GGUF files of [Llama-3-Magenta-Instruct-4x8B-MoE](https://huggingface.co/RDson/Llama-3-Magenta-Instruct-4x8B-MoE)
<img src="https://i.imgur.com/c1Mv8cy.png" width="640"/>
# Llama-3-Magenta-Instruct-4x8B-MoE
This is a experimental MoE created from [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct), [nvidia/Llama3-ChatQA-1.5-8B](https://huggingface.co/nvidia/Llama3-ChatQA-1.5-8B), [Salesforce/SFR-Iterative-DPO-LLaMA-3-8B-R](https://huggingface.co/Salesforce/SFR-Iterative-DPO-LLaMA-3-8B-R) and [Muhammad2003/Llama3-8B-OpenHermes-DPO](https://huggingface.co/Muhammad2003/Llama3-8B-OpenHermes-DPO) using Mergekit.
<img src="https://i.imgur.com/c1Mv8cy.png" width="640"/>
## You should also check out the updated [Llama-3-Peach-Instruct-4x8B-MoE](https://huggingface.co/RDson/Llama-3-Peach-Instruct-4x8B-MoE)!
This is a experimental MoE using Mergekit, created from
* [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)
* [nvidia/Llama3-ChatQA-1.5-8B](https://huggingface.co/nvidia/Llama3-ChatQA-1.5-8B)
* [Salesforce/SFR-Iterative-DPO-LLaMA-3-8B-R](https://huggingface.co/Salesforce/SFR-Iterative-DPO-LLaMA-3-8B-R)
* [Muhammad2003/Llama3-8B-OpenHermes-DPO](https://huggingface.co/Muhammad2003/Llama3-8B-OpenHermes-DPO)
Mergekit yaml file:
```