commit 7008a811d2d475519928512447c1375fb1585d37 Author: ModelHub XC Date: Wed Jun 24 23:53:21 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: Deeokay/llama3-8B-Instruct-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..92bb8e4 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,42 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +unsloth.BF16.gguf filter=lfs diff=lfs merge=lfs -text +unsloth.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +unsloth.F16.gguf filter=lfs diff=lfs merge=lfs -text +unsloth.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +unsloth.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +unsloth.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +unsloth.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..403a566 --- /dev/null +++ b/README.md @@ -0,0 +1,107 @@ +--- +base_model: unsloth/llama-3-8b-Instruct-bnb-4bit +language: +- en +license: apache-2.0 +tags: +- text-generation-inference +- transformers +- unsloth +- llama +- gguf +--- + +# Uploaded model + +- **Developed by:** Deeokay +- **License:** apache-2.0 +- **Finetuned from model :** unsloth/llama-3-8b-Instruct-bnb-4bit + +This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library. + +[](https://github.com/unslothai/unsloth) + + +# README + +This is a test model on a the following +- a private dataset +- slight customization on llama3 template (no new tokens | no new configs) +- Works with Ollama create with just "FROM path/to/model" as Modelfile (llama3 template works no issues) + +# HOW TO USE + +The whole point of conversion for me was I wanted to be able to to use it through Ollama or (other local options) +For Ollama, it required to be a GGUF file. Once you have this it is pretty straight forward (if it is in llama3 which this model is) + +Quick Start: +- You must already have Ollama running in your setting +- Download the unsloth.Q4_K_M.gguf model from Files +- In the same directory create a file call "Modelfile" +- Inside the "Modelfile" type + +```python +FROM ./unsloth.Q4_K_M.gguf + +``` +- Save a go back to the folder (folder where model + Modelfile exisit) +- Now in terminal make sure you are in the same location of the folder and type in the following command + +```python +ollama create mycustomai # "mycustomai" <- you can name it anything u want +``` + +This GGUF is based on llama3-3-8B-Instruct thus ollama doesn't need anything else to auto configure this model + +After than you should be able to use this model to chat! + +Model is also available in Ollama +- deeokay/minillama -> Q2_K version +- deeokay/mediumllama -> Q3_K_M version +- deeokay/customllama -> Q4_K_M version + +In the terminal just +```pthon +ollama pull deeokay/customllama +``` + +and you can use the model. + + +# NOTE: DISCLAIMER + +Please note this is not for the purpose of production, but result of Fine Tuning through self learning + +The llama3 Special Tokens where kept the same, however the format was slight customized using the available tokens + +I have foregone the {{.System}} part as this would be updated when converting the llama3. + +I wanted to test if the model would understand additional headers that I created such as what my datasets has +- Analaysis, Classification, Sentiment + +Mulitple pass through my ~70K personalized customized dataset. + +If would like to know how I started creating my dataset, you can check this link +[Crafting GPT2 for Personalized AI-Preparing Data the Long Way (Part1)](https://medium.com/@deeokay/the-soul-in-the-machine-crafting-gpt2-for-personalized-ai-9d38be3f635f) + +As the data was getting created with custom GPT2 special tokens, I had to convert that to the llama3 Template. + +However I got creative again.. the training data has the following Template: + +``` +<|begin_of_text|> +<|start_header_id|>user<|end_header_id|> +{{.Prompt}}<|eot_id|><|start_header_id|>analysis<|end_header_id|> +{{.Analysis}}<|eot_id|><|start_header_id|>assistant<|end_header_id|> +{{.Response}}<|eot_id|><|start_header_id|>classification<|end_header_id|> +{{.Classification}}<|eot_id|><|start_header_id|>sentiment<|end_header_id|> +{{.Sentiment}}<|eot_id|> <|start_header_id|>user<|end_header_id|> +<|end_of_text|> + +``` + +The llama3 standard template holds, and can be created in Ollama through normal llama3 template + +Will be updating this periodically.. as I have limited colab resources.. + + diff --git a/config.json b/config.json new file mode 100644 index 0000000..a4ba21b --- /dev/null +++ b/config.json @@ -0,0 +1,3 @@ +{ + "model_type": "llama" +} \ No newline at end of file diff --git a/unsloth.BF16.gguf b/unsloth.BF16.gguf new file mode 100644 index 0000000..eba8b08 --- /dev/null +++ b/unsloth.BF16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3a166266917ed07f141ccbaf66f63ce0e0cac4986ee72ae5d274a2f85fd742a0 +size 16068890944 diff --git a/unsloth.F16.gguf b/unsloth.F16.gguf new file mode 100644 index 0000000..fa5e302 --- /dev/null +++ b/unsloth.F16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:46bac5c13e3f3635b1487055472df11f040af05509eef975055326cf92504591 +size 16068891168 diff --git a/unsloth.Q2_K.gguf b/unsloth.Q2_K.gguf new file mode 100644 index 0000000..c9f83b5 --- /dev/null +++ b/unsloth.Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d95c47024e6cdc05bf9f5c26ee0b9feb0fa4ab9d3988ea3fd439f9d59804841f +size 3179131424 diff --git a/unsloth.Q3_K_M.gguf b/unsloth.Q3_K_M.gguf new file mode 100644 index 0000000..25bed1e --- /dev/null +++ b/unsloth.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b48eb534a0f8eb25d28ca15c67877ddb4077b5bdc8ccf53f3e660377470efdfb +size 4018917920 diff --git a/unsloth.Q4_K_M.gguf b/unsloth.Q4_K_M.gguf new file mode 100644 index 0000000..3a31469 --- /dev/null +++ b/unsloth.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:18c0fcb06ceda91f329151603d41fd742289943f53826e003166cf71ab5c3a18 +size 4920734240 diff --git a/unsloth.Q5_K_M.gguf b/unsloth.Q5_K_M.gguf new file mode 100644 index 0000000..e53e505 --- /dev/null +++ b/unsloth.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f19d10cb5b3fbdf5f342ff98c8ba4142f02c16819573778834b2b554b620131f +size 5732987424 diff --git a/unsloth.Q8_0.gguf b/unsloth.Q8_0.gguf new file mode 100644 index 0000000..d5a878e --- /dev/null +++ b/unsloth.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:28c9c1fda23ae0baf1e3d1f4f051eeb8a21a6656ccfa8bf05b1308f7dd8c55fb +size 8540770848