初始化项目，由ModelHub XC社区提供模型

Model: afrideva/Mermaid-Llama-6.7B-RAG-Code-Instruct-GGUF Source: Original Platform
2026-10-28 07:46:26 +08:00
commit 345894bbcf
9 changed files with 293 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,42 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+mermaid-llama-6.7b-rag-code-instruct.fp16.gguf filter=lfs diff=lfs merge=lfs -text
+mermaid-llama-6.7b-rag-code-instruct.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+mermaid-llama-6.7b-rag-code-instruct.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+mermaid-llama-6.7b-rag-code-instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+mermaid-llama-6.7b-rag-code-instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+mermaid-llama-6.7b-rag-code-instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+mermaid-llama-6.7b-rag-code-instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,230 @@
+---
+base_model: TroyDoesAI/Mermaid-Llama-6.7B-RAG-Code-Instruct
+inference: true
+license: cc-by-4.0
+model_creator: TroyDoesAI
+model_name: Mermaid-Llama-6.7B-RAG-Code-Instruct
+pipeline_tag: text-generation
+quantized_by: afrideva
+tags:
+- gguf
+- ggml
+- quantized
+---
+
+# Mermaid-Llama-6.7B-RAG-Code-Instruct-GGUF
+
+Quantized GGUF model files for [Mermaid-Llama-6.7B-RAG-Code-Instruct](https://huggingface.co/TroyDoesAI/Mermaid-Llama-6.7B-RAG-Code-Instruct) from [TroyDoesAI](https://huggingface.co/TroyDoesAI)
+
+## Original Model Card:
+
+# Introducing Mermaid-Llama-6.7B-RAG
+
+Powered by 6.7 billion parameters, this model sets the bar for excellence in 
+AI-driven code comprehension and narrative visualization now with further reduction of hallucinations inspired by https://huggingface.co/jondurbin
+who created the "Context-Obedient" chat template. We stand on the shoulders of Giants, so we thank you Jon Durbin the original RAG pioneer for LLM's.
+Special Thanks to Eric Hartford for sharing his intuition with me personally on prompt templates, your shared wisdom has helped me innovate my own style that works for my own specialized Mermaid Models.
+
+Beyond turning input into Flow Diagrams this RAG Model excels in Formatted Knowledge Graph utilization in the Mermaid JS Syntax.
+Now includes an enhanced capability called "Code Instruct," which allows the model to generate executable code snippets directly from user instructions. 
+
+See more Mermaid Here : https://www.mermaidchart.com
+
+![MermaidLlama GIF](Mermaid_ShowCase/MermaidLlama.webp)
+
+---
+```
+Note: I have been informed over this past 2 months that my models are being used in production.
+Through insights gathered on how my models are being used effectively in business environments 
+I have tailored this model to the needs of those that have reached out to me.
+
+So please enjoy, and feedback is always welcome, good or bad. I prefer bad actually.
+- Current Issue is lack of compute - I will solve once I get a job / money to train : Context length of 4096 is very limiting for those that want full system diagrams without using aggregation strategies.
+```
+
+### Key Features
+
+1. **Code Understanding:**
+   - Masters Python's intricacies.
+   - Generates accurate Mermaid Diagram Flow Charts.
+   - Ideal for developers visualizing code logic.
+
+2. **Storytelling Capabilities:**
+   - Converts narratives into captivating Mermaid Diagrams.
+   - Maps character interactions, plot developments, and narrative arcs.
+
+3. **Unmatched Performance:**
+   - Surpasses GPT-4 in generating well-organized Mermaid Diagrams.
+
+4. **Enhanced Adherence to Context (New):**
+   - Incorporates contextual prompts to improve adherence and reduce hallucinations.
+   - Supports the airoboros context-obedient format.
+
+### Collaboration
+
+For collaboration opportunities to enhance Mermaid's capabilities, contact troydoesai@gmail.com.
+
+### Use Cases
+
+- **Retrieval-Augmented Generation (RAG):** Creates condensed knowledge graphs to enhance retrieval using vector databases for efficient information retrieval. Combines knowledge graphs and context-aware RAG capabilities for better knowledge condensation.
+- **Code Documentation:** Generates automatic visual flow charts from Python code.
+- **Storyboarding:** Creates visually appealing diagrams for storytelling.
+- **Project Planning:** Generates visual project flow maps for effective team communication.
+- **Learning Python:** Assists students in visualizing Python code structures.
+- **Game Design:** Visualizes game storylines for coherent narrative structure.
+
+### Dataset Format (New)
+To enhance contextual adherence and reduce hallucinations, the dataset follows the format below:
+
+```
+BEGININPUT
+BEGINCONTEXT
+[key0: value0]
+[key1: value1]
+ENDCONTEXT
+[insert your text blocks here]
+ENDINPUT
+BEGININSTRUCTION
+[insert your instruction(s)]
+ENDINSTRUCTION
+```
+
+This structure, while verbose, helps models understand specific responses and sources.
+
+### Example
+
+**Prompt:**
+```
+BEGININPUT
+BEGINCONTEXT
+date: 2021-01-01
+url: https://web.site/123
+ENDCONTEXT
+Blueberries are now green.
+ENDINPUT
+BEGININSTRUCTION
+What color are blueberries? Source?
+ENDINSTRUCTION
+```
+
+**Expected Response:**
+```
+Blueberries are now green.
+Source:
+date: 2021-01-01
+url: https://web.site/123
+```
+
+---
+
+## New Feature: Code Instruct
+
+The Mermaid-Llama-6.7B-RAG model now includes an enhanced capability called "Code Instruct," which allows the model to generate executable code snippets directly from user instructions. This feature is particularly useful for developers and educators who need to quickly generate code examples or want to automate coding tasks.
+
+### How It Works
+
+To use the Code Instruct feature, provide a clear and concise instruction in the following template:
+
+```markdown
+Code-Assistant-Request:
+{instruction}
+
+### Code:
+```
+
+The model interprets the instruction and generates a corresponding code block that attempts to fulfill the specified task. This feature leverages the model's deep understanding of code syntax and semantics, enhanced by training on a diverse range of programming tasks.
+
+### Example Usage
+
+**Instruction:**
+```
+Code-Assistant-Request:
+Create a Python function to calculate the factorial of a number
+
+### Code:
+```
+
+**Generated Code:**
+```python
+def factorial(n):
+    if n == 0:
+        return 1
+    else:
+        return n * factorial(n-1)
+
+print(factorial(5))  # Output: 120
+```
+
+![5 CODE_INSTRUCT GIF](Mermaid_ShowCase/5_Code_Instruct.gif)
+
+---
+
+### Proof of Concept
+
+A VSCode Extension is forthcoming, providing a live flow map upon pausing for more than 10 seconds.
+
+### Training Specifications
+
+- **LoRA Rank:** 2048
+- **LoRA Alpha:** 4096
+- **Batch Size:** 1
+- **Micro Batch Size:** 1
+- **Cutoff Length:** 4096
+- **Save every n steps:** 1000
+- **Epochs:** 3
+- **Learning Rate:** 1e-6
+- **LR Scheduler:** Cosine
+
+**Target Modules:**
+- Enable q_proj
+- Enable v_proj
+- Enable k_proj
+- Enable o_proj
+- Enable gate_proj
+- Enable down_proj
+- Enable up_proj
+
+---
+
+## Getting Started
+
+Start by downloading one of my models.
+
+![0 TroyDoesAI GIF](Mermaid_ShowCase/0_TroyDoesAI.gif)
+
+Load the model.
+
+![1 Load Model in 4-bit Show Example Use GIF](Mermaid_ShowCase/1_LoadModel_in_4bit_Show_Example_Use.gif)
+
+Use my prompt template to generate a Mermaid code block, which can be viewed in the Mermaid Live Editor or using the Mermaid CLI tool.
+
+![2 Loaded Model in Full Precision 16-bit Show Inference and Mermaid Live Editor GIF](Mermaid_ShowCase/2_Loaded_Model_in_Full_Precision_16bit_Show_Inference_and_Mermaid_Live_editor.gif)
+
+Here we open the VLLM GUI Program while still running in Vram the Mermaid-Llama-8B to compare the flow diagram to the actual program and show the lightweight capabilites of small models on consumer hardware.
+
+![3 Open The Program VLLM Program With Full Precision Mermaid-Llama-8B Running to Evaluate Flow Map GIF](Mermaid_ShowCase/3_Open_The_Program_VLLM_Program_With_Full_Precision_Mermaid-Llama-8B-Running_to_evaluate_flow_map.gif)
+
+## More on my VLLM Class and inference GUI : https://github.com/Troys-Code/VLLM
+
+![Python RtdBsaz8gy GIF](Mermaid_ShowCase/python_RtdBsaz8gy.gif)
+---
+
+Note: This model should be treated as an Auto-Complete Model, Do not try talking to it in chat you are gonna get garbage, those layers have been pruned and replaced, that is all you will hear of my secret sauce on training on small < 1000 entry datasets.
+
+```
+ԅ(≖‿≖ԅ)
+STAY TUNED: THERES MORE TO COME, SOON MERMAID MODELS WILL BE ABLE TO TURN "MERMAID" --> "CODE"
+
+This new dataset is gonna be a game changer for refactoring code blocks if it works.
+I am interviewing like crazy so this may take some time as my days have been hectic, imaging studying for finals week every week. 
+```
+
+video on how to use colab notebook and inference the model in the simplest example:
+'''
+https://m.youtube.com/watch?v=fdwoOmiA2d0
+'''
+
+colab notebook: 
+'''
+https://colab.research.google.com/github/Troys-Code/MermaidEngine/blob/main/Mermaid_Llama_RAG_Colab_TextGen_GPU.ipynb
+'''
--- a/mermaid-llama-6.7b-rag-code-instruct.Q2_K.gguf
+++ b/mermaid-llama-6.7b-rag-code-instruct.Q2_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc645e34baf055db4186d424b08f83e6fbcb2d513f97812a8122d61e1ab48e7a
+size 2533399168
--- a/mermaid-llama-6.7b-rag-code-instruct.Q3_K_M.gguf
+++ b/mermaid-llama-6.7b-rag-code-instruct.Q3_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d591989dab5115fb6c73de93a92b67d7bcda434da087443c59bca3bc1faee584
+size 3298549728
--- a/mermaid-llama-6.7b-rag-code-instruct.Q4_K_M.gguf
+++ b/mermaid-llama-6.7b-rag-code-instruct.Q4_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1f6ad3ed9a5f975d89877238f6d764b5ec9ace33648bc3e0105524f8414bae9
+size 4081561792
--- a/mermaid-llama-6.7b-rag-code-instruct.Q5_K_M.gguf
+++ b/mermaid-llama-6.7b-rag-code-instruct.Q5_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c4ec91eb6de0d62fd23f5ad4a07a0baa109584337ccfd4c1f1a53dbfe357cad
+size 4783726272
--- a/mermaid-llama-6.7b-rag-code-instruct.Q6_K.gguf
+++ b/mermaid-llama-6.7b-rag-code-instruct.Q6_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2eba9c9119a2720b80e6a7a48c090d2b276d7e45aefbf3057d7edc584f19a1b
+size 5529776032
--- a/mermaid-llama-6.7b-rag-code-instruct.Q8_0.gguf
+++ b/mermaid-llama-6.7b-rag-code-instruct.Q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:274407e2ab7b50ac3c2f0e3946059e18ceca4dfd751837fe264e7efeba31011d
+size 7161717216
--- a/mermaid-llama-6.7b-rag-code-instruct.fp16.gguf
+++ b/mermaid-llama-6.7b-rag-code-instruct.fp16.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff2601887afbef8346f4b37dbac614378266729f70999bb7f12a5680f701eebc
+size 13478908896