commit 8e6ce77c300addfaaca163b60dc1f3ad6c4acab8 Author: ModelHub XC Date: Fri Apr 24 22:02:07 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: Ayansk11/FinSenti-DeepSeek-R1-1.5B-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..043cb7e --- /dev/null +++ b/.gitattributes @@ -0,0 +1,38 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +FinSenti-DeepSeek-R1-1.5B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +FinSenti-DeepSeek-R1-1.5B.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf b/FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf new file mode 100644 index 0000000..92e9c03 --- /dev/null +++ b/FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:781c6abdc7d384252fec25bc92d466d516d626d00e0745a5d1d0ae9cfbdc9f5e +size 1117320448 diff --git a/FinSenti-DeepSeek-R1-1.5B.Q5_K_M.gguf b/FinSenti-DeepSeek-R1-1.5B.Q5_K_M.gguf new file mode 100644 index 0000000..b4f7353 --- /dev/null +++ b/FinSenti-DeepSeek-R1-1.5B.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d94279237fb3ec8bcb5abc887ba2e15191faab1a554fc4a216fa6a3a7660c127 +size 1285494016 diff --git a/FinSenti-DeepSeek-R1-1.5B.Q8_0.gguf b/FinSenti-DeepSeek-R1-1.5B.Q8_0.gguf new file mode 100644 index 0000000..c93a3d4 --- /dev/null +++ b/FinSenti-DeepSeek-R1-1.5B.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fb71cfe5a8469e51d6b20d4eb1008d349d58c23997b39131a36c60f2349ceca3 +size 1894531840 diff --git a/Modelfile.Q4_K_M b/Modelfile.Q4_K_M new file mode 100644 index 0000000..233656e --- /dev/null +++ b/Modelfile.Q4_K_M @@ -0,0 +1,31 @@ +FROM ./FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf + +TEMPLATE \"\"\"<|im_start|>system +You are a financial sentiment analyst. Analyze the given financial text and provide: +1. Your reasoning in tags +2. Your sentiment classification (positive, negative, or neutral) in tags + +Always use this exact format: + +[Your step-by-step analysis] + +[positive/negative/neutral]<|im_end|> +<|im_start|>user +{{ .Prompt }}<|im_end|> +<|im_start|>assistant + + + +\"\"\" + +PARAMETER stop "<|im_end|>" +PARAMETER stop "" +PARAMETER stop "<|endoftext|>" +PARAMETER stop "<|im_start|>" + +PARAMETER temperature 0.3 +PARAMETER top_p 0.9 +PARAMETER top_k 40 +PARAMETER repeat_penalty 1.15 +PARAMETER num_ctx 1024 +PARAMETER num_predict 512 diff --git a/Modelfile.Q5_K_M b/Modelfile.Q5_K_M new file mode 100644 index 0000000..8ced9b8 --- /dev/null +++ b/Modelfile.Q5_K_M @@ -0,0 +1,31 @@ +FROM ./FinSenti-DeepSeek-R1-1.5B.Q5_K_M.gguf + +TEMPLATE \"\"\"<|im_start|>system +You are a financial sentiment analyst. Analyze the given financial text and provide: +1. Your reasoning in tags +2. Your sentiment classification (positive, negative, or neutral) in tags + +Always use this exact format: + +[Your step-by-step analysis] + +[positive/negative/neutral]<|im_end|> +<|im_start|>user +{{ .Prompt }}<|im_end|> +<|im_start|>assistant + + + +\"\"\" + +PARAMETER stop "<|im_end|>" +PARAMETER stop "" +PARAMETER stop "<|endoftext|>" +PARAMETER stop "<|im_start|>" + +PARAMETER temperature 0.3 +PARAMETER top_p 0.9 +PARAMETER top_k 40 +PARAMETER repeat_penalty 1.15 +PARAMETER num_ctx 1024 +PARAMETER num_predict 512 diff --git a/Modelfile.Q8_0 b/Modelfile.Q8_0 new file mode 100644 index 0000000..2727ff5 --- /dev/null +++ b/Modelfile.Q8_0 @@ -0,0 +1,31 @@ +FROM ./FinSenti-DeepSeek-R1-1.5B.Q8_0.gguf + +TEMPLATE \"\"\"<|im_start|>system +You are a financial sentiment analyst. Analyze the given financial text and provide: +1. Your reasoning in tags +2. Your sentiment classification (positive, negative, or neutral) in tags + +Always use this exact format: + +[Your step-by-step analysis] + +[positive/negative/neutral]<|im_end|> +<|im_start|>user +{{ .Prompt }}<|im_end|> +<|im_start|>assistant + + + +\"\"\" + +PARAMETER stop "<|im_end|>" +PARAMETER stop "" +PARAMETER stop "<|endoftext|>" +PARAMETER stop "<|im_start|>" + +PARAMETER temperature 0.3 +PARAMETER top_p 0.9 +PARAMETER top_k 40 +PARAMETER repeat_penalty 1.15 +PARAMETER num_ctx 1024 +PARAMETER num_predict 512 diff --git a/README.md b/README.md new file mode 100644 index 0000000..a0406bc --- /dev/null +++ b/README.md @@ -0,0 +1,227 @@ +--- +license: apache-2.0 +language: + - en +base_model: Ayansk11/FinSenti-DeepSeek-R1-1.5B +datasets: + - Ayansk11/FinSenti-Dataset +pipeline_tag: text-generation +library_name: gguf +tags: + - finance + - financial-sentiment + - chain-of-thought + - reasoning + - gguf + - llama-cpp + - ollama + - quantized + - finsenti +--- +# FinSenti-DeepSeek-R1-1.5B - GGUF + +GGUF builds of [FinSenti-DeepSeek-R1-1.5B](https://huggingface.co/Ayansk11/FinSenti-DeepSeek-R1-1.5B) +for use with [Ollama](https://ollama.com), [llama.cpp](https://github.com/ggerganov/llama.cpp), +LM Studio, KoboldCpp, and other GGUF-compatible runtimes. + +This is the same model as the SafeTensors repo, just converted and +quantized so you can run it on a CPU or a small GPU without pulling in +PyTorch. + +## Files in this repo + +| File | Quant | Size | Notes | +|------|-------|------|-------| +| `FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf` | Q4_K_M | 1.00 GB | Smallest, mild quality dip. Default pick for laptops. | +| `FinSenti-DeepSeek-R1-1.5B.Q5_K_M.gguf` | Q5_K_M | 1.16 GB | Balanced quality and size. | +| `FinSenti-DeepSeek-R1-1.5B.Q8_0.gguf` | Q8_0 | 1.70 GB | Closest to bf16, biggest file. | + +If you're not sure which to pick: **start with Q4_K_M**. It's the smallest +file, it runs everywhere, and the quality drop versus the original bf16 +weights is small for a model this size. + +## Quick start (llama.cpp) + +```bash +# Download the Q4_K_M file (or pick a different quant from the table above) +huggingface-cli download Ayansk11/FinSenti-DeepSeek-R1-1.5B-GGUF FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf --local-dir . + +# Run it +./llama-cli -m FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf \ + --system "You are a financial sentiment analyst. For each headline you receive, write a short reasoning chain inside ... tags, then give a single label inside ... tags. The label must be exactly one of: positive, negative, neutral." \ + -p "Apple beats Q4 estimates as iPhone sales jump 12% year over year." \ + -n 256 +``` + +## Quick start (Ollama) + +This repo ships a `Modelfile` for each quant. To register the Q4_K_M build +under the name `finsenti-deepseek-r1-1-5b`: + +```bash +huggingface-cli download Ayansk11/FinSenti-DeepSeek-R1-1.5B-GGUF \ + FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf Modelfile.Q4_K_M --local-dir ./finsenti-tmp +cd finsenti-tmp +ollama create finsenti-deepseek-r1-1-5b -f Modelfile.Q4_K_M + +# Then chat with it +ollama run finsenti-deepseek-r1-1-5b "Apple beats Q4 estimates as iPhone sales jump 12% year over year." +``` + +You should see output like: + +``` + +Beating estimates is a positive earnings surprise. A 12% YoY iPhone sales jump in the company's biggest product line points to demand strength. Both signals push the read positive. + +positive +``` + +## Quick start (Python via llama-cpp-python) + +```python +from llama_cpp import Llama + +llm = Llama( + model_path="./FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf", + n_ctx=2048, + n_threads=8, +) + +system = ( + "You are a financial sentiment analyst. For each headline you receive, " + "write a short reasoning chain inside ... tags, " + "then give a single label inside ... tags. The label " + "must be exactly one of: positive, negative, neutral." +) + +resp = llm.create_chat_completion( + messages=[ + {"role": "system", "content": system}, + {"role": "user", "content": "Apple beats Q4 estimates as iPhone sales jump 12% year over year."}, + ], + max_tokens=256, + temperature=0.0, +) +print(resp["choices"][0]["message"]["content"]) +``` + +## Hardware + +The Q4_K_M build is about 1.00 GB on disk and needs +roughly 2 GB of free RAM at runtime. On a modern laptop +CPU you should see 15-40 tokens per second depending on the size of the +model and your core count. Throwing it on a small GPU (Apple Silicon, a +6-8 GB NVIDIA card) gets you considerably faster generation. + +If you need more headroom, the Q5_K_M and Q8_0 files are progressively +closer to the original bf16 quality at the cost of size. + +## Picking a quant + +- **Q4_K_M** (1.00 GB): the default for laptops + and small servers. Mild quality dip versus full precision but fits + almost anywhere. +- **Q5_K_M** (1.16 GB): a step up if you have + the RAM. Most people won't notice the difference from Q8. +- **Q8_0** (1.70 GB): closest to the bf16 weights. + Use this if you want the cleanest output and have the disk space. + +## Run it on your phone + +This model is small enough to run entirely on-device. The Q4_K_M build is +1.00 GB on disk and needs roughly 1.6 GB of free RAM +during inference, so it fits on most phones with 4 GB+ RAM (roughly any +Android flagship from 2020 onward, or iPhone 11 and newer). + +### iOS + +The easiest path is [PocketPal AI](https://apps.apple.com/app/id6502579498) +(free, App Store): + +1. Install PocketPal AI from the App Store. +2. Open the app and go to **Models** -> **+** -> **Add from Hugging Face**. +3. Search for `Ayansk11/FinSenti-DeepSeek-R1-1.5B-GGUF` and select `FinSenti-DeepSeek-R1-1.5B.Q4_K_M.gguf`. +4. Tap download; the file is 1.00 GB. +5. Once downloaded, tap the model to load it. Open the chat tab. +6. Set the system prompt (gear icon) to: + > You are a financial sentiment analyst. For each headline you receive, + > write a short reasoning chain inside `...` tags, + > then give a single label inside `...` tags. The label + > must be exactly one of: positive, negative, neutral. +7. Send a headline like *"Apple beats Q4 estimates as iPhone sales jump 12% YoY"* + and you'll get back the reasoning chain plus the label. + +[LLMFarm](https://apps.apple.com/app/id6443968971) and +[Private LLM](https://privatellm.app/) work too if you already use them. + +### Android + +PocketPal AI is on +[Google Play](https://play.google.com/store/apps/details?id=com.pocketpalai) +as well, with the same flow as the iOS version. + +If you'd rather avoid the Play Store, +[ChatterUI](https://github.com/Vali-98/ChatterUI) is a free, open-source +client. Install the APK from the GitHub Releases page, then add the model +from Hugging Face inside the app. + +### Tips for phone usage + +- **Keep max output tokens around 256.** A reasoning chain plus an answer + rarely needs more than that. +- **Inference is fully offline** once the model is downloaded. No data + leaves your phone. +- **Heat and battery:** one classification finishes in a few seconds, but + running hundreds in a loop will warm the device up. Charge while batching. +- **Stick with Q4_K_M on phones.** The quality difference vs Q5/Q8 for + sentiment labels is small, and the smaller file leaves more headroom for + the OS. + +## Prompt format + +Same as the base model. Use the system prompt verbatim, put the headline +or short snippet in the user turn, and parse the `...` +block for the label. + +## Limitations + +GGUF is a faithful conversion of the base model, so the same caveats apply: + +- English only +- Short text only (training context was 2048 tokens) +- Three labels: positive, negative, neutral +- It explains its read but it isn't doing finance research; don't use the + reasoning chain as investment advice + +Quantization adds a small extra error on top of the base model. For +Q4_K_M on a model this size you'll see occasional disagreement with the +bf16 model on borderline headlines, usually neutral-vs-positive flips. + +## Related FinSenti models + +Other sizes and bases trained with the same recipe: + +- **Qwen3**: [Qwen3-0.6B](https://huggingface.co/Ayansk11/FinSenti-Qwen3-0.6B), [Qwen3-1.7B](https://huggingface.co/Ayansk11/FinSenti-Qwen3-1.7B), [Qwen3-4B](https://huggingface.co/Ayansk11/FinSenti-Qwen3-4B), [Qwen3-8B](https://huggingface.co/Ayansk11/FinSenti-Qwen3-8B) +- **Qwen3.5**: [Qwen3.5-0.8B](https://huggingface.co/Ayansk11/FinSenti-Qwen3.5-0.8B), [Qwen3.5-2B](https://huggingface.co/Ayansk11/FinSenti-Qwen3.5-2B), [Qwen3.5-4B](https://huggingface.co/Ayansk11/FinSenti-Qwen3.5-4B), [Qwen3.5-9B](https://huggingface.co/Ayansk11/FinSenti-Qwen3.5-9B) + +The full-precision SafeTensors version of this model is at +[Ayansk11/FinSenti-DeepSeek-R1-1.5B](https://huggingface.co/Ayansk11/FinSenti-DeepSeek-R1-1.5B), and the +training data is at +[Ayansk11/FinSenti-Dataset](https://huggingface.co/datasets/Ayansk11/FinSenti-Dataset). + +## Citation + +```bibtex +@misc{shaikh2026finsenti, + title = {FinSenti: Small Language Models for Financial Sentiment with Chain-of-Thought Reasoning}, + author = {Shaikh, Ayan}, + year = {2026}, + url = {https://huggingface.co/collections/Ayansk11/finsenti}, + note = {Indiana University} +} +``` + +## License + +Apache 2.0.