初始化项目，由ModelHub XC社区提供模型

Model: Swicked86/phi4-mm-gguf Source: Original Platform
2026-10-25 21:24:21 +08:00
commit d9d3041c89
10 changed files with 829 additions and 0 deletions
--- a/deploy/Modelfile
+++ b/deploy/Modelfile
@@ -0,0 +1,36 @@
+# Ollama Modelfile — Phi-4 Multimodal Instruct Q4_K_M
+# Optimised for: Intel 11th Gen NUC, 8 GB RAM, CPU-only
+#
+# Source model : microsoft/Phi-4-multimodal-instruct
+# License      : MIT  https://huggingface.co/microsoft/Phi-4-multimodal-instruct/blob/main/LICENSE
+# Quantization : Q4_K_M via llama.cpp llama-quantize
+# Architecture : phi3 (3.8B LLM backbone + vision/speech adapters in base GGUF)
+
+FROM ./phi4-mm-Q4_K_M.gguf
+
+# ── Context & KV cache ───────────────────────────────────────────────────────
+# 8 192 tokens balances capability vs RAM on 8 GB hardware.
+# Lower to 4096 if you observe OOM / heavy swapping.
+PARAMETER num_ctx 8192
+
+# ── CPU tuning ───────────────────────────────────────────────────────────────
+# 11th Gen NUC typically 4 cores / 8 logical threads (i5/i7-1135G7 / 1165G7).
+# Reduce to 4 if the NUC is a Core i3 variant.
+PARAMETER num_thread 8
+
+# No discrete GPU — all layers run on CPU.
+PARAMETER num_gpu 0
+
+# Flash attention is a GPU feature; disable for CPU inference.
+PARAMETER flash_attn false
+
+# ── Generation defaults ───────────────────────────────────────────────────────
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.1
+PARAMETER stop "<|end|>"
+PARAMETER stop "<|user|>"
+PARAMETER stop "<|assistant|>"
+
+# ── System prompt ─────────────────────────────────────────────────────────────
+SYSTEM """You are a helpful, accurate, and concise AI assistant. You excel at reasoning, analysis, writing, coding, and answering questions. Be direct and thorough."""
--- a/deploy/README-deployment.md
+++ b/deploy/README-deployment.md
@@ -0,0 +1,150 @@
+---
+license: mit
+base_model: microsoft/Phi-4-multimodal-instruct
+tags:
+  - phi4mm
+  - gguf
+  - quantized
+  - q4_k_m
+  - cpu
+  - ollama
+  - llama-cpp
+language:
+  - multilingual
+  - ar
+  - zh
+  - cs
+  - da
+  - nl
+  - en
+  - fi
+  - fr
+  - de
+  - he
+  - hu
+  - it
+  - ja
+  - ko
+  - "no"
+  - pl
+  - pt
+  - ru
+  - es
+  - sv
+  - th
+  - tr
+  - uk
+pipeline_tag: text-generation
+---
+
+# Phi-4 Multimodal Instruct — GGUF Quantizations
+
+CPU-optimised GGUF quantizations of [microsoft/Phi-4-multimodal-instruct](https://huggingface.co/microsoft/Phi-4-multimodal-instruct) produced with [llama.cpp](https://github.com/ggerganov/llama.cpp).
+
+## Model Summary
+
+Phi-4-multimodal-instruct is a **5.6 B parameter** lightweight multimodal foundation model by Microsoft that processes text, image, and audio inputs. The backbone LLM is **Phi-4-Mini** (3.8 B). This repository contains GGUF quantizations for local CPU and GPU deployment.
+
+- **Context length:** 128 K tokens (131 072)
+- **Architecture:** `phi3` (GGUF)  
+- **License:** [MIT](https://huggingface.co/microsoft/Phi-4-multimodal-instruct/blob/main/LICENSE) — © Microsoft Corporation
+
+## Available Files
+
+| File | Quant | Size | BPW | Best for |
+|------|-------|------|-----|----------|
+| `phi4-mm-Q4_K_M.gguf` | Q4_K_M | 2.37 GB | 5.18 | CPU inference, 8 GB RAM systems |
+| `phi4-mm-Q8_0.gguf`   | Q8_0   | 3.90 GB | 8.00 | GPU/high-RAM systems, near-lossless |
+| `phi4-mm-f16.gguf`    | F16    | 7.17 GB  | 16.00 | Source / re-quantization base |
+
+## Quantization Method
+
+Quantized using `llama-quantize` from [ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp) (build 8334).
+
+```bash
+llama-quantize phi4-mm-f16.gguf phi4-mm-Q4_K_M.gguf Q4_K_M 16
+```
+
+Source weights converted from the original safetensors using `convert_hf_to_gguf.py`.
+
+## Recommended Usage
+
+### Ollama (CPU — Intel NUC / low-power hardware)
+
+```bash
+# Pull and run (once uploaded to Ollama registry)
+ollama run phi4-mm-nuc
+
+# Or import from GGUF directly with a Modelfile:
+ollama create phi4-mm-nuc -f Modelfile
+ollama run phi4-mm-nuc
+```
+
+**Modelfile for 8 GB RAM / no GPU:**
+
+```
+FROM phi4-mm-Q4_K_M.gguf
+PARAMETER num_ctx 8192
+PARAMETER num_thread 8
+PARAMETER num_gpu 0
+PARAMETER flash_attn false
+PARAMETER temperature 0.7
+PARAMETER repeat_penalty 1.1
+SYSTEM "You are a helpful, accurate, and concise AI assistant."
+```
+
+### llama.cpp CLI (GPU, full quality)
+
+```bash
+./build/bin/llama-cli \
+  -m phi4-mm-Q8_0.gguf \
+  --ctx-size 65536 \
+  --flash-attn on \
+  --kv-offload \
+  -ngl 99 \
+  --threads 16
+```
+
+### OpenClaw Integration
+
+In `~/.openclaw/openclaw.json`:
+
+```json
+{
+  "agent": {
+    "model": "ollama/phi4-mm-nuc"
+  },
+  "modelConfigs": {
+    "ollama/phi4-mm-nuc": {
+      "provider": "ollama",
+      "model": "phi4-mm-nuc",
+      "baseUrl": "http://localhost:11434"
+    }
+  }
+}
+```
+
+## Hardware Notes
+
+| Hardware | Recommended quant | Context | Notes |
+|---|---|---|---|
+| Intel NUC 11th Gen, 8 GB RAM | Q4_K_M | 8 192 | CPU-only, `num_gpu 0` |
+| Laptop / desktop, 16 GB RAM | Q5_K_M or Q8_0 | 16 384 | CPU or iGPU |
+| GPU with ≥ 8 GB VRAM | Q8_0 or F16 | 32 768–65 536 | Full `-ngl 99` offload |
+
+## License
+
+This repository redistributes quantized weights derived from [microsoft/Phi-4-multimodal-instruct](https://huggingface.co/microsoft/Phi-4-multimodal-instruct) under the original **MIT License**.
+
+```
+MIT License
+Copyright (c) Microsoft Corporation.
+```
+
+Quantization tooling (llama.cpp) is also MIT licensed. See [llama.cpp LICENSE](https://github.com/ggerganov/llama.cpp/blob/master/LICENSE).
+
+## Attribution
+
+- Original model: Microsoft Research  
+- Quantization: produced with [llama.cpp](https://github.com/ggerganov/llama.cpp) by Georgi Gerganov et al.  
+- Technical report: [arXiv:2503.01743](https://arxiv.org/abs/2503.01743)
--- a/deploy/install-nuc.sh
+++ b/deploy/install-nuc.sh
@@ -0,0 +1,89 @@
+#!/usr/bin/env bash
+# =============================================================================
+# install-nuc.sh
+# Deploy Phi-4 Multimodal Q4_K_M to an Intel NUC (8 GB RAM, CPU-only)
+# and register it with Ollama for use with OpenClaw.
+#
+# Usage:
+#   On the NUC, run:
+#       bash install-nuc.sh [path/to/phi4-mm-Q4_K_M.gguf]
+#
+#   Or from the build machine, copy files first then run:
+#       scp phi4-mm-Q4_K_M.gguf Modelfile openclaw-config.json user@nuc-ip:~/phi4-deploy/
+#       ssh user@nuc-ip "cd ~/phi4-deploy && bash install-nuc.sh"
+# =============================================================================
+set -euo pipefail
+
+GGUF_DEFAULT="/home/$(whoami)/phi4-deploy/phi4-mm-Q4_K_M.gguf"
+GGUF_PATH="${1:-$GGUF_DEFAULT}"
+MODELFILE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+MODEL_NAME="phi4-mm-nuc"
+
+echo "============================================================"
+echo " Phi-4 Multimodal NUC Deployment"
+echo "============================================================"
+echo " GGUF    : $GGUF_PATH"
+echo " Model   : $MODEL_NAME"
+echo " Host    : $(hostname)"
+echo "============================================================"
+
+# ── 1. Verify GGUF exists ─────────────────────────────────────────────────────
+if [[ ! -f "$GGUF_PATH" ]]; then
+  echo "ERROR: GGUF not found at $GGUF_PATH"
+  echo "Transfer it first:  scp <build-host>:~/models/phi4-mm-Q4_K_M.gguf $GGUF_PATH"
+  exit 1
+fi
+
+# ── 2. Install Ollama if missing ──────────────────────────────────────────────
+if ! command -v ollama &>/dev/null; then
+  echo ""
+  echo ">>> Installing Ollama..."
+  curl -fsSL https://ollama.com/install.sh | sh
+  echo ">>> Ollama installed."
+else
+  echo ">>> Ollama already installed: $(ollama --version)"
+fi
+
+# ── 3. Ensure Ollama daemon is running ────────────────────────────────────────
+if ! pgrep -x ollama &>/dev/null; then
+  echo ">>> Starting Ollama daemon..."
+  ollama serve &>/dev/null &
+  sleep 3
+fi
+
+# ── 4. Patch Modelfile with absolute GGUF path ───────────────────────────────
+MODELFILE_TMP="$(mktemp)"
+sed "s|FROM ./phi4-mm-Q4_K_M.gguf|FROM ${GGUF_PATH}|" \
+  "${MODELFILE_DIR}/Modelfile" > "$MODELFILE_TMP"
+
+# ── 5. Register model with Ollama ─────────────────────────────────────────────
+echo ""
+echo ">>> Creating Ollama model: $MODEL_NAME"
+ollama create "$MODEL_NAME" -f "$MODELFILE_TMP"
+rm "$MODELFILE_TMP"
+echo ">>> Model registered."
+
+# ── 6. Smoke-test ─────────────────────────────────────────────────────────────
+echo ""
+echo ">>> Running smoke test (one token)..."
+RESPONSE=$(ollama run "$MODEL_NAME" "Reply with exactly: OK" --nowordwrap 2>&1 | head -5)
+echo "    Response: $RESPONSE"
+
+# ── 7. Systemd user service for Ollama (optional) ─────────────────────────────
+if command -v systemctl &>/dev/null && systemctl --user status ollama.service &>/dev/null 2>&1; then
+  echo ">>> Ollama systemd service already configured."
+else
+  echo ""
+  echo ">>> To keep Ollama running at startup, enable the service:"
+  echo "    sudo systemctl enable --now ollama"
+fi
+
+# ── Done ──────────────────────────────────────────────────────────────────────
+echo ""
+echo "============================================================"
+echo " DONE. Model is ready."
+echo ""
+echo " Test:           ollama run $MODEL_NAME"
+echo " API endpoint:   http://localhost:11434/v1"
+echo " OpenClaw model: ollama/$MODEL_NAME"
+echo "============================================================"
--- a/deploy/openclaw-config.json
+++ b/deploy/openclaw-config.json
@@ -0,0 +1,12 @@
+{
+  "agent": {
+    "model": "ollama/phi4-mm-nuc"
+  },
+  "modelConfigs": {
+    "ollama/phi4-mm-nuc": {
+      "provider": "ollama",
+      "model": "phi4-mm-nuc",
+      "baseUrl": "http://localhost:11434"
+    }
+  }
+}