初始化项目,由ModelHub XC社区提供模型
Model: Swicked86/phi4-mm-gguf Source: Original Platform
This commit is contained in:
36
deploy/Modelfile
Normal file
36
deploy/Modelfile
Normal file
@@ -0,0 +1,36 @@
|
||||
# Ollama Modelfile — Phi-4 Multimodal Instruct Q4_K_M
|
||||
# Optimised for: Intel 11th Gen NUC, 8 GB RAM, CPU-only
|
||||
#
|
||||
# Source model : microsoft/Phi-4-multimodal-instruct
|
||||
# License : MIT https://huggingface.co/microsoft/Phi-4-multimodal-instruct/blob/main/LICENSE
|
||||
# Quantization : Q4_K_M via llama.cpp llama-quantize
|
||||
# Architecture : phi3 (3.8B LLM backbone + vision/speech adapters in base GGUF)
|
||||
|
||||
FROM ./phi4-mm-Q4_K_M.gguf
|
||||
|
||||
# ── Context & KV cache ───────────────────────────────────────────────────────
|
||||
# 8 192 tokens balances capability vs RAM on 8 GB hardware.
|
||||
# Lower to 4096 if you observe OOM / heavy swapping.
|
||||
PARAMETER num_ctx 8192
|
||||
|
||||
# ── CPU tuning ───────────────────────────────────────────────────────────────
|
||||
# 11th Gen NUC typically 4 cores / 8 logical threads (i5/i7-1135G7 / 1165G7).
|
||||
# Reduce to 4 if the NUC is a Core i3 variant.
|
||||
PARAMETER num_thread 8
|
||||
|
||||
# No discrete GPU — all layers run on CPU.
|
||||
PARAMETER num_gpu 0
|
||||
|
||||
# Flash attention is a GPU feature; disable for CPU inference.
|
||||
PARAMETER flash_attn false
|
||||
|
||||
# ── Generation defaults ───────────────────────────────────────────────────────
|
||||
PARAMETER temperature 0.7
|
||||
PARAMETER top_p 0.9
|
||||
PARAMETER repeat_penalty 1.1
|
||||
PARAMETER stop "<|end|>"
|
||||
PARAMETER stop "<|user|>"
|
||||
PARAMETER stop "<|assistant|>"
|
||||
|
||||
# ── System prompt ─────────────────────────────────────────────────────────────
|
||||
SYSTEM """You are a helpful, accurate, and concise AI assistant. You excel at reasoning, analysis, writing, coding, and answering questions. Be direct and thorough."""
|
||||
Reference in New Issue
Block a user