71 lines
3.8 KiB
Markdown
71 lines
3.8 KiB
Markdown
# ModelHub GPU memory boundaries — 2026-08-10
|
||
|
||
## Scope and method
|
||
|
||
The live ModelHub machine-info endpoint returned 18 GPU types on 2026-08-10.
|
||
Four were disabled (`canVerify=false`): `Mthreads_s5000`,
|
||
`Kunlunxin_r-200-8f`, `Cambricon_mlu-590`, and `Ascend_950`. They are not current
|
||
submission targets. The remaining 14 are covered below.
|
||
|
||
ModelHub `PREFLIGHT_OOM` logs reveal the platform's actual rule:
|
||
|
||
```text
|
||
required memory = complete recursive repository size × 1.20
|
||
maximum repository size = allocated GPU memory ÷ 1.20
|
||
```
|
||
|
||
This was cross-checked against `Kwaipilot/KAT-Coder-V2.5-Dev`: the recursive
|
||
ModelScope tree was 64.5916 GiB and ModelHub reported 64.6 GiB on disk, then
|
||
required 77.5 GiB after the 20% multiplier. Consequently, parameter count or a
|
||
single selected weight format is not a valid substitute.
|
||
|
||
## Current boundaries
|
||
|
||
| ModelHub GPU type | Allocated memory | Maximum complete repository | Primary evidence |
|
||
| --- | ---: | ---: | --- |
|
||
| `Cambricon_mlu-370-x4` | 24 GiB | 20.000 GiB | 49 ModelHub OOM logs; manufacturer says 24 GB |
|
||
| `Ascend_910-b4` | 32 GiB | 26.667 GiB | 8 ModelHub OOM logs |
|
||
| `Iluvatar_bi-100` | 32 GiB | 26.667 GiB | 48 ModelHub OOM logs |
|
||
| `Iluvatar_bi-150` | 32 GiB | 26.667 GiB | 24 ModelHub OOM logs |
|
||
| `Iluvatar_mrv-100` | 32 GiB | 26.667 GiB | 50 ModelHub OOM logs |
|
||
| `Vastai_va16` | 32 GiB | 26.667 GiB | 40 ModelHub OOM logs |
|
||
| `Cambricon_mlu-370-x8` | 48 GiB | 40.000 GiB | manufacturer specification |
|
||
| `Mthreads_s4000` | 48 GiB | 40.000 GiB | manufacturer specification |
|
||
| `Ascend_910-b3` | 64 GiB | 53.333 GiB | published deployment specification |
|
||
| `Biren_166m` | 64 GiB | 53.333 GiB | manufacturer publication |
|
||
| `hygon_k100-ai` | 64 GiB | 53.333 GiB | 15 ModelHub OOM logs |
|
||
| `MetaX_c-500` | 64 GiB | 53.333 GiB | 29 ModelHub OOM logs; manufacturer says 64 GB |
|
||
| `Sunrise_pt-200-x1` | 64 GiB | 53.333 GiB | 36 ModelHub OOM logs |
|
||
| `Kunlunxin_p-800` | 96 GiB | 80.000 GiB | public procurement specification |
|
||
|
||
Published sources:
|
||
|
||
- ModelHub machine inventory: <https://modelhub.org.cn/api/computility/power/machine/list/machine-info>
|
||
- Cambricon MLU370-X4 (24 GB): <https://cambricon.com/index.php?a=lists&c=index&catid=371&m=content>
|
||
- Cambricon MLU370-X8 (48 GB): <https://cambricon.com/index.php?a=lists&c=index&catid=406&m=content>
|
||
- Iluvatar TianGai/ZhiKai series (32 GB): <https://www.iluvatar.com/serias?fullCode=cpjs-yj-xlxl>
|
||
- MetaX C500 (64 GB): <https://www.metax-tech.com/prod.html?cid=107&id=21>
|
||
- MTT S4000 (48 GB): <https://docs.mthreads.com/s4000/s4000-doc-online/product_specifications/>
|
||
- Biren 166M (64 GB): <https://www.birentech.com/news/id6rz98v3obczy77cmxzfgk3/>
|
||
- Ascend 910B3/B4 deployment capacities (64/32 GB): <https://aclanthology.org/2025.emnlp-main.1630.pdf>
|
||
- Kunlunxin P800 procurement requirement (at least 96 GB per card): <https://pms2g.shrcb.com/cms/cmscaigougg/1d93b4b8166041e096c65d739073ade1.html>
|
||
|
||
## Runtime behavior
|
||
|
||
The code uses the evidence hierarchy `local ModelHub OOM > explicit environment
|
||
override > historical ModelHub OOM > published specification`. A newly
|
||
downloaded structured OOM log records the actual allocation for that ModelHub
|
||
GPU type and replaces the published fallback on subsequent planning cycles.
|
||
|
||
For a known GPU, missing even one file size causes
|
||
`preflight_model_size_unknown`; the candidate is deferred rather than estimated.
|
||
An exact boundary is accepted, while anything larger is rejected locally as
|
||
`preflight_predicted_oom` before consuming a platform queue slot.
|
||
If ModelHub later introduces another GPU, it is deferred as
|
||
`preflight_gpu_memory_unknown` until a platform observation, published capacity,
|
||
or explicit override supplies evidence.
|
||
|
||
This boundary reproduces ModelHub's repository preflight. It does not promise
|
||
that every model below the boundary will run: framework support, operators,
|
||
quantization, context length, and runtime KV-cache memory remain separate checks.
|