21 lines
949 B
Markdown
21 lines
949 B
Markdown
|
---
|
||
|
|
base_model: prashanthsura/gemma-2-2b-legal-financial-sft
|
||
|
|
library_name: transformers
|
||
|
|
---
|
||
|
|
|
||
|
|
# Gemma-2-2B legal/financial SFT -- RQ-compressed embedding
|
||
|
|
|
||
|
|
Same weights as [prashanthsura/gemma-2-2b-legal-financial-sft](https://huggingface.co/prashanthsura/gemma-2-2b-legal-financial-sft) except the tied
|
||
|
|
token-embedding matrix (256000 x 2304, ~590M params, ~22.6% of the model) has
|
||
|
|
been replaced by its 2-stage Residual Product Quantization reconstruction
|
||
|
|
(M=384 subspaces, K=1024 codewords per stage, L=2 stages ->
|
||
|
|
**3.59 bits/param** for that matrix vs 16 in bf16). Because Gemma ties
|
||
|
|
its output head to the embedding, one codebook covers both roles ("unified
|
||
|
|
codebook").
|
||
|
|
|
||
|
|
The main model files store the *reconstructed* embedding for drop-in
|
||
|
|
`from_pretrained` loading. The genuinely compressed artifact (int16 codes +
|
||
|
|
fp32 codebooks) is in `rq_M384_K1024_artifact.pt`.
|
||
|
|
|
||
|
|
Demo + head-to-head evaluation: https://prashanthsura-gemma-2-2b-rq.vercel.app
|