sqft-phi-3.5-mini-instruct-wikitext2-awq-64g-ppl10.41/README.md

language, license, library_name

language	license	library_name
en	apache-2.0	transformers

Quantized Fine-tuned Model: sqft-phi-3.5-mini-instruct-wikitext2-awq-64g-ppl10.41

• Source Model: IntelLabs/sqft-phi-3.5-mini-instruct-wikitext2-ppl9.78
• Finetuning Method: NLS
• Adapter Version: Heuristic
• Quantization: AWQ-INT4 (group size: 64)

Evaluation

CUDA_VISIBLE_DEVICES=$DEVICES lm_eval --model hf --model_args pretrained=IntelLabs/sqft-phi-3.5-mini-instruct-wikitext2-awq-64g-ppl10.41,max_length=4096 --tasks wikitext --batch_size auto:4 --output_path result.json

License

Apache-2.0