pipeline_tag, inference, license, library_name, model-index

pipeline_tag	inference	license	library_name	model-index
text-generation	false	apache-2.0	transformers	name results ibm/PowerLM-3b task dataset metrics type text-generation type name lm-eval-harness ARC name type value verified accuracy-norm accuracy-norm 60.5 false task dataset metrics type text-generation type name lm-eval-harness BoolQ name type value verified accuracy accuracy 72.0 false task dataset metrics type text-generation type name lm-eval-harness Hellaswag name type value verified accuracy-norm accuracy-norm 74.6 false task dataset metrics type text-generation type name lm-eval-harness OpenBookQA name type value verified accuracy-norm accuracy-norm 43.6 false task dataset metrics type text-generation type name lm-eval-harness PIQA name type value verified accuracy-norm accuracy-norm 79.9 false task dataset metrics type text-generation type name lm-eval-harness Winogrande name type value verified accuracy-norm accuracy-norm 70.0 false task dataset metrics type text-generation type name lm-eval-harness MMLU (5 shot) name type value verified accuracy accuracy 49.2 false task dataset metrics type text-generation type name lm-eval-harness GSM8k (5 shot) name type value verified accuracy accuracy 34.9 false task dataset metrics type text-generation type name lm-eval-harness math (4 shot) name type value verified accuracy accuracy 15.2 false task dataset metrics type text-generation type name bigcode-eval humaneval name type value verified pass@1 pass@1 26.8 false task dataset metrics type text-generation type name bigcode-eval MBPP name type value verified pass@1 pass@1 33.6 false

Model Summary

PowerLM-3B is a 3B state-of-the-art small language model trained with the Power learning rate scheduler. It is trained on a mix of open-source and proprietary datasets. PowerLM-3B has shown promising results compared to other models in the size categories across various benchmarks, including natural language multi-choices, code generation, and math reasoning. Paper: https://arxiv.org/abs/2408.13359

Usage

Note: Requires installing HF transformers from source.

Generation

This is a simple example of how to use PowerLM-3b model.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # or "cpu"
model_path = "ibm/PowerLM-3b"
tokenizer = AutoTokenizer.from_pretrained(model_path)
# drop device_map if running on CPU
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()
# change input text as desired
prompt = "Write a code to find the maximum value in a list of numbers."
# tokenize the text
input_tokens = tokenizer(prompt, return_tensors="pt")
# transfer tokenized inputs to the device
for i in input_tokens:
    input_tokens[i] = input_tokens[i].to(device)
# generate output tokens
output = model.generate(**input_tokens, max_new_tokens=100)
# decode output tokens into text
output = tokenizer.batch_decode(output)
# loop over the batch to print, in this example the batch size is 1
for i in output:
    print(i)