llm-traffic-controller/README.md

tags, widget, metrics, pipeline_tag, library_name, inference, base_model

tags	widget	metrics	pipeline_tag	library_name	inference	base_model
setfit sentence-transformers text-classification generated_from_setfit_trainer	text What are the deadlines and deliverables listed in this project plan summary? text Quickly, just give me the dates and locations mentioned in this travel itinerary. text Convert this list of configuration parameters into a JSON object. Keys are parameter names, values are their settings. text My GitLab CI/CD pipeline fails at `npm install`. The error log is `[log snippet]`. What's wrong? text Three friends, Alice, Bob, and Carol, each have a favorite color: red, blue, or green. Alice doesn't like red. Bob doesn't like green. The person who likes blue is not Carol. Who likes which color?	accuracy	text-classification	setfit	true	sentence-transformers/all-MiniLM-L6-v2

SetFit with sentence-transformers/all-MiniLM-L6-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-MiniLM-L6-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/all-MiniLM-L6-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 256 tokens
• Number of Classes: 4 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
simple_chat	'What precisely is your nature?' 'What is your primary function?' 'Good morning.'
extraction	'Summarize the main benefits of this service based on the provided marketing copy.' "Can you just summarize the key findings from this research data list? I don't need all the numbers." 'Convert this list of configuration parameters into a JSON object. Keys are parameter names, values are their settings.'
reasoning	"What's the best way to pivot our struggling brick-and-mortar bookstore to survive in the digital age?" 'Develop a decision tree for purchasing a new company car, considering budget, fuel efficiency, maintenance costs, and resale value.' "What's 15% of 250?"
coding	'Refactor this C++ legacy code to use std::unique_ptr and std::shared_ptr instead of raw pointers.' 'Refactor this spaghetti PHP script to separate business logic, presentation, and data access layers.' "What's the fundamental difference between SQL and NoSQL databases, and when should I use each?"

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("What are the deadlines and deliverables listed in this project plan summary?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	13.5101	41

Label	Training Sample Count
simple_chat	48
extraction	50
reasoning	50
coding	50

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 10
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: True
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• evaluation_strategy: no
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0040	1	0.5538	-
0.2016	50	0.2712	-
0.4032	100	0.1337	-
0.6048	150	0.0604	-
0.8065	200	0.0284	-

Framework Versions

• Python: 3.9.6
• SetFit: 1.1.3
• Sentence Transformers: 5.1.2
• Transformers: 4.57.6
• PyTorch: 2.8.0
• Datasets: 4.5.0
• Tokenizers: 0.22.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}