56 lines
3.2 KiB
Markdown
56 lines
3.2 KiB
Markdown
---
|
|
license: apache-2.0
|
|
base_model:
|
|
- HuggingFaceTB/SmolLM2-135M-Instruct
|
|
language:
|
|
- en
|
|
pipeline_tag: text-generation
|
|
datasets:
|
|
- zenglhardt/convfill-dataset
|
|
---
|
|
# Model Card for SmolLM135M-IT-ConvFill
|
|
|
|
This model card implements a finetune of [HuggingFaceTB/SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) for the conversational infill task described in the [ConvFill paper](https://arxiv.org/abs/2511.07397).
|
|
|
|
## Model Details
|
|
|
|
This model should be used respecting the original license of the base model, HuggingFaceTB/SmolLM2-135M-Instruct. The dataset that was used to finetune this model can be found [here](https://huggingface.co/datasets/zenglhardt/convfill-dataset).
|
|
|
|
### Model Description
|
|
Deploying responsive, multi-turn conversational voice agents with large language models poses a critical challenge: cloud-based foundation models utilize reasoning, information retrieval, and tool use for high-value tasks, but introduce latency that disrupts natural conversation. In contrast, small models can respond quickly but lack capabilities needed in real-world tasks. We propose _conversational infill_, a task where a small, local model generates prompt, contextually appropriate dialogue and seamlessly incorporates delayed, external knowledge produced in parallel by a foundation model backend. This finetune trains HuggingFaceTB/SmolLM2-135M-Instruct to perform the conversational infill task.
|
|
|
|
- **Finetuned from model:** [HuggingFaceTB/SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct)
|
|
- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
|
|
|
|
### Model Sources [optional]
|
|
|
|
- **Repository:** https://github.com/vysri/conversational-infill
|
|
- **Paper:** [Thinking While Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents](https://arxiv.org/abs/2511.07397)
|
|
- **Demo:** TBD
|
|
|
|
### Direct Use
|
|
|
|
This model is intended to be used with the infrastructure in the [ConvFill repository](https://github.com/vysri/conversational-infill).
|
|
|
|
## Bias, Risks, and Limitations
|
|
|
|
This model is not explicitly tuned for guardrailed behavior. Please use with caution.
|
|
|
|
## How to Get Started with the Model
|
|
Use the code in the [ConvFill repository](https://github.com/vysri/conversational-infill) to get started with this model.
|
|
|
|
### Training Data
|
|
A link to the training data for this model can be found [here](https://huggingface.co/datasets/zenglhardt/convfill-dataset). The dataset generation procedure can be found [here](https://github.com/zenglhardt/convfill-dataset). Information on training procedures can be found in the [ConvFill paper](https://arxiv.org/abs/2511.07397). Training code and scripts can be found in the [ConvFill repository](https://github.com/vysri/conversational-infill).
|
|
|
|
## Citation
|
|
```
|
|
@misc{srinivas2026thinkingspeakinginferencetimeknowledge,
|
|
title={Thinking While Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents},
|
|
author={Vidya Srinivas and Zachary Englhardt and Shwetak Patel and Vikram Iyer},
|
|
year={2026},
|
|
eprint={2511.07397},
|
|
archivePrefix={arXiv},
|
|
primaryClass={cs.CL},
|
|
url={https://arxiv.org/abs/2511.07397},
|
|
}
|
|
``` |