---
library_name: transformers
tags: []
---

# qwen2.5-coder-7b-pandas-dpo-aligned

<!-- Provide a quick summary of what the model is/does. -->

## Introduction

Fine-tuned version of Qwen's Code-Specific large language model, Qwen2.5-coder-7b. Qwen-2.5 Coder has six mainstream model sizes: 0.5, 1.5, 3, 7, 14 and 32 billion 
parameter models to meet developer needs. The 7B model version provides a great balance between model representationl capacity and runtime requirements. This 7 Billion parameter
model was fine-tuned using offline DPO and a preference dataset for Python data analysis.

Available in GGUF format.
- Q4_K_M recommended for systems with at least 8GB of RAM. Model file size: 4.68GB

**Context Window:** up to 128K tokens
  
### Model Description

<!-- Provide a longer summary of what this model is. -->




- **License:** Apache 2.0
- **Finetuned from model:** [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)




## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Provides a good foundation for real-world applications such as Coding Agents. It has enhanced coding capabilities and
maintains strengths in mathematics and general competencies.



## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->




### Compute Infrastructure

The memory you need depends heavily on precision/quantization and context length. *Always consider headroom for runtime and cache!*
KV/Key-Value cache (scales with context length and batch size).

Considering standard Memory (bytes)≈params×bytes per parameter rule of thumb:

**Example 1 (CPU):** Running on CPU (llama.cpp / GGUF):

A 7B model at 4‑bit needs about 3.5 – 4GB (7×10^9x0.5) for weights. With runtime overhead, KV cache, and your app, a practical minimum is:

- 16 GB RAM (barely)
- 32 GB RAM recommended for smoother operation

**Example 2 (GPU):** For local GPU inference with decent context and speed:

- Aim for 10–12 GB VRAM (7×10^9×0.5) for a 4‑bit Qwen2.5‑Coder‑7B.
- Aim for 16 GB VRAM (7×10^9×2) if you want FP16 or very long contexts.


#### Hardware

No GPU offload:

**Absolute Minimum:** 4 vCPUs (4 cores / threads) to run 7B 4‑bit model, but generation will likely feel sluggish, especially with longer responses.

**Realistic Minimum:** For heavier coding uses with multiple users and long context: 8–12 vCPUs, allows:

- 8–10 threads for the model.
- Remaining cores for the app and OS.

With GPU offload performing all the matrix multiplication 4 vCPUs are sufficient.


## More Information [optional]

[More Information Needed]