multilingual_model/EVAL_REPORT.md

# Automated MNLP evaluation report

- **Model repo:** [`cs-552-2026-4neurons/multilingual_model`](https://huggingface.co/cs-552-2026-4neurons/multilingual_model)
- **Owner(s):** group **4neurons**
- **Generated at:** 2026-06-09T08:56:40+00:00 (UTC)
- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)

_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._

## Evaluated checkpoint

- **Commit:** [`c1b9ac8`](https://huggingface.co/cs-552-2026-4neurons/multilingual_model/commit/c1b9ac861378e189045611415a6259003b80494f)
- **Message:** Upload config
- **Committed:** 2026-05-29T08:04:02+00:00

## Summary

| Benchmark | Accuracy | Status |
|---|---:|---|
| Math | — | not run |
| Knowledge | — | not run |
| Multilingual | 0.6000 | ok |
| Safety | — | not run |

## Sample completions

_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._

### Multilingual

**Correct** (1 shown)

- **reference**: `D`
- **overall** (1/1 completions correct)
- **extracted** (✓): `D`
- **completion**:

    ```text
    <think>
    Okay, let's see. The question is in Hindi, and it's about a type of deception in marketing where a product or service is presented as something else. The options are A to D.
    
    First, I need to recall the different types of deceptive practices in marketing. The question is asking when a product or service is presented with another complementary product or service. 
    
    Option D is "शुद्न मूल्य
    
    ... [truncated 403 chars] ...
    
    is presented as part of a bundle with another product, that's more about price bundling where both are sold together. But sometimes, "bundling" can refer to selling multiple products together. However, the options given are in Hindi, so maybe the correct term is "price bundling" (D) where the product is sold with another, making it seem like a package deal. 
    
    Alternatively, "price discrimination" (B) is when different prices are charged for the same product or service. "Purely price discrimination" (B) is a type where the seller doesn't consider the buyer's willingness to pay. 
    
    "Purely price bundling" (D) is when a product is sold with another product at a higher price than the sum. So if a product is presented as part of a bundle with another, that's bundling. But the question is about presenting a product with another complementary product. So the answer would be D. 
    
    Wait, but sometimes "bundling" can be a form of deception if the customer isn't getting the additional product they paid for. So the correct answer here is D, which is "purely price bundling," where the product is sold as a bundle with another, leading to deception. 
    
    So the answer should be D.
    </think>
    
    \boxed{D}
    ```

**Incorrect** (1 shown)

- **reference**: `A`
- **overall** (0/1 completions correct)
- **extracted** (✗): `B`
- **completion**:

    ```text
    <think>
    Okay, let's see. The question is in Russian, and it's asking about the term for a set of three formal objects. The options are A to D.
    
    First, I need to recall what "тройка формальных объектов" means. "Формальные объекты" translates to "formal objects," so the question is about three formal objects. The term here is likely a specific term in a particular field. Let's look at the options.
    
    ... [truncated 823 chars] ...
    
    he options:
    
    - **A. Квалиметрической шкалой** refers to a *qualimetric scale*, which is a type of measurement system that combines both qualitative and quantitative data. While it involves multiple types of data, it is not specifically defined as a "set of three formal objects."
    
    - **B. Шкалированием** translates to **"scaling"**, which is a general term for assigning numerical values to data. While scaling can involve multiple data points, it is not a precise term for a *set of three formal objects*.
    
    - **C. Ранжирование** means **"ranging"**, which typically refers to measuring or comparing distances or quantities in a linear context. It is not a formal term for a group of three objects.
    
    - **D. Размерность** translates to **"dimensionality"**, which refers to the number of dimensions in a dataset or a system. While it can imply the number of objects or variables, it is not a term that refers to a *set of three formal objects*.
    
    ---
    
    Given that none of the options perfectly match the phrasing of the question, and considering that **"шкалированием"** (option B) is the most likely interpretation of the phrase in a technical or formal context, the correct answer is:
    
    $$
    \boxed{B}
    $$
    ```