---
library_name: transformers
tags:
- axolotl
- edge
- bitnet
license: apache-2.0
---

# Falcon-E-1.2-3B-Exp-prequantized

This is the model card of `Falcon-E-1.2-3B-Exp`, a ternary (1.58bits) language model trained on SFT agentic, and STEM data using [`axolotl`](https://github.com/axolotl-ai-cloud/axolotl) framework combined with [`onebitllm`](https://github.com/tiiuae/onebitllms) library.

The model has been trained starting from [`tiiuae/Falcon-E-3B-Base-prequantized`](https://huggingface.co/tiiuae/Falcon-E-3B-Base-prequantized) checkpoint using full-finetuning for 3 epochs. Below are the hyper-parameters used for fine-tuning:

This model is the prequantized version of [`axolotl-ai-co/Falcon-E-1.2-3B-Exp`](https://huggingface.co/axolotl-ai-co/Falcon-E-1.2-3B-Exp) that can be used for further fine-tuning.

```yaml
micro_batch_size: 1
num_epochs: 3
optimizer: adamw_torch
lr_scheduler: cosine
learning_rate: 8.0e-4
# adamw hyperparams
adam_beta1: 0.9
adam_beta2: 0.95
warmup_steps: 128
```

And we used a context parallel size of `8`.

## Usage

The model uses `think` mode by default, this can be disabled and switched to non-thiking mode. You can use the model with different frameworks such as HF `transformers`, `llama.cpp` or `mlx-lm`

### transformers 

```bash
transformers chat axolotl-ai-co/Falcon-E-1.2-3B-Exp
```

### llama.cpp

```bash
# thinking mode
llama-cli -m axolotl-ai-co/Falcon-E-1.2-3B-Exp-GGUF:TQ2_0 --reasoning-format auto --temp 0.2 -cnv

# non thinking mode
llama-cli -m axolotl-ai-co/Falcon-E-1.2-3B-Exp-GGUF:TQ2_0 --reasoning-format auto --temp 0.2 -cnv --reasoning-budget 0.0
```

### mlx-lm

```bash
mlx_lm.chat axolotl-ai-co/Falcon-E-1.2-3B-Exp --temperature 0.2
```

## Further fine-tuning the model

You can further fine-tune this model, or the base model using their prequantized version. Refer to the [`axolotl` config](https://github.com/axolotl-ai-cloud/axolotl-ai-co/tree/main/examples/falcon-e/falcon-e-3b-ft.yaml) to get started on fine-tuning these models:

- [Base prequantized model](https://huggingface.co/tiiuae/Falcon-E-3B-Base-prequantized)
- [`Falcon-E-1.2-3B-Exp` base model](https://huggingface.co/axolotl-ai/Falcon-E-1.2-3B-Exp-prequantized)
- [`DPO fine-tuning config`](https://github.com/axolotl-ai-cloud/axolotl-ai-co/tree/main/examples/falcon-e/falcon-e-3b-dpo.yaml)

## Aknowledgement

Falcon-E-3B-Chat-Exp models are built using Falcon LLM technology from the [Technology Innovation Institute](https://huggingface.co/tiiuae).