commit 9a52d1b74b076933020ad0f08cd77b1781af6f27 Author: ModelHub XC Date: Fri Aug 28 00:21:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..4d5fb8a --- /dev/null +++ b/.gitattributes @@ -0,0 +1,57 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q3_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q4_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q5_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Llama-3-8B-dutch.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Llama-3-8B-dutch.IQ3_M.gguf b/Llama-3-8B-dutch.IQ3_M.gguf new file mode 100644 index 0000000..ab4d1c5 --- /dev/null +++ b/Llama-3-8B-dutch.IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e58d328bb95e88ad1a688da3704996aa193126cc05d34d310fe19b46554bb6e7 +size 3784833728 diff --git a/Llama-3-8B-dutch.IQ3_S.gguf b/Llama-3-8B-dutch.IQ3_S.gguf new file mode 100644 index 0000000..ce18fd5 --- /dev/null +++ b/Llama-3-8B-dutch.IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b7cda973b392de1f430a7dd4376d9b0860b7a0c6edcb43f197b6e4eeacf2395 +size 3682335424 diff --git a/Llama-3-8B-dutch.IQ3_XS.gguf b/Llama-3-8B-dutch.IQ3_XS.gguf new file mode 100644 index 0000000..dd71494 --- /dev/null +++ b/Llama-3-8B-dutch.IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:12ee0061bcd07b33138832f0abff7170b57507b596f73feef7bb75ee22fd631d +size 3518757568 diff --git a/Llama-3-8B-dutch.IQ4_NL.gguf b/Llama-3-8B-dutch.IQ4_NL.gguf new file mode 100644 index 0000000..3cd1eac --- /dev/null +++ b/Llama-3-8B-dutch.IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7eb7c29740c7d59e16b507ac9b380f0c8ad70e7436b04818d315ffa5a75e9b89 +size 4707360512 diff --git a/Llama-3-8B-dutch.IQ4_XS.gguf b/Llama-3-8B-dutch.IQ4_XS.gguf new file mode 100644 index 0000000..d0bca52 --- /dev/null +++ b/Llama-3-8B-dutch.IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:175b56aaa269ef69ff6cc47a29c95394b8762eed23cbb2c6edec857307066a6b +size 4484374016 diff --git a/Llama-3-8B-dutch.Q2_K.gguf b/Llama-3-8B-dutch.Q2_K.gguf new file mode 100644 index 0000000..8789989 --- /dev/null +++ b/Llama-3-8B-dutch.Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9f5ba482adb21ee054faf283cc719d9fc7063226791589dbaf8cdab7a8df66db +size 3179140992 diff --git a/Llama-3-8B-dutch.Q3_K.gguf b/Llama-3-8B-dutch.Q3_K.gguf new file mode 100644 index 0000000..03db7a9 --- /dev/null +++ b/Llama-3-8B-dutch.Q3_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d66c476f92047bcb6f284a6405cd0b458e477cdcdc0cce4d8dbb6bb930ab39a9 +size 4018928320 diff --git a/Llama-3-8B-dutch.Q3_K_L.gguf b/Llama-3-8B-dutch.Q3_K_L.gguf new file mode 100644 index 0000000..69fdc6a --- /dev/null +++ b/Llama-3-8B-dutch.Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:178d55eb7283b397af63933667ba2374482056895db5d24c3c471b99708e4efa +size 4321966784 diff --git a/Llama-3-8B-dutch.Q3_K_M.gguf b/Llama-3-8B-dutch.Q3_K_M.gguf new file mode 100644 index 0000000..03db7a9 --- /dev/null +++ b/Llama-3-8B-dutch.Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d66c476f92047bcb6f284a6405cd0b458e477cdcdc0cce4d8dbb6bb930ab39a9 +size 4018928320 diff --git a/Llama-3-8B-dutch.Q3_K_S.gguf b/Llama-3-8B-dutch.Q3_K_S.gguf new file mode 100644 index 0000000..217a4c3 --- /dev/null +++ b/Llama-3-8B-dutch.Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:11288467461320a1bb3cf8799bcd75e0ff54cbecd40ff83ad2aeb175cc24b7bd +size 3664509632 diff --git a/Llama-3-8B-dutch.Q4_0.gguf b/Llama-3-8B-dutch.Q4_0.gguf new file mode 100644 index 0000000..13d1e3a --- /dev/null +++ b/Llama-3-8B-dutch.Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:51e10c43fbc9dbfcb8be1310933485d45ae521acc9b5fae9de7f6af18f173e1e +size 4661223168 diff --git a/Llama-3-8B-dutch.Q4_1.gguf b/Llama-3-8B-dutch.Q4_1.gguf new file mode 100644 index 0000000..558ceb1 --- /dev/null +++ b/Llama-3-8B-dutch.Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bded0100467775d4f6a56afb6fcae42a003967d534155a1d0d1ccdbd9a200725 +size 5130264832 diff --git a/Llama-3-8B-dutch.Q4_K.gguf b/Llama-3-8B-dutch.Q4_K.gguf new file mode 100644 index 0000000..ce35554 --- /dev/null +++ b/Llama-3-8B-dutch.Q4_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:beea14c58f5fe28c0726be84552f7b23383b28aed4d48ae0dfaf8b6f35df47b5 +size 4920745728 diff --git a/Llama-3-8B-dutch.Q4_K_M.gguf b/Llama-3-8B-dutch.Q4_K_M.gguf new file mode 100644 index 0000000..ce35554 --- /dev/null +++ b/Llama-3-8B-dutch.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:beea14c58f5fe28c0726be84552f7b23383b28aed4d48ae0dfaf8b6f35df47b5 +size 4920745728 diff --git a/Llama-3-8B-dutch.Q4_K_S.gguf b/Llama-3-8B-dutch.Q4_K_S.gguf new file mode 100644 index 0000000..4bab406 --- /dev/null +++ b/Llama-3-8B-dutch.Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb470cdf1ced53ffd3ba8e7aff2a410458ef6d53909f4a8901071d61a00dd538 +size 4692680448 diff --git a/Llama-3-8B-dutch.Q5_0.gguf b/Llama-3-8B-dutch.Q5_0.gguf new file mode 100644 index 0000000..309aafd --- /dev/null +++ b/Llama-3-8B-dutch.Q5_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4cea2d4182199b6376417a01d8fb59d9f6a3d671273150f701c0a91756a41283 +size 5599306496 diff --git a/Llama-3-8B-dutch.Q5_1.gguf b/Llama-3-8B-dutch.Q5_1.gguf new file mode 100644 index 0000000..ad151b4 --- /dev/null +++ b/Llama-3-8B-dutch.Q5_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed3f0ff14396b71b962e34eaf2e99b43e052a0107a7c7bc86d7dfa0b0ae226f2 +size 6068348160 diff --git a/Llama-3-8B-dutch.Q5_K.gguf b/Llama-3-8B-dutch.Q5_K.gguf new file mode 100644 index 0000000..0c924ae --- /dev/null +++ b/Llama-3-8B-dutch.Q5_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dfc6dadba3acb6802179fe24c5712600cbeaccc0d3bcda1f66fccc7e026f4f04 +size 5732999936 diff --git a/Llama-3-8B-dutch.Q5_K_M.gguf b/Llama-3-8B-dutch.Q5_K_M.gguf new file mode 100644 index 0000000..0c924ae --- /dev/null +++ b/Llama-3-8B-dutch.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dfc6dadba3acb6802179fe24c5712600cbeaccc0d3bcda1f66fccc7e026f4f04 +size 5732999936 diff --git a/Llama-3-8B-dutch.Q5_K_S.gguf b/Llama-3-8B-dutch.Q5_K_S.gguf new file mode 100644 index 0000000..968fc64 --- /dev/null +++ b/Llama-3-8B-dutch.Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4a111f0cd9f57b8c1c066b552de0eb5bda235c765d260a53148413c00fe10f1c +size 5599306496 diff --git a/Llama-3-8B-dutch.Q6_K.gguf b/Llama-3-8B-dutch.Q6_K.gguf new file mode 100644 index 0000000..10c317b --- /dev/null +++ b/Llama-3-8B-dutch.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1c82c59999e62206025866429902d0ef3e4cebd991f5404b93d75b1f44216d2 +size 6596020032 diff --git a/Llama-3-8B-dutch.Q8_0.gguf b/Llama-3-8B-dutch.Q8_0.gguf new file mode 100644 index 0000000..525b306 --- /dev/null +++ b/Llama-3-8B-dutch.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:34852d0f86102118238a1d281520e842fa912602489da57e5de92b3bc090e1d9 +size 8540788416 diff --git a/README.md b/README.md new file mode 100644 index 0000000..a564de8 --- /dev/null +++ b/README.md @@ -0,0 +1,114 @@ +Quantization made by Richard Erkhov. + +[Github](https://github.com/RichardErkhov) + +[Discord](https://discord.gg/pvy7H8DZMG) + +[Request more models](https://github.com/RichardErkhov/quant_request) + + +Llama-3-8B-dutch - GGUF +- Model creator: https://huggingface.co/ReBatch/ +- Original model: https://huggingface.co/ReBatch/Llama-3-8B-dutch/ + + +| Name | Quant method | Size | +| ---- | ---- | ---- | +| [Llama-3-8B-dutch.Q2_K.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q2_K.gguf) | Q2_K | 2.96GB | +| [Llama-3-8B-dutch.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.IQ3_XS.gguf) | IQ3_XS | 3.28GB | +| [Llama-3-8B-dutch.IQ3_S.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.IQ3_S.gguf) | IQ3_S | 3.43GB | +| [Llama-3-8B-dutch.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q3_K_S.gguf) | Q3_K_S | 3.41GB | +| [Llama-3-8B-dutch.IQ3_M.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.IQ3_M.gguf) | IQ3_M | 3.52GB | +| [Llama-3-8B-dutch.Q3_K.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q3_K.gguf) | Q3_K | 3.74GB | +| [Llama-3-8B-dutch.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q3_K_M.gguf) | Q3_K_M | 3.74GB | +| [Llama-3-8B-dutch.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q3_K_L.gguf) | Q3_K_L | 4.03GB | +| [Llama-3-8B-dutch.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.IQ4_XS.gguf) | IQ4_XS | 4.18GB | +| [Llama-3-8B-dutch.Q4_0.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q4_0.gguf) | Q4_0 | 4.34GB | +| [Llama-3-8B-dutch.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.IQ4_NL.gguf) | IQ4_NL | 4.38GB | +| [Llama-3-8B-dutch.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q4_K_S.gguf) | Q4_K_S | 4.37GB | +| [Llama-3-8B-dutch.Q4_K.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q4_K.gguf) | Q4_K | 4.58GB | +| [Llama-3-8B-dutch.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q4_K_M.gguf) | Q4_K_M | 4.58GB | +| [Llama-3-8B-dutch.Q4_1.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q4_1.gguf) | Q4_1 | 4.78GB | +| [Llama-3-8B-dutch.Q5_0.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q5_0.gguf) | Q5_0 | 5.21GB | +| [Llama-3-8B-dutch.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q5_K_S.gguf) | Q5_K_S | 5.21GB | +| [Llama-3-8B-dutch.Q5_K.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q5_K.gguf) | Q5_K | 5.34GB | +| [Llama-3-8B-dutch.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q5_K_M.gguf) | Q5_K_M | 5.34GB | +| [Llama-3-8B-dutch.Q5_1.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q5_1.gguf) | Q5_1 | 5.65GB | +| [Llama-3-8B-dutch.Q6_K.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q6_K.gguf) | Q6_K | 6.14GB | +| [Llama-3-8B-dutch.Q8_0.gguf](https://huggingface.co/RichardErkhov/ReBatch_-_Llama-3-8B-dutch-gguf/blob/main/Llama-3-8B-dutch.Q8_0.gguf) | Q8_0 | 7.95GB | + + + + +Original model description: +--- +license: llama3 +base_model: meta-llama/Meta-Llama-3-8B +tags: +- ORPO +- llama 3 8B +- conversational +datasets: +- BramVanroy/ultra_feedback_dutch +model-index: +- name: ReBatch/Llama-3-8B-dutch + results: [] +language: +- nl +pipeline_tag: text-generation +--- + +

+ Llama 3 dutch banner +

+ +
+

Llama 3 8B - Dutch

+A conversational model for Dutch, based on Llama 3 8B +

Try chatting with the model!

+
+ +This model is a [QLORA](https://huggingface.co/blog/4bit-transformers-bitsandbytes) and [ORPO](https://huggingface.co/docs/trl/main/en/orpo_trainer) fine-tuned version of [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on the synthetic feedback dataset [BramVanroy/ultra_feedback_dutch](https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch) + + +## Model description +This model is a Dutch chat model, originally developed from Llama 3 8B and further refined through a feedback dataset with [ORPO](https://huggingface.co/docs/trl/main/en/orpo_trainer) and trained on [BramVanroy/ultra_feedback_dutch](https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch) + + + +## Intended uses & limitations +Although the model has been aligned with gpt-4-turbo output, which has strong content filters, the model could still generate wrong, misleading, and potentially even offensive content. Use at your own risk. + + +## Training procedure + +The model was trained in bfloat16 with QLORA with flash attention 2 on one GPU - H100 80GB SXM5 for around 24 hours on RunPod. + +## Evaluation Results + +The model was evaluated using [scandeval](https://scandeval.com/dutch-nlg/) + +The model showed mixed results across different benchmarks; it exhibited slight improvements on some while experiencing a decrease in scores on others. This occurred despite being trained on only 200,000 samples for a single epoch. We are curious to see whether its performance could be enhanced by training with more data or additional epochs. + +| Model| conll_nl | dutch_social | scala_nl | squad_nl | wiki_lingua_nl | mmlu_nl | hellaswag_nl | +|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------: +meta-llama/Meta-Llama-3-8B-Instruct | 68.72 | 14.67 | 32.91 | 45.36 | 67.62 | 36.18 | 33.91 +ReBatch/Llama-3-8B-dutch | 58.85 | 11.14 | 15.58 | 59.96 | 64.51 | 36.27 | 28.34 +meta-llama/Meta-Llama-3-8B | 62.26 | 10.45| 30.3| 62.99| 65.17 | 36.38| 28.33 + +### Training hyperparameters + +The following hyperparameters were used during training: +- learning_rate: 8e-06 +- train_batch_size: 2 +- eval_batch_size: 2 +- num_devices: 1 +- gradient_accumulation_steps: 4 +- optimizer: paged_adamw_8bit +- lr_scheduler_type: linear +- warmup_steps: 10 +- num_epochs: 1.0 +- r: 16 +- lora_alpha: 32 +- lora_dropout: 0.05 +