1 line
701 B
Plaintext
1 line
701 B
Plaintext
Atlas-Flash (Preview) is a fine-tune of the Deepseek's R1 Qwen 2.5 Distills. Since the model that Deepseek used was Qwen 2.5 Math and Qwen themselves have advised to not use this for any other tasks, I have decided to use a variety of datasets to help make this model as close as possible to a regular one. To do this I have used high quality datasets (BAAI/TACO, codeparrot/apps, rubenroy/GammaCorpus-v1-70k-UNFILTERED and hand-collected synthetic data). The hand-collected synthetic data was generated by models such as Gemini-2.0, GPT-4o and Claude 3.5 Sonnet. Using Unsloth as an efficient fine-tuning framework and using Lambda Labs' 1x H100 (80 GB SXM5), Atlas-Flash 1215 went under fine-tuning. |