← Back to Home

Apertus: Democratizing Open and Compliant LLMs for Global Language Environments

Alejandro Hernández-Cano, ... Ilia Badanin ... Imanol Schlag [Show all]
arXiv preprint arXiv:2509.14233, 2025
arXiv PDF
Apertus Project Overview

Abstract

The paper introduces Apertus, the biggest European open-source language model, a comprehensive suite addressing two key gaps in the current ecosystem: data compliance and multilingual support.

The models are trained exclusively on openly available data with retroactively respecting robots.txt exclusions and filtering for non-permissive, toxic, and personally identifiable content. To reduce memorization risks, the team employs the Goldfish objective during pretraining.

The models expand language representation by training on 15 trillion tokens across over 1,800 languages, dedicating roughly 40% of pretraining data to non-English content. Available in 8B and 70B parameter sizes, Apertus achieves competitive results among fully open models on multilingual benchmarks.

Beyond model weights, the team releases all scientific artifacts from our development cycle with a permissive license, including data preparation scripts, checkpoints, evaluation suites, and training code, enabling transparent audit and extension. This approach emphasizes reproducibility, ethical data practices, and broad linguistic accessibility in open-source language model development.

BibTeX

@misc{apertus2025apertusdemocratizingopencompliant, title={Apertus: Democratizing Open and Compliant LLMs for Global Language Environments}, author={Project Apertus and Alejandro Hernández-Cano and Alexander Hägele and Allen Hao Huang and Angelika Romanou and Antoni-Joan Solergibert and Barna Pasztor and Bettina Messmer and Dhia Garbaya and Eduard Frank Ďurech and Ido Hakimi and Juan García Giraldo and Mete Ismayilzada and Negar Foroutan and Skander Moalla and Tiancheng Chen and Vinko Sabolčec and Yixuan Xu and Michael Aerni and Badr AlKhamissi and Inés Altemir Mariñas and Mohammad Hossein Amani and Matin Ansaripour and Ilia Badanin and Harold Benoit and Emanuela Boros and Nicholas Browning and Fabian Bösch and Maximilian Böther and Niklas Canova and Camille Challier and Clement Charmillot and Jonathan Coles and Jan Deriu and Arnout Devos and Lukas Drescher and Daniil Dzenhaliou and Maud Ehrmann and Dongyang Fan and Simin Fan and Silin Gao and Miguel Gila and María Grandury and Diba Hashemi and Alexander Hoyle and Jiaming Jiang and Mark Klein and Andrei Kucharavy and Anastasiia Kucherenko and Frederike Lübeck and Roman Machacek and Theofilos Manitaras and Andreas Marfurt and Kyle Matoba and Simon Matrenok and Henrique Mendonça and Fawzi Roberto Mohamed and Syrielle Montariol and Luca Mouchel and Sven Najem-Meyer and Jingwei Ni and Gennaro Oliva and Matteo Pagliardini and Elia Palme and Andrei Panferov and Léo Paoletti and Marco Passerini and Ivan Pavlov and Auguste Poiroux and Kaustubh Ponkshe and Nathan Ranchin and Javi Rando and Mathieu Sauser and Jakhongir Saydaliev and Muhammad Ali Sayfiddinov and Marian Schneider and Stefano Schuppli and Marco Scialanga and Andrei Semenov and Kumar Shridhar and Raghav Singhal and Anna Sotnikova and Alexander Sternfeld and Ayush Kumar Tarun and Paul Teiletche and Jannis Vamvas and Xiaozhe Yao and Hao Zhao and Alexander Ilic and Ana Klimovic and Andreas Krause and Caglar Gulcehre and David Rosenthal and Elliott Ash and Florian Tramèr and Joost VandeVondele and Livio Veraldi and Martin Rajman and Thomas Schulthess and Torsten Hoefler and Antoine Bosselut and Martin Jaggi and Imanol Schlag}, year={2025}, eprint={2509.14233}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2509.14233}, }