Can Continual Pretraining Bridge the Performance Gap between General-purpose and Specialized Language Models in the Medical Domain?

Jasper Schulze Buschhoff Niclas Doll Jasper Schulze Buschhoff Shalaka Satheesh Hammam Abdelwahab Héctor Allende-Cid Katrin Klug Can Continual Pretraining Bridge the Performance Gap between General-purpose and Specialized Language Models in the Medical Domain? 427-444 2026 ACL (1) https://aclanthology.org/2026.acl-long.17/ conf/acl/2026-1 db/conf/acl/acl2026-1.html#DollBSAAK26

Niclas Doll Jasper Schulze Buschhoff Shalaka Satheesh Hammam Abdelwahab Héctor Allende-Cid Katrin Klug Can Continual Pre-training Bridge the Performance Gap between General-purpose and Specialized Language Models in the Medical Domain? 2026 April abs/2604.19394 CoRR https://doi.org/10.48550/arXiv.2604.19394 db/journals/corr/corr2604.html#abs-2604-19394 streams/journals/corr

Mehdi Ali Michael Fromm 0001 Klaudia Thellmann Jan Ebert Alexander Arno Weber Richard Rutmann Charvi Jain Max Lübbering Daniel Steinigen Johannes Leveling Katrin Klug Jasper Schulze Buschhoff Lena Jurkschat Hammam Abdelwahab Benny Jörg Stein Karl-Heinz Sylla Pavel Denisov Nicolo' Brandizzi Qasid Saleem Anirban Bhowmick Lennard Helmer Chelsea Maria John Pedro Ortiz Suarez Malte Ostendorff Alex Jude Lalith Manjunath Samuel Weinbach Carolin Penke Oleg Filatov Fabio Barth Paramita Mirza Lucas Weber Ines Wendler Rafet Sifa Fabian Küch Andreas Herten René Jäkel Georg Rehm Stefan Kesselheim Joachim Köhler Nicolas Flores-Herr Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs. 4321-4329 2025 ECAI https://doi.org/10.3233/FAIA251328 conf/ecai/2025 db/conf/ecai/ecai2025.html#Ali0TEWRJLSLKBJ25 Mehdi Ali Michael Fromm 0001 Klaudia Thellmann Richard Rutmann Max Lübbering Johannes Leveling Katrin Klug Jan Ebert Niclas Doll Jasper Schulze Buschhoff Charvi Jain Alexander Arno Weber Lena Jurkschat Hammam Abdelwahab Chelsea Maria John Pedro Ortiz Suarez Malte Ostendorff Samuel Weinbach Rafet Sifa Stefan Kesselheim Nicolas Flores-Herr Tokenizer Choice For LLM Training: Negligible or Crucial? 3907-3924 2024 NAACL-HLT (Findings) https://doi.org/10.18653/v1/2024.findings-naacl.247 conf/naacl/2024f db/conf/naacl/naacl2024f.html#AliFTRLLKEDBJWJAJSOWSKF24 streams/conf/naacl

Klaudia Thellmann Bernhard Stadler 0001 Michael Fromm 0001 Jasper Schulze Buschhoff Alex Jude Fabio Barth Johannes Leveling Nicolas Flores-Herr Joachim Köhler René Jäkel Mehdi Ali Towards Multilingual LLM Evaluation for European Languages. 2024 abs/2410.08928 CoRR https://doi.org/10.48550/arXiv.2410.08928 db/journals/corr/corr2410.html#abs-2410-08928 streams/journals/corr

Mehdi Ali Michael Fromm 0001 Klaudia Thellmann Richard Rutmann Max Lübbering Johannes Leveling Katrin Klug Jan Ebert Niclas Doll Jasper Schulze Buschhoff Charvi Jain Alexander Arno Weber Lena Jurkschat Hammam Abdelwahab Chelsea Maria John Pedro Ortiz Suarez Malte Ostendorff Samuel Weinbach Rafet Sifa Stefan Kesselheim Nicolas Flores-Herr Tokenizer Choice For LLM Training: Negligible or Crucial? 2023 abs/2310.08754 CoRR https://doi.org/10.48550/arXiv.2310.08754 db/journals/corr/corr2310.html#abs-2310-08754

Hammam Abdelwahab Mehdi Ali Héctor Allende-Cid Shima Asaadi Fabio Barth Anirban Bhowmick Nicolo' Brandizzi Pavel Denisov Niclas Doll Jan Ebert Oleg Filatov Nicolas Flores-Herr Michael Fromm 0001 Lennard Helmer Andreas Herten Charvi Jain René Jäkel Chelsea Maria John Alex Jude Lena Jurkschat Stefan Kesselheim Katrin Klug Joachim Köhler Fabian Küch Johannes Leveling Max Lübbering Lalith Manjunath Paramita Mirza Malte Ostendorff Carolin Penke Georg Rehm Richard Rutmann Qasid Saleem Shalaka Satheesh Rafet Sifa Bernhard Stadler 0001 Benny Jörg Stein Daniel Steinigen Pedro Ortiz Suarez Karl-Heinz Sylla Klaudia Thellmann Alexander Arno Weber Lucas Weber Samuel Weinbach Ines Wendler