Training Language Models to Self-Correct via Reinforcement Learning.

Cosmin Paduraru Aviral Kumar Vincent Zhuang Rishabh Agarwal Yi Su John D. Co-Reyes Avi Singh Kate Baumli Shariq Iqbal Colton Bishop Rebecca Roelofs Lei M. Zhang Kay McKinney Disha Shrivastava Cosmin Paduraru George Tucker Doina Precup Feryal M. P. Behbahani Aleksandra Faust Training Language Models to Self-Correct via Reinforcement Learning. 2025 ICLR https://openreview.net/forum?id=CjwERcAU7w conf/iclr/2025 db/conf/iclr/iclr2025.html#KumarZASCSBIBRZ25

Aviral Kumar Vincent Zhuang Rishabh Agarwal Yi Su John D. Co-Reyes Avi Singh Kate Baumli Shariq Iqbal Colton Bishop Rebecca Roelofs Lei M. Zhang Kay McKinney Disha Shrivastava Cosmin Paduraru George Tucker Doina Precup Feryal M. P. Behbahani Aleksandra Faust Training Language Models to Self-Correct via Reinforcement Learning. 2024 abs/2409.12917 CoRR https://doi.org/10.48550/arXiv.2409.12917 db/journals/corr/corr2409.html#abs-2409-12917 streams/journals/corr

Daniel J. Mankowitz Andrea Michi Anton Zhernov Marco Gelmi Marco Selvi Cosmin Paduraru Edouard Leurent Shariq Iqbal Jean-Baptiste Lespiau Alex Ahern Thomas Köppe Kevin Millikin Stephen Gaffney Sophie Elster Jackson Broshear Chris Gamble Kieran Milan Robert Tung Minjae Hwang A. Taylan Cemgil Mohammadamin Barekatain Yujia Li 0001 Amol Mandhane Thomas Hubert Julian Schrittwieser Demis Hassabis Pushmeet Kohli Martin A. Riedmiller Oriol Vinyals David Silver 0001 Faster sorting algorithms discovered using deep reinforcement learning. 257-263 2023 618 Nat. 7964 https://doi.org/10.1038/s41586-023-06004-9 db/journals/nature/nature618.html#MankowitzMZGSPL23

Simon Geisler Yujia Li 0001 Daniel J. Mankowitz Ali Taylan Cemgil Stephan Günnemann Cosmin Paduraru Transformers Meet Directed Graphs. 11144-11172 2023 ICML https://proceedings.mlr.press/v202/geisler23a.html conf/icml/2023 db/conf/icml/icml2023.html#GeislerLMCGP23

Simon Geisler Yujia Li 0001 Daniel J. Mankowitz Ali Taylan Cemgil Stephan Günnemann Cosmin Paduraru Transformers Meet Directed Graphs. 2023 abs/2302.00049 CoRR https://doi.org/10.48550/arXiv.2302.00049 db/journals/corr/corr2302.html#abs-2302-00049

Pengming Wang 0001 Mikita Sazanovich Berkin Ilbeyi Phitchaya Mangpo Phothilimthana Manish Purohit Han Yang Tay Ngân Vu Miaosen Wang Cosmin Paduraru Edouard Leurent Anton Zhernov Julian Schrittwieser Thomas Hubert Robert Tung Paula Kurylowicz Kieran Milan Oriol Vinyals Daniel J. Mankowitz Optimizing Memory Mapping Using Deep Reinforcement Learning. 2023 abs/2305.07440 CoRR https://doi.org/10.48550/arXiv.2305.07440 db/journals/corr/corr2305.html#abs-2305-07440

Brendan D. Tracey Andrea Michi Yuri Chervonyi Ian Davies Cosmin Paduraru Nevena Lazic Federico Felici Timo Ewalds Craig Donner Cristian Galperti Jonas Buchli Michael Neunert Andrea Huber Jonathan Evens Paula Kurylowicz Daniel J. Mankowitz Martin A. Riedmiller TCV Team Towards practical reinforcement learning for tokamak magnetic control. 2023 abs/2307.11546 CoRR https://doi.org/10.48550/arXiv.2307.11546 db/journals/corr/corr2307.html#abs-2307-11546

Jongmin Lee 0004 Cosmin Paduraru Daniel J. Mankowitz Nicolas Heess Doina Precup Kee-Eung Kim Arthur Guez COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation. 2022 ICLR https://openreview.net/forum?id=FLA55mBee6Q conf/iclr/2022 db/conf/iclr/iclr2022.html#0004PMHPKG22

Jongmin Lee 0004 Cosmin Paduraru Daniel J. Mankowitz Nicolas Heess Doina Precup Kee-Eung Kim Arthur Guez COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation. 2022 abs/2204.08957 CoRR https://doi.org/10.48550/arXiv.2204.08957 db/journals/corr/corr2204.html#abs-2204-08957

Yuri Chervonyi Praneet Dutta Piotr Trochim Octavian Voicu Cosmin Paduraru Crystal Qian Emre Karagozler Jared Quincy Davis Richard Chippendale Gautam Bajaj Sims Witherspoon Jerry Luo Semi-analytical Industrial Cooling System Model for Reinforcement Learning. 2022 abs/2207.13131 CoRR https://doi.org/10.48550/arXiv.2207.13131 db/journals/corr/corr2207.html#abs-2207-13131

William Wong Praneet Dutta Octavian Voicu Yuri Chervonyi Cosmin Paduraru Jerry Luo Optimizing Industrial HVAC Systems with Hierarchical Reinforcement Learning. 2022 abs/2209.08112 CoRR https://doi.org/10.48550/arXiv.2209.08112 db/journals/corr/corr2209.html#abs-2209-08112

Jerry Luo Cosmin Paduraru Octavian Voicu Yuri Chervonyi Scott Munns Jerry Li Crystal Qian Praneet Dutta Jared Quincy Davis Ningjia Wu Xingwei Yang Chu-Ming Chang Ted Li Rob Rose Mingyan Fan Hootan Nakhost Tinglin Liu Brian Kirkman Frank Altamura Lee Cline Patrick Tonker Joel Gouker Dave Uden Warren Buddy Bryan Jason Law Deeni Fatiha Neil Satra Juliet Rothenberg Molly Carlin Satish Tallapaka Sims Witherspoon David Parish Peter Dolan Chenyu Zhao Daniel J. Mankowitz Controlling Commercial Cooling Systems Using Reinforcement Learning. 2022 abs/2211.07357 CoRR https://doi.org/10.48550/arXiv.2211.07357 db/journals/corr/corr2211.html#abs-2211-07357

Gabriel Dulac-Arnold Nir Levine Daniel J. Mankowitz Jerry Li Cosmin Paduraru Sven Gowal Todd Hester Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. 2419-2468 2021 110 Mach. Learn. 9 https://doi.org/10.1007/s10994-021-05961-4 https://www.wikidata.org/entity/Q114224937 db/journals/ml/ml110.html#Dulac-ArnoldLML21

Justin Fu Mohammad Norouzi 0002 Ofir Nachum George Tucker Ziyu Wang 0001 Alexander Novikov 0001 Mengjiao Yang 0001 Michael R. Zhang Yutian Chen 0001 Aviral Kumar Cosmin Paduraru Sergey Levine Tom Le Paine Benchmarks for Deep Off-Policy Evaluation. 2021 ICLR https://openreview.net/forum?id=kWSeGEeHvF8 conf/iclr/2021 db/conf/iclr/iclr2021.html#Fu0NTw0YZCKPLP21 Michael R. Zhang Thomas Paine Ofir Nachum Cosmin Paduraru George Tucker Ziyu Wang 0001 Mohammad Norouzi 0002 Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization. 2021 ICLR https://openreview.net/forum?id=kmqjgSNXby conf/iclr/2021 db/conf/iclr/iclr2021.html#ZhangPNPT0021 Ksenia Konyushkova Yutian Chen 0001 Thomas Paine Çaglar Gülçehre Cosmin Paduraru Daniel J. Mankowitz Misha Denil Nando de Freitas Active Offline Policy Selection. 24631-24644 2021 NeurIPS https://proceedings.neurips.cc/paper/2021/hash/cec2346566ba8ecd04bfd992fd193fb3-Abstract.html conf/nips/2021 db/conf/nips/neurips2021.html#KonyushkovaCPGP21

Justin Fu Mohammad Norouzi 0002 Ofir Nachum George Tucker Ziyu Wang 0001 Alexander Novikov 0001 Mengjiao Yang 0001 Michael R. Zhang Yutian Chen 0001 Aviral Kumar Cosmin Paduraru Sergey Levine Tom Le Paine Benchmarks for Deep Off-Policy Evaluation. 2021 abs/2103.16596 CoRR https://arxiv.org/abs/2103.16596 db/journals/corr/corr2103.html#abs-2103-16596

Michael R. Zhang Tom Le Paine Ofir Nachum Cosmin Paduraru George Tucker Ziyu Wang 0001 Mohammad Norouzi 0002 Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization. 2021 abs/2104.13877 CoRR https://arxiv.org/abs/2104.13877 db/journals/corr/corr2104.html#abs-2104-13877

Ksenia Konyushkova Yutian Chen 0001 Thomas Paine Çaglar Gülçehre Cosmin Paduraru Daniel J. Mankowitz Misha Denil Nando de Freitas Active Offline Policy Selection. 2021 abs/2106.10251 CoRR https://arxiv.org/abs/2106.10251 db/journals/corr/corr2106.html#abs-2106-10251

Çaglar Gülçehre Ziyu Wang 0001 Alexander Novikov 0001 Thomas Paine Sergio Gómez Colmenarejo Konrad Zolna Rishabh Agarwal Josh Merel Daniel J. Mankowitz Cosmin Paduraru Gabriel Dulac-Arnold Jerry Li Mohammad Norouzi 0002 Matthew Hoffman 0002 Nicolas Heess Nando de Freitas RL Unplugged: A Collection of Benchmarks for Offline Reinforcement Learning. 2020 NeurIPS https://proceedings.neurips.cc/paper/2020/hash/51200d29d1fc15f5a71c1dab4bb54f7c-Abstract.html conf/nips/2020 db/conf/nips/neurips2020.html#Gulcehre0NPCZAM20

Gabriel Dulac-Arnold Nir Levine Daniel J. Mankowitz Jerry Li Cosmin Paduraru Sven Gowal Todd Hester An empirical investigation of the challenges of real-world reinforcement learning. 2020 abs/2003.11881 CoRR https://arxiv.org/abs/2003.11881 db/journals/corr/corr2003.html#abs-2003-11881

Çaglar Gülçehre Ziyu Wang 0001 Alexander Novikov 0001 Tom Le Paine Sergio Gómez Colmenarejo Konrad Zolna Rishabh Agarwal Josh Merel Daniel J. Mankowitz Cosmin Paduraru Gabriel Dulac-Arnold Jerry Li Mohammad Norouzi 0002 Matt Hoffman 0001 Ofir Nachum George Tucker Nicolas Heess Nando de Freitas RL Unplugged: Benchmarks for Offline Reinforcement Learning. 2020 abs/2006.13888 CoRR https://arxiv.org/abs/2006.13888 db/journals/corr/corr2006.html#abs-2006-13888

Tom Le Paine Cosmin Paduraru Andrea Michi Çaglar Gülçehre Konrad Zolna Alexander Novikov 0001 Ziyu Wang 0001 Nando de Freitas Hyperparameter Selection for Offline Reinforcement Learning. 2020 abs/2007.09055 CoRR https://arxiv.org/abs/2007.09055 db/journals/corr/corr2007.html#abs-2007-09055

Daniel J. Mankowitz Dan A. Calian Rae Jeong Cosmin Paduraru Nicolas Heess Sumanth Dathathri Martin A. Riedmiller Timothy A. Mann Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecification. 2020 abs/2010.10644 CoRR https://arxiv.org/abs/2010.10644 db/journals/corr/corr2010.html#abs-2010-10644

Gal Dalal Krishnamurthy Dvijotham Matej Vecerík Todd Hester Cosmin Paduraru Yuval Tassa Safe Exploration in Continuous Action Spaces. 2018 abs/1801.08757 CoRR http://arxiv.org/abs/1801.08757 db/journals/corr/corr1801.html#abs-1801-08757

Cosmin Paduraru Doina Precup Joelle Pineau Gheorghe Comanici An Empirical Analysis of Off-policy Learning in Discrete MDPs. 89-102 2012 EWRL http://proceedings.mlr.press/v24/paduraru12a.html conf/ewrl/2012 db/conf/ewrl/ewrl2012.html#PaduraruPPC12 Cosmin Paduraru Doina Precup Joelle Pineau A Framework for Computing Bounds for the Return of a Policy. 201-212 2011 EWRL https://doi.org/10.1007/978-3-642-29946-9_21 conf/ewrl/2011 db/conf/ewrl/ewrl2011.html#PaduraruPP11 Brian Tanner Vadim Bulitko Anna Koop Cosmin Paduraru Grounding Abstractions in Predictive State Representations. 1077-1082 2007 conf/ijcai/2007 IJCAI http://ijcai.org/Proceedings/07/Papers/174.pdf db/conf/ijcai/ijcai2007.html#TannerBKP07 Doina Precup Richard S. Sutton Cosmin Paduraru Anna Koop Satinder Singh 0001 Off-policy Learning with Options and Recognizers. 2005 conf/nips/2005 NIPS https://proceedings.neurips.cc/paper/2005/hash/f75526659f31040afeb61cb7133e4e6d-Abstract.html http://papers.nips.cc/paper/2767-off-policy-learning-with-options-and-recognizers db/conf/nips/nips2005.html#PrecupSPKS05 1097-1104 Rishabh Agarwal Alex Ahern Frank Altamura Gautam Bajaj Mohammadamin Barekatain Kate Baumli Feryal M. P. Behbahani Colton Bishop Jackson Broshear Warren Buddy Bryan Jonas Buchli Vadim Bulitko Dan Andrei CalianDan A. Calian Molly Carlin A. Taylan CemgilAli Taylan Cemgil Chu-Ming Chang Yutian Chen 0001 Yuri Chervonyi Richard Chippendale Lee Cline John D. Co-Reyes Sergio Gomez ColmenarejoSergio Gómez Colmenarejo Gheorghe Comanici Gal Dalal Sumanth Dathathri Ian Davies Jared DavisJared Quincy Davis Misha Denil Peter Dolan Craig Donner Gabriel Dulac-Arnold Praneet Dutta Krishnamurthy Dvijotham Sophie Elster Jonathan Evens Timo Ewalds Mingyan Fan Deeni Fatiha Aleksandra Faust Federico Felici Nando de Freitas Justin Fu Stephen Gaffney Cristian Galperti Chris Gamble Simon Geisler Marco Gelmi Joel Gouker Sven Gowal Arthur Guez Caglar GulcehreÇaglar Gülçehre Stephan Günnemann Demis Hassabis Nicolas Heess Todd Hester Matt Hoffman 0001Matthew Hoffman 0002 Andrea Huber Thomas Hubert Minjae Hwang Berkin Ilbeyi Shariq Iqbal Rae Jeong Emre Karagozler Kee-Eung Kim Brian Kirkman Pushmeet Kohli Ksenia Konyushkova Anna Koop Thomas Köppe Aviral Kumar Paula Kurylowicz Jason Law Nevena Lazic Jongmin Lee 0004 Jean-Baptiste Lespiau Edouard Leurent Nir Levine Sergey Levine Jerry Li Ted Li Yujia Li 0001 Tinglin Liu Jerry Luo Amol Mandhane Daniel J. Mankowitz Timothy A. Mann Kay McKinney Josh Merel Andrea Michi Kieran Milan Kevin Millikin Scott Munns Ofir Nachum Hootan Nakhost Michael Neunert Mohammad Norouzi 0002 Alexander Novikov 0001 Thomas Paine Tom Le Paine David Parish Phitchaya Mangpo Phothilimthana Joelle Pineau Doina Precup Manish Purohit Crystal Qian Martin A. Riedmiller Rebecca Roelofs Rob Rose Juliet Rothenberg Neil Satra Mikita Sazanovich Julian Schrittwieser Marco Selvi Disha Shrivastava David Silver 0001 Avi Singh Satinder Singh 0001 Yi Su Richard S. Sutton Satish Tallapaka Brian Tanner Yuval Tassa Han Yang Tay TCV Team Patrick Tonker Brendan D. Tracey Piotr Trochim George Tucker Robert Tung Dave Uden Matej Vecerík Oriol Vinyals Octavian Voicu Ngân Vu Miaosen Wang Pengming Wang 0001 Ziyu Wang 0001 Sims Witherspoon William Wong Ningjia Wu Sherry Yang 0001Mengjiao Yang 0001 Xingwei Yang Lei M. Zhang Michael R. Zhang Chenyu Zhao Anton Zhernov Vincent Zhuang Konrad Zolna