Confronting Reward Model Overoptimization with Constrained RLHF.

DJ Strouse Daniel Strouse http://djstrouse.com/ https://www.linkedin.com/pub/dj-strouse/7/228/463/ https://twitter.com/djstrouse https://github.com/djstrouse Princeton University, Physics Department Ted Moskovitz Aaditya K. Singh DJ Strouse Tuomas Sandholm Ruslan Salakhutdinov Anca D. Dragan Stephen Marcus McAleer Confronting Reward Model Overoptimization with Constrained RLHF. 2024 ICLR https://openreview.net/forum?id=gkfUvn0fLU conf/iclr/2024 db/conf/iclr/iclr2024.html#MoskovitzSSSSDM24

Aaditya K. Singh DJ Strouse Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs. 2024 abs/2402.14903 CoRR https://doi.org/10.48550/arXiv.2402.14903 db/journals/corr/corr2402.html#abs-2402-14903

Albert S. Yue Lovish Madaan Ted Moskovitz DJ Strouse Aaditya K. Singh HARP: A challenging human-annotated math reasoning benchmark. 2024 abs/2412.08819 CoRR https://doi.org/10.48550/arXiv.2412.08819 db/journals/corr/corr2412.html#abs-2412-08819 streams/journals/corr

Michael Laskin Luyu Wang Junhyuk Oh Emilio Parisotto Stephen Spencer Richie Steigerwald DJ Strouse Steven Stenberg Hansen Angelos Filos Ethan Brooks Maxime Gazeau Himanshu Sahni Satinder Singh 0001 Volodymyr Mnih In-context Reinforcement Learning with Algorithm Distillation. 2023 ICLR https://openreview.net/forum?id=hy0a5MMPUv conf/iclr/2023 db/conf/iclr/iclr2023.html#LaskinWOPSSSHFB23

Ted Moskovitz Aaditya K. Singh DJ Strouse Tuomas Sandholm Ruslan Salakhutdinov Anca D. Dragan Stephen McAleer Confronting Reward Model Overoptimization with Constrained RLHF. 2023 abs/2310.04373 CoRR https://doi.org/10.48550/arXiv.2310.04373 db/journals/corr/corr2310.html#abs-2310-04373

DJ Strouse Kate Baumli David Warde-Farley Volodymyr Mnih Steven Stenberg Hansen Learning more skills through optimistic exploration. 2022 ICLR https://openreview.net/forum?id=cU8rknuhxc conf/iclr/2022 db/conf/iclr/iclr2022.html#StrouseBWMH22 Allison C. Tam Neil C. Rabinowitz Andrew K. Lampinen Nicholas A. Roy Stephanie C. Y. Chan DJ Strouse Jane Wang 0001 Andrea Banino Felix Hill Semantic Exploration from Language Abstractions and Pretrained Representations. 2022 conf/nips/2022 NeurIPS http://papers.nips.cc/paper_files/paper/2022/hash/a28e024ccd623ed113fb19683fa0910d-Abstract-Conference.html db/conf/nips/neurips2022.html#TamRLRCS0BH22

Allison C. Tam Neil C. Rabinowitz Andrew K. Lampinen Nicholas A. Roy Stephanie C. Y. Chan DJ Strouse Jane X. Wang Andrea Banino Felix Hill Semantic Exploration from Language Abstractions and Pretrained Representations. 2022 abs/2204.05080 CoRR https://doi.org/10.48550/arXiv.2204.05080 db/journals/corr/corr2204.html#abs-2204-05080

Michael Laskin Luyu Wang Junhyuk Oh Emilio Parisotto Stephen Spencer Richie Steigerwald DJ Strouse Steven Hansen 0001 Angelos Filos Ethan Brooks Maxime Gazeau Himanshu Sahni Satinder Singh 0001 Volodymyr Mnih In-context Reinforcement Learning with Algorithm Distillation. 2022 abs/2210.14215 CoRR https://doi.org/10.48550/arXiv.2210.14215 db/journals/corr/corr2210.html#abs-2210-14215

John P. Agapiou Alexander Sasha Vezhnevets Edgar A. Duéñez-Guzmán Jayd Matyas Yiran Mao Peter Sunehag Raphael Köster Udari Madhushani Kavya Kopparapu Ramona Comanescu DJ Strouse Michael Bradley Johanson Sukhdeep Singh Julia Haas Igor Mordatch Dean Mobbs Joel Z. Leibo Melting Pot 2.0. 2022 abs/2211.13746 CoRR https://doi.org/10.48550/arXiv.2211.13746 db/journals/corr/corr2211.html#abs-2211-13746

DJ Strouse Kevin R. McKee Matt M. Botvinick Edward Hughes 0001 Richard Everett 0001 Collaborating with Humans without Human Data. 14502-14515 2021 NeurIPS https://proceedings.neurips.cc/paper/2021/hash/797134c3e42371bb4979a462eb2f042a-Abstract.html conf/nips/2021 db/conf/nips/neurips2021.html#StrouseMBHE21

DJ Strouse Kate Baumli David Warde-Farley Vlad Mnih Steven Hansen 0001 Learning more skills through optimistic exploration. 2021 abs/2107.14226 CoRR https://arxiv.org/abs/2107.14226 db/journals/corr/corr2107.html#abs-2107-14226

DJ Strouse Kevin R. McKee Matt M. Botvinick Edward Hughes 0001 Richard Everett 0001 Collaborating with Humans without Human Data. 2021 abs/2110.08176 CoRR https://arxiv.org/abs/2110.08176 db/journals/corr/corr2110.html#abs-2110-08176

DJ Strouse David J. Schwab The Information Bottleneck and Geometric Clustering. 2019 31 Neural Comput. 3 https://doi.org/10.1162/neco_a_01136 https://www.wikidata.org/entity/Q57472002 db/journals/neco/neco31.html#StrouseS19

Xundong Wu Gabriel C. Mel DJ Strouse Bartlett W. Mel How Dendrites Affect Online Recognition Memory. 2019 15 PLoS Comput. Biol. 5 https://doi.org/10.1371/journal.pcbi.1006892 https://www.wikidata.org/entity/Q91728885 db/journals/ploscb/ploscb15.html#WuMSM19

Anirudh Goyal Riashat Islam Daniel Strouse Zafarali Ahmed Hugo Larochelle Matthew M. Botvinick Yoshua Bengio Sergey Levine InfoBot: Transfer and Exploration via the Information Bottleneck. 2019 ICLR (Poster) https://openreview.net/forum?id=rJg8yhAqKm conf/iclr/2019 db/conf/iclr/iclr2019.html#GoyalISALBBL19 Natasha Jaques Angeliki Lazaridou Edward Hughes 0001 Çaglar Gülçehre Pedro A. Ortega DJ Strouse Joel Z. Leibo Nando de Freitas Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning. 3040-3049 2019 ICML http://proceedings.mlr.press/v97/jaques19a.html conf/icml/2019 db/conf/icml/icml2019.html#JaquesLHGOSLF19

Anirudh Goyal Riashat Islam Daniel Strouse Zafarali Ahmed Matthew M. Botvinick Hugo Larochelle Sergey Levine Yoshua Bengio InfoBot: Transfer and Exploration via the Information Bottleneck. 2019 abs/1901.10902 CoRR http://arxiv.org/abs/1901.10902 db/journals/corr/corr1901.html#abs-1901-10902

Andrea Tacchetti DJ Strouse Marta Garnelo Thore Graepel Yoram Bachrach A Neural Architecture for Designing Truthful and Efficient Auctions. 2019 abs/1907.05181 CoRR http://arxiv.org/abs/1907.05181 db/journals/corr/corr1907.html#abs-1907-05181

Daniel Strouse Max Kleiman-Weiner Josh Tenenbaum 0001 Matthew M. Botvinick David J. Schwab Learning to Share and Hide Intentions using Information Regularization. 10270-10281 2018 NeurIPS https://proceedings.neurips.cc/paper/2018/hash/1ef03ed0cd5863c550128836b28ec3e9-Abstract.html http://papers.nips.cc/paper/8227-learning-to-share-and-hide-intentions-using-information-regularization conf/nips/2018 db/conf/nips/nips2018.html#StrouseKTBS18

DJ Strouse Max Kleiman-Weiner Josh Tenenbaum 0001 Matthew M. Botvinick David J. Schwab Learning to Share and Hide Intentions using Information Regularization. 2018 abs/1808.02093 CoRR http://arxiv.org/abs/1808.02093 db/journals/corr/corr1808.html#abs-1808-02093

Natasha Jaques Angeliki Lazaridou Edward Hughes 0001 Çaglar Gülçehre Pedro A. Ortega DJ Strouse Joel Z. Leibo Nando de Freitas Intrinsic Social Motivation via Causal Influence in Multi-Agent RL. 2018 abs/1810.08647 CoRR http://arxiv.org/abs/1810.08647 db/journals/corr/corr1810.html#abs-1810-08647

DJ Strouse David J. Schwab The Deterministic Information Bottleneck. 1611-1630 2017 29 Neural Comput. 6 https://doi.org/10.1162/NECO_a_00961 https://www.wikidata.org/entity/Q38836891 db/journals/neco/neco29.html#StrouseS17

DJ Strouse David J. Schwab The information bottleneck and geometric clustering. 2017 abs/1712.09657 CoRR http://arxiv.org/abs/1712.09657 db/journals/corr/corr1712.html#abs-1712-09657

DJ Strouse David J. Schwab The Deterministic Information Bottleneck. 2016 UAI http://auai.org/uai2016/proceedings/papers/319.pdf conf/uai/2016 db/conf/uai/uai2016.html#StrouseS16

DJ Strouse David J. Schwab The deterministic information bottleneck. 2016 abs/1604.00268 CoRR http://arxiv.org/abs/1604.00268 db/journals/corr/corr1604.html#StrouseS16

John P. Agapiou Zafarali Ahmed Yoram Bachrach Andrea Banino Kate Baumli Yoshua Bengio Matt M. BotvinickMatthew M. Botvinick Ethan Brooks Stephanie C. Y. Chan Ramona Comanescu Anca D. Dragan Edgar A. Duéñez-Guzmán Richard Everett 0001 Angelos Filos Nando de Freitas Marta Garnelo Maxime Gazeau Anirudh Goyal Thore Graepel Caglar GulcehreÇaglar Gülçehre Julia Haas Steven Hansen 0001Steven Stenberg Hansen Felix Hill Edward Hughes 0001 Riashat Islam Natasha Jaques Michael JohansonMichael Bradley Johanson Max Kleiman-Weiner Kavya Kopparapu Raphael KosterRaphael Köster Andrew K. Lampinen Hugo Larochelle Michael Laskin Angeliki Lazaridou Joel Z. Leibo Sergey Levine Lovish Madaan T. W. U. MadhushaniUdari Madhushani Yiran Mao Jayd Matyas Stephen McAleerStephen Marcus McAleer Kevin R. McKee Bartlett W. Mel Gabriel C. Mel Volodymyr MnihVlad Mnih Dean Mobbs Igor Mordatch Ted Moskovitz Junhyuk Oh Pedro A. Ortega Emilio Parisotto Neil C. Rabinowitz Nicholas A. Roy Himanshu Sahni Ruslan Salakhutdinov Tuomas Sandholm David J. Schwab Aaditya K. Singh Satinder Singh 0001 Sukhdeep Singh Stephen Spencer Richie Steigerwald Peter Sunehag Andrea Tacchetti Allison C. Tam Joshua B. TenenbaumJosh Tenenbaum 0001 Alexander VezhnevetsAlexander Sasha Vezhnevets Jane X. WangJane Wang 0001 Luyu Wang David Warde-Farley Xundong Wu Albert S. Yue