Feature learning and generalization in deep networks with orthogonal weights.

Daniel A. Roberts

Hannah Day Yonatan Kahn Daniel A. Roberts Feature learning and generalization in deep networks with orthogonal weights. 35027 2025 6 Mach. Learn. Sci. Technol. 2 https://doi.org/10.1088/2632-2153/adf278 db/journals/mlst/mlst6.html#DayKR25 streams/journals/mlst

Andrey Gromov Kushal Tirumala Hassan Shapourian Paolo Glorioso Daniel A. Roberts The Unreasonable Ineffectiveness of the Deeper Layers. 2025 ICLR https://openreview.net/forum?id=ngmEcEer8a conf/iclr/2025 db/conf/iclr/iclr2025.html#GromovTSGR25

Andrey Gromov Kushal Tirumala Hassan Shapourian Paolo Glorioso Daniel A. Roberts The Unreasonable Ineffectiveness of the Deeper Layers. 2024 abs/2403.17887 CoRR https://doi.org/10.48550/arXiv.2403.17887 db/journals/corr/corr2403.html#abs-2403-17887

Matthias Gerstgrasser Rylan Schaeffer Apratim Dey Rafael Rafailov Henry Sleight John Hughes Tomasz Korbak Rajashree Agrawal Dhruv Pai Andrey Gromov Daniel A. Roberts Diyi Yang David L. Donoho Sanmi Koyejo Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data. 2024 abs/2404.01413 CoRR https://doi.org/10.48550/arXiv.2404.01413 db/journals/corr/corr2404.html#abs-2404-01413

Hannah Day Yonatan Kahn Daniel A. Roberts Feature Learning and Generalization in Deep Networks with Orthogonal Weights. 2023 abs/2310.07765 CoRR https://doi.org/10.48550/arXiv.2310.07765 db/journals/corr/corr2310.html#abs-2310-07765

Alexander Maloney Daniel A. Roberts James Sully A Solvable Model of Neural Scaling Laws. 2022 abs/2210.16859 CoRR https://doi.org/10.48550/arXiv.2210.16859 db/journals/corr/corr2210.html#abs-2210-16859

Joshua Batson C. Grace Haaf Yonatan Kahn Daniel A. Roberts Topological Obstructions to Autoencoding. 2021 abs/2102.08380 CoRR https://arxiv.org/abs/2102.08380 db/journals/corr/corr2102.html#abs-2102-08380

Daniel A. Roberts Why is AI hard and Physics simple? 2021 abs/2104.00008 CoRR https://arxiv.org/abs/2104.00008 db/journals/corr/corr2104.html#abs-2104-00008

Daniel A. Roberts SGD Implicitly Regularizes Generalization Error. 2021 abs/2104.04874 CoRR https://arxiv.org/abs/2104.04874 db/journals/corr/corr2104.html#abs-2104-04874

Daniel A. Roberts Sho Yaida Boris Hanin The Principles of Deep Learning Theory. 2021 abs/2106.10165 CoRR https://arxiv.org/abs/2106.10165 db/journals/corr/corr2106.html#abs-2106-10165

Judy Hoffman Daniel A. Roberts Sho Yaida Robust Learning with Jacobian Regularization. 2019 abs/1908.02729 CoRR http://arxiv.org/abs/1908.02729 db/journals/corr/corr1908.html#abs-1908-02729

Guy Gur-Ari Daniel A. Roberts Ethan Dyer Gradient Descent Happens in a Tiny Subspace. 2018 abs/1812.04754 CoRR http://arxiv.org/abs/1812.04754 db/journals/corr/corr1812.html#abs-1812-04754

John R. Frank Max Kleiman-Weiner Daniel A. Roberts Ellen M. Voorhees Ian Soboroff Evaluating Stream Filtering for Entity Profile Updates in TREC 2012, 2013, and 2014. 2014 TREC http://trec.nist.gov/pubs/trec23/papers/overview-kba.pdf conf/trec/2014 db/conf/trec/trec2014.html#FrankKRVS14 John R. Frank Steven J. Bauer Max Kleiman-Weiner Daniel A. Roberts Nilesh Tripuraneni Ce Zhang 0001 Christopher Ré Ellen M. Voorhees Ian Soboroff Evaluating Stream Filtering for Entity Profile Updates for TREC 2013. 2013 TREC http://trec.nist.gov/pubs/trec22/papers/KBA.OVERVIEW.pdf conf/trec/2013 db/conf/trec/trec2013.html#FrankBKRTZRVS13 John R. Frank Max Kleiman-Weiner Daniel A. Roberts Feng Niu Ce Zhang 0001 Christopher Ré Ian Soboroff Building an Entity-Centric Stream Filtering Test Collection for TREC 2012. 2012 TREC http://trec.nist.gov/pubs/trec21/papers/KBA.OVERVIEW.pdf conf/trec/2012 db/conf/trec/trec2012.html#FrankKRNZRS12 Rajashree Agrawal Joshua Batson Steven J. Bauer Hannah Day Apratim Dey David L. Donoho Ethan Dyer John R. Frank Matthias Gerstgrasser Paolo Glorioso Andrey Gromov Guy Gur-Ari C. Grace Haaf Boris Hanin Judy Hoffman John Hughes Yonatan Kahn Max Kleiman-Weiner Tomasz Korbak Oluwasanmi KoyejoSanmi Koyejo Alexander Maloney Feng Niu Dhruv Pai Rafael Rafailov Christopher Ré Rylan Schaeffer Hassan Shapourian Henry Sleight Ian Soboroff James Sully Kushal Tirumala Nilesh Tripuraneni Ellen M. Voorhees Sho Yaida Diyi Yang Ce Zhang 0001