Transient learning dynamics drive escape from sharp valleys in Stochastic Gradient Descent.

Yuhai Tu

Ning Yang Yikuan Zhang Qi Ouyang Chao Tang Yuhai Tu Transient learning dynamics drive escape from sharp valleys in Stochastic Gradient Descent. 2026 January abs/2601.10962 CoRR https://doi.org/10.48550/arXiv.2601.10962 db/journals/corr/corr2601.html#abs-2601-10962 streams/journals/corr

Yikuan Zhang Ning Yang Yuhai Tu On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature. 2026 February abs/2602.05600 CoRR https://doi.org/10.48550/arXiv.2602.05600 db/journals/corr/corr2602.html#abs-2602-05600 streams/journals/corr

Adrian Shuai Li Elisa Bertino Xuan-Hong Dang Ankush Singla Yuhai Tu Mark N. Wegman Maximizing Information in Domain-Invariant Representation Improves Transfer Learning. CIC 2025 21-32 conf/coinco/2025 https://doi.org/10.1109/CIC67133.2025.00013 db/conf/coinco/cic2025.html#LiBDSTW25 streams/conf/coinco

Yu Feng 0004 Wei Zhang 0022 Yuhai Tu Activity-weight duality in feed-forward neural networks reveals two co-determinants for generalization. 908-918 2023 August 5 Nat. Mac. Intell. 8 https://doi.org/10.1038/s42256-023-00700-x db/journals/natmi/natmi5.html#FengZT23

Adrian Shuai Li Elisa Bertino Xuan-Hong Dang Ankush Singla Yuhai Tu Mark N. Wegman Maximal Domain Independent Representations Improve Transfer Learning. 2023 abs/2306.00262 CoRR https://doi.org/10.48550/arXiv.2306.00262 db/journals/corr/corr2306.html#abs-2306-00262

Yu Feng Yuhai Tu The activity-weight duality in feed forward neural networks: The geometric determinants of generalization. 2022 abs/2203.10736 CoRR https://doi.org/10.48550/arXiv.2203.10736 db/journals/corr/corr2203.html#abs-2203-10736

Ning Yang Chao Tang Yuhai Tu Stochastic gradient descent introduces an effective landscape-dependent regularization favoring flat solutions. 2022 abs/2206.01246 CoRR https://doi.org/10.48550/arXiv.2206.01246 db/journals/corr/corr2206.html#abs-2206-01246

Steven Durr Youssef Mroueh Yuhai Tu Shenshen Wang Effective Dynamics of Generative Adversarial Networks. 2022 abs/2212.04580 CoRR https://doi.org/10.48550/arXiv.2212.04580 db/journals/corr/corr2212.html#abs-2212-04580

Yuxuan Liu Qianyi Li Chao Tang Shanshan Qin Yuhai Tu Short-Term Plasticity Regulates Both Divisive Normalization and Adaptive Responses in Drosophila Olfactory System. 730431 2021 15 Frontiers Comput. Neurosci. https://doi.org/10.3389/fncom.2021.730431 db/journals/ficn/ficn15.html#LiuLTQT21

Yu Feng 0004 Yuhai Tu Phases of learning dynamics in artificial neural networks in the absence or presence of mislabeled data. 43001 2021 2 Mach. Learn. Sci. Technol. 4 https://doi.org/10.1088/2632-2153/abf5b9 db/journals/mlst/mlst2.html#FengT21

Yu Feng 0004 Yuhai Tu The inverse variance-flatness relation in stochastic gradient descent is critical for finding flat minima. e2015617118 2021 118 Proc. Natl. Acad. Sci. USA 9 https://doi.org/10.1073/pnas.2015617118 https://www.wikidata.org/entity/Q128831625 db/journals/pnas/pnas118.html#FengT21

Yu Feng 0004 Yuhai Tu Phases of learning dynamics in artificial neural networks: with or without mislabeled data. 2021 abs/2101.06509 CoRR https://arxiv.org/abs/2101.06509 db/journals/corr/corr2101.html#abs-2101-06509

Wei Zhang 0022 Mingrui Liu Yu Feng 0004 Xiaodong Cui Brian Kingsbury Yuhai Tu Loss Landscape Dependent Self-Adjusting Learning Rates in Decentralized Stochastic Gradient Descent. 2021 abs/2112.01433 CoRR https://arxiv.org/abs/2112.01433 db/journals/corr/corr2112.html#abs-2112-01433

Yu Feng 0004 Yuhai Tu How neural networks find generalizable solutions: Self-tuned annealing in deep learning. 2020 abs/2001.01678 CoRR http://arxiv.org/abs/2001.01678 db/journals/corr/corr2001.html#abs-2001-01678

Matthew Riemer Ignacio Cases Robert Ajemian Miao Liu 0001 Irina Rish Yuhai Tu Gerald Tesauro Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference. 2019 ICLR (Poster) https://openreview.net/forum?id=B1gTShAct7 conf/iclr/2019 db/conf/iclr/iclr2019.html#RiemerCALRTT19

Pouya Bashivan Martin Schrimpf Robert Ajemian Irina Rish Matthew Riemer Yuhai Tu Continual Learning with Self-Organizing Maps. 2019 abs/1904.09330 CoRR http://arxiv.org/abs/1904.09330 db/journals/corr/corr1904.html#abs-1904-09330

Bernardo A. Mello Wenlin Pan Gerald L. Hazelbauer Yuhai Tu A dual regulation mechanism of histidine kinase CheA identified by combining network-dynamics modeling and system-level input-output data. 2018 14 PLoS Comput. Biol. 7 https://doi.org/10.1371/journal.pcbi.1006305 https://www.wikidata.org/entity/Q89387204 db/journals/ploscb/ploscb14.html#MelloPHT18

Matthew Riemer Ignacio Cases Robert Ajemian Miao Liu 0001 Irina Rish Yuhai Tu Gerald Tesauro Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference. 2018 abs/1810.11910 CoRR http://arxiv.org/abs/1810.11910 db/journals/corr/corr1810.html#abs-1810-11910

Bo Hu Yuhai Tu Behaviors and Strategies of Bacterial Navigation in Chemical and Nonchemical Gradients. 2014 10 PLoS Comput. Biol. 6 https://doi.org/10.1371/journal.pcbi.1003672 https://www.wikidata.org/entity/Q28539832 db/journals/ploscb/ploscb10.html#HuT14

Yuhai Tu Microfluidics and its applications in quantitative biology. 272-280 2013 1 Quant. Biol. 4 https://doi.org/10.1007/s40484-014-0024-8 db/journals/quanbio/quanbio1.html#Tu13

Nikita Vladimirov Yuhai Tu Roger D. Traub Shortest Loops are Pacemakers in Random Networks of Electrically Coupled Axons. 2012 6 Frontiers Comput. Neurosci. https://doi.org/10.3389/fncom.2012.00017 https://www.wikidata.org/entity/Q30512977 db/journals/ficn/ficn6.html#VladimirovTT12 17

Lili Jiang Qi Ouyang Yuhai Tu Quantitative Modeling of <i>Escherichia coli</i> Chemotactic Motion in Environments Varying in Space and Time. 2010 6 PLoS Comput. Biol. 4 https://doi.org/10.1371/journal.pcbi.1000735 https://www.wikidata.org/entity/Q33553425 db/journals/ploscb/ploscb6.html#JiangOT10

John Jeremy Rice Yuhai Tu Corrado Poggesi Pieter P. deTombe Spatially-Compressed Cardiac Myofilament Models Generate Hysteresis that Is Not Found in Real Muscle. 366-377 2008 Pacific Symposium on Biocomputing http://psb.stanford.edu/psb-online/proceedings/psb08/rice.pdf conf/psb/2008 db/conf/psb/psb2008.html#RiceTPT08

John Jeremy Rice Yuhai Tu Gustavo Stolovitzky Reconstructing biological networks using conditional correlation analysis. 765-773 2005 21 Bioinform. 6 https://doi.org/10.1093/bioinformatics/bti064 https://www.wikidata.org/entity/Q48518444 db/journals/bioinformatics/bioinformatics21.html#RiceTS05

Jorge Lepre John Jeremy Rice Yuhai Tu Gustavo Stolovitzky Genes@Work: an efficient algorithm for pattern discovery and multivariate feature selection in gene expression data. 1033-1044 2004 20 Bioinform. 7 https://doi.org/10.1093/bioinformatics/bth035 https://www.wikidata.org/entity/Q30892202 db/journals/bioinformatics/bioinformatics20.html#LepreRTS04

Nigam Shah Jorge Lepre Yuhai Tu Gustavo Stolovitzky Can We Identify Cellular Pathways Implicated in Cancer Using Gene Expression Data? 94-103 2003 conf/csb/2003 CSB https://doi.org/10.1109/CSB.2003.1227308 https://doi.ieeecomputersociety.org/10.1109/CSB.2003.1227308 db/conf/csb/csb2003.html#ShahLTS03 Yiming Ye Yuhai Tu Dynamics of Coalition Formation in Combinatorial Trading. 625-632 2003 conf/ijcai/2003 IJCAI db/conf/ijcai/ijcai2003.html#YeT03 http://ijcai.org/Proceedings/03/Papers/092.pdf Andrea Califano Gustavo Stolovitzky Yuhai Tu Analysis of Gene Expression Microarrays for Phenotype Classification. 75-85 2000 conf/ismb/2000 ISMB db/conf/ismb/ismb2000.html#CalifanoST00 http://www.aaai.org/Library/ISMB/2000/ismb00-008.php Robert Ajemian Pouya Bashivan Elisa Bertino Andrea Califano Ignacio Cases Xiaodong Cui Xuan-Hong Dang Pieter P. deTombe Steven Durr Yu Feng Yu Feng 0004 Gerald L. Hazelbauer Bo Hu Lili Jiang Brian Kingsbury Jorge Lepre Adrian Shuai Li Qianyi Li Miao Liu 0001 Mingrui Liu Yuxuan Liu Bernardo A. Mello Youssef Mroueh Qi Ouyang Wenlin Pan Corrado Poggesi Shanshan Qin John Jeremy Rice Matthew Riemer Irina Rish Martin Schrimpf Nigam H. ShahNigam Shah Ankush Singla Gustavo Stolovitzky Chao Tang Gerald Tesauro Roger D. Traub Nikita Vladimirov Shenshen Wang Mark N. Wegman Ning Yang Yiming Ye Wei Zhang 0022 Yi-Kuan ZhangYikuan Zhang