← Authors

Tengyu Ma

95 papers · 2014–2025 · 14 conferences · across top CS/AI conferences

Achievements

🐣 Hot Topic Early Bird πŸ—ΊοΈ Taxonomy Completionist (14) 🧭 Keyword Pioneer 🌍 Conference Polyglot (14) 🌈 Renaissance Researcher (9) 🐝 Cross-Pollinator (9) πŸŒ‰ Interdisciplinary Bridge πŸƒ Academic Marathon (11) 🏠 Conference Loyalist (30) 🌟 Keyword Trendsetter Combo (3) πŸ“› The Namer 🀝 Dynamic Duo (14) πŸ‘‘ Triple Crown πŸ† Grand Slam πŸ”¬ Deep Specialist (13) πŸ† Keyword Champion (2) πŸ”₯ Unstoppable (12) πŸš€ Conference Pioneer ⚑ Prolific Year (14) πŸ—ƒοΈ Keyword Collector (273) ❓ The Questioner (4) πŸ’Ž Century Club (95) πŸ“ˆ Trend Setter

Conferences

NIPS (30) ICLR (26) ICML (16) COLT (8) AAAI (2) ACL (2) AISTATS (2) CVPR (2) JMLR (2) ECCV (1) EMNLP (1) IJCNLP (1) NAACL (1) UAI (1)

Top co-authors

Colin Wei (14) Percy Liang (12) Rong Ge (9) Zhiyuan Li (8) Ananya Kumar (8) Sang Michael Xie (8) Kefan Dong (7) Yining Chen (6) Jeff Z. HaoChen (6) Sanjeev Arora (6)

Keywords

neural network (13) stochastic gradient descent (7) representation learning (6) sample complexity (5) gradient descent (5) non-convex optimization (4) transfer learning (4) implicit regularization (4) contrastive learning (3) language model (3) pretrained language model (3) document-level relation extraction (3) model-based reinforcement learning (3) generalization bound (3) kernel methods (3) neural tangent kernel (2) unsupervised domain adaptation (2) learning theory (2) communication complexity (2) distributed optimization (2)

Papers

SAM 2: Segment Anything in Images and Videos ICLR 2025
Rethinking Reconstruction and Denoising in the Dark: New Perspective, General Architecture and Beyond CVPR 2025
STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Proving ICML 2025
Non-Asymptotic Length Generalization ICML 2025
Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View ICLR 2025
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training ICLR 2024
Linguistic Calibration of Long-Form Generations ICML 2024
One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention ICLR 2024
Chain of Thought Empowers Transformers to Solve Inherently Serial Problems ICLR 2024
Trash to Treasure: Low-Light Object Detection via Decomposition-and-Aggregation AAAI 2024
Large Language Models as Tool Makers ICLR 2024
Symbol tuning improves in-context learning in language models EMNLP 2023
Toward L_∞Recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields COLT 2023
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining NIPS 2023
Asymptotic Instance-Optimal Algorithms for Interactive Decision Making ICLR 2023
What is the Inductive Bias of Flatness Regularization? A Study of Deep Matrix Factorization Models NIPS 2023
Data Selection for Language Models via Importance Resampling NIPS 2023
Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time NIPS 2023
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models ICML 2023
​​What learning algorithm is in-context learning? Investigations with linear models ICLR 2023
Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization NIPS 2023
Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence ICLR 2023
How Sharpness-Aware Minimization Minimizes Sharpness? ICLR 2023
First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains ICLR 2023
A theoretical study of inductive biases in contrastive learning ICLR 2023
Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift UAI 2022
Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments NIPS 2022
Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers NIPS 2022
Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations NIPS 2022
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective AISTATS 2022
Toward Fast, Flexible, and Robust Low-Light Image Enhancement CVPR 2022
An Explanation of In-context Learning as Implicit Bayesian Inference ICLR 2022
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution ICLR 2022
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization ICLR 2022
Self-supervised Learning is More Robust to Dataset Imbalance ICLR 2022
Near-Optimal Algorithms for Autonomous Exploration and Multi-Goal Stochastic Shortest Path ICML 2022
Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification ICML 2022
Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation ICML 2022
Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data ICLR 2021
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss NIPS 2021
Safe Reinforcement Learning by Imagining the Near Future NIPS 2021
Entity and Evidence Guided Document-Level Relation Extraction IJCNLP 2021
Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap COLT 2021
Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations NIPS 2021
Entity and Evidence Guided Document-Level Relation Extraction ACL 2021
Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration NIPS 2021
Active Online Learning with Hidden Shifting Domains AISTATS 2021
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning NIPS 2021
Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature NIPS 2021
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling AAAI 2021
Label Noise SGD Provably Prefers Flat Global Minimizers NIPS 2021
Variance-reduced First-order Meta-learning for Natural Language Processing Tasks NAACL 2021
Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization ICML 2021
Shape Matters: Understanding the Implicit Bias of the Noise Covariance COLT 2021
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness ICLR 2021
Optimal Regularization can Mitigate Double Descent ICLR 2021
Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization ICLR 2021
Improved Sample Complexities for Deep Neural Networks and Robust Classification via an All-Layer Margin ICLR 2020
Learning Over-Parametrized Two-Layer Neural Networks beyond NTK COLT 2020
Robust and On-the-fly Dataset Denoising for Image Classification ECCV 2020
Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative Sampling ICLR 2020
Beyond Lazy Training for Over-parameterized Tensor Decomposition NIPS 2020
Self-training Avoids Using Spurious Features Under Domain Shift NIPS 2020
MOPO: Model-based Offline Policy Optimization NIPS 2020
Model-based Adversarial Meta-Reinforcement Learning NIPS 2020
Federated Accelerated Stochastic Gradient Descent NIPS 2020
On the Expressivity of Neural Networks for Deep Reinforcement Learning ICML 2020
Understanding Self-Training for Gradual Domain Adaptation ICML 2020
The Implicit and Explicit Regularization Effects of Dropout ICML 2020
Individual Calibration with Randomized Forecasting ICML 2020
Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks NIPS 2019
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel NIPS 2019
Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss NIPS 2019
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation NIPS 2019
On the Performance of Thompson Sampling on Logistic Bandits COLT 2019
Approximability of Discriminators Implies Diversity in GANs ICLR 2019
Fixup Initialization: Residual Learning Without Normalization ICLR 2019
Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees ICLR 2019
Verified Uncertainty Calibration NIPS 2019
A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors ACL 2018
Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations COLT 2018
Gradient Descent Learns Linear Dynamical Systems JMLR 2018
Learning One-hidden-layer Neural Networks with Landscape Design ICLR 2018
Generalization and Equilibrium in Generative Adversarial Nets (GANs) ICML 2017
On the Optimization Landscape of Tensor Decompositions NIPS 2017
On the Ability of Neural Nets to Express Distributions COLT 2017
Distributed Stochastic Variance Reduced Gradient Methods by Sampling Extra Data with Replacement JMLR 2017
A Non-generative Framework and Convex Relaxations for Unsupervised Learning NIPS 2016
Matrix Completion has No Spurious Local Minimum NIPS 2016
Provable Algorithms for Inference in Topic Models ICML 2016
Online Learning of Eigenvectors ICML 2015
Simple, Efficient, and Neural Algorithms for Sparse Coding COLT 2015
Sum-of-Squares Lower Bounds for Sparse PCA NIPS 2015
Provable Bounds for Learning Some Deep Representations ICML 2014
On Communication Cost of Distributed Statistical Estimation and Dimensionality NIPS 2014