Lee Lab @ UW Madison

We sincerely appreciate the support provided by our sponsors: NSF, Amazon, and FuriosaAI!

Research focus: LLMs (theoretical/empirical analysis to understand how/why they work and improve them)

New Preprints

• Arxiv’25 / In-Context Learning with Hypothesis-Class Guidance
• Arxiv’25 / LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization

Selected Work

• ICML’25 (spotlight) / VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data / Summary / Github / HuggingFace
• ICML’25 (spotlight) / Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
• ICML’25 / Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
• ICLR’25 / Looped Transformers for Length Generalization / Summary / Github
• ICML’24 / Dual operating modes of ICL / Summary / Github
• ICLR’24 / Expressive power of LoRA / Summary / Github
• ICLR’24 / Teaching arithmetic to small TFs / Summary / Github
• NeurIPS’22 / LIFT (Finetuning LLMs on non-textual data) / Summary / Github

Selected Talks

• (Mar. 2025) Helmholtz/ELLIS Workshop on Foundation Models in Science, Berlin, Germany
  Title: Bridging Large Language Models and Classical Machine Learning: From LIFT to LLM-Lasso for Predictive Modeling
• (Mar. 2025) EnCORE Workshop on Theoretical Perspective on LLMs, San Diego, CA
  Title: Beyond Decoder-Only Next Token Prediction (video)
• (Feb. 2025) ECE Grad Seminar at the University of Pittsburgh, Feb. 2025.
  Title: Beyond Decoder-Only Next Token Prediction
• (Nov. 2024) Seminars on AI Core and Applications, Seoul National University
• (Oct. 2024) Mathematical Principles in Foundation Models, 2024 SIAM Conference on Mathematics of Data Science
  Title: Dual Operating Modes of ICL
• (Apr. 2024) The Johns Hopkins University CIS/MINDS seminar
  Title: Theoretical Exploration of Foundation Model Adaptation Methods
• (Mar. 2024) The 58th Annual Conference on Information Sciences and Systems @ Princeton University
  Title: A Probabilistic Framework for Understanding In-Context Task Learning and Retrieval
• (Feb. 2024) 2024 Information Theory and Applications Workshop
  Title: The Expressive Power of Low-Rank Adaptation (LoRA)
• (Feb. 2024) Foundations of Data Science - Virtual Talk Series @ UCSD/NSF TRIPODS Institute on Emerging CORE Methods in Data Science (EnCORE)
  Title: Theoretical Exploration of Foundation Model Adaptation Methods (video)
• (Dec. 2023) CSP Seminar @ University of Michigan
  Title: Towards a Theoretical Understanding of Parameter-Efficient Fine-Tuning (and Beyond) (video)
• (Nov. 2023) Efficient ML workshop @ Google Research New York
  Title: The Expressive Power of Low-Rank Adaptation (LoRA)

Openings

• [postdocs] We are looking for a postdoc interested in the theoretical and algorithmic aspects of LLMs. If you want to work with us, please email me your CV and a research statement. I strongly recommend reading our lab’s recent papers.
• [PhD students] We are looking for PhD students (ECE or CS) interested in the theoretical and algorithmic aspects of LLMs. If you want to work with us, please email me your CV and a research statement. I strongly recommend reading our lab’s recent papers.
• [MS students] I am not currently looking for MS students.
• [undergraduate students] Please take my courses first. I teach various machine learning courses (ECE 532, 539, 561, 570, 761, 901, …).