Lee Lab @ UW Madison

Research focus: LLMs (theoretical/empirical analysis to understand how/why they work and improve them)

New Preprints

• Arxiv’25 / In-Context Learning with Hypothesis-Class Guidance
• Arxiv’25 / LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization
• Arxiv’25 / VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data
• Arxiv’25 / Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges

Selected Work

• ICLR’25 / Looped Transformers for Length Generalization / Summary / Github
• ICML’24 / Dual operating modes of ICL / Summary / Github
• ICLR’24 / Expressive power of LoRA / Summary / Github
• ICLR’24 / Teaching arithmetic to small TFs / Summary / Github
• NeurIPS’22 / LIFT (Finetuning LLMs on non-textual data) / Summary / Github

Selected Talks

• (Mar. 2025) Helmholtz/ELLIS Workshop on Foundation Models in Science, Berlin, Germany
  Title: Bridging Large Language Models and Classical Machine Learning: From LIFT to LLM-Lasso for Predictive Modeling
• (Mar. 2025) EnCORE Workshop on Theoretical Perspective on LLMs, San Diego, CA
  Title: Beyond Decoder-Only Next Token Prediction (video)
• (Feb. 2025) ECE Grad Seminar at the University of Pittsburgh, Feb. 2025.
  Title: Beyond Decoder-Only Next Token Prediction
• (Nov. 2024) Seminars on AI Core and Applications, Seoul National University
• (Oct. 2024) Mathematical Principles in Foundation Models, 2024 SIAM Conference on Mathematics of Data Science
  Title: Dual Operating Modes of ICL
• (Apr. 2024) The Johns Hopkins University CIS/MINDS seminar
  Title: Theoretical Exploration of Foundation Model Adaptation Methods
• (Mar. 2024) The 58th Annual Conference on Information Sciences and Systems @ Princeton University
  Title: A Probabilistic Framework for Understanding In-Context Task Learning and Retrieval
• (Feb. 2024) 2024 Information Theory and Applications Workshop
  Title: The Expressive Power of Low-Rank Adaptation (LoRA)
• (Feb. 2024) Foundations of Data Science - Virtual Talk Series @ UCSD/NSF TRIPODS Institute on Emerging CORE Methods in Data Science (EnCORE)
  Title: Theoretical Exploration of Foundation Model Adaptation Methods (video)
• (Dec. 2023) CSP Seminar @ University of Michigan
  Title: Towards a Theoretical Understanding of Parameter-Efficient Fine-Tuning (and Beyond) (video)
• (Nov. 2023) Efficient ML workshop @ Google Research New York
  Title: The Expressive Power of Low-Rank Adaptation (LoRA)

Openings

• [postdocs] We are looking for a postdoc interested in the theoretical and algorithmic aspects of foundation models, particularly LLMs. If you want to work with us, please email me your CV and a research statement. I strongly recommend reading our lab’s recent papers.
• [PhD students] We are looking for PhD students (ECE or CS) interested in the theoretical and algorithmic aspects of foundation models, particularly LLMs. If you want to work with us, please email me your CV and a research statement. I strongly recommend reading our lab’s recent papers.
• [MS students] I am not currently looking for MS students.
• [undergraduate students] Please take my courses first. I teach various machine learning courses (ECE 532, 539, 561, 570, 761, 901, …).

We sincerely appreciate the support provided by our sponsors: NSF, Amazon, and FuriosaAI.