Lee Lab @ UW Madison
We sincerely appreciate the support provided by our sponsors: NSF, Amazon, and FuriosaAI!
Research focus: LLMs (theoretical/empirical analysis to understand how/why they work and improve them)
New Preprints
- Arxiv’25 / In-Context Learning with Hypothesis-Class Guidance
- Arxiv’25 / LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization
- Arxiv’25 / VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data
- Arxiv’25 / Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
Selected Work
- ICLR’25 / Looped Transformers for Length Generalization / Summary / Github
- ICML’24 / Dual operating modes of ICL / Summary / Github
- ICLR’24 / Expressive power of LoRA / Summary / Github
- ICLR’24 / Teaching arithmetic to small TFs / Summary / Github
- NeurIPS’22 / LIFT (Finetuning LLMs on non-textual data) / Summary / Github
Selected Talks
- (Mar. 2025) Helmholtz/ELLIS Workshop on Foundation Models in Science, Berlin, Germany
Title: Bridging Large Language Models and Classical Machine Learning: From LIFT to LLM-Lasso for Predictive Modeling - (Mar. 2025) EnCORE Workshop on Theoretical Perspective on LLMs, San Diego, CA
Title: Beyond Decoder-Only Next Token Prediction (video) - (Feb. 2025) ECE Grad Seminar at the University of Pittsburgh, Feb. 2025.
Title: Beyond Decoder-Only Next Token Prediction - (Nov. 2024) Seminars on AI Core and Applications, Seoul National University
- (Oct. 2024) Mathematical Principles in Foundation Models, 2024 SIAM Conference on Mathematics of Data Science
Title: Dual Operating Modes of ICL - (Apr. 2024) The Johns Hopkins University CIS/MINDS seminar
Title: Theoretical Exploration of Foundation Model Adaptation Methods - (Mar. 2024) The 58th Annual Conference on Information Sciences and Systems @ Princeton University
Title: A Probabilistic Framework for Understanding In-Context Task Learning and Retrieval - (Feb. 2024) 2024 Information Theory and Applications Workshop
Title: The Expressive Power of Low-Rank Adaptation (LoRA) - (Feb. 2024) Foundations of Data Science - Virtual Talk Series @ UCSD/NSF TRIPODS Institute on Emerging CORE Methods in Data Science (EnCORE)
Title: Theoretical Exploration of Foundation Model Adaptation Methods (video) - (Dec. 2023) CSP Seminar @ University of Michigan
Title: Towards a Theoretical Understanding of Parameter-Efficient Fine-Tuning (and Beyond) (video) - (Nov. 2023) Efficient ML workshop @ Google Research New York
Title: The Expressive Power of Low-Rank Adaptation (LoRA)
Openings
- [postdocs] We are looking for a postdoc interested in the theoretical and algorithmic aspects of LLMs. If you want to work with us, please email me your CV and a research statement. I strongly recommend reading our lab’s recent papers.
- [PhD students] We are looking for PhD students (ECE or CS) interested in the theoretical and algorithmic aspects of LLMs. If you want to work with us, please email me your CV and a research statement. I strongly recommend reading our lab’s recent papers.
- [MS students] I am not currently looking for MS students.
- [undergraduate students] Please take my courses first. I teach various machine learning courses (ECE 532, 539, 561, 570, 761, 901, …).