Link Search Menu Expand Document

Lee Lab @ UW Madison

Research focus: Theory and algorithms for deep learning with foundation models.

We sincerely appreciate the support provided by our sponsors: NSF, Amazon, American Family Insurance, and FuriosaAI.

Openings

  • [postdocs] We are looking for a postdoc interested in the theoretical and algorithmic aspects of foundation models, particularly LLMs. If you want to work with us, please email me your CV and a short research statement. I strongly recommend reading our lab’s recent papers.
  • [PhD students] We are looking for PhD students (ECE or CS) interested in the theoretical and algorithmic aspects of foundation models, particularly LLMs. If you want to work with us, please email me your CV and a short research statement. I strongly recommend reading our lab’s recent papers.
  • [undergraduate students] You might want to consider taking my class first. I teach various machine learning courses (ECE 532, 539, 561, 570, 761, …).

News

  • (Apr. 2024) Amazon Research Awards
    Our group will develop principled approaches to prompt engineering through the information/coding-theoretic lens. Huge thanks to Amazon and my amazing collaborators and students!
  • (Mar. 2024) NSF CAREER Award
    Our group will develop a unified theory and new algorithms with provable guarantees for learning with frozen pretrained models, also known as foundation models. Huge thanks to NSF and my amazing collaborators and students!

Selected Work

LinkTopic/TypeTLDRSummaryGithub
ICML’24LLM/TheoryDual Operating Modes of In-Context LearningSummaryGithub
ICML’24LLM/AlgorithmCan Mamba Learn How To Learn? A Comparative Study on In-Context Learning TasksSummaryGithub
Arxiv’24LLM/AlgorithmCan MLLMs Perform Text-to-Image In-Context Learning?SummaryGithub
UAI’24PEFT/TheoryMemorization Capacity for Additive Fine-TuningSummaryGithub
ICLR’24PEFT/TheoryThe Expressive Power of Low-Rank Adaptation (LoRA)SummaryGithub
ICLR’24LLM/AlgorithmImage Clustering Conditioned on Text CriteriaSummaryGithub
ICLR’24LLM/AlgorithmTeaching Arithmetic to a Small TransformerSummaryGithub
ICLR’24LLM/AlgorithmA Looped-Transformer Architecture for Efficient Meta-learningSummaryGithub
NeurIPS’23Diffusion/AlgorithmReinforcement learning for improved text-to-image alignmentSummaryGithub
ICML’23LLM/TheoryLooped Transformers as Programmable ComputersSummaryGithub
ICML’23Diffusion/AlgorithmReinforcement learning for faster DDPM samplingSummaryGithub
NeurIPS’22LLM/AlgorithmLIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning TasksSummaryGithub
NeurIPS’22Diffusion/TheoryScore-based Generative Modeling Secretly Minimizes the Wasserstein DistanceSummaryGithub

Selected Talks on Deep Learning with Foundation Models

  • (Apr. 2024) The Johns Hopkins University CIS/MINDS seminar
    Title: Theoretical Exploration of Foundation Model Adaptation Methods
  • (Mar. 2024) The 58th Annual Conference on Information Sciences and Systems @ Princeton University
    Title: A Probabilistic Framework for Understanding In-Context Task Learning and Retrieval
  • (Feb. 2024) 2024 Information Theory and Applications Workshop
    Title: The Expressive Power of Low-Rank Adaptation (LoRA)
  • (Feb. 2024) Foundations of Data Science - Virtual Talk Series @ UCSD/NSF TRIPODS Institute on Emerging CORE Methods in Data Science (EnCORE)
    Title: Theoretical Exploration of Foundation Model Adaptation Methods
  • (Dec. 2023) CSP Seminar @ University of Michigan
    Title: Towards a Theoretical Understanding of Parameter-Efficient Fine-Tuning (and Beyond)
  • (Nov. 2023) Efficient ML workshop @ Google Research New York
    Title: The Expressive Power of Low-Rank Adaptation (LoRA)