Link Search Menu Expand Document

Lee Lab @ UW Madison

Research focus: Theory and algorithms for deep learning with foundation models.

Selected Work

LinkTopic/TypeTLDRSummaryGithub
Arxiv’24LLM/TheoryDual Operating Modes of In-Context LearningSummaryGithub
Arxiv’24LLM/AlgorithmCan MLLMs Perform Text-to-Image In-Context Learning?SummaryGithub
ICLR’24PEFT/TheoryThe Expressive Power of Low-Rank Adaptation (LoRA)SummaryGithub
ICLR’24LLM/AlgorithmImage Clustering Conditioned on Text CriteriaSummaryGithub
ICLR’24LLM/AlgorithmTeaching Arithmetic to a Small TransformerSummaryGithub
ICLR’24LLM/AlgorithmA Looped-Transformer Architecture for Efficient Meta-learningSummaryGithub
NeurIPS’23Diffusion/AlgorithmReinforcement learning for improved text-to-image alignmentSummaryGithub
ICML’23LLM/TheoryLooped Transformers as Programmable ComputersSummaryGithub
ICML’23Diffusion/AlgorithmReinforcement learning for faster DDPM samplingSummaryGithub
NeurIPS’22LLM/AlgorithmLIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning TasksSummaryGithub
NeurIPS’22Diffusion/TheoryScore-based Generative Modeling Secretly Minimizes the Wasserstein DistanceSummaryGithub

Selected Talks on Deep Learning with Foundation Models

  • (Feb. 2024) 2024 Information Theory and ApplicationsWorkshop
    Title: The Expressive Power of Low-Rank Adaptation (LoRA)
  • (Feb. 2024) Foundations of Data Science - Virtual Talk Series @ UCSD/NSF TRIPODS Institute on Emerging CORE Methods in Data Science (EnCORE)
    Title: Theoretical Exploration of Foundation Model Adaptation Methods
  • (Dec. 2023) CSP Seminar @ University of Michigan
    Title: Towards a Theoretical Understanding of Parameter-Efficient Fine-Tuning (and Beyond)
  • (Nov. 2023) Efficient ML workshop @ Google Research New York
    Title: The Expressive Power of Low-Rank Adaptation (LoRA)
  • (Oct. 2023) Trust Perspectives in Machine Learning, Law, and Public Policy at the Institute for Data, Econometrics, Algorithms, and Learning (IDEAL) @ Northwestern University
  • (Oct. 2023) Distinguished Lectures in Microbiology @ University of Wisconsin-Madison
  • (May 2023) KSEA Distinguished Guest Series
  • (Feb. 2023) Information Theory and Applications Workshop
  • (Feb. 2023) The Coordinated Science Laboratory Student Conference @ UIUC
  • (Jan. 2023) Information Theory and Data Science Workshop @ National University of Singapore
  • (Jan. 2023) Systems, Information, Learning and Optimization (SILO) Seminar @ University of Wisconsin-Madison

News

  • (Mar. 2024) NSF CAREER Award
    Our group will develop a unified theory and new algorithms with provable guarantees for learning with frozen pretrained models, also known as foundation models. Huge thanks to NSF and my amazing collaborators and students!
  • (Feb. 2024) One paper is accepted to [TMLR]
  • (Jan. 2024) Four papers are accepted to [ICLR’24]
  • (Sep. 2023) One paper is accepted to [NeurIPS’23]
  • (May 2023) One paper is accepted to [ACL’23 (Findings)]
  • (Apr. 2023) Three papers are accepted to [ICML’23]