Full Publications
Conference Papers and Journal Papers.
Publications on Theory and Algorithms for Foundation Models.
2025
- VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data
Thomas Zeng, Shuibai Zhang, Shutong Wu, Christian Classen, Daewon Chae, Ethan Ewer, Minjae Lee, Heeju Kim, Wonjun Kang, Jackson Kunde, Ying Fan, Jungtaek Kim, Hyung Il Koo, Kannan Ramchandran, Dimitris Papailiopoulos, and Kangwook Lee
ICML 2025 (spotlight) | Summary / Github / HuggingFace - Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
Zheyang Xiong, Ziyang Cai, John Cooper, Albert Ge, Vasilis Papageorgiou, Zack Sifakis, Angeliki Giannou, Ziqian Lin, Liu Yang, Saurabh Agarwal, Grigorios Chrysos, Samet Oymak, Kangwook Lee, and Dimitris Papailiopoulos
ICML 2025 (spotlight) - Parameter-Efficient Fine-Tuning of State Space Models
Kevin Galim, Wonjun Kang, Yuchen Zeng, Hyung Il Koo, and Kangwook Lee
ICML 2025 - Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
Nayoung Lee, Ziyang Cai, Avi Schwarzschild, Kangwook Lee, and Dimitris Papailiopoulos
ICML 2025 - Looped Transformers for Length Generalization
Ying Fan, Yilun Du, Kannan Ramchandran, and Kangwook Lee
ICLR 2025 | Summary | Github - From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data
Zheyang Xiong, Vasilis Papageorgiou, Kangwook Lee, and Dimitris Papailiopoulos
ICLR 2025 | Summary | Github - Rare-to-Frequent: Unlocking Compositional Generation Power of Diffusion Models on Rare Concepts with LLM Guidance
Dongmin Park, Sebin Kim, Taehong Moon, Minkyu Kim, Kangwook Lee, and Jaewoong Cho
ICLR 2025 (spotlight) | Summary | Github - Improving CLIP Counting Accuracy via Parameter-Efficient Fine-Tuning
Ruisu Zhang, Bryce Chen, and Kangwook Lee
Transactions on Machine Learning Research (TMLR) 2025 | Github
2024
- Can MLLMs Perform Text-to-Image In-Context Learning?
Yuchen Zeng*, Wonjun Kang*, Yicong Chen, Hyung Il Koo, and Kangwook Lee
COLM 2024 | Summary | Github - Mini-Batch Optimization of Contrastive Loss
Jaewoong Cho*, Kartik Sreenivasan*, Keon Lee, Kyunghoo Mun, Soheun Yi, Jeong-Gwan Lee, Anna Lee, Jy-yong Sohn, Dimitris Papailiopoulos, Kangwook Lee
Transactions on Machine Learning Research (TMLR) 2024 | Summary | Github - Dual Operating Modes of In-Context Learning
Ziqian Lin and Kangwook Lee
ICML 2024 | Summary | Github - Can Mamba Learn How To Learn? A Comparative Study on In-Context Learning Tasks
Jongho Park, Jaeseung Park, Zheyang Xiong, Nayoung Lee, Jaewoong Cho, Samet Oymak, Kangwook Lee, Dimitris Papailiopoulos
ICML 2024 | Summary | Github - Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding
Seongjun Yang, Gibbeum Lee, Jaewoong Cho, Dimitris Papailiopoulos, Kangwook Lee
Transactions on Machine Learning Research (TMLR) 2024 - Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks
Jy-yong Sohn, Dohyun Kwon, Seoyeon An, Kangwook Lee
UAI 2024 | Summary | Github - The Expressive Power of Low-Rank Adaptation
Yuchen Zeng and Kangwook Lee
ICLR 2024 | Summary | Github - Image Clustering Conditioned on Text Criteria
Sehyun Kwon, Jaeseung Park, Minkyu Kim, Jaewoong Cho, Ernest K. Ryu, and Kangwook Lee
ICLR 2024 | Summary | Github - Teaching Arithmetic to Small Transformers
Nayoung Lee, Kartik Sreenivasan, Jason Lee, Kangwook Lee, and Dimitris Papailiopoulos
ICLR 2024 | Summary | Github - Looped Transformers are Better at Learning Learning Algorithms
Liu Yang, Kangwook Lee, Robert D Nowak, and Dimitris Papailiopoulos
ICLR 2024 | Summary | Github
2023
- DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models (code)
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, Kimin Lee
NeurIPS 2023 - Prompted LLMs as Chatbot Modules for Long Open-domain Conversation (code)
Gibbeum Lee, Volker Hartmann, Jongho Park, Dimitris Papailiopoulos, and Kangwook Lee
ACL 2023 (Findings, Short) - Optimizing DDPM Sampling with Shortcut Fine-Tuning (code)
Ying Fan, Kangwook Lee
ICML 2023 - Looped Transformers as Programmable Computers (code)
Angeliki Giannou*, Shashank Rajput*, Jy-yong Sohn, Kangwook Lee, Jason D. Lee, Dimitris Papailiopoulos
ICML 2023
2022
- Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment (code)
Tuan Dinh, Jy-yong Sohn, Shashank Rajput, Tim Ossowski, Yifei Ming, Junjie Hu, Dimitris Papailiopoulos, and Kangwook Lee
EMNLP 2022 (Findings) - Score-based generative modeling secretly minimizes the Wasserstein distance (code)
Dohyun Kwon, Ying Fan, and Kangwook Lee
NeurIPS 2022 - LIFT: Language-Interfaced FineTuning for Non-Language Machine Learning Tasks (code)
Tuan Dinh*, Yuchen Zeng*, Ruisu Zhang, Ziqian Lin, Michael Gira, Shashank Rajput, Jy-yong Sohn, Dimitris Papailiopoulos, and Kangwook Lee
NeurIPS 2022 - Debiasing Pre-Trained Language Models via Efficient Fine-tuning
Michael Gira, Ruisu Zhang, and Kangwook Lee
ACL 2022 Workshop on Language Technology for Equality, Diversity, Inclusion