Full Publications
Preprints, Conference Papers, and Journal Papers.
Selected Publications on Theory and Algorithms for Deep Learning with Foundation Models.
2024
- From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data
Zheyang Xiong, Vasilis Papageorgiou, Kangwook Lee, and Dimitris Papailiopoulos
Arxiv 2024 | Summary | Github - Can MLLMs Perform Text-to-Image In-Context Learning?
Yuchen Zeng*, Wonjun Kang*, Yicong Chen, Hyung Il Koo, and Kangwook Lee
COLM 2024 | Summary | Github - Mini-Batch Optimization of Contrastive Loss
Jaewoong Cho*, Kartik Sreenivasan*, Keon Lee, Kyunghoo Mun, Soheun Yi, Jeong-Gwan Lee, Anna Lee, Jy-yong Sohn, Dimitris Papailiopoulos, Kangwook Lee
Transactions on Machine Learning Research (TMLR) 2024 | Summary | Github - Dual Operating Modes of In-Context Learning
Ziqian Lin and Kangwook Lee
ICML 2024 | Summary | Github - Can Mamba Learn How To Learn? A Comparative Study on In-Context Learning Tasks
Jongho Park, Jaeseung Park, Zheyang Xiong, Nayoung Lee, Jaewoong Cho, Samet Oymak, Kangwook Lee, Dimitris Papailiopoulos
ICML 2024 | Summary | Github - Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding
Seongjun Yang, Gibbeum Lee, Jaewoong Cho, Dimitris Papailiopoulos, Kangwook Lee
Transactions on Machine Learning Research (TMLR) 2024 - Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks
Jy-yong Sohn, Dohyun Kwon, Seoyeon An, Kangwook Lee
UAI 2024 | Summary | Github - The Expressive Power of Low-Rank Adaptation
Yuchen Zeng and Kangwook Lee
ICLR 2024 | Summary | Github - Image Clustering Conditioned on Text Criteria
Sehyun Kwon, Jaeseung Park, Minkyu Kim, Jaewoong Cho, Ernest K. Ryu, and Kangwook Lee
ICLR 2024 | Summary | Github - Teaching Arithmetic to Small Transformers
Nayoung Lee, Kartik Sreenivasan, Jason Lee, Kangwook Lee, and Dimitris Papailiopoulos
ICLR 2024 | Summary | Github - Looped Transformers are Better at Learning Learning Algorithms
Liu Yang, Kangwook Lee, Robert D Nowak, and Dimitris Papailiopoulos
ICLR 2024 | Summary | Github
2023
- DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models (code)
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, Kimin Lee
NeurIPS 2023 - Prompted LLMs as Chatbot Modules for Long Open-domain Conversation (code)
Gibbeum Lee, Volker Hartmann, Jongho Park, Dimitris Papailiopoulos, and Kangwook Lee
ACL 2023 (Findings, Short) - Optimizing DDPM Sampling with Shortcut Fine-Tuning (code)
Ying Fan, Kangwook Lee
ICML 2023 - Looped Transformers as Programmable Computers (code)
Angeliki Giannou*, Shashank Rajput*, Jy-yong Sohn, Kangwook Lee, Jason D. Lee, Dimitris Papailiopoulos
ICML 2023
2022
- Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment (code)
Tuan Dinh, Jy-yong Sohn, Shashank Rajput, Tim Ossowski, Yifei Ming, Junjie Hu, Dimitris Papailiopoulos, and Kangwook Lee
EMNLP 2022 (Findings) - Score-based generative modeling secretly minimizes the Wasserstein distance (code)
Dohyun Kwon, Ying Fan, and Kangwook Lee
NeurIPS 2022 - LIFT: Language-Interfaced FineTuning for Non-Language Machine Learning Tasks (code)
Tuan Dinh*, Yuchen Zeng*, Ruisu Zhang, Ziqian Lin, Michael Gira, Shashank Rajput, Jy-yong Sohn, Dimitris Papailiopoulos, and Kangwook Lee
NeurIPS 2022 - Debiasing Pre-Trained Language Models via Efficient Fine-tuning
Michael Gira, Ruisu Zhang, and Kangwook Lee
ACL 2022 Workshop on Language Technology for Equality, Diversity, Inclusion