Principal Investigator
Kaiyi Ji
吉凯意
Assistant Professor of Computer Science & Engineering
Department of Computer Science and Engineering
kaiyiji [AT] buffalo.edu · 338G Davis Hall, Buffalo, NY
Bio
I am an Assistant Professor in Computer Science and Engineering at the University at Buffalo. Previously, I was a Postdoctoral Researcher in Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor, and a Visiting Student Research Collaborator in Electrical Engineering at Princeton University. I received my Ph.D. in Electrical and Computer Engineering from The Ohio State University, supervised by Prof. Yingbin Liang, and my B.E. from the University of Science and Technology of China.
I received the NSF CAREER Award (2025), CSE Excellence in Research Award (2025), SEAS Early Career Researcher of the Year Award (2026), and UB’s Exceptional Scholar Award – Young Investigator (2026).
Research Interests
Foundations of Machine Learning with Multiple Objectives
- Bilevel optimizationTackling machine learning problems with hierarchical objectives.
- Multi-objective optimizationBalancing multiple, possibly conflicting objectives.
- Use-inspired domain applicationsMulti-label learning to rank for recommendation and neuro-symbolic robot learning.
Efficient Methods for Large Foundation Models
- Multi-task learningScaling LLMs to many tasks through selective parameter sharing and optimized subspaces.
- Continual learningImproving LLM robustness to catastrophic forgetting.
- Retrieval-augmented generation (RAG)Developing reliable retrieval and memory mechanisms for long-context language models.
Recent News
- June 2026 Received UB’s Exceptional Scholar Award – Young Investigator.
- June 2026 LDC-MTL (code) was accepted to ECCV 2026.
- June 2026 Ph.D. student Meng Ding successfully defended her dissertation and will join UMass Boston as an Assistant Professor.
- May 2026 Our dysglycemia risk-screening paper was accepted to the AMIA 2026 Annual Symposium.
- May 2026 Hao and Peiyao joined Amazon as summer research interns.
- May 2026 Space-Aware Crystal Prediction (code) was accepted to TMLR.
- May 2026 Two papers on bilevel-optimization lower bounds and backward knowledge transfer for LLMs (code) were accepted to ICML 2026.
- April 2026 Received the SEAS Early Career Researcher of the Year Award.
- April 2026 Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs (code) was accepted to ACL 2026 Findings.
- March 2026 Serving as an Area Chair for NeurIPS 2026.
- February 2026 Released the open-source DeepMTL2R library with Amazon (paper).
- January 2026 Received a UB Research, Innovation and Economic Development Grant for continual learning research.
- December 2025 Serving on the Organizing Committee for ICBO 2026.
- December 2025 Received the CSE Excellence in Research Award.
- May 2025 Imperative Learning was accepted to IJRR (overview, library).
- April 2025 Received an NSF CAREER Award.
Publications
* indicates equal contribution. See also Google Scholar.
Efficient Methods for Large Language Models
Turning Back Without Forgetting: Selective Backward Refinement for Parameter-Efficient Continual Learning
Anushka Tiwari and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2026.
Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
Hao Ban and Kaiyi Ji.
Association for Computational Linguistics (ACL) 2026 Findings.
Machine Learning with Multiple Objectives
Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy
Chen Wang, Kaiyi Ji, Junyi Geng, Zhongqiang Ren, Taimeng Fu, Fan Yang, Yifan Guo, Haonan He, Xiangyu Chen, Zitong Zhan, Qiwei Du, Shaoshu Su, Bowen Li, Yuheng Qiu, Yi Du, Qihang Li, Yifan Yang, Xiao Lin, Zhipeng Zhao.
The International Journal of Robotics Research (IJRR) 2025.
Fair Resource Allocation in Multi-Task Learning
Hao Ban and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2024.
Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms
Peiyao Xiao*, Hao Ban*, Kaiyi Ji.
Accepted by Conference on Neural Information Processing Systems (NeurIPS) 2023.
Fundamental Theory of Bilevel and Multi-Objective Optimization
Lower Complexity Bounds for Nonconvex-Strongly-Convex Bilevel Optimization with First-Order Oracles
Kaiyi Ji.
International Conference on Machine Learning (ICML) 2026.
Will Bilevel Optimizers Benefit from Loops
Kaiyi Ji, Mingrui Liu, Yingbin Liang, Lei Ying
Conference on Neural Information Processing Systems (NeurIPS) 2022. (Spotlight, 5% acceptance rate)
Lower Bounds and Accelerated Algorithms for Bilevel Optimization
Kaiyi Ji, Yingbin Liang
Journal of Machine Learning Research (JMLR) 2022.
Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms
Kaiyi Ji, Junjie Yang, Yingbin Liang
International Conference on Machine Learning (ICML) 2021.
GRID: Scaling Task-Agnostic Inference in Continual Prompt Tuning
Anushka Tiwari, Sayantan Pal, Rohini K Srihari, Kaiyi Ji.
Submitted, 2026.
LDC-MTL: Balancing Multi-Task Learning through Scalable Loss Discrepancy Control
Peiyao Xiao, Chaosheng Dong, Shaofeng Zou, Kaiyi Ji.
European Conference on Computer Vision (ECCV) 2026.
Beyond the Blood Draw: Explainable Machine Learning for Non-Invasive Dysglycemia Risk Screening
Black Sun, Chenyi Zhang, Kaiyi Ji, Xi Lu.
American Medical Informatics Association (AMIA) 2026 Annual Symposium.
ReciNet: Reciprocal Space-Aware Long-Range Modeling for Crystalline Property Prediction
Jianan Nie, Peiyao Xiao, Kaiyi Ji, Peng Gao.
Transactions on Machine Learning Research (TMLR) 2026.
Turning Back Without Forgetting: Selective Backward Refinement for Parameter-Efficient Continual Learning
Anushka Tiwari and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2026.
Lower Complexity Bounds for Nonconvex-Strongly-Convex Bilevel Optimization with First-Order Oracles
Kaiyi Ji.
International Conference on Machine Learning (ICML) 2026.
Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
Hao Ban and Kaiyi Ji.
Association for Computational Linguistics (ACL) 2026 Findings.
Minimax Optimal Adversarial Reinforcement Learning
Yudan Wang, Kaiyi Ji, Ming Shi and Shaofeng Zou
International Conference on Learning Representations (ICLR) 2026.
Provable Effects of Data Replay in Continual Learning: A Feature Learning Perspective
Meng Ding, Jinhui Xu and Kaiyi Ji
International Conference on Artificial Intelligence and Statistics (AISTATS) 2026.
Understanding Fine-tuning in Approximate Unlearning: A Theoretical Perspective
Meng Ding, Rohan Sharma, Changyou Chen, Jinhui Xu and Kaiyi Ji
Transactions on Machine Learning Research (TMLR) 2025.
Meta-Learning with Heterogeneous Tasks
Zhaofeng Si, Shu Hu, Kaiyi Ji and Siwei Lyu
Generalizing from Limited Resources in the Open World Workshop (GLOW) (Best Paper Candidate Award) International Joint Conferences on Artificial Intelligence Organization (IJCAI) 2025.
SAMO: A Lightweight Sharpness-Aware Approach for Multi-Task Optimization with Joint Global-Local Perturbation
Hao Ban, Gokul Ram Subramani and Kaiyi Ji.
International Conference on Computer Vision (ICCV) 2025.
Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis
Yifan Yang, Hao Ban, Minhui Huang, Shiqian Ma, Kaiyi Ji.
International Conference on Learning Representations (ICLR) 2025.
MGDA Converges under Generalized Smoothness, Provably
Qi Zhang*, Peiyao Xiao*, Shaofeng Zou, Kaiyi Ji.
International Conference on Learning Representations (ICLR) 2025.
Theoretical Study of Conflict-Avoidant Multi-Objective Reinforcement Learning
Yudan Wang, Peiyao Xiao, Hao Ban, Kaiyi Ji, Shaofeng Zou.
IEEE Transactions on Information Theory (TIT) 2025.
Efficiently Escaping Saddle Points in Bilevel Optimization
Minhui Huang, Kaiyi Ji, Shiqian Ma, Lifeng Lai.
Journal of Machine Learning Research (JMLR) 2025.
Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy
Chen Wang, Kaiyi Ji, Junyi Geng, Zhongqiang Ren, Taimeng Fu, Fan Yang, Yifan Guo, Haonan He, Xiangyu Chen, Zitong Zhan, Qiwei Du, Shaoshu Su, Bowen Li, Yuheng Qiu, Yi Du, Qihang Li, Yifan Yang, Xiao Lin, Zhipeng Zhao.
The International Journal of Robotics Research (IJRR) 2025.
First-Order Federated Bilevel Learning
Yifan Yang, Peiyao Xiao, Shiqian Ma, Kaiyi Ji.
AAAI Conference on Artificial Intelligence (AAAI) 2025.
Understanding Forgetting in Continual Learning with Linear Regression
Meng Ding, Kaiyi Ji, Di Wang, Jinhui Xu.
International Conference on Machine Learning (ICML) 2024.
Fair Resource Allocation in Multi-Task Learning
Hao Ban and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2024.
AUC-CL: A Batchsize-Robust Framework for Self-Supervised Contrastive Representation Learning
Rohan Sharma, Kaiyi Ji, Zhiqiang Xu, Changyou Chen.
International Conference on Learning Representations (ICLR) 2024.
Boosting One-Point Derivative-Free Online Optimization via Residual Feedback
Yan Zhang, Yi Zhou, Kaiyi Ji, Yi Shen, and Michael M. Zavlanos.
IEEE Transactions on Automatic Control, 2024.
First-Order Minimax Bilevel Optimization.
Yifan Yang*, Zhaofeng Si*, Siwei Lyu, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2024.
Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms
Peiyao Xiao*, Hao Ban*, Kaiyi Ji.
Accepted by Conference on Neural Information Processing Systems (NeurIPS) 2023.
SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning
Yifan Yang, Peiyao Xiao, Kaiyi Ji.
Accepted by Conference on Neural Information Processing Systems (NeurIPS) 2023. (Spotlight, 5% acceptance rate)
Achieving O(\epsilon^{-1.5}) Complexity in Hessian/Jacobian-free Stochastic Bilevel Optimization
Yifan Yang, Peiyao Xiao, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
Non-Convex Bilevel Optimization with Time-Varying Objective Functions
Sen Lin, Daouda Sow, Kaiyi Ji, Yingbin Liang, Ness Shroff.
Accepted by Conference on Neural Information Processing Systems (NeurIPS) 2023.
Bilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm
Jie Hao, Kaiyi Ji, Mingrui Liu.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
Network Utility Maximization with Unknown Utility Functions: A Distributed, Data-Driven Bilevel Optimization Approach
Kaiyi Ji and Lei Ying.
ACM MobiHoc 2023!
Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiation
. Peiyao Xiao and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2023.
Achieving Linear Speedup in Non-IID Federated Bilevel Learning
Minhui Huang*, Dewei Zhang*, Kaiyi Ji.
International Conference on Machine Learning (ICML) 2023.
On the Convergence Theory for Hessian-Free Bilevel Algorithms
Daouda Sow, Kaiyi Ji, Yingbin Liang
Conference on Neural Information Processing Systems (NeurIPS) 2022.
Will Bilevel Optimizers Benefit from Loops
Kaiyi Ji, Mingrui Liu, Yingbin Liang, Lei Ying
Conference on Neural Information Processing Systems (NeurIPS) 2022. (Spotlight, 5% acceptance rate)
Lower Bounds and Accelerated Algorithms for Bilevel Optimization
Kaiyi Ji, Yingbin Liang
Journal of Machine Learning Research (JMLR) 2022.
Data Sampling Affects the Complexity of Online SGD over Dependent Data
Shaocong Ma, Ziyi Chen, Yi Zhou, Kaiyi Ji, Yingbin Liang.
Conference on Uncertainty in Artificial Intelligence (UAI) 2022.
Provably Faster Algorithms for Bilevel Optimization
Junjie Yang, Kaiyi Ji, Yingbin Liang
Conference on Neural Information Processing Systems (NeurIPS) 2021. (Spotlight, 3% acceptance rate)
Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms
Kaiyi Ji, Junjie Yang, Yingbin Liang
International Conference on Machine Learning (ICML) 2021.
A New One-Point Residual-Feedback Oracle For Black-Box Learning and Control
Yan Zhang, Yi Zhou, Kaiyi Ji, Michael M. Zavlanos
Automatica 2021.
Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning
Kaiyi Ji, Junjie Yang, Yingbin Liang
Journal of Machine Learning Research (JMLR) 2021.
Understanding Estimation and Generalization Error of Generative Adversarial Networks
Kaiyi Ji, Yi Zhou, Yingbin Liang
IEEE Trans. on Information Theory (TIT), 2021.
When Will Gradient Methods Converge to Max-margin Classifier under ReLU Models?
Tengyu Xu, Yi Zhou, Kaiyi Ji, Yingbin Liang
Stat, Special Issue of Deep Learning from Statistical Perspective, 2021
Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters
Kaiyi Ji, Jason D. Lee, Yingbin Liang, H. Vincent Poor
Conference on Neural Information Processing Systems (NeurIPS) 2020.
History-Gradient Aided Batch Size Adaptation for Variance Reduced Algorithms
Kaiyi Ji, Zhe Wang, Bowen Weng, Yi Zhou, Wei Zhang, Yingbin Liang,
International Conference on Machine Learning (ICML) 2020.
Learning Latent Features with Pairwise Penalties in Matrix Completion
Kaiyi Ji, Jian Tan, Yuejie Chi, Jinfeng Xu
IEEE Trans. on Signal Processing (TSP), 2020 Short version as an invited paper to a special session at the IEEE SAM 2020
Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization
Yi Zhou, Zhe Wang, Kaiyi Ji, Yingbin Liang, Vahid Tarokh
International Joint Conferences on Artificial Intelligence Organization (IJCAI) 2020.
Robust Stochastic Bandit Algorithms under Probabilistic Unbounded Adversarial Attack
Ziwei Guan, Kaiyi Ji, Donald J. Bucci Jr, Timothy Hu, Joseph Palombo, Michael Liston, Yingbin Liang
AAAI Conference on Artificial Intelligence (AAAI) 2020.
SpiderBoost and Momentum: Faster Stochastic Variance Reduction Algorithms
Zhe Wang, Kaiyi Ji, Yi Zhou, Yingbin Liang, Vahid Tarokh
Conference on Neural Information Processing Systems (NeurIPS) 2019.
Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization
Kaiyi Ji, Zhe Wang, Yi Zhou, Yingbin Liang
International Conference on Machine Learning (ICML) 2019.
Minimax Estimation of Neural Net Distance
Kaiyi Ji, Yingbin Liang
Conference on Neural Information Processing Systems (NeurIPS) 2018.
Asymptotic Miss Ratio of LRU Caching with Consistent Hashing
Kaiyi Ji, Guocong Quan, Jian Tan
IEEE INFOCOM 2018.
LRU Caching with Dependent Competing Requests
Guocong Quan, Kaiyi Ji, Jian Tan
IEEE INFOCOM 2018.
On Resource Pooling and Separation for LRU Caching
Jian Tan, Guocong Quan, Kaiyi Ji, Ness Shroff
ACM SIGMETRICS 2018.
On Resource Pooling and Separation for LRU Caching
Jian Tan, Guocong Quan, Kaiyi Ji, Ness Shroff
Proceedings of the ACM on Measurement and Analysis of Computing Systems (POMACS), 2018
Group Members
Ph.D. Students
- Peiyao Xiao (2023–present)
- Hao Ban (2023–present)
- Anushka Tiwari (2025–present)
- Qiang Hu (2025–present)
- Ziming Yang (starting Fall 2026)
Student Intern
- Keqin Che
Graduated Students
- Meng Ding, Ph.D. 2026; Assistant Professor at UMass Boston
- Yujie Zhu, M.S. 2026
- Gokul Ram Subramani, M.S. 2025; Meta
Teaching
- Fall 2026 — CSE 676: Deep Learning
- Spring 2025 — CSE 676: Deep Learning
- Fall 2024 — CSE 705 SEM: Recent Progresses in Machine Learning
- Spring 2024 — CSE 676 A: Deep Learning
- Fall 2023 — CSE 676: Deep Learning
- Spring 2023 — CSE 701: Recent Progresses in Machine Learning
- Fall 2022 — CSE 712: Optimization for Modern Machine Learning
Awards
- UB Exceptional Scholar Award – Young Investigator (2026)
- SEAS Early Career Researcher of the Year Award (2026)
- CSE Excellence in Research Award (2025)
- NSF CAREER Award (2025)
- Best Paper Candidate Award, IJCAI GLOW Workshop (2025)
- UB CSE Junior Faculty Research Award (2023)
Service
- Area Chair: ICLR 2026 and NeurIPS 2026
- Senior Program Committee: AAAI 2027
- Program Committee: MobiHoc 2023–2026 and IEEE ITW 2025
- Reviewer: ICML, NeurIPS, ICLR, AISTATS, CVPR, ICCV, UAI, IJCAI, and AAAI
- Associate Chair: Graduate Student Admission Committee
- Co-organizer: CSExplore Committee for K–12 Education
Research Support
Our group is grateful for support from the National Science Foundation and the University at Buffalo.
Principal Investigator · Collaborative Research
Principal Investigator · Collaborative Research