Finetuning LLMs cost-efficiently

Abstract：

Finetuning makes the power of LLMs accessible to everyone, and is poised to serve as the “last mile” in deploying these models across a wide range of applications. This talk focuses on a line of our recent works on cost-efficient methods to further lower the barriers to finetuning, while respecting critical aspects such as computational and memory efficiency, data privacy, and human alignment.

We will explore three practical scenarios. i) For users with access to one or two commercial-grade GPUs,  we focus on how to fully exploit the capabilities of LoRA, maximizing the utility of each trainable parameter. ii) In settings where only consumer-level GPUs are available, we will discuss memory-efficient finetuning approaches that also incorporate privacy-preserving mechanisms to safeguard individual data privacy. iii) Fianally, we will talk about transfer learning methods that reduce reliance on human-annotated preference data, making alignment more feasible in low-data domains.

All approaches discussed in this talk are backed by strong theoretical guarantees, hopefully offering insights into their efficiency.

Short bio.：

Bingcong Li (https://bingcongli.github.io/) received the B.Eng degree (with highest honors) in Information Science and Engineering from Fudan University in 2017, and the Ph.D. degree in Electrical and Computer Engineering from the University of Minnesota in 2022. He is now a post-doctoral research associate with ETH Zurich, Switzerland. His research interests lie in optimization with application to pretraining and finetuning language models. He received the National Scholarship twice from China in 2014 and 2015, and UMN ECE Department Fellowship in 2017.