Best-Paper-Writing-Skill

Large language models have achieved remarkable success across a wide range of natural language processing tasks. However, as models continue to grow in size, fully fine-tuning all model parameters for each downstream task becomes increasingly impractical, making it prohibitively expensive to deploy multiple fine-tuned instances. In this paper, we propose Low-Rank Adaptation (LoRA), a novel and efficient approach that freezes the pre-trained model weights and injects trainable low-rank decomposition matrices into each layer of the Transformer architecture, significantly reducing the number of trainable parameters. Extensive experiments demonstrate that LoRA achieves comparable or even better performance than full fine-tuning on RoBERTa, DeBERTa, GPT-2, and GPT-3, while reducing trainable parameters by 10,000 times and GPU memory by 3 times, and introducing no additional inference latency. We also provide an empirical investigation into rank-deficiency in language model adaptation, providing insights into the effectiveness of LoRA. Our code and models are publicly available.

Serving one language model on many tasks means keeping one adapted copy per task, and at the scale of GPT-3 175B each copy is as large as the model itself. Adapter layers shrink the copy but run in sequence with the model, which adds inference latency, while prefix tuning pays for its trainable prefix with part of the input sequence. We observe that the weight change during adaptation has a low intrinsic rank, so a pair of small matrices can represent it. LoRA freezes the pre-trained weights, trains one such pair for each layer, and merges their product into the original weights at deployment. On GPT-3 175B, this efficient recipe cuts trainable parameters by 10,000 times and GPU memory by 3 times relative to fine-tuning with Adam, and it matches or exceeds fine-tuning quality on RoBERTa, DeBERTa, GPT-2 and GPT-3 with no added inference latency. A study of rank deficiency in adaptation explains why such small matrices are enough.

In recent years, large pre-trained language models have achieved remarkable success across a wide range of natural language processing tasks. Many real-world applications rely on adapting a single pre-trained model to multiple downstream tasks, which is typically done through fine-tuning. However, as models become larger and larger, fine-tuning all of the parameters has become increasingly challenging. For example, GPT-3 175B requires storing and deploying a separate copy of 175 billion parameters for each task, making it prohibitively expensive.

To tackle this challenge, many researchers have proposed parameter-efficient adaptation methods, such as adapter layers and prefix tuning. While these methods are effective in reducing the number of trainable parameters, they still suffer from several limitations. Adapter layers introduce additional inference latency, whereas prefix tuning is difficult to optimize and reduces the sequence length available for downstream tasks.

In this paper, we propose Low-Rank Adaptation (LoRA), a novel and efficient method that addresses these limitations. We hypothesize that the weight updates during adaptation also have a low intrinsic rank. Specifically, LoRA freezes the pre-trained weights and injects trainable low-rank matrices into each layer, thereby greatly reducing the number of trainable parameters, making it possible to switch between tasks efficiently.

Many applications adapt one large pre-trained language model to several downstream tasks, and full fine-tuning is the standard recipe. It updates every parameter, so each task ends up with its own copy of the model. The cost grows with the model. For GPT-3 175B, deploying many independent fine-tuned instances, each with 175B parameters, is prohibitively expensive.

Prior work reduces this cost in two ways, and each pays for it elsewhere. Adapter layers insert small trainable modules between Transformer blocks, but the modules must be processed one after another, so every query pays extra latency that is hard to hide when the batch is small. Prefix tuning instead trains a prefix of the input, which is difficult to optimize and occupies part of the sequence length.

Our approach starts from a property of the problem. Over-parameterized models reside on a low intrinsic dimension, so the change in weights during adaptation should also have a low intrinsic rank. LoRA freezes the pre-trained weights and represents the update of each weight matrix by the product of two small matrices. At deployment the product is added to the frozen weights, so inference adds no latency, and switching tasks means swapping the two small matrices.