| # TOWARDS A UNIFIED VIEW OF PARAMETER-EFFICIENT TRANSFER LEARNING | |
| **Junxian He[∗]** | |
| Carnegie Mellon University | |
| junxianh@cs.cmu.edu | |
| **Xuezhe Ma** | |
| University of Southern California | |
| xuezhema@isi.edu | |
| **Chunting Zhou[∗]** | |
| Carnegie Mellon University | |
| chuntinz@cs.cmu.edu | |
| **Graham Neubig** | |
| Carnegie Mellon University | |
| gneubig@cs.cmu.edu | |
| **Taylor Berg-Kirkpatrick** | |
| UC San Diego | |
| tberg@eng.ucsd.edu | |
| ABSTRACT | |
| Fine-tuning large pretrained language models on downstream tasks has become | |
| the de-facto learning paradigm in NLP. However, conventional approaches finetune all the parameters of the pretrained model, which becomes prohibitive as | |
| the model size and the number of tasks grow. Recent work has proposed a variety of parameter-efficient transfer learning methods that only fine-tune a small | |
| number of (extra) parameters to attain strong performance. While effective, the | |
| critical ingredients for success and the connections among the various methods | |
| are poorly understood. In this paper, we break down the design of state-of-the-art | |
| parameter-efficient transfer learning methods and present a unified framework that | |
| establishes connections between them. Specifically, we re-frame them as modifications to specific hidden states in pretrained models, and define a set of design | |
| dimensions along which different methods vary, such as the function to compute | |
| the modification and the position to apply the modification. Through comprehensive empirical studies across machine translation, text summarization, language | |
| understanding, and text classification benchmarks, we utilize the unified view to | |
| identify important design choices in previous methods. Furthermore, our unified | |
| framework enables the transfer of design elements across different approaches, | |
| and as a result we are able to instantiate new parameter-efficient fine-tuning methods that tune less parameters than previous methods while being more effective, | |
| achieving comparable results to fine-tuning all parameters on all four tasks.[1] | |
| 1 INTRODUCTION | |
| Transfer learning from pre-trained language models (PLMs) is now the prevalent paradigm in natural | |
| language processing, yielding strong performance on many tasks (Peters et al., 2018; Devlin et al., | |
| 2019; Qiu et al., 2020). The most common way to adapt general-purpose PLMs to downstream tasks | |
| is to fine-tune all the model parameters (full fine-tuning). However, this results in a separate copy of | |
| fine-tuned model parameters for each task, which is prohibitively expensive when serving models | |
| that perform a large number of tasks. This issue is particularly salient with the ever-increasing size | |
| of PLMs, which now range from hundreds of millions (Radford et al., 2019; Lewis et al., 2020) to | |
| hundreds of billions (Brown et al., 2020) or even trillions of parameters (Fedus et al., 2021). | |
| To mitigate this issue, a few lightweight alternatives have been proposed to update only a small number of extra parameters while keeping most pretrained parameters frozen. For example, adapter tun_ing (Houlsby et al., 2019) inserts small neural modules called adapters to each layer of the pretrained_ | |
| network and only the adapters are trained at fine-tuning time. Inspired by the success of prompting | |
| methods that control PLMs through textual prompts (Brown et al., 2020; Liu et al., 2021a), prefix | |
| _tuning (Li & Liang, 2021) and prompt tuning (Lester et al., 2021) prepend an additional l tunable_ | |
| _∗Equal Contribution. Order determined by random dice rolling._ | |
| [1Code is available at https://github.com/jxhe/unify-parameter-efficient-tuning.](https://github.com/jxhe/unify-parameter-efficient-tuning) | |
| ----- | |
| **Adapter** | |
| + | |
| **_WgkUfBdVXrwPGIAf+EOMszp6GBfgP6wbL1CGYyzke6wN7YZgaLv0C5z/Bw39/esPG9vN/6m0enHzfmv7X7a2Lz5s/G3+l9Mv1n7h7V/XHuztr32r2t/WDta61drwVrs7X/Xvuftf/9+OHj/Uf/Y2BMf/2rqs/frzk/H+P/B9Wg0mE=</latexit>_** up | |
| **Nonlinear** | |
| **_WxHv7r65+/2dhu/k+l1Yvbt1vbv9/avxu4+71f9i+vnar9Z+vfZqbXvtv9b+uHa0drF2sxasPa39z9r/rv313e/f9d8F78CY/vQnVZ/WHN+3qX/B3kw0g=</latexit>_** down | |
| **Prefix Tuning** | |
| 22 | |
| 21 | |
| 20 | |
| 19 | |
| **_Pnh0i9w/BM8/NfXP3+zsd38n0qrFzdvt7b/c2v74ruNP+5W/4vpF2v/vabtVdr2v/tfbHtaO13tr1WrAWr/3f2v+v/fndq3dn767f3ZrQn/6kqvNva87Pu/+KxhAzTo=</latexit>_** _k_ | |
| **_P6t3Zu+t3tyb0pz+p6vzbmvPz7r/CmHnzU=</latexit>_** _v_ | |
| 18 | |
| |P2v/tfbHtaO13tr1WrAWr/3f2v+v/fndq3dn767f3ZrQn/6kqvNva87Pu/+KxhAzTo=</latexit> k|KRhzFc=</latexit>| | |
| |---|---| | |
| |Pq+G7j7vV/2L6xdq/r/1m7dXa9tp/rf1x7Wit3a9FqzFa/+39v9rf3736t3Zu+t3tyb0pz+p6vzbmvPz7r/CmHnzU=</latexit> v|VefDXwGeE8xi</latexit>| | |
| |---|---| | |
| |W27ebm3/59b2xXcbf9yt/hfTL9b+fe03a6/Wtf+a+2Pa0drvbXrtWAtXvu/tf9f+/O7V+/O3l2/uzWhP/1JVef1pyfd/9V8xyzUc=</latexit> q|̴LoRA| | |
| |---|---| | |
| |Wvu/tf9f+/O7V+/O3l2/uzWhP/1JVef1pyfd/9V40BzUE=</latexit> k|̴LoR| | |
| |---|---| | |
| **_Wz9r/rv313e/f9d8F78CY/vQnVZ/WHN+3qX/B3kw0g=</latexit>_** down **_WgkUfBdVXrwPGIAf+EOMszp6GBfgP6wbL1CGYyzke6wN7YZgaLv0C5z/Bw39/esPG9vN/6m0enHzfmv7X7a2Lz5s/G3+l9Mv1n7h7V/XHuztr32r2t/WDta61drwVrs7X/Xvuftf/9+OHj/Uf/Y2BMf/2rqs/frzk/H+P/B9Wg0mE=</latexit>_** up | |
| _⇥_ | |
| Add & Layer Norm x L | |
| Adapter | |
| Feed Forward | |
| Add & Layer Norm | |
| Adapter | |
| Attention | |
| **_Q7tVdr2t/XPvL2tHaxdr1WrD2Ze1/1v537f/e/eu7rveu2sT+tOfVHX+Zc35efDXwGT2Mxd</latexit>_** **_P2v/tfbHtaO13tr1WrAWr/3f2v+v/fndq3dn767f3ZrQn/6kqvNva87Pu/+KxhAzTo=</latexit>_** _k_ **_KRhzFc=</latexit>_** **_Pq+G7j7vV/2L6xdq/r/1m7dXa9tp/rf1x7Wit3a9FqzFa/+39v9rf3736t3Zu+t3tyb0pz+p6vzbmvPz7r/CmHnzU=</latexit>_** _v_ **_VefDXwGeE8xi</latexit>_** | |
| **_W27ebm3/59b2xXcbf9yt/hfTL9b+fe03a6/Wtf+a+2Pa0drvbXrtWAtXvu/tf9f+/O7V+/O3l2/uzWhP/1JVef1pyfd/9V8xyzUc=</latexit>_** _q_ +̴LoRA **_Wvu/tf9f+/O7V+/O3l2/uzWhP/1JVef1pyfd/9V40BzUE=</latexit>_** _k_ +̴LoRA **_Wvj2tFab+16LViL1/5v7f/X/vzu1buzd9fvbk3oT39S1fm3Nefn3X/FdaozUw=</latexit>_** _v_ | |
| Hidden States | |
| **Multi-Head** | |
| Figure 1: Illustration of the transformer architecture | |
| and several state-of-the-art parameter-efficient tuning | |
| methods. We use blocks with dashed borderlines to | |
| represent the added modules by those methods. | |
| |in|e-tuning 21.9|4 Ours 21.90|Col4|Col5| | |
| |---|---|---|---|---| | |
| |P|refix Tuning 2|Adapte 0.46|r 20.98 LoRA 20.50|| | |
| |||||| | |
| |||||| | |
| ||BitFit 17.32|||| | |
| 10 15 | |
| Fine-tuned Parameters (%) | |
| Figure 2: Performance of different methods on the | |
| XSum (Narayan et al., 2018) summarization task. | |
| The number of fine-tuned parameters is relative to | |
| the tuned parameters in full fine-tuning. | |
| prefix tokens to the input or hidden layers and only train these soft prompts when fine-tuning on | |
| downstream tasks. More recently, Hu et al. (2021) learn low-rank matrices to approximate parameter updates. We illustrate these methods in Figure 1. These approaches have all been reported to | |
| demonstrate comparable performance to full fine-tuning on different sets of tasks, often through updating less than 1% of the original model parameters. Besides parameter savings, parameter-efficient | |
| tuning makes it possible to quickly adapt to new tasks without catastrophic forgetting (Pfeiffer et al., | |
| 2021) and often exhibits superior robustness in out-of-distribution evaluation (Li & Liang, 2021). | |
| However, we contend that the important ingredients that contribute to the success of these parameterefficient tuning methods are poorly understood, and the connections between them are still unclear. | |
| In this paper, we aim to answer three questions: (1) How are these methods connected? (2) Do these | |
| methods share design elements that are essential for their effectiveness, and what are they? (3) Can | |
| the effective ingredients of each method be transferred to others to yield more effective variants? | |
| In order to answer these questions, we first derive an alternative form of prefix tuning that reveals | |
| prefix tuning’s close connections with adapters (§3.1). Based on this we then devise a unified framework that frames the aforementioned methods as different ways to modify the hidden representations | |
| of frozen PLMs (§3.2). Our unified framework decomposes previous methods along a shared set | |
| of design dimensions, such as the function used to perform the modification, the position in which | |
| to impose this modification, and how to integrate the modification. This framework allows us to | |
| transfer design choices across approaches to propose new variants such as adapters with multiple | |
| heads (§3.3). In experiments, we first show that existing parameter-efficient tuning methods still | |
| lag behind full fine-tuning on higher-resource and challenging tasks (§4.2), as exemplified in Figure 2. Then we utilize the unified framework to identify critical design choices and validate the | |
| proposed variants empirically (§4.3-4.6). Our experiments on four NLP benchmarks covering text | |
| summarization, machine translation (MT), text classification, and general language understanding, | |
| demonstrate that the proposed variant uses less parameters than existing methods while being more | |
| effective, matching full fine-tuning results on all four tasks. | |
| 2 PRELIMINARIES | |
| 2.1 RECAP OF THE TRANSFORMER ARCHITECTURE | |
| The transformer model (Vaswani et al., 2017) is now the workhorse architecture behind most stateof-the-art PLMs. In this section we recap the equations of this model for completeness. Transformer | |
| models are composed of L stacked blocks, where each block (Figure 1) contains two types of sub | |
| ----- | |
| layers: multi-head self-attention and a fully connected feed-forward network (FFN).[2] The conventional attention function maps queries Q ∈ R[n][×][d][k] and key-value pairs K ∈ R[m][×][d][k] _, V ∈_ R[m][×][d][v] : | |
| Attn(Q, K, V ) = softmax( **_[QK]√dk[T]_** )V, (1) | |
| where n and m are the number of queries and key-value pairs respectively. Multi-head attention performs the attention function in parallel over Nh heads, where each head is separately parameterized | |
| by Wq[(][i][)][,][ W][ (]k[i][)][,][ W][ (]v _[i][)]_ _∈_ R[d][×][d][h] to project inputs to queries, keys, and values. Given a sequence of | |
| _m vectors C ∈_ R[m][×][d] over which we would like to perform attention and a query vector x ∈ R[d], | |
| multi-head attention (MHA) computes the output on each head and concatenates them:[3] | |
| MHA(C, x) = Concat(head1, · · ·, headh)Wo, headi = Attn(xWq[(][i][)][,][ CW]k[ (][i][)][,][ CW]v[ (][i][)][)][,] (2) | |
| whereparameters, which indicates that each attention head is operating on a lower-dimensional space. The Wo ∈ R[d][×][d]. d is the model dimension, and in MHA dh is typically set to d/Nh to save | |
| other important sublayer is the fully connected feed-forward network (FFN) which consists of two | |
| linear transformations with a ReLU activation function in between: | |
| FFN(x) = ReLU(xW1 + b1)W2 + b2, (3) | |
| wherea residual connection is used followed by layer normalization (Ba et al., 2016). W1 ∈ R[d][×][d][m], W2 ∈ R[d][m][×][d]. Transformers typically use a large dm, e.g. dm = 4d. Finally, | |
| 2.2 OVERVIEW OF PREVIOUS PARAMETER-EFFICIENT TUNING METHODS | |
| Below and in Figure 1, we introduce several state-of-the-art parameter-efficient tuning methods. | |
| Unless otherwise specified, they only tune the added parameters while the PLM’s are frozen. | |
| **Adapters (Houlsby et al., 2019):** The adapter approach inserts small modules (adapters) between | |
| transformer layers. The adapter layer generally uses a down-projection withproject the input h to a lower-dimensional space specified by bottleneck dimension Wdown r, followed by ∈ R[d][×][r] to | |
| a nonlinear activation functionsurrounded by a residual connection, leading to a final form: f (·), and a up-projection with Wup ∈ R[r][×][d]. These adapters are | |
| **_h ←_** **_h + f_** (hWdown)Wup. (4) | |
| Houlsby et al. (2019) places two adapters sequentially within one layer of the transformer, one after | |
| the multi-head attention and one after the FFN sub-layer. Pfeiffer et al. (2021) have proposed a more | |
| efficient adapter variant that is inserted only after the FFN “add & layer norm” sub-layer. | |
| **Prefix Tuning (Li & Liang, 2021):** Inspired by the success of textual prompting methods (Liu | |
| et al., 2021a), prefix tuning prepends l tunable prefix vectors to the keys and values of the multihead attention at every layer. Specifically, two sets of prefix vectorswith the original key K and value V . Then multi-head attention is performed on the new prefixed Pk, Pv ∈ R[l][×][d] are concatenated | |
| keys and values. The computation of headi in Eq. 2 becomes: | |
| headi = Attn(xWq[(][i][)][,][ concat][(][P][ (]k[i][)][,][ CW]k[ (][i][)][)][,][ concat][(][P][ (]v _[i][)][,][ CW]v[ (][i][)][))][,]_ (5) | |
| **_Pk and Pv are split into Nh head vectors respectively and Pk[(][i][)][,][ P][ (]v_** _[i][)]_ _∈_ R[l][×][d/N][h] denote the i-th | |
| head vector. Prompt-tuning (Lester et al., 2021) simplifies prefix-tuning by only prepending to the | |
| input word embeddings in the first layer; similar work also includes P-tuning (Liu et al., 2021b). | |
| **LoRA (Hu et al., 2021):** LoRA injects trainable low-rank matrices into transformer layers to | |
| approximate the weight updates. For a pre-trained weight matrix W ∈ R[d][×][k], LoRA represents its | |
| update with a low-rank decomposition W +∆W = W +WdownWup, where Wdown ∈ R[d][×][r], Wup ∈ | |
| R[r][×][k] are tunable parameters. LoRA applies this update to the query and value projection matrices | |
| (Wq, Wv) in the multi-head attention sub-layer, as shown in Figure 1. For a specific input x to the | |
| linear projection in multi-head attention, LoRA modifies the projection output h as: | |
| **_h ←_** **_h + s · xWdownWup,_** (6) | |
| 2In an encoder-decoder architecture, the transformer decoder usually has another multi-head cross-attention | |
| module between the self-attention and FFN, which we omit here for simplicity. | |
| 3Below, we sometimes ignore the head index i to simplify notation when there is no confusion. | |
| ----- | |
| |Add Scaling hE9/UtX5lzXn590PfwVch8x0</latexit> W9dKsKvMlzLa8vUkm1PJkqgAMJt4+fiJO4fCG2bZw9a685DnPFenc1b8M2q8CPUA8O9gU19z1Cdk8YQr9pdarWimf79gkUfBdVXrwPGIAf+EOMszp6GBfgP6wbL1CGYyzke6wN7YZgaLv0C5z/Bw39/esPG9vN/6m0enHzfmv7X7a2Lz5s/G3+l9Mv1n7h7V/XHuztr32r2t/WDta61drwVrs7X/Xvuftf/9+OHj/Uf/Y2BMf/2rqs/frzk/H+P/B9Wg0mE=</latexit> up PLM module Wz9r/rv313e/f9d8F78CY/vQnVZ/WHN+3qX/B3kw0g=</latexit> down x1/WXN+3v3wV7B0zIQ=</latexit>|Col2|Scaling| | |
| |---|---|---| | |
| ||hE9/UtX5lzXn590PfwVch8x0</latexit>|| | |
| |Add Scaling hE9/UtX5lzXn590PfwVch8x0</latexit> W9dKsKvMlzLa8vUkm1PJkqgAMJt4+fiJO4fCG2bZw9a685DnPFenc1b8M2q8CPUA8O9gU19z1Cdk8YQr9pdarWimf79gkUfBdVXrwPGIAf+EOMszp6GBfgP6wbL1CGYyzke6wN7YZgaLv0C5z/Bw39/esPG9vN/6m0enHzfmv7X7a2Lz5s/G3+l9Mv1n7h7V/XHuztr32r2t/WDta61drwVrs7X/Xvuftf/9+OHj/Uf/Y2BMf/2rqs/frzk/H+P/B9Wg0mE=</latexit> up PLM module ReLU Wz9r/rv313e/f9d8F78CY/vQnVZ/WHN+3qX/B3kw0g=</latexit> down x1/WXN+3v3wV7B0zIQ=</latexit>|Col2|Scaling| | |
| |---|---|---| | |
| ||hE9/UtX5lzXn590PfwVch8x0</latexit>|| | |
| Add Gating & Add Add Add Add | |
| Scaling Scaling | |
| **_hF2vXa8Hal7X/Wfvftf9796/vu96765N6E9/UtX5lzXn590PfwVch8x0</latexit>_** **_WIAf+EOMszp6GBfgP6wbL1CGYyzke6wN7YZgaLv0C5z/Bw39/esPG9vN/6m0enHzfmv7X7a2Lz5s/G3+l9Mv1n7h7V/XHuztr32r2t/WDta61drwVrs7X/Xvuftf/9+OHj/Uf/Y2BMf/2rqs/frzk/H+P/B9Wg0mE=</latexit>_** up **_ht/GW3+l9Mv1j73drv16tba/9ce0va0drF2vXa8Hal7X/Wfvftf9796/vu96765N6E9/UtX5lzXn590PfwVch8x0</latexit>_** **_WIAf+EOMszp6GBfgP6wbL1CGYyzke6wN7YZgaLv0C5z/Bw39/esPG9vN/6m0enHzfmv7X7a2Lz5s/G3+l9Mv1n7h7V/XHuztr32r2t/WDta61drwVrs7X/Xvuftf/9+OHj/Uf/Y2BMf/2rqs/frzk/H+P/B9Wg0mE=</latexit>_** up **_hE9/UtX5lzXn590PfwVch8x0</latexit>_** **_W9dKsKvMlzLa8vUkm1PJkqgAMJt4+fiJO4fCG2bZw9a685DnPFenc1b8M2q8CPUA8O9gU19z1Cdk8YQr9pdarWimf79gkUfBdVXrwPGIAf+EOMszp6GBfgP6wbL1CGYyzke6wN7YZgaLv0C5z/Bw39/esPG9vN/6m0enHzfmv7X7a2Lz5s/G3+l9Mv1n7h7V/XHuztr32r2t/WDta61drwVrs7X/Xvuftf/9+OHj/Uf/Y2BMf/2rqs/frzk/H+P/B9Wg0mE=</latexit>_** up **_hE9/UtX5lzXn590PfwVch8x0</latexit>_** **_W9dKsKvMlzLa8vUkm1PJkqgAMJt4+fiJO4fCG2bZw9a685DnPFenc1b8M2q8CPUA8O9gU19z1Cdk8YQr9pdarWimf79gkUfBdVXrwPGIAf+EOMszp6GBfgP6wbL1CGYyzke6wN7YZgaLv0C5z/Bw39/esPG9vN/6m0enHzfmv7X7a2Lz5s/G3+l9Mv1n7h7V/XHuztr32r2t/WDta61drwVrs7X/Xvuftf/9+OHj/Uf/Y2BMf/2rqs/frzk/H+P/B9Wg0mE=</latexit>_** up **_hE9/UtX5lzXn590PfwVch8x0</latexit>_** **_W9dKsKvMlzLa8vUkm1PJkqgAMJt4+fiJO4fCG2bZw9a685DnPFenc1b8M2q8CPUA8O9gU19z1Cdk8YQr9pdarWimf79gkUfBdVXrwPGIAf+EOMszp6GBfgP6wbL1CGYyzke6wN7YZgaLv0C5z/Bw39/esPG9vN/6m0enHzfmv7X7a2Lz5s/G3+l9Mv1n7h7V/XHuztr32r2t/WDta61drwVrs7X/Xvuftf/9+OHj/Uf/Y2BMf/2rqs/frzk/H+P/B9Wg0mE=</latexit>_** up | |
| PLM module ReLU PLM module Softmax PLM module PLM module ReLU PLM module ReLU | |
| **_Wpg7BXM7xWO/bC8PUcOkXOP8xHv7r65+/2dhu/k+l1Yvbt1vbv9/avxu4+71f9i+vnar9Z+vfZqbXvtv9b+uHa0drF2sxasPa39z9r/rv313e/f9d8F78CY/vQnVZ/WHN+3qX/B3kw0g=</latexit>_** down **_Wpg7BXM7xWO/bC8PUcOkXOP8xHv7r65+/2dhu/k+l1Yvbt1vbv9/avxu4+71f9i+vnar9Z+vfZqbXvtv9b+uHa0drF2sxasPa39z9r/rv313e/f9d8F78CY/vQnVZ/WHN+3qX/B3kw0g=</latexit>_** down **_Wz9r/rv313e/f9d8F78CY/vQnVZ/WHN+3qX/B3kw0g=</latexit>_** down **_Wz9r/rv313e/f9d8F78CY/vQnVZ/WHN+3qX/B3kw0g=</latexit>_** down **_Wz9r/rv313e/f9d8F78CY/vQnVZ/WHN+3qX/B3kw0g=</latexit>_** down | |
| **_xx9Wfuftf9d+793/qu+6737tqE/vQnVZ1/WXN+3v3wV7B0zIQ=</latexit>_** **_x0i7Xfrf1+7dXa9tof1/6ydrR2sXa9Fqx9Wfuftf9d+793/qu+6737tqE/vQnVZ1/WXN+3v3wV7B0zIQ=</latexit>_** **_x1/WXN+3v3wV7B0zIQ=</latexit>_** **_x1/WXN+3v3wV7B0zIQ=</latexit>_** **_x1/WXN+3v3wV7B0zIQ=</latexit>_** | |
| (a) Adapter (b) Prefix Tuning (c) LoRA (d) Parallel Adapter (e) Scaled PA | |
| Figure 3: Graphical illustration of existing methods and the proposed variants. “PLM module” represents a | |
| certain sublayer of the PLM (e.g. attention or FFN) that is frozen. “Scaled PA” denotes scaled parallel adapter. | |
| We do not include multi-head parallel adapter here to save space. | |
| where s 1 is a tunable scalar hyperparameter.[4] | |
| **Others:** Other parameter-efficient tuning methods include BitFit (Ben Zaken et al., 2021), which | |
| only fine-tunes bias vectors in the pre-trained model, and diff-pruning (Guo et al., 2021), which | |
| learns a sparse parameter update vector. | |
| 3 BRIDGING THE GAP – A UNIFIED VIEW | |
| We first derive an equivalent form of prefix tuning to establish its connection with adapters. We | |
| then propose a unified framework for parameter-efficient tuning that includes several state-of-the-art | |
| methods as instantiations. | |
| 3.1 A CLOSER LOOK AT PREFIX TUNING | |
| Eq. 5 describes the mechanism of prefix tuning which changes the attention module through | |
| prepending l learnable vectors to the original attention keys and values. Here, we derive an equivalent form of Eq. 5 and provide an alternative view of prefix tuning:[5] | |
| head = Attn(xWq, concat(Pk, CWk), concat(Pv, CWv)) | |
| = softmax **_xWqconcat(Pk, CWk)[⊤][ ]CWPv_** _v_ | |
| = (1 − _λ(x))softmax(xWqWk[⊤][C]_ _[⊤][)][CW][v]_ [+][ λ][(][x][)][softmax][(][x][W][q][P][ ⊤]k [)][P][v] | |
| = (1 _λ(x)) Attn(xWq, CWk, CWv)_ +λ(x) Attn(xWq, Pk, Pv), | |
| _−_ | |
| standard attention independent of C | |
| where λ(x) is a scalar that represents the sum of normalized attention weights on the prefixes:| {z } | {z } | |
| (7) | |
| _i_ [exp(][xW][q][P][ ⊤]k [)][i] | |
| _λ(x) =_ _._ (8) | |
| _i_ [exp(][xW][q]P[P][ ⊤]k [)][i][ +][ P]j [exp(][xW][q][W][ ⊤]k **_[C]_** _[⊤][)][j]_ | |
| Note that the first term in Eq. 7, AttnP (xWq, CWk, CWv), is the original attention without prefixes, | |
| whereas the second term is a position-wise modification independent of C. Eq. 7 gives an alternative view of prefix tuning that essentially applies a position-wise modification to the original head | |
| attention output h through linear interpolation: | |
| **_h ←_** (1 − _λ(x))h + λ(x)∆h,_ ∆h := softmax(xWqPk[⊤][)][P][v][.] (9) | |
| **The Connection with Adapters:** We define W1=WqPk[⊤][,][ W][2][=][P][v][,][ f] [=softmax, and rewrite Eq. 9:] | |
| **_h_** (1 _λ(x))h + λ(x)f_ (xW1)W2, (10) | |
| _←_ _−_ | |
| which reaches a very similar form to the adapter function in Eq. 4, except that prefix tuning is | |
| performing weighted addition while the adapter one is unweighted.[6] Figure 3b demonstrates the | |
| [4The public code of LoRA at https://github.com/microsoft/LoRA uses different s in different datasets, and](https://github.com/microsoft/LoRA) | |
| we have verified the value of s could have a significant effect on the results. | |
| 5 | |
| Without loss of generalization, we ignore the softmax scaling factor _√d for ease of notation._ | |
| 6h in adapters and prefix tuning are usually different, as described more below. However, here we mainly | |
| discuss the functional form as adapters can, in principle, be inserted at any position. | |
| ----- | |
| Table 1: Parameter-efficient tuning methods decomposed along the defined design dimensions. Here, for clarity, | |
| we directly write the adapter nonlinear function as ReLU which is commonly used. The bottom part of the table | |
| exemplifies new variants by transferring design choices of existing approaches. | |
| Method ∆h functional form insertion form modified representation composition function | |
| **Existing Methods** | |
| Prefix Tuning softmax(xWqPk[⊤][)][P][v] parallel head attn **_h ←_** (1 − _λ)h + λ∆h_ | |
| Adapter ReLU(hWdown)Wup sequential ffn/attn **_h ←_** **_h + ∆h_** | |
| LoRA **_xWdownWup_** parallel attn key/val **_h ←_** **_h + s · ∆h_** | |
| **Proposed Variants** | |
| Parallel adapter ReLU(hWdown)Wup parallel ffn/attn **_h ←_** **_h + ∆h_** | |
| Muti-head parallel adapter ReLU(hWdown)Wup parallel head attn **_h ←_** **_h + ∆h_** | |
| Scaled parallel adapter ReLU(hWdown)Wup parallel ffn/attn **_h ←_** **_h + s · ∆h_** | |
| computation graph of prefix tuning from this view, which allows for abstraction of prefix tuning | |
| as a plug-in module like adapters. Further, we note thatrank matrices when l is small, and thus they function similarly to the W1 ∈ R[d][h][×][l] and W Wdown2 and ∈ R W[l][×][d]up[h] matricesare lowin adapters. This view also suggests that the number of prefix vectors, l, plays a similar role to | |
| the bottleneck dimension r in adapters: they both represent the rank limitation of computing the | |
| modification vector ∆h. Thus we also refer l as the bottleneck dimension. Intuitively, the rank | |
| limitation implies that ∆h is a linear combination of the same l (or ≤ _l) basis vectors for any x._ | |
| **The Difference from Adapters:** In addition to the gating variable λ, we emphasize three differences between prefix tuning and adapters. (1) As demonstrated in Figure 3, prefix tuning uses x, the | |
| input of the PLM layer, to compute ∆h, while adapters use h, the output of the PLM layer. Thus, | |
| prefix tuning can be thought of as a “parallel” computation to the PLM layer, whereas the typical | |
| adapter is “sequential” computation. (2) Adapters are more flexible with respect to where they are | |
| inserted than prefix tuning: adapters typically modify attention or FFN outputs, while prefix tuning | |
| only modifies the attention output of each head. Empirically, this makes a large difference as we will | |
| show in §4.4. (3) Eq. 10 applies to each attention head, while adapters are always single-headed, | |
| which makes prefix tuning more expressive: head attention is of dimension d/Nh – basically we | |
| have full rank updates to each attention head if l _d/Nh, but we only get full-rank updates to the_ | |
| _≥_ | |
| whole attention output with adapters if r ≥ _d. Notably, prefix tuning is not adding more parameters_ | |
| than adapters when l = r.[7] We empirically validate such multi-head influence in §4.4. | |
| 3.2 THE UNIFIED FRAMEWORK | |
| Inspired by the connections between prefix tuning and adapters, we propose a general framework | |
| that aims to unify several state-of-the-art parameter-efficient tuning methods. Specifically, we cast | |
| them as learning a modification vector ∆h, which is applied to various hidden representations. | |
| Formally, we denote the hidden representation to be directly modified as h, and the direct input | |
| to the PLM sub-module that computes h as x (e.g. h and x can be the attention output and input | |
| respectively). To characterize this modification process, we define a set of design dimensions, and | |
| different methods can be instantiated by varying values along these dimensions. We detail the design | |
| dimensions below, and illustrate how adapters, prefix tuning, and LoRA fall along them in Table 1: | |
| **Functional Form is the specific function that computes ∆h. We have detailed the functional form** | |
| for adapters, prefix tuning, and LoRA in Eq. 4, 6, and 10 respectively. The functional forms of all | |
| these methods are similar with a proj down → nonlinear → proj up architecture, while | |
| “nonlinear” degenerates to the identity function in LoRA. | |
| **Modified Representation indicates which hidden representation is directly modified.[8]** | |
| **Insertion Form is how the added module is inserted into the network. As mentioned in the previous** | |
| section and shown in Figure 3, traditionally adapters are inserted at a position in a sequential manner, | |
| where both the input and output are h. Prefix tuning and LoRA – although not originally described | |
| in this way – turn out to be equivalent to a parallel insertion where x is the input. | |
| 7We will detail in §4.1 the number of parameters added of different methods. | |
| 8Strictly speaking, all the hidden representations would be indirectly influenced by modifying the ones | |
| before them. Here we refer to the position being directly modified by the added module. | |
| ----- | |
| **Composition Function is how the modified vector ∆h is composed with the original hidden repre-** | |
| sentation h to form the new hidden representation. For example, adapters perform simple additive | |
| composition, prefix tuning uses a gated additive composition as shown in Eq. 10, and LoRA scales | |
| ∆h by a constant factor and adds it to the original hidden representation as in Eq. 6. | |
| We note that many other methods not present in Table 1 fit into this framework as well. For example, | |
| prompt tuning modifies the head attention in the first layer in a way similar to prefix tuning, and | |
| various adapter variants (Pfeiffer et al., 2021; Mahabadi et al., 2021) can be represented in a similar | |
| way as adapters. Critically, the unified framework allows us to study parameter-efficient tuning | |
| methods along these design dimensions, identify the critical design choices, and potentially transfer | |
| design elements across approaches, as in the following section. | |
| 3.3 TRANSFERRING DESIGN ELEMENTS | |
| Here, and in Figure 3, we describe just a few novel methods that can be derived through our unified view above by transferring design elements across methods: (1) Parallel Adapter is the variant | |
| by transferring the parallel insertion of prefix tuning into adapters. Interestingly, while we motivate the parallel adapter due to its similarity to prefix tuning, concurrent work (Zhu et al., 2021) | |
| independently proposed this variant and studied it empirically; (2) Multi-head Parallel Adapter is | |
| a further step to make adapters more similar to prefix tuning: we apply parallel adapters to modify | |
| head attention outputs as prefix tuning. This way the variant improves the capacity for free by utilizing the multi-head projections as we discuss in §3.1. (3) Scaled Parallel Adapter is the variant by | |
| transferring the composition and insertion form of LoRA into adapters, as shown in Figure 3e. | |
| Our discussion and formulation so far raise a few questions: Do methods varying the design elements | |
| above exhibit distinct properties? Which design dimensions are particularly important? Do the novel | |
| methods described above yield better performance? We answer these questions next. | |
| 4 EXPERIMENTS | |
| 4.1 GENERAL SETUP | |
| **Datasets:** We study four downstream tasks: (1) XSum (Narayan et al., 2018) is an English summarization dataset where models predict a summary given a news article; (2) English to Romanian | |
| translation using the WMT 2016 en-ro dataset (Bojar et al., 2016); (3) MNLI (Williams et al., 2018) | |
| is an English natural language inference dataset where models predict whether one sentence entails, | |
| contradicts, or is neutral to another. (4) SST2 (Socher et al., 2013) is an English sentiment classification benchmark where models predict whether a sentence’s sentiment is positive or negative. | |
| **Setup:** We use BARTLARGE (Lewis et al., 2020) and a multilingual version of it, mBARTLARGE (Liu | |
| et al., 2020a), as the underlying pretrained models for XSum and en-ro translation respectively, and | |
| we use RoBERTaBASE (Liu et al., 2019) for MNLI and SST2. We vary the bottleneck dimension | |
| within {1, 30, 200, 512, 1024} if needed.[9] We mainly study adapters, prefix tuning (prefix), and | |
| LoRA which greatly outperform bitfit and prompt tuning in our experiments. In the analysis sections | |
| (§4.3-4.5) we insert adapters either at the attention or FFN layers for easier analysis, but include the | |
| results of inserting at both places in the final comparison (§4.6). We re-implement these methods | |
| based on their respective public code.[10] We use the huggingface transformers library (Wolf et al., | |
| 2020) for our implementation. Complete setup details can be found in Appendix A. | |
| **Evaluation:** We report ROUGE 1/2/L scores (R-1/2/L, Lin (2004)) on the XSum test set, BLEU | |
| scores (Papineni et al., 2002) on the en-ro test set, and accuracy on the MNLI and SST2 dev set. | |
| For MNLI and SST2, we take the median of five random runs. We also report the number of tuned | |
| parameters relative to that in full fine-tuning (#params). | |
| **Number of Tunable Parameters:** BART and mBART have an encoder-decoder structure that has | |
| three types of attention: encoder self-attention, decoder self-attention, and decoder cross-attention. | |
| RoBERTa only has encoder self-attention. For each attention sub-layer, the number of parameters | |
| 9In some settings we use other values to match the number of added parameters of different methods. | |
| 10We verify that our re-implementation can reproduce adapter and prefix tuning on XSum, and LoRA on | |
| MNLI, by comparing with the results of running the original released code. | |
| ----- | |
| 22 | |
| 21 | |
| Table 2: Accuracy on the dev set of | |
| MNLI and SST2. MAM Adapter is | |
| proposed in §4.6. Bitfit numbers are | |
| from Ben Zaken et al. (2021). | |
| Method (# params) MNLI SST2 | |
| Full-FT (100%) 87.6±.4 94.6±.4 | |
| Bitfit (0.1 %) 84.7 93.7 | |
| Prefix (0.5%) 86.3±.4 94.0±.1 | |
| LoRA (0.5%) 87.2±.4 94.2±.2 | |
| Adapter (0.5%) 87.2±.2 94.2±.1 | |
| MAM Adapter (0.5%) 87.4±.3 94.2±.3 | |
| 36 | |
| 34 | |
| 32 | |
| 30 | |
| 28 | |
| 26 | |
| 20 | |
| 19 | |
| 18 | |
| |Col1|Col2|Col3|Col4| | |
| |---|---|---|---| | |
| |1.94||20.98|| | |
| ||||| | |
| ||||| | |
| ||||| | |
| ||||| | |
| |Col1|Col2|Col3|Col4|Col5| | |
| |---|---|---|---|---| | |
| |7.3||36.6||| | |
| |||||| | |
| |||||| | |
| |||||| | |
| ||LoRA|||| | |
| ||Adap Prefix|ter Tuning||| | |
| ||BitFit|||| | |
| ||Full F|ine-tuning||| | |
| |||||| | |
| 37.3 | |
| 36.6 | |
| LoRA | |
| Adapter | |
| PrefixTuning | |
| BitFit | |
| Full Fine-tuning | |
| 5 10 15 | |
| Fine-tuned Parameters (%) | |
| 5 10 15 | |
| Fine-tuned Parameters (%) | |
| Figure 4: Performance of previous state-of-the-art parameterefficient tuning methods on XSum (left) and en-ro (right). | |
| Table 3: Comparison of different insertion forms for adapters, | |
| i.e. sequential adapter (SA) and parallel adapter (PA). We include the results of prefix tuning as a reference point. | |
| Table 4: Results on en-ro dataset. | |
| Method # params MT (BLEU) | |
| PA (attn), r=200 3.6% 35.6 | |
| Prefix, l=200 3.6% 35.6 | |
| MH PA (attn), r=200 3.6% 35.8 | |
| Prefix, l=30 0.1% 35.2 | |
| -gating, l=30 0.1% 34.9 | |
| PA (ffn), r=30 0.1% 33.0 | |
| PA (attn), r=30 0.1% 33.7 | |
| MH PA (attn), r=30 0.1% **35.3** | |
| Method # params XSum (R-1/2/L) MT (BLEU) | |
| Prefix, l=200 3.6% 43.40/20.46/35.51 35.6 | |
| SA (attn), r=200 3.6% 42.01/19.30/34.40 35.3 | |
| SA (ffn), r=200 2.4% 43.21/19.98/35.08 35.6 | |
| PA (attn), r=200 3.6% 43.58/20.31/35.34 35.6 | |
| PA (ffn), r=200 2.4% **43.93/20.66/35.63** **36.4** | |
| used of each method is: (1) prefix tuning prepends l vectors to the keys and values and uses 2 _×_ _l_ _×_ _d_ | |
| parameters; (2) adapter has Wdown and Wup thus uses 2×r _×d parameters; (3) LoRA employs a pair_ | |
| of Wdown and Wup for query and value projections, hence uses 4 _×_ _r ×_ _d parameters. For the adapter_ | |
| modification at ffn, it uses 2 _×_ _r ×_ _d parameters which is the same as adapter at attention. Therefore,_ | |
| for a specific value of r or l, prefix tuning uses the same number of parameters as adapters, while | |
| LoRA uses more parameters. More details can be found in Appendix B. | |
| 4.2 THE RESULTS OF EXISTING METHODS | |
| We first overview the results of existing methods on the four tasks. As shown in Figure 4 and Table 2, | |
| while existing methods can achieve competitive performance on MNLI and SST2 by tuning fewer | |
| than 1% parameters, a large gap is still present if we add 5% parameters in XSum and en-ro. The gap | |
| remains significant even though we increase the relative parameter size to >10%. Even larger gaps | |
| have been observed in Raffel et al. (2020) on high-resource MT tasks. This shows that many methods | |
| that claimed comparable results to full fine-tuning on the GLUE benchmark with an encoder-only | |
| model (Guo et al., 2021; Ben Zaken et al., 2021; Mahabadi et al., 2021), or on relatively simple | |
| generation benchmarks such as E2E (Novikova et al., 2017) with an encoder-decoder model (Li & | |
| Liang, 2021), may not generalize well to other standard benchmarks. The influencing factors could | |
| be complicated including the number of training samples, task complexity, or model architecture. | |
| We thus advocate for future research on this line to report results on more diverse benchmarks to | |
| exhibit a more complete picture of their performance profile. Below, our analysis will mainly focus | |
| on the XSum and en-ro datasets to better distinguish different design choices. We note that these two | |
| benchmarks are relatively high-resource performed with an encoder-decoder model (BART), while | |
| we will discuss the results on MNLI and SST2 with an encoder-only model (RoBERTa) in §4.6. | |
| 4.3 WHICH INSERTION FORM – SEQUENTIAL OR PARALLEL? | |
| We first study the insertion form design dimension, comparing the proposed parallel adapter (PA) | |
| variant to the conventional sequential adapter (SA) over both the attention (att) and FFN modification. We also include prefix tuning as a reference point. As shown in Table 3, prefix tuning, which | |
| uses parallel insertion, outperforms attention sequential adapters. Further, the parallel adapter is able | |
| to beat sequential adapters in all cases,[11] with PA (ffn) outperforming SA (ffn) by 1.7 R-2 points on | |
| 11More results with different r can be found in Appendix C, which exhibits similar observations. | |
| ----- | |
| 37.0 | |
| 36.5 | |
| 36.0 | |
| 35.5 | |
| 35.0 | |
| |Col1|Col2|Col3|Col4|Col5|Col6|Col7|Col8| | |
| |---|---|---|---|---|---|---|---| | |
| ||||||||| | |
| ||||||Prefix|(a|ttn) ) ttn) n)| | |
| ||||||PA (a LoRA|ttn (a|| | |
| ||||||PA (ff LoRA|n) (ff|| | |
| 2.5 5.0 7.5 10.0 12.5 | |
| Fine-tuned Parameters (%) Fine-tuned Parameters (%) | |
| Figure 5: Results on XSum (left) and en-ro (right). PA represents parallel adapter. Blue and red markers apply | |
| 21.25 | |
| 21.00 | |
| 20.75 | |
| XSum ROUGE-2 20.50 | |
| 20.25 | |
| 2.5 5.0 7.5 10.0 12.5 | |
| Fine-tuned Parameters (%) | |
| modifications at attention and FFN sub-layers respectively (best viewed in color). | |
| XSum and 0.8 BLEU points on en-ro respectively. Given the superior results of parallel adapters | |
| over sequential adapters, we focus on parallel adapter results in following sections. | |
| 4.4 WHICH MODIFIED REPRESENTATION – ATTENTION OR FFN? | |
| **Setup:** We now study the effect of modifying different representations. We mainly compare attention and FFN modification. For easier analysis we categorize methods that modifies any hidden | |
| representations in the attention sub-layer (e.g. the head output, query, etc) as modifying the attention module. We compare parallel adapters at attention and FFN and prefix tuning. We also transfer | |
| the FFN modification to LoRA to have a LoRA (ffn) variant for a complete comparison. Specifi**_Wcally, we use LoRA to approximate the parameter updates for the FFN weightsof r2 ∈ × dRm[d][m], where[×][d]. In this case dm = 4 Wd as described in §2.1. Thus we typically use smallerup in LoRA for W1 (similar for Wdown of W2) would have dimensions W r1 for LoRA (ffn) ∈_** R[d][×][d][m] and | |
| than other methods to match their overall parameter size in later experiments. | |
| **Results:** As shown in Figure 5, any method with FFN modification outperforms all the methods | |
| with attention modification in all cases (the red markers are generally above all the blue ones, the | |
| only exception is ffn-PA with 2.4% params), often with fewer parameters. Second, the same method | |
| applied at FFN always improves over its attention counterpart. For example, LoRA (ffn) improves | |
| LoRA (attn) by 1 R-2 points on XSum. We also highlight that prefix tuning does not keep improving | |
| when we further increase the capacity, which is also observed in Li & Liang (2021). These results | |
| suggest that FFN modification can utilize the added parameters more effectively than attention, no | |
| _matter what the functional form or composition function is. We hypothesize that this is because_ | |
| the FFN learns task-specific textual patterns (Geva et al., 2021), while attention learns pairwise | |
| positional interactions which do not require large capacity for adapting to new tasks. | |
| **Is the story different when we use 0.1% parameters?** In §3.1 we reason that prefix tuning is | |
| more expressive than adapters (attn), which, however, is not reflected in Figure 5. We conjecture | |
| that this is because multi-head attention is only superior when the parameter budget is small. To | |
| validate this hypothesis, we compare prefix tuning to parallel adapters when they add 0.1% of the | |
| pretrained parameters. To ablate the impact of the composition function, we also report the results | |
| of removing the gating in prefix tuning as h + ∆h. We include the results of the multi-head parallel | |
| adapter variant (MH PA) described in §3.3. As shown in Table 4, the multi-head methods – prefix | |
| tuning and MH PA (attn) – outperform all others by at least 1.6 BLEU points when using 0.1% of | |
| the parameters. Surprisingly, reducing l from 200 to 30 only causes 0.4 BLEU loss for prefix tuning | |
| while PA (attn) loses 1.9 points. The gating composition function in prefix tuning slightly helps the | |
| results by 0.3 points. We highlight that the MH parallel adapter improves the single-headed version | |
| by 1.6 points, which again verifies the effectiveness of the multi-head formulation. | |
| Combining the results in Figure 5 and Table 4, we conclude that modifying head attention shows the | |
| _best results when the parameter budget is very small, while the FFN can better utilize modifications_ | |
| _at larger capacities. This suggests that it may be effective to allocate a larger parameter budget to_ | |
| FFN modification instead of treating attention and FFN equally as in Houlsby et al. (2019). | |
| 4.5 WHICH COMPOSITION FUNCTION? | |
| We have presented three composition functions in §3.2: simple addition (adapter), gated addition | |
| (prefix tuning) and scaled addition (LoRA). As it is unnatural to incorporate the exact gated addition into methods whose functional form does not use softmax, we examine the other two by | |
| ----- | |
| Table 6: Comparison of various parameter-efficient tuning methods and the proposed variants. “†” are results | |
| copied from Lewis et al. (2020) and Liu et al. (2020b). We could not reproduce exactly the same full finetuning numbers with the same hyperparameters or even searching them. The reason may be the different | |
| libraries which the training code is based on – full fine-tuning is very sensitive to training hyperparameters. For | |
| the most performant methods we run with 3 random seeds and report mean and standard deviation. | |
| Method # params XSum (R-1/2/L) MT (BLEU) | |
| Full fine-tuning[†] 100% 45.14/22.27/37.25 37.7 | |
| Full fine-tuning (our run) 100% 44.81/21.94/36.83 37.3 | |
| Bitfit (Ben Zaken et al., 2021) 0.1% 40.64/17.32/32.19 26.4 | |
| Prompt tuning (Lester et al., 2021) 0.1% 38.91/15.98/30.83 21.0 | |
| Prefix tuning (Li & Liang, 2021), l=200 3.6% 43.40/20.46/35.51 35.6 | |
| Pfeiffer adapter (Pfeiffer et al., 2021), r=600 7.2% 44.03/20.89/35.89±.13/.10/.08 36.9±.1 | |
| LoRA (ffn), r=102 7.2% 44.53/21.29/36.28±.14/.07/.10 36.8±.3 | |
| Parallel adapter (PA, ffn), r=1024 12.3% 44.71/21.41/36.41±.16/.17/.16 37.2±.1 | |
| PA (attn, r=30) + PA (ffn, r=512) 6.7% 44.29/21.06/36.12±.31/.19/.18 37.2±.1 | |
| Prefix tuning (attn, l=30) + LoRA (ffn, r=102) 6.7% 44.84/21.71/36.77±.07/.05/.03 37.0±.1 | |
| MAM Adapter (our variant, l=30, r=512) 6.7% **45.06/21.90/36.87** .08/.01/.04 **37.5** .1 | |
| _±_ _±_ | |
| ablating on LoRA and comparing with the proposed scaled parallel adapter (Scaled PA), we constrain modified representation to be FFN since it is generally more effective as shown in §4.4. | |
| Table 5 reports the results on XSum. We set r as 512 | |
| for adapters and 102 for LoRA so that their tuned Table 5: Results on XSum when using different | |
| composition functions. The modified representa | |
| parameter sizes are the same. We select s based | |
| tion is FFN. The bottleneck dimension r = 512 | |
| on the R-2 score on the dev set. We observe that | |
| for (Scaled) PA and r = 102 for LoRA. | |
| LoRA (s = 4) performs better than parallel adapter. | |
| However, the advantage disappears if we remove the Method (# params) XSum (R-1/2/LSum) | |
| scaling by setting s = 1. Through plugging the LoRA (6.1%), s=4 44.59/21.31/36.25 | |
| composition function of LoRA into parallel adapter, LoRA (6.1%), s=1 44.17/20.83/35.74 | |
| the resulted Scaled PA improves the vanilla parallel PA (6.1%) 44.35/20.98/35.98 | |
| adapter by 0.56 ROUGE-2 points. We also experi- Scaled PA (6.1%), s=4 **44.85/21.54/36.58** | |
| ment with a learned scalar which does not give bet- Scaled PA (6.1%), trainable s 44.56/21.31/36.29 | |
| ter results. Therefore, we conclude that the scaling | |
| _composition function is better than the vanilla additive one while being easily applicable._ | |
| 4.6 AN EFFECTIVE INTEGRATION BY TRANSFERRING FAVORABLE DESIGN ELEMENTS | |
| We first highlight three findings in previous sections: (1) Scaled parallel adapter is the best variant | |
| to modify FFN; (2) FFN can better utilize modification at larger capacities; and (3) modifying head | |
| attentions like prefix tuning can achieve strong performance with only 0.1% parameters. Inspired | |
| by them, we mix and match the favorable designs behind these findings: specifically, we use prefix | |
| tuning with a small bottleneck dimension (l = 30) at the attention sub-layers and allocate more | |
| parameter budgets to modify FFN representation using the scaled parallel adapter (r = 512). Since | |
| prefix tuning can be viewed as a form of adapter in our unified framework, we name this variant | |
| as Mix-And-Match adapter (MAM Adapter). In Table 6, we compare MAM adapter with various | |
| parameter-efficient tuning methods. For completeness, we also present results of other combination | |
| versions in Table 6: using parallel adapters at both attention and FFN layers and combining prefix | |
| tuning (attn) with LoRA (ffn) – both of these combined versions can improve over their respective | |
| prototypes. However, MAM Adapter achieves the best performance on both tasks and is able to | |
| match the results of our full fine-tuning by only updating 6.7% of the pre-trained parameters. In | |
| Table 2, we present the results of MAM Adapter on MNLI and SST2 as well, where MAM Adapter | |
| achieves comparable results to full fine-tuning by adding only 0.5% of pretrained parameters. | |
| 5 DISCUSSION | |
| We provide a unified framework for several performant parameter-tuning methods, which enables | |
| us to instantiate a more effective model that matches the performance of full fine-tuning method | |
| through transferring techniques across approaches. We hope our work can provide insights and | |
| guidance for future research on parameter-efficient tuning. | |
| ----- | |
| ETHICS STATEMENT | |
| Our work proposes a method for efficient fine-tuning of pre-trained models, in particular language | |
| models. Pre-trained language models have a wide variety of positive applications, such as the applications to summarization, translation, or language understanding described in our paper. At the same | |
| time, there are a number of ethical concerns with language models in general, including concerns | |
| regarding the generation of biased or discriminative text (Bordia & Bowman, 2019), the leakage of | |
| private information from training data (Carlini et al., 2020), and environmental impact of training or | |
| tuning them (Strubell et al., 2019). | |
| Our method attempts to train language models making minimal changes to their pre-existing parameters. While it is an interesting research question whether parameter-efficient fine-tuning methods | |
| exacerbate, mitigate, or make little change to issues such as bias or information leakage, to our | |
| knowledge no previous work has examined this topic. It is an interesting avenue for future work. | |
| With respect to environmental impact, the methods proposed in this paper add a small number of | |
| extra parameters and components to existing models, and thus they have a nominal negative impact | |
| on training and inference time – for example, the final MAM Adapter needs 100% - 150% training | |
| time of full fine-tuning in our four benchmarks since parameter-efficient tuning typically needs more | |
| epochs to converge; the inference time is roughly the same as the model obtained by full fine-tuning. | |
| On the other hand, as the methods proposed in this paper may obviate the need for full fine-tuning, | |
| this may also significantly reduce the cost (in terms of memory/deployed servers) of serving models. | |
| Notably, the great majority of the experimentation done for this paper was performed on a data center | |
| powered entirely by renewable energy. | |
| REPRODUCIBILITY STATEMENT | |
| In addition to the setup description in §4.1, we have detailed the complete experiments setup such | |
| as batch size, optimizer, learning rates in Appendix A. Besides, we have publicized our source code. | |
| These resources should be sufficient to reproduce results of the paper. | |
| ACKNOWLEDGEMENT | |
| We thank the anonymous reviewers for their comments. This work was supported in part by the | |
| CMU-Portugal MAIA Project, a Baidu PhD Fellowship for Junxian He, and a CMU Presidential | |
| Fellowship for Chunting Zhou. | |
| REFERENCES | |
| Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint | |
| _arXiv:1607.06450, 2016._ | |
| Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple parameter-efficient fine-tuning | |
| for transformer-based masked language-models. arXiv e-prints, pp. arXiv–2106, 2021. | |
| Ondˇrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias | |
| Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, et al. Findings | |
| of the 2016 conference on machine translation. In Proceedings of the First Conference on Machine | |
| _Translation: Volume 2, Shared Task Papers, 2016._ | |
| Shikha Bordia and Samuel R. Bowman. Identifying and reducing gender bias in word-level language | |
| models. In Proceedings of the 2019 NAACL: Student Research Workshop, 2019. | |
| Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, | |
| Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are | |
| few-shot learners. arXiv preprint arXiv:2005.14165, 2020. | |
| Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine | |
| Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data | |
| from large language models. arXiv preprint arXiv:2012.07805, 2020. | |
| ----- | |
| Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep | |
| bidirectional transformers for language understanding. In Proceedings of NAACL, 2019. | |
| William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter | |
| models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961, 2021. | |
| Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are | |
| key-value memories. In Proceedings of EMNLP, 2021. | |
| Demi Guo, Alexander M Rush, and Yoon Kim. Parameter-efficient transfer learning with diff pruning. In Proceedings of ACL, 2021. | |
| Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing | |
| human-level performance on imagenet classification. In Proceedings of ICCV, 2015. | |
| Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. | |
| In Proceedings of ICML, 2019. | |
| Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu | |
| Chen. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, | |
| 2021. | |
| Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of | |
| _ICLR, 2015._ | |
| Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt | |
| tuning. In Proceedings of EMNLP, 2021. | |
| Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer | |
| Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. In Proceedings of ACL, | |
| 2020. | |
| Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In | |
| _Proceedings of ACL, 2021._ | |
| Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization | |
| _Branches Out, 2004._ | |
| Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pretrain, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586, 2021a. | |
| Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. GPT | |
| understands, too. arXiv:2103.10385, 2021b. | |
| Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike | |
| Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized bert pretraining | |
| approach. arXiv preprint arXiv:1907.11692, 2019. | |
| Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, | |
| and Luke Zettlemoyer. Multilingual denoising pre-training for neural machine translation. Trans_actions of the Association for Computational Linguistics, 2020a._ | |
| Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike | |
| Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8:726–742, 2020b. doi: | |
| [10.1162/tacl a 00343. URL https://aclanthology.org/2020.tacl-1.47.](https://aclanthology.org/2020.tacl-1.47) | |
| Rabeeh Karimi Mahabadi, James Henderson, and Sebastian Ruder. Compacter: Efficient low-rank | |
| hypercomplex adapter layers. In Proceedings of NeurIPS, 2021. | |
| ----- | |
| Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don’t give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization. In Proceedings of | |
| _EMNLP, 2018._ | |
| Jekaterina Novikova, Ondˇrej Duˇsek, and Verena Rieser. The E2E dataset: New challenges for | |
| end-to-end generation. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and | |
| _Dialogue, pp. 201–206, Saarbr¨ucken, Germany, August 2017. doi: 10.18653/v1/W17-5525._ | |
| Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic | |
| evaluation of machine translation. In Proceedings of ACL, 2002. | |
| Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and | |
| Luke Zettlemoyer. Deep contextualized word representations. In Proceedings of NAACL, 2018. | |
| Jonas Pfeiffer, Aishwarya Kamath, Andreas R¨uckl´e, Kyunghyun Cho, and Iryna Gurevych. AdapterFusion: Non-destructive task composition for transfer learning. In Proceedings of EACL, 2021. | |
| Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. Pre-trained | |
| models for natural language processing: A survey. Science China Technological Sciences, 2020. | |
| Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language | |
| models are unsupervised multitask learners. OpenAI blog, 2019. | |
| Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi | |
| Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text | |
| transformer. Journal of Machine Learning Research, 2020. | |
| Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, | |
| and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment | |
| treebank. In Proceedings of EMNLP, 2013. | |
| Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for deep | |
| learning in NLP. In Proceedings of ACL, 2019. | |
| Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, | |
| Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of NeurIPS, | |
| 2017. | |
| Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of NAACL, 2018. | |
| Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, | |
| Pierric Cistac, Tim Rault, R´emi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick | |
| von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, | |
| Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural | |
| language processing. In Proceedings of EMNLP: System Demonstrations, 2020. | |
| Yaoming Zhu, Jiangtao Feng, Chengqi Zhao, Mingxuan Wang, and Lei Li. Serial or parallel? plugable adapter for multilingual machine translation. arXiv preprint arXiv:2104.08154, 2021. | |
| ----- | |
| A EXPERIMENTS | |
| A.1 SETUPS | |
| Table 7: Dataset Statistics of the four tasks. | |
| Dataset #train #dev #test | |
| XSum 204,045 113,332 113,334 | |
| WMT16 en-ro 610,320 1,999 1,999 | |
| MNLI 392,702 9815 9832 | |
| SST-2 67,349 872 1,821 | |
| We implement all the parameter-efficient tuning methods using the huggingface transformers library (Wolf et al., 2020). We use BARTLARGE(Lewis et al., 2020) and mBARTLARGE (Liu et al., | |
| 2020b) (mBART-cc25) for the summarization and machine translation tasks respectively, and we | |
| use RoBERTaBASE (Liu et al., 2019) for MNLI and SST2. BARTLARGE and mBARTLARGE have the | |
| same encoder-decoder architectures. mBARTLARGE is pre-trained on 25 languages. We use their | |
| public checkpoints from the transformers library in experiments. For MT and classifications tasks, | |
| the max token lengths of training data are set to be 150 and 512 respectively. For XSum, we set the | |
| max length of source articles to be 512 and the max length of the target summary to be 128. The | |
| detailed dataset statistics is present in Table 7. In our summarization experiments, we only use 1600 | |
| examples for validation to save time. | |
| While we vary the bottleneck dimension within {1, 30, 512, 1024} as mentioned in §4.1, we test | |
| bottleneck dimension 1024 only when the modified representation is FFN, because the training of | |
| prefix tuning does not fit into 48GB GPU memory when l = 1024. While other methods do not have | |
| memory issues, we keep the bottleneck dimension of attention modification at most 512 to have a | |
| relatively fair comparison with prefix tuning. For LoRA we always tune its scaling hyperparameters | |
| _s on the dev set._ | |
| A.2 TRAINING AND EVALUATION | |
| We present some training hyperparameters of parameter-efficient tuning methods in Table 8. For all | |
| the tasks, we train with the Adam optimizer (Kingma & Ba, 2015), and use a polynomial learning | |
| rate scheduler that linearly decays the learning rate throughout training. We set the warm up steps of | |
| learning rate to be 0 for both MT and summarization tasks, and for the classification tasks, learning | |
| rate is linearly warmed up from 0 for the first 6% of the total training steps before decay. For full | |
| fine-tuning we set these training hyperparameters following Lewis et al. (2020) (XSum), Liu et al. | |
| (2020b) (en-ro), and (Liu et al., 2019) (MNLI and SST2). We also did hyperparameter search in the | |
| full fine-tuning case to try to reproduce their results. We set dropout rate to be 0.1 for all the tasks. | |
| We use ROUGE-2 and perplexity as the validation metrics for summarization and MT respectively. | |
| For MT and text summarization, we use beam search for decoding and set the number of beams to be | |
| 6 and 5 following previous work (Li & Liang, 2021; Liu et al., 2020b). The min and max generation | |
| lengths for summarization and MT are set to be (10, 60) and (1, 200) respectively. | |
| A.3 OTHER EXPERIMENTAL DETAILS | |
| **Prefix Tuning:** Following Li & Liang (2021), we reparameterize the prefix vectors by a MLP | |
| network which is composed of a small embedding matrix and a large feedforward neural network. | |
| This is conducive for learning due to the shared parameters across all layers. | |
| **LoRA:** LoRA and adapter employ different parameter initialization methods: LoRA uses a random Kaiming uniform (He et al., 2015) initialization for Wdown and zero for Wup (LoRA init), | |
| while adapters use the same initialization as BERT (Devlin et al., 2019). We found it beneficial to | |
| use the same initialization method as LoRA in scaled PA. | |
| ----- | |
| Table 8: Training hyperparameters of parameter-efficient tuning methods on the four tasks. lr and ls represents | |
| learning rate and label smoothing respectively. | |
| Tasks lr batch size ls max grad norm weight decay train steps | |
| XSum 5e-5 64 sents 0.1 0.1 0.01 100K | |
| enro MT 5e-5 16384 tokens 0.1 1.0 0.01 50K | |
| MNLI/SST2 1e-4 32 sents 0 1.0 0.1 10 epochs | |
| B COMPUTATION OF TUNABLE PARAMETERS | |
| Table 10: Number of parameters used at each sub-layer for different methods. | |
| _NW[attn]_ _NW[ffn]_ | |
| Prefix Tuning 2ld – | |
| Adapter variants 2rd 2rd | |
| LoRA 2 × 2rd = 4rd 2 × (rd + 4dr) = 10rd | |
| Table 9: Number of attention or FFN sublayers in each layer of the pre-trained models. | |
| BART/mBARTLARGE RoBERTaBASE | |
| _Nattn_ 3 1 | |
| _Nffn_ 2 1 | |
| We compute the number of tunable parameters based on where the tunable module is inserted into | |
| and how it is parameterized. The pretrained-models for summarization or MT have an encoderdecoder structure and each has L layers, whereas RoBERTaBASE for classification tasks only has | |
| _L encoder layers. To simplify the computation of tunable parameters, we compute the sum of_ | |
| parameter used in one encoder layer and one decoder layer as the parameter overhead of one single | |
| layer of the pre-trained encoder-decoder model. Each layer has Nattn sub-layers and Nffn sublayers. For the encoder-decoder models, Nattn = 3: the encoder self-attention, the decoder selfattention and the decoder cross-attention. For the classification tasks, RoBERTaBASE only has the | |
| encoder self-attention, thus Nattn = 1. We present the number of attention and ffn sub-layers | |
| for different pre-trained models in Table 10. For modifications applied at the attention sub-layers, | |
| the number of tunable parameters is computed by Θ attn = NW[attn] _Nattn_ _L, where NW[attn]_ | |
| denotes the number of parameters (Wdown or Wup |) used for one attention sub-layer. Similarly,| _×_ _×_ | |
| the number of tunable parameters for the FFN sub-layers is computed by Θ ffn = NW[ffn] | |
| _L. In Table 10, we show the number of parameters for one sub-layer. As we have explained in |_ _|_ _[×][ N][ffn][ ×]_ | |
| §4.4, LoRA approximates the update of each weight matrix with a pair of Wdown and Wup, thus | |
| LoRA typically uses more parameters with the same r as other methods. Finally, the total number | |
| of tunable parameters for prefix tuning, adapter variants and LoRA is |Θ| = |Θ|attn + |Θ|ffn as | |
| applicable. Prompt tuning prepends l tunable vectors at the input layer and uses l × d number of | |
| parameters. Using MBART/BART as an example, we present the number of parameters used by | |
| several representative methods throughout our paper in Table 11, where adapter variants include | |
| sequential adapter, parallel adapter, scaled adapter and multi-head adapter. | |
| Table 11: Number of tunable parameters of various parameter-efficient tuning methods with BART/MBART | |
| models (L = 12) as an example. | |
| Method number of parameters | |
| Prompt Tuning _l × d_ | |
| Prefix Tuning (attn) 2ld × 3 × 12 | |
| Adapter variants (attn) 2rd × 3 × 12 | |
| Adapter variants (ffn) 2rd × 2 × 12 | |
| LoRA (attn) 4rd × 3 × 12 | |
| LoRA (ffn) 10rd × 2 × 12 | |
| MAM Adapter (our proposed model) 2ld × 3 × 12 + 2rd × 2 × 12 | |
| C FULL RESULTS ON DIFFERENT BOTTLENECK DIMENSIONS | |
| ----- | |
| Table 12: Performance on the test sets of abstractive summarization (XSum) and WMT EN-RO translation. | |
| Method # params (%) XSum (R-1/2/L) MT BLEU | |
| Modified Representation: attention | |
| Prefix Tuning, r = 200 3.6 43.40/20.46/35.51 35.6 | |
| Prefix Tuning, r = 512 9.2 43.29/20.40/35.37 35.1 | |
| LoRA, r = 200 7.2 43.09/20.29/35.37 36.2 | |
| Sequential Adapter, r = 200 3.6 42.01/19.30/34.40 35.3 | |
| Sequential Adapter, r = 512 9.2 41.05/18.87/33.71 34.7 | |
| Parallel Adapter, r = 200 3.6 43.58/20.31/35.34 35.6 | |
| Parallel Adapter, r = 512 9.2 43.99/20.83/35.77 36.2 | |
| Modified Representation: FFN | |
| LoRA, r = 102 6.1 44.59/21.31/36.25 36.5 | |
| Sequential Adapter, r = 200 2.4 43.21/19.98/35.08 35.6 | |
| Sequential Adapter, r = 512 6.1 43.72/20.75/35.64 36.3 | |
| Sequential Adapter, r = 1024 12.3 43.95/21.00/35.90 36.7 | |
| Parallel Adapter, r = 200 2.4 43.93/20.66/35.63 36.4 | |
| Parallel Adapter, r = 512 6.1 44.35/20.98/35.98 37.1 | |
| Parallel Adapter, r = 1024 12.3 44.53/21.24/36.23 37.3 | |
| ----- | |