LoRA-Learns-Less-and-Forgets-Less

community

https://arxiv.org/abs/2405.09673

AI & ML interests

None defined yet.

Organization Card

Community About org cards

These are the model weights associated with the TMLR 2024 publication LoRA Learns Less and Forgets Less (Biderman et al. 2024). This work was done in collaboration with Databricks Mosaic AI Research.

Collections 4

models 17

LoRA-TMLR-2024/magicoder-lora-rank-16-alpha-32

Updated Oct 16 • 65

LoRA-TMLR-2024/starcoder-full-finetuning-lr-1e-05-20B-token

LoRA-TMLR-2024/starcoder-lora-rank-256-20B-tokens

Updated Sep 27 • 2

LoRA-TMLR-2024/openwebmath-full-finetuning-lr-1e-05-20B-tokens

LoRA-TMLR-2024/openwebmath-lora-rank-256-20B-tokens

Updated Sep 27 • 2

LoRA-TMLR-2024/openwebmath-lora-rank-16-20B-tokens

Updated Sep 27 • 1

LoRA-TMLR-2024/starcoder-lora-rank-16-20B-tokens

Updated Sep 27 • 62

LoRA-TMLR-2024/magicoder-lora-rank-64-alpha-128

Updated Sep 27 • 102

LoRA-TMLR-2024/magicoder-lora-rank-256-alpha-512

Updated Sep 27 • 1

LoRA-TMLR-2024/magicoder-full-finetuning-lr-5e-05

datasets

None public yet