Spaces:
Sleeping
Sleeping
File size: 2,185 Bytes
f512012 54e2e91 f512012 ea92fac 5ee7bf3 037a4b0 df0bf53 25a5236 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
---
title: SLM Instruction Tuning using Unsloth
emoji: 🐨
colorFrom: yellow
colorTo: indigo
sdk: gradio
sdk_version: 4.38.1
app_file: app.py
pinned: false
license: apache-2.0
---
-----
# LoRA Instruction Tuning using Unsloth
### What?
This Gradio app is a simple interface to access [unsloth AI's](https://github.com/unslothai) fine-tuning methods but leveraging the A100 GPUs provided by [Huggingface Spaces](https://huggingface.co/docs/hub/en/spaces-overview). This outputs of this fine-tuning will be instruction tuned LoRA weights that will be uploaded into your personal huggingface models repository.
### Why?
The goal of this demo is to show how you can tune your own language models leveraging industry standard compute and fine tuning methods using a simple point-and-click UI.
In addition, compute, even on Google Colab's free tier is tight even with a T4 and rate limits are uncertain. This makes the use of the A100s on this demo useful for a small added boost to compute performance. For those looking to reduce the costs associated with training datasets can pull down the spaces repository to train their models at speed for $9 on The Huggingface Pro License.
This is a demo and not a production application. This application is subject a demand queue.
### How?
Just start by following the guide below:
1) Flip to the Train Model tab at the top.
2) Populate your username, repository, token, and model details
3) Upload data from transformers or your local jsonl file. Please view [this guide](https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset) for best practices.
4) Eat a snack and wait as you train the model.
### Coming soon!
- More models and added flexibility with guardrails on hyperparameter tuning.
- Downloads for a [WandB](https://wandb.ai/home) logger for training monitoring.
### Other resources.
- [Unsloth's notebooks](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing) to look at what is going on under the hood.
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|