Spaces:
Sleeping
Sleeping
| title: SLM Instruction Tuning using Unsloth | |
| emoji: 🐨 | |
| colorFrom: yellow | |
| colorTo: indigo | |
| sdk: gradio | |
| sdk_version: 4.38.1 | |
| app_file: app.py | |
| pinned: false | |
| license: apache-2.0 | |
| ----- | |
| # LoRA Instruction Tuning using Unsloth | |
| ### What? | |
| This Gradio app is a simple interface to access [unsloth AI's](https://github.com/unslothai) fine-tuning methods but leveraging the A100 GPUs provided by [Huggingface Spaces](https://huggingface.co/docs/hub/en/spaces-overview). This outputs of this fine-tuning will be instruction tuned LoRA weights that will be uploaded into your personal huggingface models repository. | |
| ### Why? | |
| The goal of this demo is to show how you can tune your own language models leveraging industry standard compute and fine tuning methods using a simple point-and-click UI. | |
| In addition, compute, even on Google Colab's free tier is tight even with a T4 and rate limits are uncertain. This makes the use of the A100s on this demo useful for a small added boost to compute performance. For those looking to reduce the costs associated with training datasets can pull down the spaces repository to train their models at speed for $9 on The Huggingface Pro License. | |
| This is a demo and not a production application. This application is subject a demand queue. | |
| ### How? | |
| Just start by following the guide below: | |
| 1) Flip to the Train Model tab at the top. | |
| 2) Populate your username, repository, token, and model details | |
| 3) Upload data from transformers or your local jsonl file. Please view [this guide](https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset) for best practices. | |
| 4) Eat a snack and wait as you train the model. | |
| ### Coming soon! | |
| - More models and added flexibility with guardrails on hyperparameter tuning. | |
| - Downloads for a [WandB](https://wandb.ai/home) logger for training monitoring. | |
| ### Other resources. | |
| - [Unsloth's notebooks](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing) to look at what is going on under the hood. | |
| [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth) | |