---
library_name: peft
---
language: en
thumbnail: 
tags:
- peft
- text-generation
- chatbot
- ecommerce
- fine-tuned
pipeline_tag: text-generation
license: apache-2.0
datasets:
- kaggle
metrics:
- 
---

# Falcon 7B LLM Fine Tune Model

## Model description

This model is a fine-tuned version of the `tiiuae/falcon-7b` model using the QLoRa library and the PEFT library. It was fine-tuned on the [Ecommerce-FAQ-Chatbot-Dataset](https://kaggle.com/datasets/saadmakhdoom/ecommerce-faq-chatbot-dataset) from Kaggle.

## Intended uses & limitations

#### How to use

```python
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch

model_id = "hipnologo/Falcon-7B-FineTune-Chatbot"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# generate text
input_prompt = "Hello, Bot!"
input_ids = tokenizer.encode(input_prompt, return_tensors='pt')
output = model.generate(input_ids)
output_text = tokenizer.decode(output[:, input_ids.shape[-1]:][0], skip_special_tokens=True)

## Training procedure

The model was fine-tuned on the [Ecommerce-FAQ-Chatbot-Dataset](https://kaggle.com/datasets/saadmakhdoom/ecommerce-faq-chatbot-dataset) using the `bitsandbytes` quantization config:
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: bfloat16

### Framework versions

- PEFT 0.4.0.dev0

## Evaluation results

The model was trained for 80 steps, with the training loss decreasing from 0.184 to nearly 0. The final training loss was 0.03094411873175886.

- Trainable params: 2359296
- All params: 3611104128
- Trainable%: 0.06533447711203746

## License

This model is licensed under Apache 2.0. Please see the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) for more information.