metadata

base_model: mrfakename/refusal
datasets:
  - mrfakename/refusal
inference: true
language:
  - en
library_name: transformers
model_creator: mrfakename
model_name: refusal
pipeline_tag: text-generation
quantized_by: afrideva
tags:
  - gguf
  - ggml
  - quantized

refusal-GGUF

Quantized GGUF model files for refusal from mrfakename

Original Model Card:

I messed up on the previous model. This is a fixed version.

A tiny 1B model that refuses basically anything you ask it! Trained on the refusal dataset. Prompt format is ChatML.

Training results:

Training Loss	Epoch	Step	Validation Loss
2.4352	0.0580	1	2.4462
1.5741	0.5217	9	1.4304
1.5204	1.0435	18	1.3701
1.0794	1.5217	27	1.3505
1.1275	2.0435	36	1.3344
0.6652	2.5217	45	1.4360
0.6248	3.0435	54	1.4313
0.6142	3.5072	63	1.4934

Training hyperparemeters:

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
num_epochs: 4

Base model: https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T