output

This model is a fine-tuned version of EleutherAI/gpt-j-6B on the None dataset. It achieves the following results on the evaluation set:

• Loss: nan

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 14
• eval_batch_size: 14
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.03	10	nan
No log	0.06	20	nan
No log	0.08	30	nan
No log	0.11	40	nan
No log	0.14	50	nan
No log	0.17	60	nan
No log	0.2	70	nan
No log	0.23	80	nan
No log	0.25	90	nan
0.9839	0.28	100	nan
0.9839	0.31	110	nan
0.9839	0.34	120	nan
0.9839	0.37	130	nan
0.9839	0.4	140	nan
0.9839	0.42	150	nan
0.9839	0.45	160	nan
0.9839	0.48	170	nan
0.9839	0.51	180	nan
0.9839	0.54	190	nan
0.0	0.56	200	nan
0.0	0.59	210	nan
0.0	0.62	220	nan
0.0	0.65	230	nan
0.0	0.68	240	nan
0.0	0.71	250	nan
0.0	0.73	260	nan
0.0	0.76	270	nan
0.0	0.79	280	nan
0.0	0.82	290	nan
0.0	0.85	300	nan
0.0	0.88	310	nan
0.0	0.9	320	nan
0.0	0.93	330	nan
0.0	0.96	340	nan
0.0	0.99	350	nan

Framework versions

• Transformers 4.26.0
• Pytorch 1.11.0+cu113
• Datasets 2.9.0
• Tokenizers 0.13.2