ultra-feedback-dutch-cleaned-hq-spin-geitje-7b-ultra-sft_iter2

This model is a fine-tuned version of davidberenstein1957/ultra-feedback-dutch-cleaned-hq-spin-geitje-7b-ultra-sft_iter1 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0162
• Rewards/real: -8.1731
• Rewards/generated: -31.3826
• Rewards/accuracies: 0.9917
• Rewards/margins: 23.2095
• Logps/generated: -956.3063
• Logps/real: -525.1735
• Logits/generated: -1.5719
• Logits/real: -1.7813

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-07
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• total_eval_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/real	Rewards/generated	Rewards/accuracies	Rewards/margins	Logps/generated	Logps/real	Logits/generated	Logits/real
0.6097	0.04	25	0.4147	-0.6192	-1.4312	0.9250	0.8120	-656.7919	-449.6341	-2.0004	-2.0773
0.2137	0.08	50	0.1745	-2.0300	-5.0060	0.9519	2.9761	-692.5404	-463.7422	-1.9306	-2.0237
0.1292	0.12	75	0.1012	-2.8227	-7.4967	0.9685	4.6740	-717.4471	-471.6697	-1.8843	-1.9887
0.0665	0.16	100	0.0676	-3.2936	-9.3177	0.9778	6.0240	-735.6567	-476.3786	-1.8508	-1.9628
0.0429	0.21	125	0.0477	-3.7328	-11.2722	0.9824	7.5395	-755.2025	-480.7701	-1.8123	-1.9332
0.0299	0.25	150	0.0369	-4.2161	-13.2599	0.9870	9.0437	-775.0787	-485.6039	-1.7938	-1.9226
0.0252	0.29	175	0.0320	-4.7201	-15.0489	0.9880	10.3288	-792.9691	-490.6432	-1.7758	-1.9116
0.0249	0.33	200	0.0301	-5.0757	-16.3570	0.9880	11.2813	-806.0497	-494.1995	-1.7515	-1.8923
0.0175	0.37	225	0.0273	-5.4299	-17.6751	0.9880	12.2451	-819.2310	-497.7419	-1.7362	-1.8821
0.0183	0.41	250	0.0254	-5.4183	-18.3899	0.9889	12.9715	-826.3791	-497.6259	-1.7300	-1.8793
0.0182	0.45	275	0.0245	-6.0900	-20.5760	0.9889	14.4860	-848.2401	-504.3426	-1.6961	-1.8564
0.0253	0.49	300	0.0224	-5.9239	-20.7184	0.9898	14.7944	-849.6640	-502.6819	-1.6938	-1.8573
0.0075	0.53	325	0.0234	-7.0436	-24.1126	0.9898	17.0691	-883.6064	-513.8781	-1.6522	-1.8252
0.0141	0.58	350	0.0212	-5.5696	-20.9714	0.9898	15.4017	-852.1937	-499.1387	-1.7082	-1.8693
0.0135	0.62	375	0.0182	-5.2646	-20.3901	0.9907	15.1254	-846.3809	-496.0890	-1.7285	-1.8897
0.014	0.66	400	0.0182	-5.5057	-21.1579	0.9907	15.6522	-854.0594	-498.4994	-1.7137	-1.8783
0.0122	0.7	425	0.0172	-5.3398	-20.7520	0.9907	15.4122	-849.9997	-496.8405	-1.7231	-1.8857
0.0144	0.74	450	0.0164	-4.6606	-19.3766	0.9917	14.7160	-836.2463	-490.0483	-1.7465	-1.9042
0.0103	0.78	475	0.0160	-4.8739	-20.1058	0.9907	15.2319	-843.5385	-492.1819	-1.7445	-1.9064
0.0147	0.82	500	0.0156	-5.1220	-20.9607	0.9917	15.8387	-852.0875	-494.6623	-1.7434	-1.9092
0.0154	0.86	525	0.0155	-5.1481	-21.3994	0.9917	16.2513	-856.4740	-494.9235	-1.7357	-1.9040
0.0158	0.91	550	0.0151	-5.6088	-22.9532	0.9917	17.3444	-872.0123	-499.5304	-1.7139	-1.8881
0.0053	0.95	575	0.0149	-5.7209	-23.5217	0.9917	17.8008	-877.6972	-500.6515	-1.7113	-1.8888
0.008	0.99	600	0.0147	-5.7523	-23.7474	0.9917	17.9952	-879.9544	-500.9651	-1.7086	-1.8878
0.0049	1.03	625	0.0154	-6.1839	-24.8883	0.9907	18.7044	-891.3632	-505.2818	-1.6731	-1.8585
0.0057	1.07	650	0.0155	-6.4947	-25.8924	0.9917	19.3977	-901.4037	-508.3892	-1.6592	-1.8484
0.0076	1.11	675	0.0158	-6.8543	-26.9217	0.9917	20.0674	-911.6970	-511.9859	-1.6407	-1.8339
0.004	1.15	700	0.0158	-7.1325	-27.7743	0.9917	20.6418	-920.2236	-514.7678	-1.6269	-1.8236
0.0168	1.19	725	0.0157	-6.9019	-26.2791	0.9917	19.3772	-905.2711	-512.4611	-1.6566	-1.8448
0.0022	1.23	750	0.0163	-6.9586	-26.5145	0.9917	19.5559	-907.6251	-513.0281	-1.6533	-1.8423
0.0039	1.28	775	0.0165	-7.5386	-28.2224	0.9917	20.6837	-924.7038	-518.8289	-1.6369	-1.8327
0.002	1.32	800	0.0165	-7.6568	-28.6441	0.9907	20.9872	-928.9208	-520.0109	-1.6365	-1.8344
0.002	1.36	825	0.0165	-7.7989	-29.2028	0.9917	21.4038	-934.5078	-521.4318	-1.6348	-1.8352
0.0019	1.4	850	0.0165	-7.8978	-29.5958	0.9917	21.6980	-938.4382	-522.4203	-1.6166	-1.8169
0.0041	1.44	875	0.0162	-7.9696	-29.7930	0.9917	21.8234	-940.4100	-523.1380	-1.6165	-1.8176
0.0023	1.48	900	0.0164	-8.2086	-30.6909	0.9917	22.4823	-949.3892	-525.5286	-1.6045	-1.8093
0.0038	1.52	925	0.0166	-8.1217	-30.6727	0.9917	22.5510	-949.2076	-524.6597	-1.5919	-1.7978
0.0096	1.56	950	0.0162	-7.8257	-30.1144	0.9917	22.2887	-943.6237	-521.6992	-1.5909	-1.7956
0.0057	1.6	975	0.0166	-8.0335	-30.6654	0.9917	22.6319	-949.1342	-523.7775	-1.5854	-1.7919
0.0046	1.65	1000	0.0165	-8.1757	-31.0139	0.9917	22.8382	-952.6191	-525.2000	-1.5768	-1.7852
0.0009	1.69	1025	0.0165	-8.0553	-30.7565	0.9917	22.7012	-950.0453	-523.9951	-1.5757	-1.7830
0.002	1.73	1050	0.0164	-8.1838	-31.3365	0.9917	23.1528	-955.8453	-525.2800	-1.5692	-1.7790
0.0069	1.77	1075	0.0163	-8.1908	-31.4118	0.9917	23.2210	-956.5981	-525.3508	-1.5749	-1.7850
0.0029	1.81	1100	0.0166	-8.4138	-32.0830	0.9917	23.6692	-963.3098	-527.5802	-1.5624	-1.7752
0.0047	1.85	1125	0.0166	-8.4223	-32.1526	0.9917	23.7304	-964.0065	-527.6652	-1.5631	-1.7759
0.0037	1.89	1150	0.0163	-8.1563	-31.3209	0.9917	23.1646	-955.6895	-525.0057	-1.5739	-1.7832
0.0026	1.93	1175	0.0163	-8.2107	-31.5009	0.9917	23.2901	-957.4888	-525.5498	-1.5708	-1.7807
0.0058	1.98	1200	0.0162	-8.1731	-31.3826	0.9917	23.2095	-956.3063	-525.1735	-1.5719	-1.7813

Framework versions

• Transformers 4.37.0
• Pytorch 2.1.2+cu121
• Datasets 2.14.6
• Tokenizers 0.15.2