zephyr-7b-dpo-lora-pubmedqa

This model is a fine-tuned version of EllieS/zephyr-7b-sft-qlora on the EllieS/pubmedqa_dpo_data dataset. It achieves the following results on the evaluation set:

• Loss: 0.0603
• Rewards/chosen: 0.3150
• Rewards/rejected: -2.4832
• Rewards/accuracies: 1.0
• Rewards/margins: 2.7982
• Logps/rejected: -284.3339
• Logps/chosen: -0.1594
• Logits/rejected: -3.0974
• Logits/chosen: -3.0713

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6917	0.0	100	0.6905	-0.0001	-0.0055	1.0	0.0054	-36.5632	-31.6725	-2.8964	-2.8908
0.6767	0.01	200	0.6758	0.0015	-0.0337	0.8000	0.0352	-39.3843	-31.5100	-2.8965	-2.8896
0.6195	0.01	300	0.6289	0.0879	-0.0489	0.8000	0.1368	-40.9016	-22.8663	-2.8974	-2.8895
0.5265	0.02	400	0.5498	0.1648	-0.1775	0.8000	0.3423	-53.7617	-15.1788	-2.8898	-2.8791
0.4265	0.02	500	0.4915	0.1800	-0.3847	0.8000	0.5647	-74.4904	-13.6600	-2.8837	-2.8704
0.3822	0.02	600	0.4624	0.1435	-0.5997	0.8000	0.7433	-95.9890	-17.3068	-2.8766	-2.8639
0.3164	0.03	700	0.4349	0.1388	-0.7367	0.8000	0.8755	-109.6869	-17.7766	-2.8752	-2.8643
0.2646	0.03	800	0.4327	0.1176	-0.8370	0.8000	0.9546	-119.7166	-19.9026	-2.8746	-2.8661
0.2358	0.04	900	0.4248	0.1070	-0.9301	0.8000	1.0371	-129.0235	-20.9559	-2.8792	-2.8718
0.2427	0.04	1000	0.4057	0.1150	-1.0071	0.8000	1.1221	-136.7261	-20.1632	-2.8785	-2.8730
0.2167	0.04	1100	0.3903	0.1183	-1.0874	0.8000	1.2057	-144.7528	-19.8280	-2.8804	-2.8753
0.1888	0.05	1200	0.3909	0.1086	-1.1508	0.8000	1.2593	-151.0921	-20.8049	-2.8784	-2.8719
0.17	0.05	1300	0.4013	0.0826	-1.2194	0.8000	1.3020	-157.9523	-23.4004	-2.8826	-2.8756
0.139	0.06	1400	0.3722	0.1104	-1.2731	0.8000	1.3834	-163.3211	-20.6215	-2.8859	-2.8795
0.1525	0.06	1500	0.3450	0.1389	-1.3306	0.8000	1.4695	-169.0768	-17.7689	-2.8962	-2.8891
0.1622	0.06	1600	0.3556	0.1096	-1.3911	0.8000	1.5007	-175.1273	-20.7011	-2.9043	-2.8991
0.1242	0.07	1700	0.3780	0.0727	-1.4364	0.8000	1.5091	-179.6512	-24.3892	-2.9111	-2.9064
0.1232	0.07	1800	0.3637	0.0774	-1.4857	0.8000	1.5631	-184.5862	-23.9169	-2.9201	-2.9152
0.1141	0.08	1900	0.3232	0.1026	-1.5624	0.8000	1.6651	-192.2589	-21.3979	-2.9257	-2.9200
0.1525	0.08	2000	0.3077	0.0918	-1.6428	0.8000	1.7346	-200.2968	-22.4783	-2.9355	-2.9305
0.1236	0.08	2100	0.4024	0.0066	-1.6566	0.8000	1.6632	-201.6738	-31.0022	-2.9432	-2.9440
0.1467	0.09	2200	0.3299	0.0719	-1.7345	0.8000	1.8064	-209.4659	-24.4665	-2.9485	-2.9427
0.1314	0.09	2300	0.3409	0.0424	-1.8004	0.8000	1.8428	-216.0566	-27.4246	-2.9606	-2.9549
0.0827	0.1	2400	0.3526	0.0279	-1.8371	0.8000	1.8651	-219.7282	-28.8650	-2.9690	-2.9643
0.132	0.1	2500	0.2030	0.1635	-1.9493	0.8000	2.1128	-230.9491	-15.3147	-2.9840	-2.9775
0.0851	0.1	2600	0.1688	0.2903	-1.9136	1.0	2.2039	-227.3723	-2.6290	-2.9917	-2.9828
0.1549	0.11	2700	0.1753	0.2794	-1.9249	1.0	2.2043	-228.5043	-3.7234	-2.9974	-2.9883
0.0822	0.11	2800	0.2224	0.1933	-1.9332	0.8000	2.1265	-229.3374	-12.3327	-3.0048	-2.9948
0.0862	0.12	2900	0.2089	0.2191	-1.9430	0.8000	2.1621	-230.3155	-9.7481	-3.0075	-2.9997
0.0702	0.12	3000	0.2735	0.1033	-1.9629	0.8000	2.0662	-232.3102	-21.3305	-3.0130	-3.0048
0.0865	0.12	3100	0.0754	0.3161	-2.2881	1.0	2.6042	-264.8222	-0.0494	-3.0220	-3.0058
0.0763	0.13	3200	0.0745	0.3162	-2.2987	1.0	2.6149	-265.8853	-0.0435	-3.0279	-3.0137
0.11	0.13	3300	0.1464	0.3107	-2.0080	1.0	2.3186	-236.8128	-0.5936	-3.0307	-3.0167
0.1114	0.14	3400	0.1534	0.2652	-2.0338	1.0	2.2989	-239.3922	-5.1433	-3.0296	-3.0148
0.0934	0.14	3500	0.1702	0.2508	-2.0232	1.0	2.2741	-238.3399	-6.5776	-3.0312	-3.0213
0.063	0.14	3600	0.0717	0.3162	-2.3330	1.0	2.6492	-269.3163	-0.0426	-3.0421	-3.0220
0.0673	0.15	3700	0.1843	0.2409	-2.0103	1.0	2.2512	-237.0486	-7.5727	-3.0412	-3.0296
0.1292	0.15	3800	0.0723	0.3161	-2.3246	1.0	2.6407	-268.4736	-0.0498	-3.0497	-3.0278
0.0683	0.16	3900	0.0699	0.3161	-2.3559	1.0	2.6720	-271.6014	-0.0451	-3.0506	-3.0321
0.1472	0.16	4000	0.0697	0.3161	-2.3568	1.0	2.6729	-271.6938	-0.0454	-3.0525	-3.0307
0.104	0.16	4100	0.0977	0.1714	-2.3361	1.0	2.5074	-269.6208	-14.5249	-3.0556	-3.0361
0.103	0.17	4200	0.0802	0.3159	-2.2830	1.0	2.5990	-264.3194	-0.0656	-3.0572	-3.0348
0.0939	0.17	4300	0.0705	0.3161	-2.3567	1.0	2.6728	-271.6851	-0.0492	-3.0577	-3.0352
0.1012	0.18	4400	0.0687	0.3161	-2.3729	1.0	2.6889	-273.3021	-0.0526	-3.0618	-3.0388
0.0928	0.18	4500	0.0678	0.3159	-2.3822	1.0	2.6981	-274.2347	-0.0664	-3.0640	-3.0422
0.092	0.18	4600	0.0681	0.3161	-2.3780	1.0	2.6941	-273.8116	-0.0472	-3.0627	-3.0396
0.0966	0.19	4700	0.0685	0.3159	-2.3792	1.0	2.6952	-273.9405	-0.0664	-3.0634	-3.0394
0.0605	0.19	4800	0.0679	0.3130	-2.3880	1.0	2.7011	-274.8187	-0.3556	-3.0658	-3.0441
0.1031	0.2	4900	0.0748	0.2545	-2.3915	1.0	2.6460	-275.1633	-6.2069	-3.0700	-3.0534
0.093	0.2	5000	0.0670	0.3159	-2.3964	1.0	2.7123	-275.6591	-0.0692	-3.0697	-3.0463
0.1176	0.2	5100	0.0728	0.2609	-2.3988	1.0	2.6597	-275.8960	-5.5717	-3.0660	-3.0456
0.1232	0.21	5200	0.0659	0.3157	-2.4090	1.0	2.7246	-276.9131	-0.0931	-3.0683	-3.0432
0.1606	0.21	5300	0.0654	0.3158	-2.4143	1.0	2.7300	-277.4427	-0.0841	-3.0748	-3.0497
0.0604	0.22	5400	0.0655	0.3159	-2.4142	1.0	2.7301	-277.4360	-0.0707	-3.0763	-3.0505
0.0966	0.22	5500	0.0653	0.3158	-2.4151	1.0	2.7308	-277.5210	-0.0842	-3.0783	-3.0537
0.0609	0.22	5600	0.0662	0.3159	-2.4082	1.0	2.7241	-276.8354	-0.0684	-3.0757	-3.0549
0.0583	0.23	5700	0.0661	0.3157	-2.4106	1.0	2.7263	-277.0789	-0.0922	-3.0779	-3.0542
0.0721	0.23	5800	0.0648	0.3157	-2.4224	1.0	2.7381	-278.2588	-0.0944	-3.0766	-3.0521
0.0935	0.24	5900	0.0649	0.3158	-2.4218	1.0	2.7376	-278.1963	-0.0777	-3.0794	-3.0554
0.1559	0.24	6000	0.0648	0.3158	-2.4243	1.0	2.7401	-278.4456	-0.0830	-3.0778	-3.0567
0.0973	0.24	6100	0.0645	0.3159	-2.4224	1.0	2.7384	-278.2578	-0.0666	-3.0870	-3.0614
0.1298	0.25	6200	0.0640	0.3155	-2.4313	1.0	2.7469	-279.1472	-0.1059	-3.0851	-3.0611
0.1037	0.25	6300	0.0648	0.3153	-2.4239	1.0	2.7393	-278.4086	-0.1257	-3.0885	-3.0585
0.0939	0.26	6400	0.0633	0.3154	-2.4417	1.0	2.7571	-280.1844	-0.1197	-3.0841	-3.0594
0.0588	0.26	6500	0.0635	0.3152	-2.4399	1.0	2.7551	-280.0093	-0.1416	-3.0841	-3.0580
0.1025	0.26	6600	0.0637	0.3143	-2.4371	1.0	2.7514	-279.7303	-0.2316	-3.0867	-3.0576
0.0583	0.27	6700	0.0636	0.3155	-2.4383	1.0	2.7538	-279.8492	-0.1092	-3.0834	-3.0596
0.0882	0.27	6800	0.0630	0.3154	-2.4476	1.0	2.7631	-280.7805	-0.1187	-3.0858	-3.0619
0.0744	0.28	6900	0.0637	0.3152	-2.4389	1.0	2.7541	-279.9033	-0.1358	-3.0874	-3.0588
0.0555	0.28	7000	0.0632	0.3153	-2.4449	1.0	2.7603	-280.5067	-0.1259	-3.0870	-3.0640
0.0745	0.28	7100	0.0644	0.3148	-2.4314	1.0	2.7463	-279.1595	-0.1782	-3.0864	-3.0591
0.112	0.29	7200	0.0634	0.3152	-2.4435	1.0	2.7587	-280.3658	-0.1356	-3.0870	-3.0601
0.0566	0.29	7300	0.0630	0.3154	-2.4478	1.0	2.7632	-280.7963	-0.1159	-3.0888	-3.0650
0.0632	0.3	7400	0.0632	0.3150	-2.4466	1.0	2.7616	-280.6743	-0.1563	-3.0852	-3.0647
0.1134	0.3	7500	0.0626	0.3151	-2.4519	1.0	2.7670	-281.2076	-0.1470	-3.0885	-3.0592
0.0802	0.3	7600	0.0626	0.3153	-2.4532	1.0	2.7685	-281.3376	-0.1320	-3.0896	-3.0624
0.0577	0.31	7700	0.1374	0.3151	-2.0953	1.0	2.4105	-245.5485	-0.1450	-3.0868	-3.0604
0.1363	0.31	7800	0.0626	0.3150	-2.4535	1.0	2.7685	-281.3674	-0.1637	-3.0920	-3.0651
0.1268	0.32	7900	0.0621	0.3151	-2.4577	1.0	2.7727	-281.7813	-0.1512	-3.0910	-3.0626
0.0593	0.32	8000	0.0618	0.3153	-2.4621	1.0	2.7774	-282.2267	-0.1342	-3.0912	-3.0626
0.1578	0.32	8100	0.0621	0.3154	-2.4563	1.0	2.7717	-281.6464	-0.1229	-3.0946	-3.0659
0.1307	0.33	8200	0.0620	0.3153	-2.4588	1.0	2.7741	-281.8946	-0.1311	-3.0971	-3.0699
0.0743	0.33	8300	0.0613	0.3152	-2.4694	1.0	2.7846	-282.9548	-0.1430	-3.0962	-3.0691
0.0892	0.34	8400	0.0614	0.3152	-2.4670	1.0	2.7822	-282.7147	-0.1406	-3.0969	-3.0686
0.091	0.34	8500	0.0625	0.3151	-2.4544	1.0	2.7696	-281.4597	-0.1490	-3.1052	-3.0734
0.1245	0.34	8600	0.0618	0.3153	-2.4570	1.0	2.7723	-281.7121	-0.1273	-3.1034	-3.0742
0.1212	0.35	8700	0.0609	0.3151	-2.4725	1.0	2.7876	-283.2621	-0.1509	-3.1016	-3.0750
0.062	0.35	8800	0.0609	0.3148	-2.4739	1.0	2.7887	-283.4044	-0.1800	-3.1021	-3.0740
0.1407	0.36	8900	0.0609	0.3151	-2.4735	1.0	2.7886	-283.3669	-0.1549	-3.1023	-3.0728
0.0594	0.36	9000	0.0605	0.3150	-2.4794	1.0	2.7944	-283.9577	-0.1585	-3.1003	-3.0703
0.065	0.36	9100	0.0604	0.3153	-2.4803	1.0	2.7956	-284.0407	-0.1291	-3.1035	-3.0733
0.0942	0.37	9200	0.0605	0.3152	-2.4780	1.0	2.7932	-283.8191	-0.1447	-3.1043	-3.0751
0.0863	0.37	9300	0.0608	0.3152	-2.4752	1.0	2.7904	-283.5404	-0.1411	-3.1016	-3.0714
0.084	0.38	9400	0.0606	0.3153	-2.4782	1.0	2.7934	-283.8343	-0.1349	-3.1007	-3.0726
0.1181	0.38	9500	0.0605	0.3150	-2.4798	1.0	2.7948	-283.9931	-0.1562	-3.1032	-3.0748
0.1231	0.38	9600	0.0603	0.3148	-2.4829	1.0	2.7977	-284.3057	-0.1813	-3.1009	-3.0725
0.1187	0.39	9700	0.0603	0.3151	-2.4813	1.0	2.7964	-284.1463	-0.1474	-3.1023	-3.0728
0.1663	0.39	9800	0.0604	0.3149	-2.4799	1.0	2.7948	-284.0080	-0.1717	-3.1038	-3.0729
0.082	0.4	9900	0.0603	0.3150	-2.4808	1.0	2.7958	-284.0976	-0.1608	-3.1028	-3.0724
0.0561	0.4	10000	0.0601	0.3150	-2.4849	1.0	2.7999	-284.5103	-0.1611	-3.1023	-3.0735
0.1215	0.4	10100	0.0602	0.3149	-2.4835	1.0	2.7983	-284.3610	-0.1741	-3.1025	-3.0729
0.0617	0.41	10200	0.0600	0.3152	-2.4852	1.0	2.8004	-284.5347	-0.1360	-3.1022	-3.0739
0.0834	0.41	10300	0.0599	0.3152	-2.4869	1.0	2.8021	-284.7045	-0.1425	-3.1017	-3.0739
0.0931	0.42	10400	0.0604	0.3149	-2.4814	1.0	2.7963	-284.1569	-0.1700	-3.1008	-3.0727
0.0576	0.42	10500	0.0600	0.3152	-2.4850	1.0	2.8002	-284.5181	-0.1442	-3.0986	-3.0707
0.0882	0.42	10600	0.0606	0.3146	-2.4799	1.0	2.7946	-284.0095	-0.1975	-3.0995	-3.0712
0.0893	0.43	10700	0.0599	0.3149	-2.4865	1.0	2.8014	-284.6656	-0.1668	-3.1019	-3.0728
0.0551	0.43	10800	0.0597	0.3145	-2.4914	1.0	2.8059	-285.1605	-0.2109	-3.1032	-3.0748
0.0875	0.44	10900	0.0598	0.3148	-2.4889	1.0	2.8037	-284.9028	-0.1783	-3.1026	-3.0747
0.0902	0.44	11000	0.0597	0.3150	-2.4892	1.0	2.8042	-284.9340	-0.1595	-3.1028	-3.0734
0.0619	0.44	11100	0.0600	0.3153	-2.4867	1.0	2.8019	-284.6834	-0.1342	-3.1034	-3.0740
0.1255	0.45	11200	0.0598	0.3153	-2.4891	1.0	2.8044	-284.9269	-0.1275	-3.1024	-3.0746
0.0564	0.45	11300	0.0600	0.3155	-2.4855	1.0	2.8009	-284.5616	-0.1136	-3.1028	-3.0736
0.0554	0.46	11400	0.0599	0.3151	-2.4881	1.0	2.8032	-284.8240	-0.1496	-3.0997	-3.0722
0.0543	0.46	11500	0.0604	0.3152	-2.4811	1.0	2.7963	-284.1210	-0.1361	-3.1028	-3.0740
0.0568	0.46	11600	0.0598	0.3147	-2.4902	1.0	2.8049	-285.0388	-0.1892	-3.1031	-3.0762
0.0562	0.47	11700	0.0602	0.3148	-2.4845	1.0	2.7993	-284.4625	-0.1755	-3.1001	-3.0733
0.056	0.47	11800	0.0600	0.3146	-2.4882	1.0	2.8028	-284.8389	-0.2018	-3.1000	-3.0738
0.0557	0.48	11900	0.0601	0.3148	-2.4855	1.0	2.8003	-284.5624	-0.1753	-3.0986	-3.0718
0.0913	0.48	12000	0.0600	0.3151	-2.4873	1.0	2.8024	-284.7490	-0.1495	-3.0990	-3.0730
0.1144	0.48	12100	0.0602	0.3150	-2.4849	1.0	2.7999	-284.5060	-0.1564	-3.0945	-3.0693
0.0911	0.49	12200	0.0603	0.3151	-2.4833	1.0	2.7984	-284.3466	-0.1472	-3.0992	-3.0716
0.091	0.49	12300	0.0604	0.3149	-2.4833	1.0	2.7982	-284.3491	-0.1747	-3.0975	-3.0713
0.0548	0.5	12400	0.0606	0.3143	-2.4812	1.0	2.7956	-284.1383	-0.2264	-3.0971	-3.0704
0.0622	0.5	12500	0.0604	0.3146	-2.4819	1.0	2.7964	-284.2023	-0.2027	-3.0979	-3.0709
0.056	0.5	12600	0.0601	0.3148	-2.4851	1.0	2.7999	-284.5273	-0.1777	-3.0984	-3.0740
0.1175	0.51	12700	0.0601	0.3146	-2.4868	1.0	2.8014	-284.7003	-0.2008	-3.0952	-3.0735
0.0714	0.51	12800	0.0605	0.3152	-2.4818	1.0	2.7969	-284.1913	-0.1419	-3.0964	-3.0714
0.0572	0.52	12900	0.0600	0.3151	-2.4863	1.0	2.8014	-284.6450	-0.1463	-3.0980	-3.0730
0.09	0.52	13000	0.0601	0.3152	-2.4854	1.0	2.8006	-284.5557	-0.1361	-3.0992	-3.0733
0.055	0.52	13100	0.0601	0.3147	-2.4865	1.0	2.8012	-284.6667	-0.1921	-3.0975	-3.0725
0.0937	0.53	13200	0.0608	0.3149	-2.4763	1.0	2.7912	-283.6473	-0.1729	-3.0956	-3.0706
0.0543	0.53	13300	0.0604	0.3148	-2.4826	1.0	2.7974	-284.2760	-0.1764	-3.0962	-3.0705
0.0549	0.54	13400	0.0601	0.3152	-2.4860	1.0	2.8012	-284.6145	-0.1399	-3.0965	-3.0708
0.057	0.54	13500	0.0604	0.3148	-2.4821	1.0	2.7969	-284.2305	-0.1809	-3.0939	-3.0694
0.2197	0.54	13600	0.0602	0.3152	-2.4842	1.0	2.7994	-284.4350	-0.1356	-3.0985	-3.0721
0.0871	0.55	13700	0.0601	0.3151	-2.4850	1.0	2.8002	-284.5201	-0.1456	-3.0979	-3.0717
0.1127	0.55	13800	0.0608	0.3151	-2.4771	1.0	2.7922	-283.7219	-0.1474	-3.0968	-3.0709
0.0862	0.56	13900	0.0609	0.3150	-2.4758	1.0	2.7908	-283.5951	-0.1580	-3.0982	-3.0726
0.0862	0.56	14000	0.0613	0.3149	-2.4719	1.0	2.7868	-283.2067	-0.1697	-3.0952	-3.0701
0.0904	0.56	14100	0.0611	0.3148	-2.4745	1.0	2.7893	-283.4654	-0.1784	-3.0964	-3.0712
0.0932	0.57	14200	0.0608	0.3148	-2.4784	1.0	2.7932	-283.8593	-0.1848	-3.0954	-3.0698
0.0765	0.57	14300	0.0610	0.3146	-2.4759	1.0	2.7906	-283.6089	-0.1970	-3.0971	-3.0715
0.076	0.58	14400	0.0604	0.3148	-2.4822	1.0	2.7969	-284.2308	-0.1812	-3.0974	-3.0716
0.0553	0.58	14500	0.0608	0.3144	-2.4788	1.0	2.7932	-283.8933	-0.2187	-3.0962	-3.0722
0.0975	0.58	14600	0.0604	0.3149	-2.4825	1.0	2.7974	-284.2697	-0.1702	-3.0973	-3.0710
0.0544	0.59	14700	0.0607	0.3148	-2.4793	1.0	2.7941	-283.9458	-0.1840	-3.0964	-3.0708
0.0545	0.59	14800	0.0605	0.3151	-2.4812	1.0	2.7963	-284.1385	-0.1515	-3.0972	-3.0717
0.0902	0.6	14900	0.0606	0.3148	-2.4812	1.0	2.7959	-284.1325	-0.1829	-3.0980	-3.0710
0.059	0.6	15000	0.0599	0.3150	-2.4877	1.0	2.8027	-284.7884	-0.1639	-3.0972	-3.0708
0.1524	0.6	15100	0.0601	0.3147	-2.4862	1.0	2.8009	-284.6385	-0.1890	-3.0986	-3.0718
0.0558	0.61	15200	0.0605	0.3149	-2.4815	1.0	2.7964	-284.1689	-0.1694	-3.0972	-3.0715
0.0969	0.61	15300	0.0601	0.3151	-2.4861	1.0	2.8012	-284.6226	-0.1482	-3.0963	-3.0704
0.0569	0.62	15400	0.0600	0.3151	-2.4876	1.0	2.8027	-284.7789	-0.1546	-3.0975	-3.0719
0.0935	0.62	15500	0.0603	0.3151	-2.4838	1.0	2.7989	-284.3983	-0.1488	-3.0961	-3.0698
0.0815	0.62	15600	0.0602	0.3143	-2.4861	1.0	2.8004	-284.6287	-0.2292	-3.0973	-3.0711
0.0556	0.63	15700	0.0603	0.3148	-2.4832	1.0	2.7979	-284.3312	-0.1816	-3.0971	-3.0709
0.0559	0.63	15800	0.0607	0.3147	-2.4795	1.0	2.7942	-283.9677	-0.1932	-3.0964	-3.0704
0.0871	0.64	15900	0.0606	0.3148	-2.4804	1.0	2.7952	-284.0538	-0.1793	-3.0967	-3.0718
0.0551	0.64	16000	0.0607	0.3146	-2.4792	1.0	2.7938	-283.9374	-0.2004	-3.0957	-3.0699
0.1236	0.64	16100	0.0606	0.3149	-2.4805	1.0	2.7954	-284.0651	-0.1657	-3.0981	-3.0712
0.0768	0.65	16200	0.0605	0.3152	-2.4818	1.0	2.7969	-284.1944	-0.1449	-3.0964	-3.0698
0.0923	0.65	16300	0.0606	0.3147	-2.4809	1.0	2.7957	-284.1096	-0.1852	-3.0964	-3.0711
0.0956	0.66	16400	0.0606	0.3150	-2.4811	1.0	2.7962	-284.1297	-0.1552	-3.0961	-3.0700
0.0552	0.66	16500	0.0605	0.3151	-2.4811	1.0	2.7962	-284.1294	-0.1549	-3.0966	-3.0703
0.0921	0.66	16600	0.0606	0.3150	-2.4792	1.0	2.7942	-283.9340	-0.1596	-3.0962	-3.0705
0.1013	0.67	16700	0.0605	0.3150	-2.4812	1.0	2.7962	-284.1385	-0.1641	-3.0952	-3.0693
0.0583	0.67	16800	0.0607	0.3152	-2.4790	1.0	2.7942	-283.9129	-0.1385	-3.0957	-3.0703
0.0564	0.68	16900	0.0605	0.3151	-2.4816	1.0	2.7967	-284.1773	-0.1529	-3.0970	-3.0707
0.0925	0.68	17000	0.0603	0.3149	-2.4838	1.0	2.7987	-284.3918	-0.1673	-3.0960	-3.0705
0.056	0.68	17100	0.0606	0.3151	-2.4814	1.0	2.7964	-284.1519	-0.1524	-3.0951	-3.0697
0.0923	0.69	17200	0.0605	0.3150	-2.4822	1.0	2.7972	-284.2387	-0.1643	-3.0958	-3.0703
0.0832	0.69	17300	0.0602	0.3150	-2.4859	1.0	2.8009	-284.6101	-0.1606	-3.0962	-3.0707
0.171	0.7	17400	0.0605	0.3150	-2.4817	1.0	2.7967	-284.1821	-0.1576	-3.0961	-3.0701
0.0823	0.7	17500	0.0606	0.3149	-2.4806	1.0	2.7954	-284.0712	-0.1717	-3.0964	-3.0699
0.0707	0.7	17600	0.0608	0.3150	-2.4775	1.0	2.7924	-283.7639	-0.1645	-3.0964	-3.0705
0.0906	0.71	17700	0.0607	0.3150	-2.4787	1.0	2.7937	-283.8846	-0.1602	-3.0963	-3.0701
0.0622	0.71	17800	0.0606	0.3148	-2.4811	1.0	2.7959	-284.1299	-0.1805	-3.0950	-3.0692
0.0548	0.72	17900	0.0603	0.3148	-2.4834	1.0	2.7982	-284.3509	-0.1764	-3.0954	-3.0708
0.0557	0.72	18000	0.0603	0.3148	-2.4839	1.0	2.7987	-284.4059	-0.1814	-3.0963	-3.0701
0.0551	0.72	18100	0.0603	0.3147	-2.4834	1.0	2.7982	-284.3599	-0.1855	-3.0952	-3.0702
0.0688	0.73	18200	0.0603	0.3145	-2.4840	1.0	2.7984	-284.4141	-0.2146	-3.0954	-3.0694
0.0897	0.73	18300	0.0604	0.3149	-2.4823	1.0	2.7972	-284.2420	-0.1675	-3.0961	-3.0703
0.1115	0.74	18400	0.0604	0.3147	-2.4827	1.0	2.7974	-284.2878	-0.1883	-3.0952	-3.0703
0.0914	0.74	18500	0.0603	0.3148	-2.4838	1.0	2.7987	-284.4003	-0.1758	-3.0967	-3.0712
0.092	0.74	18600	0.0604	0.3149	-2.4836	1.0	2.7984	-284.3739	-0.1744	-3.0962	-3.0709
0.0641	0.75	18700	0.0607	0.3150	-2.4795	1.0	2.7944	-283.9636	-0.1641	-3.0953	-3.0704
0.0576	0.75	18800	0.0607	0.3147	-2.4797	1.0	2.7944	-283.9855	-0.1860	-3.0961	-3.0708
0.0539	0.76	18900	0.0606	0.3150	-2.4802	1.0	2.7952	-284.0349	-0.1604	-3.0960	-3.0708
0.0935	0.76	19000	0.0606	0.3143	-2.4806	1.0	2.7949	-284.0767	-0.2273	-3.0964	-3.0711
0.0887	0.76	19100	0.0605	0.3150	-2.4812	1.0	2.7962	-284.1394	-0.1649	-3.0951	-3.0697
0.1274	0.77	19200	0.0605	0.3144	-2.4823	1.0	2.7967	-284.2411	-0.2167	-3.0961	-3.0706
0.1333	0.77	19300	0.0604	0.3150	-2.4831	1.0	2.7982	-284.3301	-0.1556	-3.0949	-3.0695
0.0551	0.78	19400	0.0603	0.3148	-2.4839	1.0	2.7987	-284.4022	-0.1777	-3.0967	-3.0713
0.0575	0.78	19500	0.0604	0.3148	-2.4824	1.0	2.7972	-284.2572	-0.1827	-3.0964	-3.0711
0.0581	0.78	19600	0.0604	0.3148	-2.4831	1.0	2.7979	-284.3292	-0.1798	-3.0967	-3.0707
0.0869	0.79	19700	0.0603	0.3146	-2.4841	1.0	2.7987	-284.4286	-0.2042	-3.0962	-3.0708
0.0724	0.79	19800	0.0603	0.3151	-2.4844	1.0	2.7994	-284.4534	-0.1539	-3.0955	-3.0712
0.1063	0.8	19900	0.0604	0.3149	-2.4830	1.0	2.7979	-284.3166	-0.1672	-3.0965	-3.0702
0.0857	0.8	20000	0.0603	0.3149	-2.4843	1.0	2.7992	-284.4483	-0.1739	-3.0958	-3.0702
0.0561	0.8	20100	0.0603	0.3145	-2.4849	1.0	2.7994	-284.5063	-0.2069	-3.0964	-3.0707
0.0556	0.81	20200	0.0604	0.3149	-2.4828	1.0	2.7977	-284.2974	-0.1730	-3.0955	-3.0696
0.091	0.81	20300	0.0603	0.3151	-2.4846	1.0	2.7997	-284.4747	-0.1502	-3.0950	-3.0689
0.0899	0.82	20400	0.0603	0.3147	-2.4847	1.0	2.7994	-284.4896	-0.1901	-3.0964	-3.0700
0.0849	0.82	20500	0.0604	0.3151	-2.4824	1.0	2.7974	-284.2542	-0.1547	-3.0959	-3.0701
0.092	0.82	20600	0.0603	0.3145	-2.4852	1.0	2.7997	-284.5341	-0.2096	-3.0964	-3.0713
0.0871	0.83	20700	0.0604	0.3151	-2.4828	1.0	2.7979	-284.3003	-0.1508	-3.0957	-3.0698
0.0915	0.83	20800	0.0603	0.3148	-2.4838	1.0	2.7987	-284.4004	-0.1760	-3.0960	-3.0707
0.0566	0.84	20900	0.0602	0.3150	-2.4845	1.0	2.7994	-284.4606	-0.1612	-3.0951	-3.0690
0.0552	0.84	21000	0.0603	0.3145	-2.4849	1.0	2.7994	-284.5057	-0.2062	-3.0963	-3.0711
0.164	0.84	21100	0.0603	0.3149	-2.4840	1.0	2.7989	-284.4156	-0.1661	-3.0961	-3.0705
0.0829	0.85	21200	0.0604	0.3145	-2.4834	1.0	2.7979	-284.3568	-0.2073	-3.0961	-3.0707
0.0552	0.85	21300	0.0602	0.3145	-2.4857	1.0	2.8002	-284.5827	-0.2082	-3.0954	-3.0706
0.0797	0.86	21400	0.0603	0.3148	-2.4839	1.0	2.7987	-284.4014	-0.1769	-3.0967	-3.0708
0.0569	0.86	21500	0.0604	0.3149	-2.4833	1.0	2.7982	-284.3410	-0.1666	-3.0953	-3.0704
0.1239	0.86	21600	0.0603	0.3152	-2.4830	1.0	2.7982	-284.3148	-0.1403	-3.0961	-3.0710
0.1285	0.87	21700	0.0602	0.3148	-2.4854	1.0	2.8002	-284.5537	-0.1793	-3.0966	-3.0700
0.0557	0.87	21800	0.0603	0.3146	-2.4843	1.0	2.7989	-284.4450	-0.1955	-3.0965	-3.0704
0.0999	0.88	21900	0.0604	0.3149	-2.4833	1.0	2.7982	-284.3460	-0.1716	-3.0962	-3.0708
0.0704	0.88	22000	0.0604	0.3151	-2.4826	1.0	2.7977	-284.2776	-0.1531	-3.0955	-3.0703
0.0902	0.88	22100	0.0603	0.3148	-2.4837	1.0	2.7984	-284.3843	-0.1848	-3.0950	-3.0700
0.0842	0.89	22200	0.0603	0.3148	-2.4848	1.0	2.7997	-284.5004	-0.1761	-3.0966	-3.0708
0.091	0.89	22300	0.0603	0.3151	-2.4831	1.0	2.7982	-284.3220	-0.1475	-3.0962	-3.0710
0.0612	0.9	22400	0.0604	0.3149	-2.4828	1.0	2.7977	-284.2958	-0.1713	-3.0969	-3.0716
0.0555	0.9	22500	0.0603	0.3148	-2.4846	1.0	2.7994	-284.4779	-0.1785	-3.0965	-3.0706
0.1258	0.9	22600	0.0602	0.3149	-2.4853	1.0	2.8002	-284.5466	-0.1720	-3.0963	-3.0704
0.091	0.91	22700	0.0604	0.3150	-2.4827	1.0	2.7977	-284.2870	-0.1626	-3.0966	-3.0709
0.0939	0.91	22800	0.0604	0.3149	-2.4823	1.0	2.7972	-284.2432	-0.1687	-3.0964	-3.0707
0.057	0.92	22900	0.0603	0.3148	-2.4836	1.0	2.7984	-284.3766	-0.1771	-3.0973	-3.0711
0.1252	0.92	23000	0.0603	0.3149	-2.4835	1.0	2.7984	-284.3660	-0.1666	-3.0956	-3.0703
0.2019	0.92	23100	0.0604	0.3147	-2.4828	1.0	2.7974	-284.2932	-0.1937	-3.0962	-3.0714
0.0905	0.93	23200	0.0603	0.3149	-2.4848	1.0	2.7997	-284.4948	-0.1704	-3.0966	-3.0713
0.1121	0.93	23300	0.0602	0.3149	-2.4850	1.0	2.7999	-284.5165	-0.1670	-3.0955	-3.0710
0.0559	0.94	23400	0.0603	0.3149	-2.4840	1.0	2.7989	-284.4164	-0.1670	-3.0968	-3.0712
0.0555	0.94	23500	0.0603	0.3148	-2.4847	1.0	2.7994	-284.4821	-0.1826	-3.0958	-3.0703
0.093	0.94	23600	0.0602	0.3147	-2.4854	1.0	2.8002	-284.5602	-0.1857	-3.0959	-3.0706
0.0561	0.95	23700	0.0603	0.3149	-2.4838	1.0	2.7987	-284.3966	-0.1721	-3.0959	-3.0700
0.0576	0.95	23800	0.0603	0.3147	-2.4848	1.0	2.7994	-284.4909	-0.1915	-3.0966	-3.0703
0.1255	0.96	23900	0.0603	0.3147	-2.4843	1.0	2.7989	-284.4434	-0.1940	-3.0964	-3.0707
0.094	0.96	24000	0.0603	0.3147	-2.4840	1.0	2.7987	-284.4168	-0.1923	-3.0961	-3.0701
0.0543	0.96	24100	0.0603	0.3148	-2.4844	1.0	2.7992	-284.4561	-0.1817	-3.0963	-3.0710
0.0551	0.97	24200	0.0603	0.3148	-2.4844	1.0	2.7992	-284.4572	-0.1828	-3.0959	-3.0703
0.1155	0.97	24300	0.0603	0.3149	-2.4840	1.0	2.7989	-284.4147	-0.1652	-3.0971	-3.0712
0.0549	0.98	24400	0.0604	0.3149	-2.4822	1.0	2.7972	-284.2398	-0.1654	-3.0956	-3.0701
0.0556	0.98	24500	0.0602	0.3150	-2.4854	1.0	2.8004	-284.5563	-0.1569	-3.0960	-3.0699
0.0693	0.98	24600	0.0604	0.3147	-2.4835	1.0	2.7982	-284.3652	-0.1907	-3.0963	-3.0711
0.0919	0.99	24700	0.0604	0.3147	-2.4835	1.0	2.7982	-284.3639	-0.1894	-3.0962	-3.0710
0.0924	0.99	24800	0.0604	0.3148	-2.4834	1.0	2.7982	-284.3547	-0.1803	-3.0963	-3.0707
0.0575	1.0	24900	0.0603	0.3146	-2.4841	1.0	2.7987	-284.4260	-0.2015	-3.0973	-3.0712
0.0884	1.0	25000	0.0603	0.3150	-2.4832	1.0	2.7982	-284.3339	-0.1594	-3.0974	-3.0713

Framework versions

• PEFT 0.7.1
• Transformers 4.36.2
• Pytorch 2.1.1+cu121
• Datasets 2.14.6
• Tokenizers 0.15.0