slurp-slot_baseline-xlm_r-en

This model is a fine-tuned version of xlm-roberta-base on the SLURP dataset.

It achieves the following results on the test set:

• Loss: 0.3263
• Precision: 0.7954
• Recall: 0.8413
• F1: 0.8177
• Accuracy: 0.9268

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
1.1437	1.0	720	0.5236	0.6852	0.6623	0.6736	0.8860
0.5761	2.0	1440	0.3668	0.7348	0.7829	0.7581	0.9119
0.3087	3.0	2160	0.2996	0.7925	0.8280	0.8099	0.9270
0.2631	4.0	2880	0.2959	0.7872	0.8487	0.8168	0.9275
0.1847	5.0	3600	0.3121	0.7929	0.8373	0.8145	0.9290
0.1518	6.0	4320	0.3117	0.8080	0.8601	0.8332	0.9329
0.1232	7.0	5040	0.3153	0.7961	0.8490	0.8217	0.9267
0.0994	8.0	5760	0.3125	0.8105	0.8570	0.8331	0.9332
0.0968	9.0	6480	0.3242	0.8147	0.8637	0.8385	0.9329
0.0772	10.0	7200	0.3263	0.8145	0.8641	0.8386	0.9341

Test results per slot

slot	f1	tc_size
alarm_type	0.4	4
app_name	0.42857142857142855	10
artist_name	0.8122605363984675	123
audiobook_author	0.0	9
audiobook_name	0.6021505376344087	43
business_name	0.8530259365994236	184
business_type	0.6666666666666667	41
change_amount	0.6666666666666666	9
coffee_type	0.5333333333333333	6
color_type	0.8135593220338982	28
cooking_type	0.8333333333333333	14
currency_name	0.8611111111111112	70
date	0.9034267912772587	623
definition_word	0.88	97
device_type	0.8053691275167785	71
drink_type	0.0	2
email_address	0.9599999999999999	38
email_folder	0.9523809523809523	10
event_name	0.7643504531722054	321
food_type	0.7482014388489208	121
game_name	0.7789473684210527	44
general_frequency	0.5862068965517242	21
house_place	0.8840579710144928	68
ingredient	0.0	13
joke_type	0.9411764705882353	17
list_name	0.7979274611398963	91
meal_type	0.782608695652174	18
media_type	0.8596491228070176	173
movie_name	0.0	3
movie_type	0.5	3
music_album	0.0	2
music_descriptor	0.25	8
music_genre	0.7244094488188977	58
news_topic	0.5675675675675675	64
order_type	0.7941176470588235	29
person	0.9128094725511302	438
personal_info	0.6666666666666666	16
place_name	0.8725790010193679	493
player_setting	0.5405405405405405	42
playlist_name	0.5	27
podcast_descriptor	0.4888888888888888	28
podcast_name	0.5245901639344263	31
radio_name	0.6504065040650406	53
relation	0.8478260869565218	87
song_name	0.7058823529411765	54
time	0.7914893617021276	236
time_zone	0.7804878048780488	23
timeofday	0.8396946564885496	60
transport_agency	0.8571428571428571	18
transport_descriptor	0.0	2
transport_name	0.4	7
transport_type	0.9481481481481482	68
weather_descriptor	0.789272030651341	123

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3