djstrong commited on
Commit
36e06b6
1 Parent(s): ccd6234

tasks descriptions

Browse files
Files changed (1) hide show
  1. src/about.py +24 -22
src/about.py CHANGED
@@ -70,28 +70,30 @@ or join our [Discord SpeakLeash](https://discord.gg/3G9DVM39)
70
  * add inference time
71
  * add metadata for models (e.g. #Params)
72
  * add more tasks
73
- * fix gradient when filtering
74
-
75
- ## Evaluation metrics
76
-
77
- - **belebele_pol_Latn**: accuracy
78
- - **polemo2_in**: accuracy
79
- - **polemo2_in_multiple_choice**: accuracy
80
- - **polemo2_out**: accuracy
81
- - **polemo2_out_multiple_choice**: accuracy
82
- - **polish_8tags_multiple_choice**: accuracy
83
- - **polish_8tags_regex**: accuracy
84
- - **polish_belebele_regex**: accuracy
85
- - **polish_dyk_multiple_choice**: accuracy - binary F1
86
- - **polish_dyk_regex**: accuracy - binary F1
87
- - **polish_ppc_multiple_choice**: accuracy
88
- - **polish_ppc_regex**: accuracy
89
- - **polish_psc_multiple_choice**: accuracy - binary F1
90
- - **polish_psc_regex**: accuracy - binary F1
91
- - **polish_cbd_multiple_choice**: accuracy - macro F1
92
- - **polish_cbd_regex**: accuracy - macro F1
93
- - **polish_klej_ner_multiple_choice**: accuracy
94
- - **polish_klej_ner_regex**: accuracy
 
 
95
 
96
 
97
  ## Reproducibility
 
70
  * add inference time
71
  * add metadata for models (e.g. #Params)
72
  * add more tasks
73
+ * use model templates
74
+
75
+ ## Tasks
76
+
77
+ | Task | Dataset | Metric | Type |
78
+ |---------------------------------|---------------------------------------|-----------|-----------------|
79
+ | belebele_pol_Latn | facebook/belebele | accuracy | multiple_choice |
80
+ | polemo2_in | allegro/klej-polemo2-in | accuracy | generate_until |
81
+ | polemo2_in_multiple_choice | allegro/klej-polemo2-in | accuracy | multiple_choice |
82
+ | polemo2_out | allegro/klej-polemo2-out | accuracy | generate_until |
83
+ | polemo2_out_multiple_choice | allegro/klej-polemo2-out | accuracy | multiple_choice |
84
+ | polish_8tags_multiple_choice | sdadas/8tags | accuracy | multiple_choice |
85
+ | polish_8tags_regex | sdadas/8tags | accuracy | generate_until |
86
+ | polish_belebele_regex | facebook/belebele | accuracy | generate_until |
87
+ | polish_dyk_multiple_choice | allegro/klej-dyk | binary F1 | multiple_choice |
88
+ | polish_dyk_regex | allegro/klej-dyk | binary F1 | generate_until |
89
+ | polish_ppc_multiple_choice | sdadas/ppc | accuracy | multiple_choice |
90
+ | polish_ppc_regex | sdadas/ppc | accuracy | generate_until |
91
+ | polish_psc_multiple_choice | allegro/klej-psc | binary F1 | multiple_choice |
92
+ | polish_psc_regex | allegro/klej-psc | binary F1 | generate_until |
93
+ | polish_cbd_multiple_choice | ptaszynski/PolishCyberbullyingDataset | macro F1 | multiple_choice |
94
+ | polish_cbd_regex | ptaszynski/PolishCyberbullyingDataset | macro F1 | generate_until |
95
+ | polish_klej_ner_multiple_choice | allegro/klej-nkjp-ner | accuracy | multiple_choice |
96
+ | polish_klej_ner_regex | allegro/klej-nkjp-ner | accuracy | generate_until |
97
 
98
 
99
  ## Reproducibility