model update

README.md

@@ -46,39 +46,72 @@ model-index:
     - name: MoverScore (Question Generation)
       type: moverscore_question_generation
       value: 59.06
+    - name: BLEU4 (Question & Answer Generation (with Gold Answer))
+      type: bleu4_question_answer_generation_with_gold_answer
+      value: 12.77
+    - name: ROUGE-L (Question & Answer Generation (with Gold Answer))
+      type: rouge_l_question_answer_generation_with_gold_answer
+      value: 42.77
+    - name: METEOR (Question & Answer Generation (with Gold Answer))
+      type: meteor_question_answer_generation_with_gold_answer
+      value: 37.58
+    - name: BERTScore (Question & Answer Generation (with Gold Answer))
+      type: bertscore_question_answer_generation_with_gold_answer
+      value: 89.41
+    - name: MoverScore (Question & Answer Generation (with Gold Answer))
+      type: moverscore_question_answer_generation_with_gold_answer
+      value: 63.56
+    - name: QAAlignedF1Score-BERTScore (Question & Answer Generation (with Gold Answer)) [Gold Answer]
+      type: qa_aligned_f1_score_bertscore_question_answer_generation_with_gold_answer_gold_answer
+      value: 89.43
+    - name: QAAlignedRecall-BERTScore (Question & Answer Generation (with Gold Answer)) [Gold Answer]
+      type: qa_aligned_recall_bertscore_question_answer_generation_with_gold_answer_gold_answer
+      value: 89.41
+    - name: QAAlignedPrecision-BERTScore (Question & Answer Generation (with Gold Answer)) [Gold Answer]
+      type: qa_aligned_precision_bertscore_question_answer_generation_with_gold_answer_gold_answer
+      value: 89.44
+    - name: QAAlignedF1Score-MoverScore (Question & Answer Generation (with Gold Answer)) [Gold Answer]
+      type: qa_aligned_f1_score_moverscore_question_answer_generation_with_gold_answer_gold_answer
+      value: 63.73
+    - name: QAAlignedRecall-MoverScore (Question & Answer Generation (with Gold Answer)) [Gold Answer]
+      type: qa_aligned_recall_moverscore_question_answer_generation_with_gold_answer_gold_answer
+      value: 63.72
+    - name: QAAlignedPrecision-MoverScore (Question & Answer Generation (with Gold Answer)) [Gold Answer]
+      type: qa_aligned_precision_moverscore_question_answer_generation_with_gold_answer_gold_answer
+      value: 63.75
     - name: BLEU4 (Question & Answer Generation)
       type: bleu4_question_answer_generation
-      value: 12.77
+      value: 1.83
     - name: ROUGE-L (Question & Answer Generation)
       type: rouge_l_question_answer_generation
-      value: 42.77
+      value: 15.46
     - name: METEOR (Question & Answer Generation)
       type: meteor_question_answer_generation
-      value: 37.58
+      value: 22.22
     - name: BERTScore (Question & Answer Generation)
       type: bertscore_question_answer_generation
-      value: 89.41
+      value: 69.78
     - name: MoverScore (Question & Answer Generation)
       type: moverscore_question_answer_generation
-      value: 63.56
+      value: 51.8
     - name: QAAlignedF1Score-BERTScore (Question & Answer Generation) [Gold Answer]
       type: qa_aligned_f1_score_bertscore_question_answer_generation_gold_answer
-      value: 89.43
+      value: 79.89
     - name: QAAlignedRecall-BERTScore (Question & Answer Generation) [Gold Answer]
       type: qa_aligned_recall_bertscore_question_answer_generation_gold_answer
-      value: 89.41
+      value: 82.56
     - name: QAAlignedPrecision-BERTScore (Question & Answer Generation) [Gold Answer]
       type: qa_aligned_precision_bertscore_question_answer_generation_gold_answer
-      value: 89.44
+      value: 77.46
     - name: QAAlignedF1Score-MoverScore (Question & Answer Generation) [Gold Answer]
       type: qa_aligned_f1_score_moverscore_question_answer_generation_gold_answer
-      value: 63.73
+      value: 54.82
     - name: QAAlignedRecall-MoverScore (Question & Answer Generation) [Gold Answer]
       type: qa_aligned_recall_moverscore_question_answer_generation_gold_answer
-      value: 63.72
+      value: 56.52
     - name: QAAlignedPrecision-MoverScore (Question & Answer Generation) [Gold Answer]
       type: qa_aligned_precision_moverscore_question_answer_generation_gold_answer
-      value: 63.75
+      value: 53.31
 ---
 
 # Model Card of `lmqg/mt5-small-esquad-qg`
@@ -132,7 +165,7 @@ output = pipe("del <hl> Ministerio de Desarrollo Urbano <hl> , Gobierno de la In
 | ROUGE_L    |   24.62 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
 
 
-- ***Metric (Question & Answer Generation)***: QAG metrics are computed with *the gold answer* and generated question on it for this model, as the model cannot provide an answer. [raw metric file](https://huggingface.co/lmqg/mt5-small-esquad-qg/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_esquad.default.json)
+- ***Metric (Question & Answer Generation, Reference Answer)***: Each question is generated from *the gold answer*. [raw metric file](https://huggingface.co/lmqg/mt5-small-esquad-qg/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_esquad.default.json)
 
 |                                 |   Score | Type    | Dataset                                                          |
 |:--------------------------------|--------:|:--------|:-----------------------------------------------------------------|
@@ -152,6 +185,26 @@ output = pipe("del <hl> Ministerio de Desarrollo Urbano <hl> , Gobierno de la In
 | ROUGE_L                         |   42.77 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
 
 
+- ***Metric (Question & Answer Generation, Pipeline Approach)***: Each question is generated on the answer generated by [`lmqg/mt5-small-esquad-ae`](https://huggingface.co/lmqg/mt5-small-esquad-ae). [raw metric file](https://huggingface.co/lmqg/mt5-small-esquad-qg/raw/main/eval_pipeline/metric.first.answer.paragraph.questions_answers.lmqg_qg_esquad.default.lmqg_mt5-small-esquad-ae.json)
+
+|                                 |   Score | Type    | Dataset                                                          |
+|:--------------------------------|--------:|:--------|:-----------------------------------------------------------------|
+| BERTScore                       |   69.78 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| Bleu_1                          |   11.1  | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| Bleu_2                          |    5.5  | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| Bleu_3                          |    3    | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| Bleu_4                          |    1.83 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| METEOR                          |   22.22 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| MoverScore                      |   51.8  | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| QAAlignedF1Score (BERTScore)    |   79.89 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| QAAlignedF1Score (MoverScore)   |   54.82 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| QAAlignedPrecision (BERTScore)  |   77.46 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| QAAlignedPrecision (MoverScore) |   53.31 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| QAAlignedRecall (BERTScore)     |   82.56 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| QAAlignedRecall (MoverScore)    |   56.52 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+| ROUGE_L                         |   15.46 | default | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) |
+
+
 
 ## Training hyperparameters