ragarwal commited on
Commit
4d26b82
1 Parent(s): 6954c81

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -0
README.md CHANGED
@@ -4,8 +4,14 @@ license: apache-2.0
4
 
5
  [Original Repo](https://github.com/raunak-agarwal/factual-consistency-eval)
6
 
 
 
7
  Inference script available [here](https://github.com/raunak-agarwal/factual-consistency-eval/blob/main/ft-t5/inference-t5.py)
8
 
 
 
 
 
9
 
10
  ### Results
11
  | Method | Rank | Mean Win Rate (%) | Average AUC |
@@ -24,3 +30,19 @@ Inference script available [here](https://github.com/raunak-agarwal/factual-cons
24
  | ROUGE-L | 12 | 6.73 | 54.678 |
25
 
26
  *Comparison of different factuality evaluation methods across all test datasets. The methods are ranked based on the Mean Win Rate, which measures overall performance on factuality tasks. The Average AUC column represents the average of all individual AUC-ROC scores.*
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
 
5
  [Original Repo](https://github.com/raunak-agarwal/factual-consistency-eval)
6
 
7
+ [Paper](https://arxiv.org/abs/2408.04114)
8
+
9
  Inference script available [here](https://github.com/raunak-agarwal/factual-consistency-eval/blob/main/ft-t5/inference-t5.py)
10
 
11
+ Data:
12
+ - [Training Data](https://huggingface.co/datasets/ragarwal/factual-consistency-training-mix)
13
+ - [Evaluation Benchmark](https://huggingface.co/datasets/ragarwal/factual-consistency-evaluation-benchmark)
14
+
15
 
16
  ### Results
17
  | Method | Rank | Mean Win Rate (%) | Average AUC |
 
30
  | ROUGE-L | 12 | 6.73 | 54.678 |
31
 
32
  *Comparison of different factuality evaluation methods across all test datasets. The methods are ranked based on the Mean Win Rate, which measures overall performance on factuality tasks. The Average AUC column represents the average of all individual AUC-ROC scores.*
33
+
34
+
35
+
36
+ Cite this work as follows:
37
+ ```
38
+ @misc{agarwal2024zeroshotfactualconsistencyevaluation,
39
+ title={Zero-shot Factual Consistency Evaluation Across Domains},
40
+ author={Raunak Agarwal},
41
+ year={2024},
42
+ eprint={2408.04114},
43
+ archivePrefix={arXiv},
44
+ primaryClass={cs.CL},
45
+ url={https://arxiv.org/abs/2408.04114},
46
+ }
47
+ ```
48
+