Spaces:
Running
Running
update readme
Browse files
README.md
CHANGED
@@ -21,12 +21,12 @@ This metric is used to evaluate code generation on the [APPS benchmark](https://
|
|
21 |
You can load the metric and use it with the following commands:
|
22 |
```
|
23 |
from evaluate import load
|
24 |
-
|
25 |
results = apps_metric.compute(predictions=generations)
|
26 |
```
|
27 |
|
28 |
### Inputs
|
29 |
-
**generations** (list(str)): List of code generations, each sub-list corresponds to the
|
30 |
|
31 |
### Output Values
|
32 |
|
|
|
21 |
You can load the metric and use it with the following commands:
|
22 |
```
|
23 |
from evaluate import load
|
24 |
+
apps_metric = load('loubnabnl/apps_metric')
|
25 |
results = apps_metric.compute(predictions=generations)
|
26 |
```
|
27 |
|
28 |
### Inputs
|
29 |
+
**generations** list(list(str)): List of code generations, each sub-list corresponds to the generations for a problem in APPS dataset, the order of the samples in the dataset must be kept (with respect to the difficulty level).
|
30 |
|
31 |
### Output Values
|
32 |
|