Update README.md

README.md

@@ -1,15 +1,9 @@
 ---
 title: CodeBLEU
-datasets:
--  
 tags:
 - evaluate
 - metric
-description: "TODO: add a description here"
-sdk: gradio
-sdk_version: 3.0.2
-app_file: app.py
-pinned: false
+description: "CodeBLEU metric for Python and C++"
 ---
 
 # Metric Card for CodeBLEU
@@ -17,34 +11,49 @@ pinned: false
 ***Module Card Instructions:*** *Fill out the following subsections. Feel free to take a look at existing metric cards if you'd like examples.*
 
 ## Metric Description
-*Give a brief overview of this metric, including what task(s) it is usually used for, if any.*
+CodeBLEU metric is used on code synthesis not only consider the surface match similar with the original BLEU, but can also consider the grammatical correctness and the logic correctness, leveraging the abstract syntax tree and the data-flow structure.
 
 ## How to Use
-*Give general statement of how to use the metric*
-
-*Provide simplest possible example for using the metric*
+* clone the repository
+```python
+git clone https://huggingface.co/spaces/giulio98/codebleu.git
+```
+* import metric
+```python
+from codebleu.calc_code_bleu import calculate
+```
+* compute score
+```python
+true_codes = [["def hello_world():\n    print("hello world!")"], ["def add(a,b)\n    return a+b"]]
+code_gens = ["def hello_world():\n    print("hello world!")", "def add(a,b)\n    return a+b"]
+codebleu = calculate(references=true_codes, predictions=code_gens, language="python", alpha=0.25, beta=0.25, gamma=0.25, theta=0.25)
+print(codebleu['code_bleu_score'])
+```
 
 ### Inputs
 *List all input arguments in the format below*
-- **input_field** *(type): Definition of input, with explanation if necessary. State any default value(s).*
+- **references** *(list of list of string): contains n possible solutions for each problem*
+- **predictions** *(list of string): contains a single prediction for each problem*
+- **language** *(string): python or cpp*
+
 
 ### Output Values
 
-*Explain what this metric outputs and provide an example of what the metric output looks like. Modules should return a dictionary with one or multiple key-value pairs, e.g. {"bleu" : 6.02}*
 
-*State the range of possible values that the metric's output can take, as well as what in that range is considered good. For example: "This metric can take on any value between 0 and 100, inclusive. Higher scores are better."*
 
 #### Values from Popular Papers
-*Give examples, preferrably with links to leaderboards or publications, to papers that have reported this metric, along with the values they have reported.*
 
-### Examples
-*Give code examples of the metric being used. Try to include examples that clear up any potential ambiguity left from the metric description above. If possible, provide a range of examples that show both typical and atypical results, as well as examples where a variety of input parameters are passed.*
 
 ## Limitations and Bias
-*Note any known limitations or biases that the metric has, with links and references if possible.*
 
-## Citation
-*Cite the source where this metric was introduced.*
 
-## Further References
-*Add any useful further references.*
+## Citation
+```
+@unknown{unknown,
+author = {Ren, Shuo and Guo, Daya and Lu, Shuai and Zhou, Long and Liu, Shujie and Tang, Duyu and Zhou, Ming and Blanco, Ambrosio and Ma, Shuai},
+year = {2020},
+month = {09},
+pages = {},
+title = {CodeBLEU: a Method for Automatic Evaluation of Code Synthesis}
+}
+```