TravelLeraLone commited on
Commit
e2b8735
1 Parent(s): 52aa54b

Update README

Browse files
Files changed (1) hide show
  1. README.md +14 -7
README.md CHANGED
@@ -9,6 +9,8 @@ language:
9
  ---
10
  # ChemDFM-13B: An LLM-Based Dialogue Foundation Model for Chemistry
11
 
 
 
12
  ChemDFM is the pioneering open-sourced dialogue foundation model for Chemistry and molecule science, which is build based on LLaMa-13B. ChemDFM outperforms the open-sourced LLMs in all the typical tasks of chemistry, and even reach comparable or higher performances of GPT-4. For more details, please refer to [our paper](https://arxiv.org/abs/2401.14818).
13
 
14
  ## News
@@ -91,18 +93,23 @@ def canonicalize_smiles(smiles):
91
 
92
  We evaluate the performance of ChemDFM-13B on multiple widely-used benchmarks in chemistry. The detail introduction of the benchmarks can be found in [our paper](https://arxiv.org/abs/2401.14818). The overall performance of ChemDFM-13B is shown below:
93
 
94
- ![Objective Performance]()
95
-
96
- [//]: # (TODO)
97
 
98
  ### Human Evaluation
99
 
100
- ![Human Evaluation of QA]()
 
 
 
 
 
 
 
 
 
101
 
102
- ![Human Evaluation of Dialogue]()
103
 
104
- [//]: # (TODO)
105
- For more examples and analysis, please refer to [our paper](https://arxiv.org/abs/2401.14818).
106
 
107
  ## Citation
108
  ```bibtex
 
9
  ---
10
  # ChemDFM-13B: An LLM-Based Dialogue Foundation Model for Chemistry
11
 
12
+ ![Main Image](https://github.com/OpenDFM/ChemDFM/raw/main/docs/static/images/main.png)
13
+
14
  ChemDFM is the pioneering open-sourced dialogue foundation model for Chemistry and molecule science, which is build based on LLaMa-13B. ChemDFM outperforms the open-sourced LLMs in all the typical tasks of chemistry, and even reach comparable or higher performances of GPT-4. For more details, please refer to [our paper](https://arxiv.org/abs/2401.14818).
15
 
16
  ## News
 
93
 
94
  We evaluate the performance of ChemDFM-13B on multiple widely-used benchmarks in chemistry. The detail introduction of the benchmarks can be found in [our paper](https://arxiv.org/abs/2401.14818). The overall performance of ChemDFM-13B is shown below:
95
 
96
+ ![Objective Performance](https://github.com/OpenDFM/ChemDFM/raw/main/docs/static/images/objective_performances.png)
 
 
97
 
98
  ### Human Evaluation
99
 
100
+ ![Human Evaluation of QA](https://github.com/OpenDFM/ChemDFM/raw/main/docs/static/images/human_evaluation_QA.png)
101
+
102
+ We mark <font color=#548235>the correct and relevant information</font> in the replies in green, <font color=#C55A11>the correct but irrelevant information</font> in yellow, and <font color=#C00000>the wrong information in red</font>. In addition, **the key points of the answer** are marked in bold if they appear in the reply.
103
+
104
+ The results show that while open-sourced LLMs perform well when asked about existing knowledge (Q1), *only ChemDFM can provide correct and comprehensive answers when questions involve new molecules and reactions* (Q2 [\[Yin et al., 2023\]](https://pubs.acs.org/doi/10.1021/jacs.3c07044) & Q3 [\[Dargo et al., 2023\]](https://www.sciencedirect.com/science/article/pii/S1385894723030966))
105
+
106
+ ![Human Evaluation of Dialogue](https://github.com/OpenDFM/ChemDFM/raw/main/docs/static/images/human_evaluation_dialogue.png)
107
+
108
+ The above conversation is also inspired by [Yin et al.\[2023\]](https://pubs.acs.org/doi/10.1021/jacs.3c07044). During the dialogue, the researcher wants to selectively oxidize one of the two carbonyl groups of a molecule. However, the
109
+ initial solution given by ChemDFM results in both carbonyl groups being oxidized. Through the correction given by the researcher, ChemDFM adjusts its proposal and provides two possible solutions. Finally, the researcher chooses to use protecting groups and ChemDFM further details its advice. *In the dialogue, ChemDFM shows promising capabilities regarding error correction (Round 2) and detailing (Round 3) when handling real-world research scenarios.*
110
 
 
111
 
112
+ **For more examples and analysis, please refer to [our paper](https://arxiv.org/abs/2401.14818).**
 
113
 
114
  ## Citation
115
  ```bibtex