Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
datasets:
|
4 |
+
- Nuo97/Dolphin-DPO
|
5 |
+
language:
|
6 |
+
- zh
|
7 |
+
metrics:
|
8 |
+
- bleu
|
9 |
+
pipeline_tag: conversational
|
10 |
+
---
|
11 |
+
|
12 |
+
# COMEDY: COmpressive Memory-Enhanced Dialogue sYstems framework.
|
13 |
+
|
14 |
+
Github: https://github.com/nuochenpku/COMEDY
|
15 |
+
|
16 |
+
|
17 |
+
### Task: Long-Term Conversation Dialogue Generation
|
18 |
+
|
19 |
+
Different from previous retrieval-based methods, COMEDY doesn't rely on any **retrieval module or database**.
|
20 |
+
|
21 |
+
Instead, COMEDY adopts a groundbreaking ''**One-for-All**'' approach, utilizing a single, unified model to manage the entire process from memory generation, compression to final response generation for long-term memory dialogue generation.
|
22 |
+
|
23 |
+
|
24 |
+
- COMEDY firstly involves distilling session-specific memory from past dialogues, encompassing fine-grained session summaries, including event recaps, and detailed user and bot portraits;
|
25 |
+
|
26 |
+
- In a break from traditional systems, COMEDY eschews the use of a memory database for storing these insights. Instead, it reprocesses and condenses memories from all past interactions, forming a *Compressive Memory*: The first part is the **concise events** that have occurred throughout all the conversations, creating a historical narrative that the system can draw upon. The second and third parts consist of a **detailed user profile** and the **dynamic relationship changes** between the user and chatbot across sessions, both derived from past conversational events.
|
27 |
+
|
28 |
+
- Finally, COMEDY skillfully integrates this compressive memory into ongoing conversations, enabling contextually memory-enhanced interactions.
|
29 |
+
|
30 |
+
|
31 |
+
### Training Dataset
|
32 |
+
|
33 |
+
**Dolphin**, the biggest Chinese long-term conversation dataset, from actual online user-chatbot interactions.
|
34 |
+
|
35 |
+
This dataset contains three tasks:
|
36 |
+
|
37 |
+
**Session-Level Memory Summarization**;
|
38 |
+
|
39 |
+
**Memory Compression**;
|
40 |
+
|
41 |
+
**Memory-Grounded Response Generation**,
|
42 |
+
|
43 |
+
comprising an extensive collection of 100k samples.
|
44 |
+
|
45 |
+
Dolphin is available at [**Dolphin**](https://huggingface.co/datasets/Nuo97/Dolphin-DPO)
|
46 |
+
|
47 |
+
### Training Strategy
|
48 |
+
|
49 |
+
Our training strategies include two stages: Mixed-task training and DPO Alignment.
|
50 |
+
|
51 |
+
<br>
|
52 |
+
<div align="center">
|
53 |
+
<img src="training_strategy.png" width="90%" title="Introduction Figure">
|
54 |
+
</div>
|
55 |
+
|
56 |
+
|
57 |
+
|
58 |
+
|
59 |
+
|
60 |
+
|
61 |
+
|
62 |
+
|