Create tokenizer/merges.txt
Browse files- tokenizer/merges.txt +8 -0
tokenizer/merges.txt
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#version: 0.2
|
| 2 |
+
a l
|
| 3 |
+
a n
|
| 4 |
+
a t
|
| 5 |
+
e n
|
| 6 |
+
i n
|
| 7 |
+
o n
|
| 8 |
+
...
|