beomi commited on
Commit
715f579
1 Parent(s): c10a8eb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -33,7 +33,7 @@ Solar-KoEn represents an advanced iteration of the upstage/SOLAR-10.7B-v1.0 mode
33
 
34
  **Model Architecture:**
35
 
36
- SOLAR-KO-10.7B is an auto-regressive language model that leverages an optimized transformer architecture derived from Llama-2.
37
 
38
  | |Training Data|Parameters|Content Length|GQA|Tokens|Learning Rate|
39
  |---|---|---|---|---|---|---|
@@ -57,7 +57,7 @@ Total token count: Approximately 15 billion tokens (*using the expanded tokenize
57
  | Model Name | Vocabulary Size | Description |
58
  | --- | --- | --- |
59
  | Original Solar | 32000 | Sentencepiece BPE |
60
- | **Expanded SOLAR-KO-10.7B** | 46336 | Sentencepiece BPE. Added Korean vocab and merges |
61
 
62
  **Tokenizing "안녕하세요, 오늘은 날씨가 좋네요."**
63
 
@@ -67,7 +67,7 @@ Total token count: Approximately 15 billion tokens (*using the expanded tokenize
67
  | Model | Tokens |
68
  | --- | --- |
69
  | SOLAR-10.7B | `['▁', '안', '<0xEB>', '<0x85>', '<0x95>', '하', '세', '요', ',', '▁', '오', '<0xEB>', '<0x8A>', '<0x98>', '은', '▁', '날', '<0xEC>', '<0x94>', '<0xA8>', '가', '▁', '좋', '네', '요', '.']` |
70
- | SOLAR-KO-10.7B | `['▁안', '녕', '하세요', ',', '▁오늘', '은', '▁날', '씨가', '▁좋네요', '.']` |
71
 
72
  **Tokenizing "Meet 10.7B Solar: Elevating Performance with Upstage Depth UP Scaling!"**
73
 
@@ -77,7 +77,7 @@ Total token count: Approximately 15 billion tokens (*using the expanded tokenize
77
  | Model | Tokens |
78
  | --- | --- |
79
  | SOLAR-10.7B | `['▁Meet', '▁', '1', '0', '.', '7', 'B', '▁Solar', ':', '▁E', 'lev', 'ating', '▁Performance', '▁with', '▁Up', 'stage', '▁Dep', 'th', '▁UP', '▁Scal', 'ing', '!']` |
80
- | SOLAR-KO-10.7B | `['▁Meet', '▁', '1', '0', '.', '7', 'B', '▁Solar', ':', '▁E', 'lev', 'ating', '▁Performance', '▁with', '▁Up', 'stage', '▁Dep', 'th', '▁UP', '▁Scal', 'ing', '!']` |
81
 
82
  # LICENSE
83
 
@@ -128,7 +128,7 @@ Apache 2.0
128
  ```
129
  @misc {solar_koen_junbum_taekyoon_2024,
130
  author = { {L. Junbum, Taekyoon Choi} },
131
- title = { Solar-KoEn-10.8b },
132
  year = 2024,
133
  url = { https://huggingface.co/beomi/SOLAR-KOEN-10.8B },
134
  publisher = { Hugging Face }
 
33
 
34
  **Model Architecture:**
35
 
36
+ SOLAR-KOEN-10.8B is an auto-regressive language model that leverages an optimized transformer architecture derived from Llama-2.
37
 
38
  | |Training Data|Parameters|Content Length|GQA|Tokens|Learning Rate|
39
  |---|---|---|---|---|---|---|
 
57
  | Model Name | Vocabulary Size | Description |
58
  | --- | --- | --- |
59
  | Original Solar | 32000 | Sentencepiece BPE |
60
+ | **Expanded SOLAR-KOEN-10.8B** | 46336 | Sentencepiece BPE. Added Korean vocab and merges |
61
 
62
  **Tokenizing "안녕하세요, 오늘은 날씨가 좋네요."**
63
 
 
67
  | Model | Tokens |
68
  | --- | --- |
69
  | SOLAR-10.7B | `['▁', '안', '<0xEB>', '<0x85>', '<0x95>', '하', '세', '요', ',', '▁', '오', '<0xEB>', '<0x8A>', '<0x98>', '은', '▁', '날', '<0xEC>', '<0x94>', '<0xA8>', '가', '▁', '좋', '네', '요', '.']` |
70
+ | SOLAR-KOEN-10.8B | `['▁안', '녕', '하세요', ',', '▁오늘', '은', '▁날', '씨가', '▁좋네요', '.']` |
71
 
72
  **Tokenizing "Meet 10.7B Solar: Elevating Performance with Upstage Depth UP Scaling!"**
73
 
 
77
  | Model | Tokens |
78
  | --- | --- |
79
  | SOLAR-10.7B | `['▁Meet', '▁', '1', '0', '.', '7', 'B', '▁Solar', ':', '▁E', 'lev', 'ating', '▁Performance', '▁with', '▁Up', 'stage', '▁Dep', 'th', '▁UP', '▁Scal', 'ing', '!']` |
80
+ | SOLAR-KOEN-10.8B | `['▁Meet', '▁', '1', '0', '.', '7', 'B', '▁Solar', ':', '▁E', 'lev', 'ating', '▁Performance', '▁with', '▁Up', 'stage', '▁Dep', 'th', '▁UP', '▁Scal', 'ing', '!']` |
81
 
82
  # LICENSE
83
 
 
128
  ```
129
  @misc {solar_koen_junbum_taekyoon_2024,
130
  author = { {L. Junbum, Taekyoon Choi} },
131
+ title = { SOLAR-KOEN-10.8B },
132
  year = 2024,
133
  url = { https://huggingface.co/beomi/SOLAR-KOEN-10.8B },
134
  publisher = { Hugging Face }