Update README.md
Browse files
README.md
CHANGED
|
@@ -45,7 +45,7 @@ model = AutoModelForTokenClassification.from_pretrained(model_name)
|
|
| 45 |
以下のことを意識している:
|
| 46 |
|
| 47 |
- 推論時の形態素解析器なし
|
| 48 |
-
- トークンが単語 (`unidic-cwj-202302`) の境界を跨がない
|
| 49 |
- Hugging Faceで使いやすい
|
| 50 |
- 大きすぎない語彙数
|
| 51 |
|
|
@@ -56,7 +56,7 @@ The tokenizer is trained using [the method introduced by Kudo](https://qiita.com
|
|
| 56 |
|
| 57 |
Key points include:
|
| 58 |
- No morphological analyzer needed during inference
|
| 59 |
-
- Tokens do not cross word boundaries (`unidic-cwj-202302`)
|
| 60 |
- Easy to use with Hugging Face
|
| 61 |
- Smaller vocabulary size
|
| 62 |
|
|
|
|
| 45 |
以下のことを意識している:
|
| 46 |
|
| 47 |
- 推論時の形態素解析器なし
|
| 48 |
+
- トークンが単語 (辞書: `unidic-cwj-202302`) の境界を跨がない
|
| 49 |
- Hugging Faceで使いやすい
|
| 50 |
- 大きすぎない語彙数
|
| 51 |
|
|
|
|
| 56 |
|
| 57 |
Key points include:
|
| 58 |
- No morphological analyzer needed during inference
|
| 59 |
+
- Tokens do not cross word boundaries (dictionary: `unidic-cwj-202302`)
|
| 60 |
- Easy to use with Hugging Face
|
| 61 |
- Smaller vocabulary size
|
| 62 |
|