Update README.md
Browse files
README.md
CHANGED
@@ -11,9 +11,14 @@ tags:
|
|
11 |
|
12 |
* Model Name: ArabianGPT
|
13 |
* Architecture: GPT-2
|
14 |
-
|
15 |
-
|
16 |
-
|
|
|
|
|
|
|
|
|
|
|
17 |
|
18 |
> [!NOTE]
|
19 |
> ArabianGPT is a custom-trained version of the GPT-2 base model, specifically tailored for the Arabic language. It is designed to understand and generate Arabic text, making it suitable for various natural language processing tasks in Arabic.
|
@@ -23,13 +28,13 @@ tags:
|
|
23 |
* Size: 15.5 GB
|
24 |
* Number of Words: 237,814,541
|
25 |
* Number of Tokens: 1,752,421,071
|
26 |
-
|
27 |
|
28 |
* # Compute
|
29 |
|
30 |
* | Model | Hardware | Num of Examples (seq len = 768) | Batch Size | Num of Steps | Time (in days) | Loss
|
31 |
|------------------|---------------|---------------------------------|------------|--------------|----------------|----------------|
|
32 |
-
| ArabianGPT-base | NDIVIA A100 | 7.5M |
|
33 |
|
34 |
|
35 |
> [!NOTE]
|
|
|
11 |
|
12 |
* Model Name: ArabianGPT
|
13 |
* Architecture: GPT-2
|
14 |
+
|
15 |
+
| Specification | Value |
|
16 |
+
|-----------------------|----------|
|
17 |
+
| Layers | 12 |
|
18 |
+
| MAL (Model Attention Layers) | 12 |
|
19 |
+
| Model Size | 134M |
|
20 |
+
| Context Window Size | 768 |
|
21 |
+
|
22 |
|
23 |
> [!NOTE]
|
24 |
> ArabianGPT is a custom-trained version of the GPT-2 base model, specifically tailored for the Arabic language. It is designed to understand and generate Arabic text, making it suitable for various natural language processing tasks in Arabic.
|
|
|
28 |
* Size: 15.5 GB
|
29 |
* Number of Words: 237,814,541
|
30 |
* Number of Tokens: 1,752,421,071
|
31 |
+
|
32 |
|
33 |
* # Compute
|
34 |
|
35 |
* | Model | Hardware | Num of Examples (seq len = 768) | Batch Size | Num of Steps | Time (in days) | Loss
|
36 |
|------------------|---------------|---------------------------------|------------|--------------|----------------|----------------|
|
37 |
+
| ArabianGPT-base | NDIVIA A100 | 7.5M | 512 | 313.5K | 3 | 3.97 |
|
38 |
|
39 |
|
40 |
> [!NOTE]
|