riotu-lab commited on
Commit
c8e8f21
1 Parent(s): 31bf9ee

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -5
README.md CHANGED
@@ -11,9 +11,14 @@ tags:
11
 
12
  * Model Name: ArabianGPT
13
  * Architecture: GPT-2
14
- * Layers: 12
15
- * Model Size: 134M
16
- * Context Window Size: 768
 
 
 
 
 
17
 
18
  > [!NOTE]
19
  > ArabianGPT is a custom-trained version of the GPT-2 base model, specifically tailored for the Arabic language. It is designed to understand and generate Arabic text, making it suitable for various natural language processing tasks in Arabic.
@@ -23,13 +28,13 @@ tags:
23
  * Size: 15.5 GB
24
  * Number of Words: 237,814,541
25
  * Number of Tokens: 1,752,421,071
26
- * Number of Parameters : 134 M Params
27
 
28
  * # Compute
29
 
30
  * | Model | Hardware | Num of Examples (seq len = 768) | Batch Size | Num of Steps | Time (in days) | Loss
31
  |------------------|---------------|---------------------------------|------------|--------------|----------------|----------------|
32
- | ArabianGPT-base | NDIVIA A100 | 7.5M | 1024 | 313.5K | 3 | 3.97 |
33
 
34
 
35
  > [!NOTE]
 
11
 
12
  * Model Name: ArabianGPT
13
  * Architecture: GPT-2
14
+
15
+ | Specification | Value |
16
+ |-----------------------|----------|
17
+ | Layers | 12 |
18
+ | MAL (Model Attention Layers) | 12 |
19
+ | Model Size | 134M |
20
+ | Context Window Size | 768 |
21
+
22
 
23
  > [!NOTE]
24
  > ArabianGPT is a custom-trained version of the GPT-2 base model, specifically tailored for the Arabic language. It is designed to understand and generate Arabic text, making it suitable for various natural language processing tasks in Arabic.
 
28
  * Size: 15.5 GB
29
  * Number of Words: 237,814,541
30
  * Number of Tokens: 1,752,421,071
31
+
32
 
33
  * # Compute
34
 
35
  * | Model | Hardware | Num of Examples (seq len = 768) | Batch Size | Num of Steps | Time (in days) | Loss
36
  |------------------|---------------|---------------------------------|------------|--------------|----------------|----------------|
37
+ | ArabianGPT-base | NDIVIA A100 | 7.5M | 512 | 313.5K | 3 | 3.97 |
38
 
39
 
40
  > [!NOTE]