Commit
•
7202730
1
Parent(s):
b0ef4a0
Update README.md
Browse files
README.md
CHANGED
@@ -35,15 +35,6 @@ Code Llama is a collection of pretrained and fine-tuned generative text models r
|
|
35 |
|
36 |
**Model Architecture** Code Llama is an auto-regressive language model that uses an optimized transformer architecture.
|
37 |
|
38 |
-
|
39 |
-
||Training Data|Params|Content Length|GQA|Tokens|LR|
|
40 |
-
|---|---|---|---|---|---|---|
|
41 |
-
|Llama 2|*A new mix of publicly available online data*|7B|4k|✗|2.0T|3.0 x 10<sup>-4</sup>|
|
42 |
-
|Llama 2|*A new mix of publicly available online data*|13B|4k|✗|2.0T|3.0 x 10<sup>-4</sup>|
|
43 |
-
|Llama 2|*A new mix of publicly available online data*|70B|4k|✔|2.0T|1.5 x 10<sup>-4</sup>|
|
44 |
-
|
45 |
-
*Llama 2 family of models.* Token counts refer to pretraining data only. All models are trained with a global batch-size of 4M tokens. Bigger models - 70B -- use Grouped-Query Attention (GQA) for improved inference scalability.
|
46 |
-
|
47 |
**Model Dates** Code Llama and its variants have been trained between January 2023 and July 2023.
|
48 |
|
49 |
**Status** This is a static model trained on an offline dataset. Future versions of Code Llama - Instruct will be released as we improve model safety with community feedback.
|
|
|
35 |
|
36 |
**Model Architecture** Code Llama is an auto-regressive language model that uses an optimized transformer architecture.
|
37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
38 |
**Model Dates** Code Llama and its variants have been trained between January 2023 and July 2023.
|
39 |
|
40 |
**Status** This is a static model trained on an offline dataset. Future versions of Code Llama - Instruct will be released as we improve model safety with community feedback.
|