PlanetDOGE commited on
Commit
a3db771
1 Parent(s): 5ac05ee

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -0
README.md CHANGED
@@ -1,3 +1,22 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6500c7c912c1442d994c36e5/rav-pUPtWF-u-_A4k9RaZ.png)
5
+
6
+ TinyKAI 3B is a fine-tuned LLM (Large Language Model) based off of [OpenLlama 3B v2](https://huggingface.co/openlm-research/open_llama_3b_v2).
7
+ The TinyKAI models are a series of lightweight LLMs under 5 Billion parameters, usually used for research.
8
+
9
+ ## Direct Use
10
+ TinyKAI 3B is optimal for research on large language models, specifically the influence of web data on the properties of large language models (fairness, safety, limitations, capabilities, etc.).
11
+
12
+ ## Training
13
+ This model was trained on a mixture of the [Falcon refined-web dataset](https://huggingface.co/datasets/tiiuae/falcon-refinedweb), the [StarCoder dataset](https://huggingface.co/datasets/bigcode/starcoderdata) and the wikipedia, arxiv, book and stackexchange part of the [RedPajama dataset](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T).
14
+
15
+ ## Banned Use
16
+ Production use without adequate assessment of risks and mitigation; any use cases which may be considered irresponsible or harmful or insulting to anyone or any certain group.
17
+
18
+ ## Limitations
19
+ TinyKAI 3B is trained on English data only, and will not generate appropriately reasonable content in other languages. Being trained on a representative of the web, it will carry the stereotypes and biases commonly encountered online.
20
+
21
+ ## Recommendations
22
+ We recommend users of TinyKAI 3B to consider finetuning it for personal use, and for precautions to be taken for any commercial use.