Edit model card

Pretrain 474M 4096 context length Mistral on Malaysian text

README at https://github.com/mesolitica/malaya/tree/5.1/pretrained-model/mistral

WandB, https://wandb.ai/malaysia-ai2020/mistral-474M?workspace=user-malaysia-ai2020

Downloads last month: 33

Safetensors

Model size

475M params

Tensor type

BF16

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Collection including mesolitica/malaysian-mistral-474M-4096

Malaysian CausalLM

Collection

Trained on 21B tokens, 91GB of cleaned texts, able to understand standard Malay, local Malay, local Mandarin, Manglish, and local Tamil. • 4 items • Updated Sep 22 • 1