metadata

license: bigscience-bloom-rail-1.0
datasets:
  - swap-uniba/itwiki-march-2024
language:
  - it
tags:
  - bloom
  - italian

Model Card for Model ID

The model is obtained by performing language adaptation on the original bloom-1b7 model. In detail, we continued the pre-training on Italian-specific data without adaptation of the vocabulary. We use about 2.8M documents obtained from Italian Wikimedia dumps (swap-uniba/itwiki-march-2024). The model is trained for one epoch using LoRA and SFT.

Model Details

Model Description

Developed by: SWAP Research Group, Department of Computer Science, University of Bari Aldo Moro
Model type: BLOOM
Language(s) (NLP): Italian
License: bigscience-bloom-rail-1.0
Finetuned from model [optional]: bloom-1b7

Training Details

Training Data

2.8M documents obtained from Italian Wikimedia dumps (swap-uniba/itwiki-march-2024).

Training Procedure

LoRA and SFT.

Training Hyperparameters

Training regime: fp16

Citation [optional]

BibTeX:

APA:

Model Card Authors [optional]

Pierpaolo Basile, University of Bari Aldo Moro, Italy.

Model Card Contact

Pierpaolo Basile, University of Bari Aldo Moro, Italy.