Narrativa/legal-longformer-base-4096-spanish -- error

#2
by Ramon2021 - opened

import and initialize the tokenizer and model from the checkpoint

from transformers import AutoTokenizer, AutoModelForMaskedLM
checkpoint = "Narrativa/legal-longformer-base-4096-spanish"

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForMaskedLM.from_pretrained(checkpoint)
.
.
.
.

for input in inputs:
output = model.generate(**input)
print(tokenizer.decode(*output, skip_special_tokens=True))

TypeError: The current model class (RobertaForMaskedLM) is not compatible with .generate(), as it doesn't have a language model head. Please use one of the following classes instead: {'RobertaForCausalLM'}

Hi, @Ramon2021 . This is an encoder-only model, so you cannot generate text with it. You could add a LM head on top of it.

Hi mrm8488 , can you advice about to do that. ( add a LM head on topo)

Sign up or log in to comment