Narrativa/legal-longformer-base-4096-spanish -- error
#2
by
Ramon2021
- opened
import and initialize the tokenizer and model from the checkpoint
from transformers import AutoTokenizer, AutoModelForMaskedLM
checkpoint = "Narrativa/legal-longformer-base-4096-spanish"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForMaskedLM.from_pretrained(checkpoint)
.
.
.
.
for input in inputs:
output = model.generate(**input)
print(tokenizer.decode(*output, skip_special_tokens=True))
TypeError: The current model class (RobertaForMaskedLM) is not compatible with .generate()
, as it doesn't have a language model head. Please use one of the following classes instead: {'RobertaForCausalLM'}
Hi,
@Ramon2021
. This is an encoder-only
model, so you cannot generate text with it. You could add a LM head on top of it.
Hi mrm8488 , can you advice about to do that. ( add a LM head on topo)