FIDO-GPT: Generative AI behind "Fidonet Cybernetic Immortality" Project

FIDONet is a historic computer network based on nightly mail exchange between servers via telephone lines, which was popular in 1990-s. In FIDONet Cybernetic Immortality Project we are looking to create exhibits that will revive now-almost-dead FIDONet by automatically writing correspondence in FIDONet style via generative large language models.

This model is based on GPT2-large model, and was fine-tuned for 2 epochs on archives of ExecPC BBS, obtained from here. This process took around 9 hours on NVidia A100 compute in Yandex Datasphere service.

This code can be used for generation:

from transformers import pipeline, AutoModelForCausalLM,AutoTokenizer
import torch

model_name = 'estonto/fido-gpt'

model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
pipe = pipeline(model=model,tokenizer=tokenizer,task="text-generation",device="cuda")
result = pipe("<s>Topic: COMPUTING",do_sample=True,max_length=500)[0]['generated_text'].replace('\\n','\n')

Project idea and model training: Dmitry Soshnikov