Feature request: correction scoring.

#4
by YokaiKoibito - opened

It would be nice to have a mode in which, for each correction span in the diff between the input and output, it gave a logit score for how much the correction decreases the perplexity. (I suspect this would mostly just be a more complex analysis of the results of the beam search.) Even nicer would be if it could output multiple, possibly overlapping candidate corrections, each with a score. So for example if the input is "My favourite colr is red" it would generate suggestions to correct 'colr' to either 'color' or 'colour', each with a logit score.

@YokaiKoibito yes that makes sense. However i suspect it to be a separate project rather than the model's ability directly, as HuggingFace platform is meant to be the hub for model sharing.
This feature involves calculations based on model outputs so I'm thinking it would be better to create a GitHub repo for this request. Will share with you once done.

Sign up or log in to comment