Why is the HumanEval SCORE LOWER than starcoder and starcoderbase?
#5
by
qftie
- opened
What's the defference between starcoder/starcoderbase/starcoderplus? Why is the HumanEval SCORE of starcoderplus LOWER than starcoder and starcoderbase?
For StarCoderPlus, we fine-tuned StarCoderBase on a lot of english data (while inclduing The Stack code dataset again), so the model seems to have forgot some coding capabilities. But the trade off between English and code performance seems reasonable.
loubnabnl
changed discussion status to
closed
Hello！Where could I get the HumanEval SCORE of starcoderplus? @loubnabnl