Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 11.7k

Running on CPU Upgrade

Hi!
Yes, if you rename a model, it's the same as if you deleted from the leaderboard's point of view, as it queries the old name which no longer exists.
However, you can use @Weyaxi 's cool model renamer here to adress the problem: it will open PRs on all files which need renaming, and we will just have to merge them.

That is a cool utility! I will wait until admins merge PR.

@leejunhyeok could you please ping @SaylorTwift or myself on all the PRs to merge so we make sure we don't miss one? :)

@leejunhyeok could you please ping @SaylorTwift or myself on all the PRs to merge so we make sure we don't miss one? :)

I can automate pinging you and @SaylorTwift if you like.

I'd rather not ^^
Atm, we ask people wanting a rename to open a discussion like this, and confirms this is a needed change first, then tags us on the PRs.
Otherwise anybody could open a renaming PR about any model - tagging us automatically would create a risk for us to accidentally merge an unwanted renaming.

while trying to make & ping PRs, I met a wierd issue.

in renaming v1.8.4, I cannot find related PR in https://huggingface.co/datasets/open-llm-leaderboard/results/discussions?status=closed&type=pull_request
(differs with 1.8.6, 1.4)

So I cannot ping you for this model's PR. can you check manually? @clefourrier

model link(current): https://huggingface.co/moreh/MoMo-72B-lora-1.8.4-DPO
result: https://huggingface.co/datasets/open-llm-leaderboard/details_moreh__MoMo-70B-lora-1.8.4-DPO

many thanks

Hi @leejunhyeok ,

Can you try entering moreh/MoMo-70B-lora-1.8.4-DPO in the from_model part, replacing LoRa with lora.

@Weyaxi thank you for your suggestion.
however this issue remains the same. PR is not created properly.

I noticed that https://huggingface.co/datasets/open-llm-leaderboard/details_moreh__MoMo-70B-lora-1.8.4-DPO does not have README.md file.
Can this be the problem?

I noticed that https://huggingface.co/datasets/open-llm-leaderboard/details_moreh__MoMo-70B-lora-1.8.4-DPO does not have README.md file.
Can this be the problem?

Yes, I believe that's the problem. This is the first time I have seen a details repo without a README file. Maybe the upload process has been interrupted etc. (I don't know) The code is designed to edit the README part, so when the README is not found, it gives an error.

I see, can you link to the request file of this model ? I will check the logs to see why the readme wasn't uploaded

hi.
https://huggingface.co/datasets/open-llm-leaderboard/results/tree/main/moreh/MoMo-70B-lora-1.8.4-DPO
this is the request file of MoMo-70B-lora-1.8.4-DPO(current: MoMo-72B-lora-1.8.4-DPO)

@SaylorTwift , just checked, there was a connection error on the hub at upload time - it's unrelated to the lb.
@leejunhyeok I'll have to edit your files manually haha! I'll do it tomorrow :)

Should be good!