Qwen compabilitiy
#6
by
keminglu
- opened
Great works! it looks like Qwen-72B is a very strong base model. Could you kindly share how you reassign the weights for leaderboard submission?
hi. we are happy to share how we converted qwen weight to llama
please refer to link
- ./llamafy_resource folder in script contains tokenizer and target configuration file for llama.
- place tokenizer of this repository(moreh/MoMo-70B-lora-1.8.6-DPO), and config_llama.json file to the folder
- run by
python llamafy_qwen.py - in input path, we expect directory of lora weight files.
this script is highly experimental, and cannot assure 100% functionality for every settings of qwen weights.
if you have other questions, feel free to ask
if you have further questions, feel free to re-open this discussion
leejunhyeok
changed discussion status to
closed