Converting from mistral-finetune to hf compatible

#79
by azimjon - opened

I couldn't find a script to convert from the result of mistral-finetune library to transformers library compatible weights. I would appreciate if someone shares it.

Here ya go:
https://github.com/huggingface/transformers/blob/main/src/transformers/models/mistral/convert_mistral_weights_to_hf.py

Example calling it from Python:

Converting to hf

convert_command = ['python3', 'transformers/src/transformers/models/mistral/convert_mistral_weights_to_hf.py', '--input_dir', model_save_path, '--model_size', '13B', '--is_v3', '--output_dir', f'{model_save_path}-hf']
with open(f"{Path.home()}/convert.log", "w") as f:
    hf_convert_result = subprocess.Popen(convert_command, stdout=subprocess.PIPE,
                                        cwd=f'{Path.home()}/')
    for c in iter(lambda: hf_convert_result.stdout.read(1), b""):
        sys.stdout.buffer.write(c)
        f.buffer.write(c)

@erik-akert but this is for mistral 7B model. I couldn't make it work for Nemo model.

Sign up or log in to comment