---
license: other
license_name: glm-4
license_link: https://huggingface.co/THUDM/LongReward-glm4-9b-DPO/blob/main/LICENSE
language:
- en
- zh
base_model:
- THUDM/glm-4-9b-chat
new_version: THUDM/glm-4-9b-chat-hf
pipeline_tag: text-generation
library_name: transformers
tags:
- chatglm
inference: false
---

# LongReward-glm4-9b-DPO

中文阅读，请看 [这里](README_zh.md).

<p align="center">
  🤗 <a href="https://huggingface.co/datasets/THUDM/LongReward-10k" target="_blank">[LongReward Dataset] </a> • 💻 <a href="https://github.com/THUDM/LongReward" target="_blank">[Github Repo]</a> • 📃 <a href="https://arxiv.org/abs/" target="_blank">[LongReward Paper]</a> 
</p>

LongReward-glm4-9b-DPO is the DPO version of [LongReward-glm4-9b-SFT](https://huggingface.co/THUDM/LongReward-glm4-9b-SFT) and supports a maximum context window of up to 64K tokens. It is trained on the `dpo_glm4_9b` split of [LongReward-10k](https://huggingface.co/datasets/THUDM/LongReward-45) datasets, which is a long-context preference dataset constructed via LongReward.

Environment: Same environment requirement as [glm-4-9b-chat](https://huggingface.co/THUDM/glm-4-9b-chat) (`transforemrs>=4.46.0`).

A simple demo for deployment of the model:



## License 

The weights of the model are available under the terms of [LICENSE](LICENSE).


## Citation

If you find our work useful, please consider citing LongReward:

```
@article{zhang2024longreward,
  title = {LongReward: Improving Long-context Large Language Models
with AI Feedback} 
  author={Jiajie Zhang and Zhongni Hou and Xin Lv and Shulin Cao and Zhenyu Hou and Yilin Niu and Lei Hou and Lei Hou and Yuxiao Dong and Ling Feng and Juanzi Li},
  journal={arXiv preprint arXiv:},
  year={2024}
}
```