wenlinyao/HDFlow-Llama-3-8B-Instruct

This is the code and data repo of paper HDFlow: Enhancing LLM Complex Problem-Solving with Hybrid Thinking and Dynamic Workflows. If you think this repo is useful, please cite us!

@article{yao2024hdflow,
  title={HDFlow: Enhancing LLM Complex Problem-Solving with Hybrid Thinking and Dynamic Workflows},
  author={Yao, Wenlin and Mi, Haitao and Yu, Dong},
  journal={arXiv preprint arXiv:2409.17433},
  year={2024}
}

Download our model into the dir /xxxx/checkpoints/wenlinyao/HDFlow-Llama-3-8B-Instruct/. Next, use vllm to host the model service. Here provides an example to host two models at the same time (using different ports).

CUDA_VISIBLE_DEVICES=0 python -m vllm.entrypoints.openai.api_server --model /xxxx/checkpoints/wenlinyao/HDFlow-Llama-3-8B-Instruct/ --tensor-parallel-size 1 --max-num-batched-tokens 8192 --dtype bfloat16 --port 4231

Refer to https://github.com/wenlinyao/HDFlow for more details.