README.md · xu-song/self-chat at main

metadata

title: Self Chat
emoji: 🤖~🤖
colorFrom: yellow
colorTo: purple
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
license: apache-2.0
tags:
  - chatbot
short_description: Generating synthetic data via self-chatting

Dependency

Install llama-cpp-python with the following script

pip install git+https://github.com/abetlen/llama-cpp-python.git -C cmake.args="-DGGML_BLAS=ON;-DGGML_BLAS_VENDOR=OpenBLAS"

Local Inference

python models/cpp_qwen2.py

Serverless Inference

python client_gradio.py

For streaming inference

python client_streaming.py