license: apache-2.0
language:
- en
base_model:
- meta-llama/Meta-Llama-3.1-8B
Empathetic teacher model
Overview
This is a LLM fine-tuned with real-life, ideally-empathetic teacher-student conversations. This model processes the recent conversation history and provides guidance on how a teacher might respond to the student's utterance.
To fine-tune an open-weighted LLM to act as this generic teacher, we have used the following datasets: the Teacher-Student Chatroom Corpus, TSCCv2 Caines et al., 2022, CIMA Stasaski et al., 2020, the Multicultural Classroom Discourse Dataset Rapanta et al., 2021, MathDial Macina et al., 2023, and Conversational Uptake [Demszky et al., 2021].
We are evaluating LLaMa-3 for this task. Instead of using programmable fine-tuning libraries such as Axolotl (link) or Huggingface TRL (link), we are employing the more general command-line LLaMA-Factory (link) toolkit that facilitates the fine-tuning of various well-known LLMs on custom data. Parameter-efficient fine-tuning is achieved via the QLoRA method Dettmers et al., 2023.
Usage Guide
This project was executed on an Ubuntu 22.04.3 system running Linux kernel 6.8.0-40-generic.
Installation
To get started, you first need to set up the environment using the LLaMA-Factory project. Please refer to the official LLaMA-Factory repository for more details.
You can install the project by running the following commands:
git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics]"
Execution
In the DeMINT project, the model was utilized to create a REST API. Below is an example of how to configure and run it.
Setting Server Configuration
To specify the port and server address, use the following environment variables:
To set the port and the address of the server:
# Default 8000
export KIND_TEACHER_PORT=8000
# Default localhost
export KIND_TEACHER_HOST="localhost"
Running the Program
Once the environment is configured, you can execute the program by running the following command:
llamafactory-cli api run_api_inference_1.yaml