A Recipe For Building a Compliant Real Estate Chatbot

TL;DR

This paper presents a method for creating a compliant real estate chatbot by fine-tuning a large language model with synthetic instruction and safety data, achieving high performance and safety standards.

Contribution

It introduces a novel approach to generate synthetic datasets for instruction-following and safety, fine-tunes a Llama-3-8B model, and open-sources the model and data.

Findings

Model performance matches GPT-4o on benchmarks.

Enhanced safety and compliance in real estate chatbot.

Open-source release supports community research.

Abstract

In recent years, there has been significant effort to align large language models with human preferences. This work focuses on developing a chatbot specialized in the real estate domain, with an emphasis on incorporating compliant behavior to ensure it can be used without perpetuating discriminatory practices like steering and redlining, which have historically plagued the real estate industry in the United States. Building on prior work, we present a method for generating a synthetic general instruction-following dataset, along with safety data. Through extensive evaluations and benchmarks, we fine-tuned a llama-3-8B-instruct model and demonstrated that we can enhance it's performance significantly to match huge closed-source models like GPT-4o while making it safer and more compliant. We open-source the model, data and code to support further development and research in the community.