Prosocial or Selfish? Agents with different behaviors for Contract Negotiation using Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning framework for training negotiation agents with prosocial and selfish behaviors, demonstrating their effectiveness against humans and the ability to emulate human-like negotiation strategies.

Contribution

It presents a novel multi-agent reinforcement learning approach for contract negotiation with behavior modeling and a meta agent that mimics human negotiation patterns.

Findings

Agents can negotiate effectively with humans.

Meta agent successfully emulates human negotiation behavior.

Agents with mixed behaviors outperform single-behavior agents.

Abstract

We present an effective technique for training deep learning agents capable of negotiating on a set of clauses in a contract agreement using a simple communication protocol. We use Multi Agent Reinforcement Learning to train both agents simultaneously as they negotiate with each other in the training environment. We also model selfish and prosocial behavior to varying degrees in these agents. Empirical evidence is provided showing consistency in agent behaviors. We further train a meta agent with a mixture of behaviors by learning an ensemble of different models using reinforcement learning. Finally, to ascertain the deployability of the negotiating agents, we conducted experiments pitting the trained agents against human players. Results demonstrate that the agents are able to hold their own against human players, often emerging as winners in the negotiation. Our experiments demonstrate that the meta agent is able to reasonably emulate human behavior.