# Feature-based behavior coding for efficient exploratory analysis using pose estimation

## TL;DR

This paper introduces a new method called feature-based behavior coding that uses pose estimation to make behavioral research more efficient and flexible.

## Contribution

FBBC introduces a novel framework combining pose estimation and dimensionality reduction for exploratory behavioral analysis.

## Key findings

- FBBC transforms video data into interpretable feature time series for efficient posture analysis.
- Behavior Senpai, an open-source tool, integrates automated feature extraction with human input for improved classification.
- A case study shows FBBC's effectiveness in classifying complex postures through combined features and manual clustering.

## Abstract

This paper introduces feature-based behavior coding (FBBC), an efficient method for exploratory analysis in behavioral research using pose estimation techniques. FBBC addresses the challenges of traditional behavioral coding methods, particularly in the exploratory stages of research when coding schemes are not yet well defined. By leveraging keypoint detection and dimensionality reduction, FBBC transforms video data into interpretable feature time series, enabling researchers to analyze diverse postural patterns more efficiently. Also presented is Behavior Senpai, an open-source software implementation of FBBC that integrates automated feature extraction with human insight. A case study demonstrates FBBC’s ability to classify complex postures by combining multiple features and manual clustering. While the current iteration focuses on instantaneous posture classification, the framework shows potential for expansion to action classification. FBBC offers increased flexibility in developing coding schemes and reduces the time-consuming nature of repetitive observations. This approach represents a considerable advancement in behavioral research, bridging traditional methods with modern machine-learning techniques. As FBBC is adopted and refined, it will contribute to more comprehensive and insightful behavioral analyses across the psychological and behavioral sciences.