# Waste-Bench: A Comprehensive Benchmark for Evaluating VLLMs in Cluttered Environments

## TL;DR

This paper introduces Waste-Bench, a new dataset and evaluation framework for assessing VLLMs in cluttered, real-world waste classification scenarios with deformed objects, revealing current limitations and guiding future improvements.

## Contribution

The paper presents a novel waste classification dataset and a comprehensive evaluation approach for VLLMs in complex, cluttered environments, addressing a gap in existing benchmarks.

## Key findings

- VLLMs show limited robustness in cluttered, deformed object scenarios.
- The new dataset exposes weaknesses of current VLLMs in real-world waste classification.
- Evaluation results highlight the need for improved model robustness in complex environments.

## Abstract

Recent advancements in Large Language Models (LLMs) have paved the way for Vision Large Language Models (VLLMs) capable of performing a wide range of visual understanding tasks. While LLMs have demonstrated impressive performance on standard natural images, their capabilities have not been thoroughly explored in cluttered datasets where there is complex environment having deformed shaped objects. In this work, we introduce a novel dataset specifically designed for waste classification in real-world scenarios, characterized by complex environments and deformed shaped objects. Along with this dataset, we present an in-depth evaluation approach to rigorously assess the robustness and accuracy of VLLMs. The introduced dataset and comprehensive analysis provide valuable insights into the performance of VLLMs under challenging conditions. Our findings highlight the critical need for further advancements in VLLM's robustness to perform better in complex environments. The dataset and code for our experiments will be made publicly available.