Chain-of-Look Spatial Reasoning for Dense Surgical Instrument Counting

TL;DR

This paper introduces Chain-of-Look, a structured visual reasoning framework that improves dense surgical instrument counting by mimicking human sequential counting and enforcing spatial constraints, outperforming existing methods.

Contribution

The paper proposes a novel visual reasoning framework with a structured visual chain and neighboring loss for accurate dense instrument counting, along with a new high-density surgical instrument dataset.

Findings

Outperforms state-of-the-art counting methods in dense scenarios

Effective in modeling spatial constraints of densely packed instruments

Demonstrates superior accuracy over multimodal large language models

Abstract

Accurate counting of surgical instruments in Operating Rooms (OR) is a critical prerequisite for ensuring patient safety during surgery. Despite recent progress of large visual-language models and agentic AI, accurately counting such instruments remains highly challenging, particularly in dense scenarios where instruments are tightly clustered. To address this problem, we introduce Chain-of-Look, a novel visual reasoning framework that mimics the sequential human counting process by enforcing a structured visual chain, rather than relying on classic object detection which is unordered. This visual chain guides the model to count along a coherent spatial trajectory, improving accuracy in complex scenes. To further enforce the physical plausibility of the visual chain, we introduce the neighboring loss function, which explicitly models the spatial constraints inherent to densely packed surgical instruments. We also present SurgCount-HD, a new dataset comprising 1,464 high-density surgical instrument images. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches for counting (e.g., CountGD, REC) as well as Multimodality Large Language Models (e.g., Qwen, ChatGPT) in the challenging task of dense surgical instrument counting.