Multidimensional Collaborative Attention-Driven Neural Network for Real-Time Neurosurgical Instrument Segmentation

The automatic identification and precise marking of neurosurgical instruments can assist physicians in optimizing their visual search and judgment processes, alleviating visual fatigue, and enhancing decision-making efficiency, enabling doctors to focus more on the precise operation process of intracranial surgery. However, limited field of view in the surgical area and complex collaboration among multiple instruments can lead to instrument jitter, local occlusion, insufficient lighting, smoke interference, and other factors, making it difficult to maintain the robustness and stability of instrument segmentation. This article proposes a lightweight neurosurgical multi-instrument real-time segmentation network driven by multidimensional collaborative attention, utilizing YOLOv8n as the backbone. We developed the concatenate-triplet attention (Cat-TA) module, which enhances global information exchange and integrates multidimensional attention to strengthen the representation of effective features of different instruments. The optimized omni-dimensional dynamic convolution C2f (ODC2f) module combines lightness with the capability of multiscale precise perception, better adapting to the online recognition and segmentation tasks for various instruments. Additionally, we employ the Wise-IoUv3 (WIoUv3) for the boundary box adjustment error function, with a variable incremental nonlinear mechanism to optimize the gradient distribution strategy, further improving the overall segmentation performance. The robustness and efficiency of the proposed method were proven by a dataset of surgical instruments for intracranial gliomas collaboratively created with professional physicians, which achieved an Mean Pixel Accuracy (mPA) of 95.2% and an Mean Intersection over Union (MIoU) of 89.8%. Extensive experiments demonstrated that our proposed method exhibits excellent performance in understanding surgical scenarios, increasing the safety and success rate of operations.