Are All Marine Species Created Equal? Performance Disparities in Underwater Object Detection

Underwater object detection is critical for monitoring marine ecosystems but poses unique challenges, including degraded image quality, imbalanced class distribution, and distinct visual characteristics. Not every species is detected equally well, yet underlying causes remain unclear. We investigate two questions: 1) What factors beyond data quantity drive class-specific performance disparities? 2) How can we systematically improve detection of under-performing marine species? Using manipulated DUO and RUOD datasets, we decouple localization from classification and investigate the scallop class as a representative case. Localization analysis using YOLO11 and TIDE shows that foreground-background discrimination is the most problematic stage regardless of data quantity. Classification experiments reveal persistent precision gaps even under balanced sampling, indicating intrinsic feature-based challenges beyond data scarcity. We recommend imbalanced distributions when prioritizing precision, and balanced distributions when prioritizing recall. Improving under-performing classes should focus on algorithmic advances, especially within localization modules. We publicly release our code and datasets.