LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Benchmark

University of Ljubljana
ICCV 2023

LaRS features diverse and challenging USV-centric scenes with per-pixel panoptic annotations.


The progress in maritime obstacle detection is hindered by the lack of a diverse dataset that adequately captures the complexity of general maritime environments. We present the first maritime panoptic obstacle detection benchmark LaRS, featuring scenes from Lakes, Rivers and Seas. Our major contribution is the new dataset, which boasts the largest diversity in recording locations, scene types, obstacle classes, and acquisition conditions among the related datasets. LaRS is composed of over 4000 per-pixel labeled key frames with nine preceding frames to allow exploiting temporal texture, amounting to over 40k frames. Each key frame is annotated with 11 thing and stuff classes and 19 global scene attributes. We report the results of 27 semantic and panoptic segmentation methods, along with several performance insights and future research directions. To enable objective evaluation, we have implemented an online evaluation server. The LaRS dataset, evaluation toolkit and benchmark are publicly available.

Dataset highlights

  • Largest and most diverse panoptic maritime obstacle detection dataset
  • Diverse scenes from manual capture, public videos and existing datasets
  • 4000+ manually labelled frames:
    • 3 stuff classes and 8 thing (dynamic obstacle) categories
    • 20 scene-level attributes
  • Temporal context for each annotated frame (9 frames, total 40k frames)

Annotation statistics

Annotation examples

LaRS annotations segment the image into sky (purple), water (blue) and static obstacle (yellow) stuff classes and several dynamic obstacle thing classes (various colors). Hover mouse over the image to see the annotations.




  title={LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Benchmark},
  author={{\v{Z}}ust, Lojze and Per{\v{s}}, Janez and Kristan, Matej},
  booktitle={International Conference on Computer Vision (ICCV)},