LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Benchmark

Lojze Žust, Janez Perš, Matej Kristan
University of Ljubljana
ICCV 2023
Paper Datasheet Download Code

LaRS features diverse and challenging USV-centric scenes with per-pixel panoptic annotations.

Abstract

The progress in maritime obstacle detection is hindered by the lack of a diverse dataset that adequately captures the complexity of general maritime environments. We present the first maritime panoptic obstacle detection benchmark LaRS, featuring scenes from Lakes, Rivers and Seas. Our major contribution is the new dataset, which boasts the largest diversity in recording locations, scene types, obstacle classes, and acquisition conditions among the related datasets. LaRS is composed of over 4000 per-pixel labeled key frames with nine preceding frames to allow exploiting temporal texture, amounting to over 40k frames. Each key frame is annotated with 11 thing and stuff classes and 19 global scene attributes. We report the results of 27 semantic and panoptic segmentation methods, along with several performance insights and future research directions. To enable objective evaluation, we have implemented an online evaluation server. The LaRS dataset, evaluation toolkit and benchmark are publicly available.

Dataset highlights

  • Largest and most diverse panoptic maritime obstacle detection dataset
  • Diverse scenes from manual capture, public videos and existing datasets
  • 4000+ manually labelled frames:
    • 3 stuff classes and 8 thing (dynamic obstacle) categories
    • 20 scene-level attributes
  • Temporal context for each annotated frame (9 frames, total 40k frames)

Annotation statistics

Annotation examples

LaRS annotations segment the image into sky (purple), water (blue) and static obstacle (yellow) stuff classes and several dynamic obstacle thing classes (various colors). Hover mouse over the image to see the annotations.

Leaderboards

Download

Annotation format

LaRS dataset comes with three types of annotations: panoptic, semantic and scene attributes.

Panoptic annotations are provided in COCO format and include the panoptic_annotations.json annotation file and masks in the panoptic_masks directory. Categories and their IDs are defined in the categories field of the annotation file and are as follows:

ID Name Type Supercategory
1 Static Obstacle Stuff obstacle
3 Water Stuff water
5 Sky Stuff sky
11 Boat/ship Thing obstacle
12 Row boats Thing obstacle
13 Paddle board Thing obstacle
14 Buoy Thing obstacle
15 Swimmer Thing obstacle
16 Animal Thing obstacle
17 Float Thing obstacle
19 Other Thing obstacle

Semantic annotations are provided as PNG masks in the semantic_masks directory. Semantic annotations are pixel-wise labels of 3 classes, obstacles, water and sky as defined by the supercategories of the panoptic categories. We use the following label IDs in the PNG files:

ID Category
0 Obstacles
1 Water
2 Sky
255 Ignore

Scene attributes are stored in the image_annotations.json file.

BibTeX

If you use this dataset, please cite our work:

@InProceedings{Zust2023LaRS,
  title={LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Benchmark},
  author={{\v{Z}}ust, Lojze and Per{\v{s}}, Janez and Kristan, Matej},
  booktitle={International Conference on Computer Vision (ICCV)},
  year={2023}
}