Skip to content

Home

The objective of SEEREP (SEmantic Environment REPresentation) is to store generated robot data and enable fast spatio-temporal-semantic queries over the data.

Context

Autonomous robotic systems must be aware of their environment, in order to safely achieve their goal-oriented actions. Especially in unstructured and changing environments, a detailed model of the environment is required for reasoning and planning.

The sensors of a robot provide spatial information via the robot's pose, temporal information is created by the point in time when a sensor is read. Semantic information always exists implicitly and can be made explicit by algorithms or manual labeling.

Most existing environment representations focus on one or two of these information types, SEEREP is able to store all three. Thereby, SEEREP enables the robot to reason on a higher level and disambiguate sensor data based on the context.

Core Features

  • Store robotic sensor data
  • Images
  • 3D-Points
  • Point Clouds
  • TF
  • Store analysis results alongside the corresponding sensor data
  • Add (bounding box based) semantic annotations with confidences to the data
  • Extend semantic annotations by object instances which are in multiple datasets
  • Store information about an instance alongside the data
  • Link 3d points to an instance to define the instance position
  • Allow multiple categories of labels (e.g. a category per CNN). Thus, enabling easy evaluation and comparison of multiple CNNs based on the same data
  • Fast spatio-temporal-semantic queries with gRPC.
  • Storage of data generated by the robotic system:
  • Offline on the robot (no or slow internet connection), currently in Progress #89
  • Online on a server-cluster, with gRPC.
  • Shifting computation loads away from the robot and into the cloud.
  • Easily switch between Protocol Buffers (PB) / Flatbuffers (FB) as the messaging format.

Architecture

The following graphic provides a broad overview of SEEREPs components. The sensor data along with results from processing the data and annotations are stored in HDF5 files. SEEREP uses projects to group common information (e.g. a scanning campaign). The data can be saved locally on the robot or sent into the cloud with gRPC. Therefore, the computational load on the robot can be reduced and algorithms can fetch the data subset which they actually need.

SEEREP-Overview

A more detailed version, with all ROS packages and message types is available in the package overview.