[Documentation] [TitleIndex] [WordIndex

Package Summary

This package contains nodes which make up the Passive-Scene-Recognition interface to Implicit Shape Model (ISM) trees. The Active-Scene-Recognition interface to ISM trees is located in asr_recognizer_prediction_ism, instead. A short outline of the functionalities provided by the nodes are: 1. Recording of scenes 2. Training of an ISM tree (Implicit shape model tree) 3. Recognition of scenes 4. Visualization of ISM (tree) data etc.

  • Maintainer: Meißner Pascal <asr-ros AT lists.kit DOT edu>
  • Author: Borella Jocelyn, Hanselmann Fabian, Heller Florian, Heizmann Heinrich, Kübler Marcel, Mehlhaus Jonas, Meißner Pascal, Qattan Mohamad, Reckling Reno, Stroh Daniel
  • License: BSD
  • Source: git https://github.com/asr-ros/asr_ism.git (branch: master)

Package Summary

This package contains nodes which make up the Passive-Scene-Recognition interface to Implicit Shape Model (ISM) trees. The Active-Scene-Recognition interface to ISM trees is located in asr_recognizer_prediction_ism, instead. A short outline of the functionalities provided by the nodes are: 1. Recording of scenes 2. Training of an ISM tree (Implicit shape model tree) 3. Recognition of scenes 4. Visualization of ISM (tree) data etc.

  • Maintainer: Meißner Pascal <asr-ros AT lists.kit DOT edu>
  • Author: Borella Jocelyn, Hanselmann Fabian, Heller Florian, Heizmann Heinrich, Kübler Marcel, Mehlhaus Jonas, Meißner Pascal, Qattan Mohamad, Reckling Reno, Stroh Daniel
  • License: BSD
  • Source: git https://github.com/asr-ros/asr_ism.git (branch: master)

Logo-white-medium.jpg

Description

This package serves as the Passive-Scene-Recognition interface to the functionality provided by the asr_lib_ism library in the ROS environment. The Active-Scene-Recognition interface to ISM trees is located in asr_recognizer_prediction_ism, instead. The core functions of this package are the training and recognition of scenes which consist of various objects using Implicit Shape Model (ISM) trees. In addition to the implementation of the tools provided by asr_lib_ism there are several visualization (based on asr_ism_visualizations) and utility tools to make the usage of the scene recognition system more convenient.

example_recognizer_viz.png

Functionality

As mentioned above the basic functionality of the scene recognition system consists of two main parts: The training of a scene model and the recognition of a trained model in a new scene afterwards.

There are two training approaches implemented, one simply being called Trainer, the other CombinatorialTrainer. They are interchangeable but as they use different algorithms the trained models result in a difference in recognition quality mostly in terms of speed as the latter has a shorter computation time. The input of both trainers is a recording of a scene consisting of multiple objects which is stored in a sqlite database and can be created using the "Recorder" tool. The "Recognizer" tool uses the trained models and tries to find the most likely match given a set of objects.

In general the steps required are the recording of a scene, training a model from this recording and then using the recognizer to get the most likely scene match from a new object configuration. Below you can see a list of all tools this package provides including the mentioned ones and tutorials on how to use them. For more information about the functionality see the asr_lib_ism package.

Wrapper Nodes

The nodes in this list wrap tools from asr_lib_ism to provide them in the ROS environment:

Visualization Nodes

Although many nodes of this package publish some kind of visualization, the purpose of these nodes is solely to visualize ism data in rviz.

recordViewer: Visualize recorded data of a given pattern as a path for each object on which these objects moved while recording.

modelViewer: Expand the recordViewer visualization with vectors from positions of an object on its path at a certain point in time to its reference path at the same point in time. This visualization can help the user to understand how the model was created from each record data point.

voteViewer: Provide two different vote visualizations, the first just visualizes all votes from a fixed pose for a selected object in a selected subpattern and the second visualizes a given configuration with each object's votes that the model of a selected pattern provides.

Additional Nodes/Classes

ObjectConverter (class): This class can be used for custom nodes to convert ros messages (asr_msgs::AsrObject) to data-types used by the asr_lib_ism library. This class is implemented in the ism_helper.hpp/.cpp file.

scene_configurator (class): This class expands a node with the functionality to configure an object configuration interactively. Main features of this class are the multi-threaded processing of incoming messages (number of threads set by the user) and the processing of keyboard input (polling on its own dedicated thread). With this approach the actual node runs on its own thread(s) and decouples the SceneConfigurator logically and computationally from the node.

fake_data_publisher (node): This node republishes recorded data from a given database as ros messages (asr_msgs::AsrObject).

object_configuration_generator (node): This node generates an object configuration by editing object data from a given database or XML-file and writes it to an XML-file.

Usage

Because this package is a collection of nodes and helper classes to enable the user to write his own nodes, it is recommended to use the tutorials on how to use the particular tool, however in most cases the description of the nodes and their parameters should be sufficient.

Needed packages

ROS Nodes

Subscribed Topics

Published Topics

Parameters

Launch files

launch/combinatorialTrainer.launch:

launch/data_cleaner.launch:

launch/data_merger.launch:

launch/fake_data_publisher.launch:

launch/marker_rotator.launch:

launch/modelViewer.launch:

launch/object_configuration_generator.launch:

launch/pose_interpolator.launch:

launch/recognizer.launch:

launch/recorded_objects_transformer.launch:

launch/recorder.launch:

launch/recordViewer.launch:

launch/rotation_invariant_objects_rotator.launch:

launch/trainer.launch:

launch/voteViewer.launch:

YAML files

param/capturing.yaml:

param/sensitivity.yaml:

param/sqlitedb.yaml:

param/visualization.yaml:

Needed services

/asr_object_database/object_meta_data: This service is only called by the ObjectConverter class and provides it with additional information about recognizable objects from the asr_object_database.

Tutorials

How to use a SceneConfigurator-Node

How to use the recorder

Training an ISM

Training an ISM using the Combinatorial-Trainer

How to use the recognizer

How to use the visualization nodes

How to use the fake_data_publisher

How to use the object_configuration_generator

Troubleshooting

Problem: Missing object estimations when using a SceneConfigurator -Node and the asr_flir_ptu_driver GUI.

If you are using the ptu_gui.launch from asr_flir_ptu_driver and you checked the option “update current angle immediately”, it may happen that the SceneConfigurator clears the object estimations in the current view, because the ptu toggles its “reached_desired_position” parameter.

Solution: Make sure to uncheck the “update current angle immediately” option if you want to work with the current view.

When using ISMs, please cite the following publication. Thank you!

When using Combinatorial Trainer, please cite the following publication. Thank you!


2020-08-08 12:28