Multi-Master [Fuerte]

Continued in Groovy Multimaster Planning

Details

SIG Coordinator: Jeff Rousseau

Mailing list:

• discussion archives.

• signup

Topics: Adding multi-master support for a couple of common use cases

Members:

• Jeff Rousseau
• Daniel Stonier
• Jon Fink
• Ken Conley
• Bill Morris
• Christopher Crick
• Nick Armstrong-Crews
• Tully Foote
• Troy Straszheim

Use Cases

Security

Ken Conley, Daniel Stonier, Jeff Rousseau

When connecting to a robot remotely (e.g. web, or even public wifi), its useful to only expose public interfaces instead of the entire robot. One mechanism for doing this is to have a 'public Master', which only has the public subset of topics that are exposed. One can then use iptables and other mechanisms to 'lock down' access to non-publicly declared ports.

This is partially implemented in app_manager. Using a 'interface' file, we are able to dynamically bring up the public master and maintain synchronization of the enumerated topics. The app_manager goes a bit further as it does launching and remapping of the topic names into robot-specific namespaces, which is mainly to support multirobot scenarios. The latter needs much more work to be usable.

Yujin is also very interested in this path - particularly because of commercialisation of products, but also because I think multi-robot stuff is at a higher level - such simplification makes sense.

Multi-Robot

Nick Armstrong-Crews

Robots talking to each other directly (I'm thinking over wifi). They may wander in and out of range of each other.

• clock time synchronization
• best practices: namespaces, tf frame names
• data sharing: how to start with and maintain a common tf frame; how to cache data locally for synchronizing to other robot when connectivity returns
• should be scalable for (small) swarms of robots (maybe allow expansion/substitution in ROS names: e.g., /robotA/*, or /$(this_robot)/tf )
• monitoring for which robots are up, which are down (via ROS messages and/or GUI)
• out-of-scope for this SIG: multi-robot coordination algorithms, MANET routing schemes

Android to Robot

Ken Conley, Daniel Stonier

Currently, an Android device can either run its own master, or attach to a remote master. Ideally, it should be able to do both, i.e. the Android device is a robot, and it is also capable of controlling one or more other robots, all without having to restart any nodes.

NAC: is this a special case of "teleop over wifi?"

• I think this is alot broader - android connectivity methods, android ported multi-robot management tools etc, teleop over wifi would be a subcase of this (DS)

  • Don't we want all that for any teleoperating device, be it android or Ubuntu laptop? (NAC)

• The Android device itself may be a separate ROS system, so it's different (kwc)

  • The teleoperating device, in general, might be a separate ROS system; android is just one such device. (NAC)

• I don't know if this is the appropriate forum, but I'd like to see Android masters and nodes communicating to nodes on a computer over USB, but allowing connections over wifi tether as well. I don't know how to bridge those two networks, but if it's possible, it would allow Android devices to serve as wifi-hotspots/ROS Master/IMU/GPS/Cameras all at at the same time. (Chad Rockey - not part of this SIG )

Faster remote debugging

Ken Conley

When interacting on a remote terminal with a robot, commands line rostopic/etc... can get sluggish as they have to poll the master for the latest information; tab-complete in particular slows down. The master information usually does not change that fast, so it would be ideal to 'cache' this information somehow. As the master DB is not very large, one way of implementing caching is simply to replicate the master. In this particular view, every machine that interacts in a ROS graph would have a master; one authoritative, the rest just syncing.

Auto-Discovery

Daniel Stonier

The ability to be able to advertise/discover ros masters or other services so connections do not have to be manually configured by hand.

Network Topology Awareness

Daniel Stonier

Possible courses of action that might be taken to provide a means of determining and distributing information about the network connection state of components in a multi-robot solution (see Network Quality of Service for implementation ideas).

Teleop over Wifi

Human->joystick->laptop /\/\--wifi--/\/\-->laptop-->robot

Motor commands go out, video feed (and maybe telemetry) comes back.

• Challenge: motor commands must be sent with low latency; ditto for video coming back. If robot roams to where wifi signal strength is low, video feed may hose network capacity and motor commands arrive with low frequency, too late, or in bursts. Then you robot runs into a wall; or worse, out of wifi range with a constant forward velocity, never to return.
• Adding support for "Network Quality-of-Service" may help: video feed could reduce quality (frame rate, resolution, etc.) as network capacity degrades; and/or motor commands could be given priority.

Technologies to support Use Cases

Network Quality-of-Service

Nick Armstrong-Crews

I'm thinking mainly for wifi, but maybe 3G, etc. This is not really multi-master, but the "flaky wifi" topic got lumped into this SIG.

• monitoring of network quality: signal strength, negotiated data rate (capacity), measured throughput, measured latency
• a standard method/library for nodes to specify required/desired network quality (throughput, frequency, latency) and fallback procedures (e.g., reduce pub rate, skip frames, etc.)
• Network Master: like ROS master maintains metadata about nodes/links in the ROS process graph, an authority could keep metadata about quality of links (akin to rostopic bw, rostopic hz, but running constantly for all nodes and not eating up resources to take the measurements). Could dole out "slices" of bandwidth to nodes, either as a suggestion (nodes could still choose to send out a firehose of data), or could interface with ipchains to enforce restrictions. Could be part of ROS master or could be separate
• a GUI (rviz, web UI, other) to view network health in real-time
• at the low level, pub/sub API should allow changing params dynamically (TCP/UDP, queue size), as well as queue flushing, queue retrieval
• network stats could come from querying appropriate linux tools, or could be integrated into pub/sub API and measured by incoming/outgoing messages

Testimonials

• Multi-Robot Mapping Nick Armstrong-Crews

• Concert Daniel Stonier (Yujin Robot)