 Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags. |
Operation on QNX
Description: Daily operation of Hironxo's Controller Box (that runs on QNX real-time OS)Keywords: Hiro, NEXTAGE OPEN, QNX, controller
Tutorial Level: ADVANCED
Next Tutorial: rtmros_nextage/Tutorials/Changing Nextage Grippers in Robot Model
Contents
NOTE: All the tutorials available under the URL http://wiki.ros.org/rtmros_nextage/Tutorials are applicable to the multiple products of Kawada Industries; Hiro (only with the one that opensource software is installed) and NEXTAGE OPEN. To simplify the notation in the rest of the tutorials, we use HiroNXO to appoint the aforementioned robots.
Introduction
All of HiroNXO software is publicly available, which doesn't mean that you can just download and run it on your HiroNXO out of the box. You need to build/compile it on the robot's controller host ControllerBox that runs on QNX operating system. And to compile on QNX you need a developer license which you can purchase from the vendor of your region.
How to install necessary software on QNX is not disclosed either. Other than the developer's license issue, QNX is a commercial software that you're required/advised to be certified for operation, or at minimum should be familliar enough with it. Also due to the lack of robust packaging infrastructure (like ROS has) on QNX, installation work can involve a long list of manual tasks and thus is very error prone.
However, for the cases as below touching inside of QNX is required:
When you need to rollback to the previous closed-source GRX controller (to use the program you wrote for it).
See log files for debugging cases that aren't clear from logs available via API on Ubuntu software.
In these cases you can obtain the logon account info from either manufacturer or the software service provider.
In this page you'll find minimum things that you may find useful about setting on QNX on Hiro/NEXTAGE OPEN.
We assume that you can log on to QNX by:
YOURHOST$ ssh -l %QNX_YOUR_USER% %IPADDR_YOUR_QNX%
Daily Operation
NOTE: Need for a GUI tool to operate everyday task is noticed. Work is in progress.
Shutdown QNX
Using the tool officially provided by Kawada Robotics called NextageOpenSupervisor, you can do so from GUI. The tool's icon should be found on the Desktop when you're logged on to UIC with the nxouser account. No root password is needed.
- Also if you prefer using commandline, do the following. Note that root password is required.
QNX$ su -c '/opt/grx/bin/shutdown.sh'
Reboot QNX
The following command only works on NXO (ref).
QNX$ su -c '/bin/shutdown'
QNX Specific Troubleshooting
Before you move on to scroll down this page, you're advised to go through the troubleshooting for Hironx / NEXTAGE OPEN page to review general ways of shooting issues or getting support.
Although we hope you found solution/workaround there, sometimes you might need to look into logs deep in the robot. Here are some ways to troubleshoot issues on your Hiro/NEXTAGE OPEN that might be rooted in software on QNX.
Log Files
Type of Available Log Files
In /opt/jsk/var/log, you'll find a few types of log files:
Nameserver.log
Issues related to OpenRTM or CORBA are most likely written here.
Modelloader.log
OpenHRP3 issues logged here.
rtcd.log
Issues concerning RT Components from hrpsys are recoreded.
Copy of each of above file is generated automatically in the same folder.
Operation with Logs
These log files above are generated automatically, and won't get removed automatically at the time of writing (3/18/2014). Also, currently there's no way to look into log files remotely other than logging in on to QNX (via ssh).
Need to retrieve logs for debug
Often while debugging you need controller's log that is only available in the log files on QNX. You can get them by either one of the following. Note that you need login info.
(RECOMMENDED) Run a script to get a zip ball of the log files (**NOTE** this feature is available from rtmros_hironx 1.1.25).
# Simplest $ rosrun hironx_ros_bridge qnx_fetch_log.sh nextage qnx_nxo_user : $ ls opt_jsk_var_logs_20170602-020236.zip # Fetch only files generated after certain date. "1" can be anything except "archive" $ rosrun hironx_ros_bridge qnx_fetch_log.sh nextage qnx_nxo_user 1 2017-01-11
Alternatively, you can ssh into QNX and access the log file under the directory /opt/jsk/var/log.
Maintain logs regularly
Log files by themselves can easily take up gigabytes of the disk space. It is advised that you occasionally remove log files under /opt/jsk/var/log either manually or by custom script, since no mechanism to delete any log files. To do so, you can either use the same script qnx_fetch_log.sh or log on to QNX and remove manually.
# Remove all raw .log files to free disk space. Same .zip file will be kept in the log folder. $ rosrun hironx_ros_bridge qnx_fetch_log.sh nextage qnx_nxo_user archive
To Rollback to Closed-source Grx Controller
(This section will be filled per request basis. See Contribution section.)
Contribution
Hironx / NEXTAGE OPEN related QNX issues are currently handled in a private repository.
Post your thoughts on rtm-ros-robotics forum (this will be replaced by a designated issue tracker).
Hint to check server status
Since currently there's not much tools for you to remotely measure the status of processes inside of QNX on HiroNXO, you need to log in and check certain log files or processes. Below is the list of phenomenons each of which section describes what you need to check.
All 4 chest lamps on the robot are still blinking after powering it up
You want to see only white and green blink.
See /opt/jsk/var/log/rtcd.log. You want to see things like this:
QNX$ more /opt/jsk/var/log/rtcd.log Logger::Logger: streambuf address = 0x805fc70 hrpExecutionContext is registered pdgains.file_name: /opt/jsk/etc/HIRONX/hrprtc/PDgains.sav dof = 15 open_iob - shmif instance at 0x80b3f58 the number of gyros = 0 the number of accelerometers = 0 the number of force sensors = 0 period = 5[ms], priority = 49
If you see something different from above, first check if /opt/jsk/var/log/Nameserver.log looks like:
Sat Jan 24 10:55:33 2015: Starting omniNames for the first time. Wrote initial log file. Read log file successfully. Root context is IOR:010000002b00000049444c3a6f6d672e6f72672f436f734e616d696e672f4e616d696e67436f6e74 6578744578743a312e300000010000000000000070000000010102000d0000003139322e3136382e312e313600009d3a0b00 00004e616d6553657276696365000300000000000000080000000100000000545441010000001c0000000100000001000100 0100000001000105090101000100000009010100035454410800000095fbc2540100001b Checkpointing Phase 1: Prepare. Checkpointing Phase 2: Commit. Checkpointing completed. Sat Jan 24 11:10:33 2015: Checkpointing Phase 1: Prepare. Checkpointing Phase 2: Commit. Checkpointing completed.
Then, check if /opt/jsk/var/log/ModelLoader.log says like:
ready loading /opt/jsk/etc/HIRONX/model/main.wrl Humanoid node Joint nodeWAIST Segment node WAIST_Link Joint nodeCHEST_JOINT0 Segment node CHEST_JOINT0_Link Joint nodeHEAD_JOINT0 Segment node HEAD_JOINT0_Link Joint nodeHEAD_JOINT1 Segment node HEAD_JOINT1_Link VisionSensorCAMERA_HEAD_R VisionSensorCAMERA_HEAD_L Joint nodeRARM_JOINT0 Segment node RARM_JOINT0_Link Joint nodeRARM_JOINT1 Segment node RARM_JOINT1_Link Joint nodeRARM_JOINT2 Segment node RARM_JOINT2_Link Joint nodeRARM_JOINT3 Segment node RARM_JOINT3_Link Joint nodeRARM_JOINT4 Segment node RARM_JOINT4_Link Joint nodeRARM_JOINT5 Segment node RARM_JOINT5_Link Joint nodeLARM_JOINT0 Segment node LARM_JOINT0_Link Joint nodeLARM_JOINT1 Segment node LARM_JOINT1_Link Joint nodeLARM_JOINT2 Segment node LARM_JOINT2_Link Joint nodeLARM_JOINT3 Segment node LARM_JOINT3_Link Joint nodeLARM_JOINT4 Segment node LARM_JOINT4_Link Joint nodeLARM_JOINT5 Segment node LARM_JOINT5_Link The model was successfully loaded !If all of above looks ok, something complicated may be happening. ask for support.