Python unit testing

catkin rosbuild

Disclaimer

(11/28/2013): Some parts of this page may be outdated, especially to the standard introduced after catkin. Until this page gets fully updated (so that this whole disclaimer section is removed by the author who completes the update), be careful to also read some other documents including:

• http://docs.ros.org/hydro/api/catkin/html/howto/index.html

• http://answers.ros.org/question/94526/any-good-examples-of-ros-unittests/

Introduction

See also:

• UnitTesting

• gtest

Python unit tests use the built-in unittest module with a small wrapper to enable XML output of the test results. In other words, Python unittest-based tests are compatible with ROS as long as you add an extra wrapper that creates the necessary XML results output.

There are two style of tests that are supported:

• ROS-Node-Level Integration Tests: in general, these test against the external topic/service API of a Node. For these tests, it is assumed that your test script is itself a node.

• Code-Level Unit Tests: in general, these tests make direct calls into your code; i.e. these are typical unit tests. Your test script is not a node.

The syntax for running these tests is different, so it's important that you properly distinguish between the two. Node-level tests will incur additional ROS resources to test whereas code-level tests are more lightweight.

ROS-Node-Level Integration Tests

Writing a unit test that acts as a ROS node is fairly simple in Python. You just have to wrap your code using the Python unittest framework.

http://docs.python.org/library/unittest.html

unittest code

A bare-bones test results looks as follows:

   1 #!/usr/bin/env python
   2 PKG = 'test_roslaunch'
   3 import roslib; roslib.load_manifest(PKG)  # This line is not needed with Catkin.
   4 
   5 import sys
   6 import unittest
   7 
   8 ## A sample python unit test
   9 class TestBareBones(unittest.TestCase):
  10     ## test 1 == 1
  11     def test_one_equals_one(self): # only functions with 'test_'-prefix will be run!
  12         self.assertEquals(1, 1, "1!=1")
  13 
  14 if __name__ == '__main__':
  15     import rostest
  16     rostest.rosrun(PKG, 'test_bare_bones', TestBareBones)

NOTE: PKG should be the name of your package, and your package will need to depend on 'rostest' in the package.xml (manifest.xml with rosbuild).

Almost everything there should be familiar to ROS developers as well as unittest writers. Everything except for the first and last two lines are a standard unittest. The first two lines are the standard ROS-python boilerplate for setting up your python path.

IMPORTANT: As this test is meant to be a ROS node launched via rostest, we need to use the rostest wrapper in our main:

   1     import rostest
   2     rostest.rosrun(PKG, 'test_bare_bones', TestBareBones)

The first line is also important:

   1 #!/usr/bin/env python

The parameters to rostest.rosrun() are:

   1 rostest.rosrun(package_name, test_name, test_case_class, sysargs=None)

• package_name

  • Name of ROS package to record these results as. Test results are aggregated by package name.

  test_name

  • Name to use for test. This name will be used in the filename of the test results as well as in the XML results reporting.

  sysargs

  • Override sys.argv.

  coverage_packages=['module1.foo', 'module2.bar']

  • List of packages that should be included in coverage report.

The rostest.rosrun method assumes that your test is a rospy node and will perform extra operations to try and make sure that your node properly runs and is shut down.

package_name and test_name control where your XML test results are placed (i.e. $ROS_ROOT/test/test_results/''package_name''/TEST-''test_name''.xml). rostest also examines sys.argv for command-line arguments such as --text and --gtest_output.

Update your manifest

You will need to depend on the rostest package in order to setup your Python path correctly.

  <depend package="rostest" />

  <build_depend>rostest</build_depend>

Create a rostest file

You will need to run your node in a rostest file. The rostest tool is based on roslaunch, so this is similar to writing a roslaunch file for your test. Please see the rostest documentation on how to integrate your node into an integration test.

Code-level Python Unit Tests

You can also write normal Python unit tests with ROS based on the Python unittest framework.

http://docs.python.org/library/unittest.html

unittest code

New in Diamondback

A bare-bones test results looks as follows:

   1 #!/usr/bin/env python
   2 PKG='test_foo'
   3 import roslib; roslib.load_manifest(PKG)  # This line is not needed with Catkin.
   4 
   5 import sys
   6 import unittest
   7 
   8 ## A sample python unit test
   9 class TestBareBones(unittest.TestCase):
  10 
  11     def test_one_equals_one(self):
  12         self.assertEquals(1, 1, "1!=1")
  13 
  14 if __name__ == '__main__':
  15     import rosunit
  16     rosunit.unitrun(PKG, 'test_bare_bones', TestBareBones)

This is almost identical to a standard Python unittest.

The last two lines wrap your unit test with rosunit in order to produce XML test results:

   1 import rosunit
   2 rosunit.unitrun('test_roslaunch', 'test_bare_bones', TestBareBones)

The parameters to rosunit.unitrun() are:

   1 rosunit.unitrun(package_name, test_name, test_case_class, sysargs=None, coverage_packages=None)

• package_name

  • Name of ROS package to record these results as. Test results are aggregated by package name.

  test_name

  • Name to use for test. This name will be used in the filename of the test results as well as in the XML results reporting.

  sysargs

  • Override sys.argv.

  coverage_packages=['module1.foo', 'module2.bar']

  • List of packages that should be included in coverage report.

Running tests as part of 'make test'/CMakeLists.txt

You can have ROS automatically run these tests when you type make test by adding the following to your CMakeLists.txt:

catkin_add_nosetests(path/to/my_test.py)

rosbuild_add_pyunit(path/to/my_test.py)

Update your manifest

You will need to depend on the proper package in order to setup your Python path correctly.

  <depend package="rosunit" />

Edit your package.xml to add rosunit as:

  <test_depend>rosunit</test_depend>

Using Test Suites

New in Indigo

Test suites are useful when you want to write separate test cases for different bits of functionality. Test suites are composed of test cases, or even other test suites, which are then run in their entirety when the top-level test suite is run.

Here is a basic example of test suite construction:

test/my_test_cases.py

   1 import unittest
   2 
   3 class CaseA(unittest.TestCase):
   4 
   5     def runTest(self):
   6         my_var = True
   7 
   8         # do some things to my_var which might change its value...
   9 
  10         self.assertTrue(my_var)
  11 
  12 class CaseB(unittest.TestCase):
  13 
  14     def runTest(self):
  15         my_var = True
  16 
  17         # do some things to my_var which might change its value...
  18 
  19         self.assertTrue(my_var)
  20 
  21 
  22 class MyTestSuite(unittest.TestSuite):
  23 
  24     def __init__(self):
  25         super(MyTestSuite, self).__init__()
  26         self.addTest(CaseA())
  27         self.addTest(CaseB())

Here, we have two test cases, A and B, which we add to MyTestSuite. We can then run the suite much like running specific test cases, except that we pass a string to rosunit or rostest rather than the test case itself:

test/run_tests.py

   1 # rosunit
   2 rosunit.unitrun('test_package', 'test_name', 'test.my_test_cases.MyTestSuite')
   3 
   4 # rostest
   5 rostest.rosrun('test_package', 'test_name', 'test.my_test_cases.MyTestSuite')

This assumes a package structure like this:

.
├── CMakeLists.txt
├── package.xml
├── src
└── test
    ├── my_test_cases.py
    └── run_tests.py

More details about test suites can be found in the unittest documentation:

https://docs.python.org/2.7/library/unittest.html#unittest.TestSuite

Loading Tests by Name

New in Indigo

You can also use the names of test cases or suites to run them with rosunit.unitrun or rostest.rosrun.

For example, if we have a test case like this:

test/my_tests.py

   1 class MyTestCase(unittest.TestCase):
   2 
   3     def test_a(self):
   4         self.assertTrue(True)
   5     
   6     def test_b(self):
   7         self.assertTrue(True)

We can run the two individual parts of the test case like so:

   1 rosunit.unitrun('test_package', 'test_name', 'test.my_tests.MyTestCase.test_a')
   2 rosunit.unitrun('test_package', 'test_name', 'test.my_tests.MyTestCase.test_b')

The following lines are equivalent:

   1 from my_tests import MyTestCase
   2 
   3 rosunit.unitrun('test_package', 'test_name', MyTestCase)
   4 rosunit.unitrun('test_package', 'test_name', 'test.my_tests.MyTestCase')

More information about what the string specifier should contain can be found here:

https://docs.python.org/2/library/unittest.html#unittest.TestLoader.loadTestsFromName

Important Tips

1. Make sure to mark your Python script as executable if it is a ROS node. You may need to use the SVN command

svn propset svn:executable ON yourscript

Instead, if the script is a unit test to be run with nosetest, make sure that it is not executable.

If no test is found, try to run nosetest inside the test directory

nosetest -vv

which will debug info on how the test scripts are searched

1. PyUnit documentation can be found here http://pyunit.sourceforge.net/pyunit.html