Please note that these Trac pages are no longer being updated. Wiki contents/documentation have moved to GitHub.

Running Repy Unit Tests



Due to references to the SVN single-repo directory structure and preparetest.py, this document is outdated. UnitTestFrameworkRunning remains relevant, and BuildInstructions contains updated information.


Overview


Seattle consists of several different components including, the Repy VM (which executes programs), the node manager (which accepts incoming connections), Seattle standard libraries, and software updater. There is a separate wiki page that describes how to run the software updater unit tests. There currently isn't a script that runs all of the unit tests for the Seattle standard libraries (but this is under development).

There are also integration tests, tests for SeattlGENI, and similar other tests which are not covered here.


Running the Repy VM tests


Repy and the unit tests cannot be run straight out of svn. Please create a new directory which will store the runnable copy of the files (if you use an existing directory, the contents will be erased).

To run the repy unit tests locally, first navigate to the trunk directory of your svn working directory in a terminal, then execute the following commands:

python preparetest.py -t /path/to/test/directory
cd /path/to/test/directory

This will set up the directory with the appropriate files. To run the tests, type:

python utf.py -m repytests

This will start all of the repy unit tests running, and will output which tests pass and fail, and give failure details intermittently for the ones that do fail.


Running Nodemanager Tests


To run the node manager unit tests locally, in a shell, navigate to the trunk directory. Create a test directory /path/to/test/directory if one is not already there. Next, type the following command:

python preparetest.py -t /path/to/test/directory
cd /path/to/test/directory
python utf.py -m nm

<test_name> refers to the name of the test file that is to be run (do not include the .mix extension). To see which tests are available to run navigate to nodemanager\tests in the trunk folder.

Known Issue 1: the test ut_nm_changeowner_validsequenceid.py fails since sequence id's are currently broken.
Known Issue 2: Occasionally the testscript freezes as it is running (no change in status for 3 minutes or more). This is rare and the cause is not yet fully known (appears to be related to connection resets in the node manager). If this occurs, restart the tests by calling the python utf.py command again.


Running Individual Nodemanager Tests


To run the repy unit tests locally, first navigate to the trunk directory of your svn working directory in a terminal. Create a test directory <test_directory_name> if one is not already there. Run:

python preparetest.py <test_directory_name>
cd <test_directory_name>
python nminit.py
python nmmain.py

Open a new terminal and navigate to the test directory in the trunk that was used previously. The following command can be used to run a test:

python repy.py restrictions.test <test_name>.py

<test_name> refers to the name of the test file that is to be run (do not include the .mix extension). To see which tests are available to run navigate to nodemanager\tests in the trunk folder.


Running Nodemanager threading error / restrictions processor check (advanced)


To run the node manager threading test locally, first navigate to the trunk directory of your svn working directory in a terminal, then execute the following commands:

python preparetest.py -t /path/to/test/directory
cd /path/to/test/directory
python testnmthreadingerror.py

This will start the test running, and will output a status report with the results of the test.


Running Seattle Library Tests


Running the Seattle library tests is much like the node manager and repy tests. First, prepare the files in the a directory. Next, type the following command:

python preparetest.py -t /path/to/test/directory
cd /path/to/test/directory
python utf.py -m seattlelibtests



Note, the below tests rarely need to be run. They may be disruptive and hard to run.


Running Network Restriction Check Locally (advanced)


To run the repy network restriction tests locally, first navigate to the trunk directory of your svn working directory in a terminal, then execute the following commands:

python preparetest.py -t /path/to/test/directory
cd /path/to/test/directory
python utf.py -m repynetworktests

This will start the test running, and will output a status report with the results of the test.


Running Threading Error Check Locally (advanced)


Be warned that this test is designed to push the system to its thread limit. This means you may experience system instability. Windows and OSX seem to remain stable, but near the end of the test Linux will exhaust its system wide thread availability.

To run the repy threading error test locally, first navigate to the trunk directory of your svn working directory in a terminal, then execute the following commands:

python preparetest.py -t /path/to/test/directory
cd /path/to/test/directory
python run_tests.py --obsolete -threaderr

This will start the test running, and will output a status report with the results of the test. The --obsolete flag is necessary because the run_tests script is being phased out, and we are encouraging the use of our new unit testing framework.


Running CPU Throttling Check Locally (advanced)


To run the repy CPU throttling test locally, first navigate to the trunk directory of your svn working directory in a terminal, then execute the following commands:

python preparetest.py -t /path/to/test/directory
cd /path/to/test/directory
python run_tests.py --obsolete -cpu

This will start the test running, and will output a status report with the results of the test. The effectiveness of CPU throttling is hard to judge into Pass and Fail, so that is left to the users judgement. The --obsolete flag is necessary because we are in the middle of transitioning our tests to our new unit testing framework. However, the cpu tests are not yet converted.