This folder contains testing routines for ForceBalance

=====================
=   Basic usage     =
=====================
To run through all tests and print results to standard out, run
python test
from a shell at the root 'forcebalance' directly.

To see options for more advanced usage, run
python test -h

To work properly, tests need to be able to "import forcebalance" from within the python
environment. This means that tests must be run on a host with forcebalance already installed. In
addition, any changes made to the forcebalance source files will not be reflected in test results
until forcebalance is reinstalled. Because of this, its a good idea to run
python setup.py install
before running the test suite to ensure tests are being run on the proper file versions. This means
possibly running setup.py to make sure the latest version of forcebalance is installed.

=====================
=   Contents        =
=====================
Contents of this directory are as follows:

    files/      : This directory holds files used to provide sample input for various test cases

    test_*.py   : Test modules named after the corresponding files in ../src/ being tested

    test_system.py: A special test module with integration tests that run through full calculations

    __init__.py : Classes that modify the python unittest framework and provide general unit testing functionality

    __main__.py : Functions providing an interface from which tests can be run from the command line

    __test__.py : some basic test cases that test result presentation
                  This is also a good template to use when creating a new test module file

=========================
=   Writing Test Cases  =
=========================

Unit tests are organized into test modules in the test folder named according to the forcebalance module they test,
prefixed by 'test_'. For example, tests for objective.py go into a file at test/test_objective.py

All test modules should start by importing the following:

    from __init__ import ForceBalanceTestCase
    import unittest
    import forcebalance

along with any other modules required by tests. Modules should end with:

    if __name__ == '__main__':           
        unittest.main()

This provides added flexibility by allowing test modules to be run as standalone scripts using a default test runner.

Within a test module, tests are split into test cases. Test cases are subclasses of ForceBalanceTestCase. They are
usually named after the class being tested, prefixed by "Test". For example, tests for the Objective class go into a
TestObjective class. The test loader will automatically instantiate and run any class derived from TestCase.

Test Cases contain several special methods which determine what happens during a test:
    setUp is run before every test that is run
    test* methods define the actual tests to be run after setUp is complete
    runTest is run if there are no test* methods and can be used if a test case only has one associated test method
    tearDown is run after tests have completed and is run regardless of test outcome as long as setUp was successful
    addCleanup allows you to dynamically add cleanup actions depending on the progress of the test
    doCleanups is automatically run when exiting from the test case and runs the actions defined by addCleanup calls.

Test methods will usually perform some action and call self.assert* functions provided by the unittest framework.
These assert methods check some condition. If the condition is not met, the test fails and an error message is printed.
The most common asserts used are self.assertEqual/self.assertNotEqual and self.assertTrue/self.assertFalse, but there
are a number of others provided by the unittest framework (http://docs.python.org/2/library/unittest.html#assert-methods).
Test cases have fail and skipTest methods which both take as input a string explaining the reason for failing or skipping.

Some forcebalance modules/classes provide base classes which are never instantiated themselves, but which provide
a framework for other more specific classes to expand upon. In this case, classes with general tests that can be run
on derived classes can be included in a *Tests class. For example, general tests for any class derived from the Target
class are located in test/test_target.py in a TargetTests class. Note that to prevent the test loader from running these
"abstract" tests directly, these classes are not derived from TestCase. To load these tests, we use the multiple inheritance
feature of Python to create classes derived from BOTH ForceBalanceTestCase and the abstract tests class. For example,
the TestCase class header for the AbInitio_GMX class looks like this:
    class TestAbInitio_GMX(ForceBalanceTestCase, TargetTests)
This way TargetTests can provide tests for all Target subclasses but the setup for specific types of targets can be
done by the individual subclass test cases.

=============================
=   Test case documentation =
=============================

When possible, try to include docstrings and comments within test cases. The test runner will automatically print
the first line of the docstring for the test method currently being run. In addition, it would probably be helpful to 
send as much information as possible to the debug logger (ForceBalanceTestCase provides an instance logger, so you can use
self.logger.debug(msg) from anywhere within a test case and not worry about having to import things in a test module.

=====================
=   Headless Config =
=====================
In headless mode, tests results are not printed to the screen. Tests will be run in this
mode when a headless config file is provided as a command line argument:
python test --headless-config=config_file.conf

the config file is a line separated list of options in the form
option=value
which determine where test results are sent

Available options to be placed in the headless config file are:
log_location (string)       : path to put test log. If not specified, log is not saved
enable_smtp (string)        : valid values are
                              'true'- send test results via email after every test
                              'error'- only send if any tests fail or have errors
smtp_server (string)        : the url of the smtp server used for sending log information
smtp_port (integer)         : the port number to use when connecting to the smtp server
smtp_user (string)          : the user to use when connecting to the smtp server
smtp_password (string)      : the password to use when connecting to the smtp server
smtp_tls (boolean)          : whether the smtp server requires TLS when authenticating
smtp_destination (string)   : the destination address where the error log will be sent
smtp_source (string)        : specifies the "from:" header in the outgoing email
smtp_subject (string)       : specifies the subject of the outgoing email
