Contents
Aims
Regression tests - tests to make sure previously fixed bugs don't regress
openModeller core classes - tests to ensure the core create and project model operations work as expected
Algorithm tests - tests to ensure that changes to the lib dont break plugins and that all plugins are working within expected norms
Unexpected parameter tests - tests to make sure the lib gracefully handles situations where unexpected parameters passed to om functions dont cause a crash.
The tests should simulate the work typically carried out by an openModeller client.
Example outputs from other projects
BuildBot - A different automated test report system (useful for reference but we should probably use CMake + CTest + Dart)
Suggested technologies
- CTest
Possible additional technologies
External links
Work Plans - Albert
- Research C++ testing frameworks
- Get Familiarity With C++ (OOP principles)
- Get to know openModeller library code base
- Get to know CMake, CTest and Dart
- Get to know SVN
- Generate prototype tests with the chosen candidate frameworks
- Document pros and cons of each framework on wiki
- Select a c++ testing framework (with peer review)
- Begin implementation using the chosen framework
- Document everything on wiki including:
- Tutorials for writing new tests
- In-source doxygen documentation
- Code templates for new tests
Unit testing Evaluation Framework
Framework |
GPL Friendly? |
Well Documented |
Tutorials |
Mailing Lists / Forum |
Linux |
Mac OSX |
Win (MSVC) |
Win (msys) |
Easy to Learn? |
Suites? |
Fixtures? |
CTest Integration |
Has Dependencies |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CppUnit Lite |
Unknown |
(x) |
(x) |
(x) |
|
Unknown |
Unknown |
Unknown |
|
(x) |
(x) |
Unknown |
(x) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Boost.Test |
(x) |
|
|
|
|
|
|
Unknown |
|
|
|
Unknown |
Unknown |
Frameworks
CppUnit: Good framework but requires some additional work for developing tests.
CppUnit Lite: It doesn´t support fixtures and suites, and has very poor documentation
CxxTest: Good framework and very easy to develop tests.
- Boost.Test: This framework is part of Boost library (very large) and has very good documentation.
Desirable Outcomes
- Unit tests grouped by functional areas: lib, console apps, env_io, occ_io
- Template tests that we can use to move towards TDD (test driven development)
Prioritised List of Classes to test
- Priority 1 = Urgent
- Priority 2 = Required
- Priority 3 = Can Wait
Least complex classes will be listed before more complex.
Class Name |
Completed? |
Priority |
Notes |
Port Ricardos tests to cxxunit |
|||
Sample.hh |
|
3 |
|
SampleExpr.hh |
|
3 |
|
|
3 |
|
|
ignore_traits.hh |
|
3 |
|
Configuration.hh |
|
3 |
|
Core Library - simple tests |
|||
AlgMetadata.hh |
x |
2 |
Contains structs for AlgParameterMetadata and AlgMetadata |
AlgParameter.hh |
|
3 |
Container for an algorithm parameter |
x |
3 |
Port Tims ctest/om_load_algs test for this. |
|
Random.hh |
|
3 |
No complex types and used to generate random numbers. |
Configuration.hh |
|
3 |
Provides capability for serialisation and deserialisation to and from XML |
Configurable.hh |
x |
3 |
Simple base class that serialiseable classes inherit. Needs a instance of Configuration for testing. |
Exceptions.hh |
|
3 |
No complex classes used as params. This file provdes for throwing custom exceptions from various points in the om lib. |
Log.hh |
|
3 |
Provides application logging to file, stdout or via a call back. No complex classes used as params |
Occurrence.hh |
|
3 |
Uses Sample. Represents a georeferences occurrence locality. Should not be too complicated to test. | |
MapFormat.hh |
|
3 |
Uses Map object but otherwise all params are simple types. MapFormat provides a representation of the properties of a raster layer. |
AreaStats.hh |
|
3 |
Basic start available in om_test_areastats - check what else needs to be tested in this class. |
Core Library - complex tests |
|||
Algorithm.hh |
x |
3 |
AlgorithmImpl base class and Algorithm class def. Uses complex objects as params. |
x |
3 |
More complex to test. Requires Model, Environment, Occurrences, Sampler |
|
Occurrences.hh |
x |
3 |
Needs GeoTransform and Environment objects. |
Environment.hh |
x |
3 |
A collection of layers with ability to fetch data for the collection. Defines EnvironmentImpl, needs Map and SampledData |
Model.hh |
x |
2 |
Not sure what this class is for?? Needs SamplePtr and EnvironmentPtr. |
Normalizable.hh |
|
0 |
An abstract base class for normalisable datasets. Has no implementation (pure virtual) so no test needs to be written. |
OpenModeller.hh |
x |
3 |
One of the most complicated to test, this is the key conroller class for modelling. Uses Log, Configuration, Sampler, Algmetadata, Algorithm etc. |
Projector.hh |
x |
3 |
Used for projecting model into a climate scenario. Uses Model, AreaStats and en_io stuff. |
RocCurve.hh |
x |
0 |
|
Sampler.hh |
x |
0 |
|
x |
0 |
||
TeUrlParser.hh |
x |
0 |
|
matrix.hh |
x |
0 |
|
minimum_distance.hh |
x |
0 |
|
om.hh |
x |
0 |
|
om_defs.hh |
|
0 |
|
om_soap.hh |
x |
0 |
|
os_specific.hh |
x |
3 |
Operating specific defines for openModeller. Not too much to test here I dont think |
refcount.hh |
x |
0 |
|
Console |
|||
consolexml.hh |
x |
0 |
|
file_parser.hh |
x |
0 |
|
request_file.hh |
x |
0 |
|
Algs |
|||
aquamaps.hh |
x |
0 |
|
bioclim.hh |
x |
0 |
|
csm.hh |
x |
0 |
|
csmbs.hh |
x |
0 |
|
csmkg.hh |
x |
0 |
|
distance_to_average.hh |
x |
0 |
|
environmental_distance.hh |
x |
0 |
|