openModeller
Version 1.5.0
|
#include <ConfusionMatrix.hh>
Public Member Functions | |
ConfusionMatrix (Scalar predictionThreshold=CONF_MATRIX_DEFAULT_THRESHOLD, bool ignoreAbsences=false) | |
~ConfusionMatrix () | |
void | reset (Scalar predictionThreshold=CONF_MATRIX_DEFAULT_THRESHOLD, bool ignoreAbsences=false) |
void | setLowestTrainingThreshold (const Model &model, const SamplerPtr &sampler) |
void | calculate (const EnvironmentPtr &env, const Model &model, const OccurrencesPtr &presences, const OccurrencesPtr &absences=OccurrencesPtr()) |
void | calculate (const Model &model, const SamplerPtr &sampler) |
int | getValue (Scalar predictionValue, Scalar actualValue) const |
double | getAccuracy () const |
double | getOmissionError () const |
double | getCommissionError () const |
double | getThreshold () const |
bool | ready () const |
ConfigurationPtr | getConfiguration () const |
Private Attributes | |
int | _confMatrix [2][2] |
Scalar | _predictionThreshold |
bool | _ignoreAbsences |
bool | _ready |
Class ConfusionMatrix tabulates the proportions of types of successes and errors generated by the model. The matrix is represented as following:
+---------------------------------+ | | Actual Values | + Predicted |---------------------+ | Values | Presence | Absence | +---------------------------------+ | Presence | a | b | +---------------------------------+ | Absence | c | d | +---------------------------------+
Values a and d are correct predictions, while b (false positives) and c (false negatives) are Type I and Type II errors.
Definition at line 61 of file ConfusionMatrix.hh.
ConfusionMatrix::ConfusionMatrix | ( | Scalar | predictionThreshold = CONF_MATRIX_DEFAULT_THRESHOLD , |
bool | ignoreAbsences = false |
||
) |
Default constructor.
predictionThreshold | Threshold for prediction values, above which a prediction value means a predicted presence. |
ignoreAbsences | Indicates if absence points should be ignored. |
Definition of ConfusionMatrix class
LICENSE INFORMATION
Copyright(c), The Center for Research, University of Kansas, 2385 Irving Hill Road, Lawrence, KS 66044-4755, USA.
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details:
http://www.gnu.org/copyleft/gpl.html
Definition at line 40 of file ConfusionMatrix.cpp.
References reset().
ConfusionMatrix::~ConfusionMatrix | ( | ) |
Destructor.
Definition at line 46 of file ConfusionMatrix.cpp.
void ConfusionMatrix::calculate | ( | const EnvironmentPtr & | env, |
const Model & | model, | ||
const OccurrencesPtr & | presences, | ||
const OccurrencesPtr & | absences = OccurrencesPtr() |
||
) |
Calculate confusion matrix based on model and sampled data from environment and occurrences objects.
env | Pointer to Environment object containing the environmental data to be used in calculation |
model | Model object to be evaluated |
presences | Pointer to an Occurrences object storing the presence points being evaluated |
absences | Pointer to an Occurrences object storing the absence points being evaluated |
Definition at line 111 of file ConfusionMatrix.cpp.
References _confMatrix, _ignoreAbsences, _predictionThreshold, _ready, Log::debug(), Log::instance(), reset(), Sample::size(), and Log::warn().
Referenced by calculate(), OpenModeller::calculateModelStatistics(), AlgorithmRun::createModel(), OpenModeller::getConfusionMatrix(), main(), and PreJackknife::runImplementation().
void ConfusionMatrix::calculate | ( | const Model & | model, |
const SamplerPtr & | sampler | ||
) |
Calculate confusion matrix based on an abstract Sampler object
model | Model object to be evaluated |
sampler | Pointer to a Sampler object that will provide data for evaluation |
Definition at line 223 of file ConfusionMatrix.cpp.
References calculate().
double ConfusionMatrix::getAccuracy | ( | ) | const |
Returns the accuracy of the model, which corresponds to the sum of successes (points predicted correctly) divided by all available points. I.e., accuracy = (a + d) / (a + b + c + d)
Definition at line 243 of file ConfusionMatrix.cpp.
References _confMatrix, and _ready.
Referenced by getConfiguration(), main(), and PreJackknife::runImplementation().
double ConfusionMatrix::getCommissionError | ( | ) | const |
Returns the commission error of the model, which corresponds to the sum of absence points predicted present divided by all available absence points. I.e., omission = b / (b + d)
Definition at line 256 of file ConfusionMatrix.cpp.
References _confMatrix, and _ready.
Referenced by AlgorithmRun::createModel(), getConfiguration(), and main().
ConfigurationPtr ConfusionMatrix::getConfiguration | ( | ) | const |
Serialize the confusion matrix
Definition at line 276 of file ConfusionMatrix.cpp.
References getAccuracy(), getCommissionError(), getOmissionError(), getThreshold(), and getValue().
Referenced by OpenModeller::getModelConfiguration(), and main().
double ConfusionMatrix::getOmissionError | ( | ) | const |
Returns the omission error of the model, which corresponds to the sum of presence points predicted absent divided by all available presence points. I.e., omission = c / (a + c)
Definition at line 266 of file ConfusionMatrix.cpp.
References _confMatrix, and _ready.
Referenced by AlgorithmRun::createModel(), getConfiguration(), and main().
|
inline |
Returns the probability threshold for distinguishing presence/absence.
Definition at line 149 of file ConfusionMatrix.hh.
Referenced by OpenModeller::calculateModelStatistics(), getConfiguration(), and main().
Returns a value from the confusion matrix.
predictionValue | Prediction value (probability returned by the model) |
actualValue | Actual value (0 for real absence, 1 for real presence) |
Definition at line 231 of file ConfusionMatrix.cpp.
References _confMatrix, and _predictionThreshold.
Referenced by getConfiguration(), and main().
|
inline |
Check whether the confusion matrix has been calculated already
Definition at line 154 of file ConfusionMatrix.hh.
Referenced by OpenModeller::getConfusionMatrix(), OpenModeller::getModelConfiguration(), and main().
void ConfusionMatrix::reset | ( | Scalar | predictionThreshold = CONF_MATRIX_DEFAULT_THRESHOLD , |
bool | ignoreAbsences = false |
||
) |
Resets area counters.
predictionThreshold | New threshold for prediction values, above which a prediction value means a predicted presence. |
ignoreAbsences | Indicates if absence points should be ignored. |
Definition at line 50 of file ConfusionMatrix.cpp.
References _confMatrix, _ignoreAbsences, _predictionThreshold, and _ready.
Referenced by calculate(), OpenModeller::calculateModelStatistics(), ConfusionMatrix(), OpenModeller::createModel(), main(), PreJackknife::runImplementation(), and OpenModeller::setModelConfiguration().
void ConfusionMatrix::setLowestTrainingThreshold | ( | const Model & | model, |
const SamplerPtr & | sampler | ||
) |
Set the threshold to the lowest probability value of all training points. Note: it is assumed that both parameters are already normalized, if necessary.
model | Model object to be evaluated |
sampler | Pointer to a Sampler object with training data |
Definition at line 58 of file ConfusionMatrix.cpp.
References _predictionThreshold, CONF_MATRIX_DEFAULT_THRESHOLD, Log::debug(), Log::instance(), Sample::size(), and Log::warn().
Referenced by OpenModeller::calculateModelStatistics(), and main().
|
private |
Definition at line 174 of file ConfusionMatrix.hh.
Referenced by calculate(), getAccuracy(), getCommissionError(), getOmissionError(), getValue(), and reset().
|
private |
Definition at line 178 of file ConfusionMatrix.hh.
Referenced by calculate(), and reset().
|
private |
Definition at line 176 of file ConfusionMatrix.hh.
Referenced by calculate(), getValue(), reset(), and setLowestTrainingThreshold().
|
private |
Definition at line 180 of file ConfusionMatrix.hh.
Referenced by calculate(), getAccuracy(), getCommissionError(), getOmissionError(), and reset().