#include <ConfusionMatrix.hh>

Collaboration diagram for ConfusionMatrix:

Public Member Functions
	ConfusionMatrix (Scalar predictionThreshold=CONF_MATRIX_DEFAULT_THRESHOLD, bool ignoreAbsences=false)

	~ConfusionMatrix ()

void	reset (Scalar predictionThreshold=CONF_MATRIX_DEFAULT_THRESHOLD, bool ignoreAbsences=false)

void	setLowestTrainingThreshold (const Model &model, const SamplerPtr &sampler)

void	calculate (const EnvironmentPtr &env, const Model &model, const OccurrencesPtr &presences, const OccurrencesPtr &absences=OccurrencesPtr())

void	calculate (const Model &model, const SamplerPtr &sampler)

int	getValue (Scalar predictionValue, Scalar actualValue) const

double	getAccuracy () const

double	getOmissionError () const

double	getCommissionError () const

double	getThreshold () const

bool	ready () const

ConfigurationPtr	getConfiguration () const

Private Attributes
int	_confMatrix [2][2]

Scalar	_predictionThreshold

bool	_ignoreAbsences

bool	_ready

Detailed Description

Class ConfusionMatrix tabulates the proportions of types of successes and errors generated by the model. The matrix is represented as following:

+---------------------------------+
|           |  Actual Values      |
+ Predicted |---------------------+
| Values    |  Presence | Absence |
+---------------------------------+
| Presence  |     a     |    b    |
+---------------------------------+
| Absence   |     c     |    d    |
+---------------------------------+

Values a and d are correct predictions, while b (false positives) and c (false negatives) are Type I and Type II errors.

Definition at line 61 of file ConfusionMatrix.hh.

Constructor & Destructor Documentation

ConfusionMatrix::ConfusionMatrix	(	Scalar	predictionThreshold = `CONF_MATRIX_DEFAULT_THRESHOLD`,
		bool	ignoreAbsences = `false`
	)

Default constructor.

Parameters

predictionThreshold	Threshold for prediction values, above which a prediction value means a predicted presence.
ignoreAbsences	Indicates if absence points should be ignored.

Definition of ConfusionMatrix class

Author: Ricardo Scachetti Pereira (rpere.nosp@m.ira@.nosp@m.ku.ed.nosp@m.u)

Date: 2004-10-18

Id:: ConfusionMatrix.cpp 5632 2012-11-13 17:32:37Z rdg

LICENSE INFORMATION

Copyright(c), The Center for Research, University of Kansas, 2385 Irving Hill Road, Lawrence, KS 66044-4755, USA.

http://www.nhm.ku.edu

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details:

http://www.gnu.org/copyleft/gpl.html

Definition at line 40 of file ConfusionMatrix.cpp.

References reset().

Here is the call graph for this function:

ConfusionMatrix::~ConfusionMatrix ( )

Destructor.

Definition at line 46 of file ConfusionMatrix.cpp.

Member Function Documentation

void ConfusionMatrix::calculate	(	const EnvironmentPtr &	env,
		const Model &	model,
		const OccurrencesPtr &	presences,
		const OccurrencesPtr &	absences = `OccurrencesPtr()`
	)

Calculate confusion matrix based on model and sampled data from environment and occurrences objects.

Parameters

env	Pointer to Environment object containing the environmental data to be used in calculation
model	Model object to be evaluated
presences	Pointer to an Occurrences object storing the presence points being evaluated
absences	Pointer to an Occurrences object storing the absence points being evaluated

Definition at line 111 of file ConfusionMatrix.cpp.

References _confMatrix, _ignoreAbsences, _predictionThreshold, _ready, Log::debug(), Log::instance(), reset(), Sample::size(), and Log::warn().

Referenced by calculate(), OpenModeller::calculateModelStatistics(), AlgorithmRun::createModel(), OpenModeller::getConfusionMatrix(), main(), and PreJackknife::runImplementation().

Here is the call graph for this function:

void ConfusionMatrix::calculate	(	const Model &	model,
		const SamplerPtr &	sampler
	)

Calculate confusion matrix based on an abstract Sampler object

Parameters

model	Model object to be evaluated
sampler	Pointer to a Sampler object that will provide data for evaluation

Definition at line 223 of file ConfusionMatrix.cpp.

References calculate().

Here is the call graph for this function:

double ConfusionMatrix::getAccuracy ( ) const

Returns the accuracy of the model, which corresponds to the sum of successes (points predicted correctly) divided by all available points. I.e., accuracy = (a + d) / (a + b + c + d)

Definition at line 243 of file ConfusionMatrix.cpp.

References _confMatrix, and _ready.

Referenced by getConfiguration(), main(), and PreJackknife::runImplementation().

double ConfusionMatrix::getCommissionError ( ) const

Returns the commission error of the model, which corresponds to the sum of absence points predicted present divided by all available absence points. I.e., omission = b / (b + d)

Definition at line 256 of file ConfusionMatrix.cpp.

References _confMatrix, and _ready.

Referenced by AlgorithmRun::createModel(), getConfiguration(), and main().

ConfigurationPtr ConfusionMatrix::getConfiguration ( ) const

Serialize the confusion matrix

Definition at line 276 of file ConfusionMatrix.cpp.

References getAccuracy(), getCommissionError(), getOmissionError(), getThreshold(), and getValue().

Referenced by OpenModeller::getModelConfiguration(), and main().

Here is the call graph for this function:

double ConfusionMatrix::getOmissionError ( ) const

Returns the omission error of the model, which corresponds to the sum of presence points predicted absent divided by all available presence points. I.e., omission = c / (a + c)

Definition at line 266 of file ConfusionMatrix.cpp.

References _confMatrix, and _ready.

Referenced by AlgorithmRun::createModel(), getConfiguration(), and main().

double ConfusionMatrix::getThreshold ( ) const

inline

Returns the probability threshold for distinguishing presence/absence.

Definition at line 149 of file ConfusionMatrix.hh.

Referenced by OpenModeller::calculateModelStatistics(), getConfiguration(), and main().

int ConfusionMatrix::getValue	(	Scalar	predictionValue,
		Scalar	actualValue
	)		const

Returns a value from the confusion matrix.

Parameters

predictionValue	Prediction value (probability returned by the model)
actualValue	Actual value (0 for real absence, 1 for real presence)

Definition at line 231 of file ConfusionMatrix.cpp.

References _confMatrix, and _predictionThreshold.

Referenced by getConfiguration(), and main().

bool ConfusionMatrix::ready ( ) const

inline

Check whether the confusion matrix has been calculated already

Definition at line 154 of file ConfusionMatrix.hh.

Referenced by OpenModeller::getConfusionMatrix(), OpenModeller::getModelConfiguration(), and main().

void ConfusionMatrix::reset	(	Scalar	predictionThreshold = `CONF_MATRIX_DEFAULT_THRESHOLD`,
		bool	ignoreAbsences = `false`
	)

Resets area counters.

Parameters

predictionThreshold	New threshold for prediction values, above which a prediction value means a predicted presence.
ignoreAbsences	Indicates if absence points should be ignored.

Definition at line 50 of file ConfusionMatrix.cpp.

References _confMatrix, _ignoreAbsences, _predictionThreshold, and _ready.

Referenced by calculate(), OpenModeller::calculateModelStatistics(), ConfusionMatrix(), OpenModeller::createModel(), main(), PreJackknife::runImplementation(), and OpenModeller::setModelConfiguration().

void ConfusionMatrix::setLowestTrainingThreshold	(	const Model &	model,
		const SamplerPtr &	sampler
	)

Set the threshold to the lowest probability value of all training points. Note: it is assumed that both parameters are already normalized, if necessary.

Parameters