de.fu_berlin.ties.eval
Class FMetrics

java.lang.Object
  extended byde.fu_berlin.ties.io.BaseStorable
      extended byde.fu_berlin.ties.eval.FMetrics
All Implemented Interfaces:
EvalInput, FMetricsView, Storable
Direct Known Subclasses:
SummaryFMetrics

public class FMetrics
extends BaseStorable
implements FMetricsView

This class manages and updates evaluation results, calculating precision (P), recall (R) and F-measure (F). It counts true positives (tp), false negatives (tn) and false positives (fp).

The metrics are defines as follows (cf. Manning and Schütze (1999): Foundations of Statistical Natural Language Processing, p. 268f):

Precision P =
tp / (tp + fp)
Recall R =
tp / (tp + fn)
F-measure F =
1 / (alpha * (1/P) + (1-alpha) * (1/R))
We define F to be 0 if P = 0 or R = 0 (not discussed by Manning and Schütze).
The factor alpha defines the weighting of precision and recall; P and R are weighted equal if alpha = 0.5 (the default value).

Instances of this class are not thread-safe and must be synchronized externally, if required.

Version:
$Revision: 1.3 $, $Date: 2004/02/19 18:15:51 $, $Author: siefkes $
Author:
Christian Siefkes

Field Summary
static String KEY_F1_MEASURE
          Serialization key for the F1-measure.
static String KEY_FALSE_NEG
          Serialization key for false negatives.
static String KEY_FALSE_POS
          Serialization key for false positives.
static String KEY_PRECISION
          Serialization key for the precision.
static String KEY_RECALL
          Serialization key for the recall.
static String KEY_TRUE_POS
          Serialization key for true positives.
 
Constructor Summary
FMetrics()
          Creates a new empty instance.
FMetrics(EvalInput input)
          Creates a new instance.
FMetrics(FieldMap fieldMap)
          Creates a new instance from a field map, fulfilling the Storable contract.
FMetrics(long initTruePos, long initFalseNeg, long initFalsePos)
          Creates a new instance.
 
Method Summary
 double getF1Measure()
          Returns the F-measure, setting alpha = 0.5 so P and R are weighted equal ("F1-measure").
 long getFalseNeg()
          Returns the number of false negatives (false rejections).
 long getFalsePos()
          Returns the number of false positives (false acceptances).
 double getFMeasure(double alpha)
          Returns the F-measure: F = 1 / (alpha * (1/P) + (1-alpha) * (1/R)).
 double getPrecision()
          Returns the precision: P = tp / (tp + fp).
 double getRecall()
          Returns the recall: R = tp / (tp + fn).
 long getTruePos()
          Returns the number of true positives (correct recognitions).
 void incFalseNeg()
          Increases the number of false negatives by 1.
 void incFalsePos()
          Increases the number of false positives by 1.
 void incTruePos()
          Increases the number of true positives by 1.
 FieldMap storeFields()
          Stores all relevant fields of this object in a field map for serialization.
 void update(EvalInput input)
          Updates the statistics, increasing the stored values as specified.
 void update(long addTruePos, long addFalseNeg, long addFalsePos)
          Updates the statistics, increasing the stored values as specified.
 
Methods inherited from class de.fu_berlin.ties.io.BaseStorable
toString, toString
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

KEY_TRUE_POS

public static final String KEY_TRUE_POS
Serialization key for true positives.

See Also:
Constant Field Values

KEY_FALSE_NEG

public static final String KEY_FALSE_NEG
Serialization key for false negatives.

See Also:
Constant Field Values

KEY_FALSE_POS

public static final String KEY_FALSE_POS
Serialization key for false positives.

See Also:
Constant Field Values

KEY_PRECISION

public static final String KEY_PRECISION
Serialization key for the precision.

See Also:
Constant Field Values

KEY_RECALL

public static final String KEY_RECALL
Serialization key for the recall.

See Also:
Constant Field Values

KEY_F1_MEASURE

public static final String KEY_F1_MEASURE
Serialization key for the F1-measure.

See Also:
Constant Field Values
Constructor Detail

FMetrics

public FMetrics()
Creates a new empty instance.


FMetrics

public FMetrics(EvalInput input)
         throws IllegalArgumentException
Creates a new instance.

Parameters:
input - contains the initial number of true and false positives and false negatives
Throws:
IllegalArgumentException - if at least one of the input value is negative

FMetrics

public FMetrics(FieldMap fieldMap)
         throws IllegalArgumentException
Creates a new instance from a field map, fulfilling the Storable contract.

Parameters:
fieldMap - map containing the serialized fields
Throws:
IllegalArgumentException - if at least one of the parameters is negative or missing

FMetrics

public FMetrics(long initTruePos,
                long initFalseNeg,
                long initFalsePos)
         throws IllegalArgumentException
Creates a new instance.

Parameters:
initTruePos - the initial number of true positives
initFalseNeg - the initial number of false negatives
initFalsePos - the initial number of false positives
Throws:
IllegalArgumentException - if at least one of the parameters is negative
Method Detail

getFalseNeg

public final long getFalseNeg()
Returns the number of false negatives (false rejections).

Specified by:
getFalseNeg in interface EvalInput
Returns:
the number of false negatives

getFalsePos

public final long getFalsePos()
Returns the number of false positives (false acceptances).

Specified by:
getFalsePos in interface EvalInput
Returns:
the number of false positives

getF1Measure

public final double getF1Measure()
Returns the F-measure, setting alpha = 0.5 so P and R are weighted equal ("F1-measure"). F1 = (2 * P * R) / (P + R).

Specified by:
getF1Measure in interface FMetricsView
Returns:
the F1-measure; a value in the range from 0.0 to 1.0

getFMeasure

public final double getFMeasure(double alpha)
                         throws IllegalArgumentException
Returns the F-measure: F = 1 / (alpha * (1/P) + (1-alpha) * (1/R)).
F is defined to be 0 if P = 0 or R = 0.

Specified by:
getFMeasure in interface FMetricsView
Parameters:
alpha - a factor in the range from 0.0 to 1.0 defining the weighting of precision and recall
Returns:
the F-measure; a value in the range from 0.0 to 1.0
Throws:
IllegalArgumentException - if alpha is smaller than 0.0 or larger than 1.0

getPrecision

public final double getPrecision()
Returns the precision: P = tp / (tp + fp).

Specified by:
getPrecision in interface FMetricsView
Returns:
the precision; a value in the range from 0.0 to 1.0

getRecall

public final double getRecall()
Returns the recall: R = tp / (tp + fn).

Specified by:
getRecall in interface FMetricsView
Returns:
the precision; a value in the range from 0.0 to 1.0

getTruePos

public final long getTruePos()
Returns the number of true positives (correct recognitions).

Specified by:
getTruePos in interface EvalInput
Returns:
the number of true positives

incFalseNeg

public final void incFalseNeg()
Increases the number of false negatives by 1.


incFalsePos

public final void incFalsePos()
Increases the number of false positives by 1.


incTruePos

public final void incTruePos()
Increases the number of true positives by 1.


storeFields

public FieldMap storeFields()
Stores all relevant fields of this object in a field map for serialization. An equivalent object can be created by calling FieldMap.createObject(Class) on the created field map. The calculated values precision, recall, and F-measure are also stored (they are ignored when deserializing a stored instance).

Specified by:
storeFields in interface Storable
Returns:
the created field map

update

public final void update(EvalInput input)
                  throws IllegalArgumentException
Updates the statistics, increasing the stored values as specified.

Parameters:
input - contains the number of true and false positives and false negatives to add
Throws:
IllegalArgumentException - if at least one of the input values is negative

update

public void update(long addTruePos,
                   long addFalseNeg,
                   long addFalsePos)
            throws IllegalArgumentException
Updates the statistics, increasing the stored values as specified.

Parameters:
addTruePos - the number of new true positives to add
addFalseNeg - the number of new false negatives to add
addFalsePos - the number of new false positives to add
Throws:
IllegalArgumentException - if at least one of the parameters is negative


Copyright © 2003-2004 Christian Siefkes. All Rights Reserved.