|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectde.fu_berlin.ties.extract.ExtractionContainer
de.fu_berlin.ties.extract.EvaluatedExtractionContainer
public class EvaluatedExtractionContainer
An extraction container that evaluates containers of predicted extractions
against containers of true extractions (answer keys) and merges their
contents, setting the evaluation states accordingly. Extractions in both containers are marked as
EvalStatus.CORRECT
(true positives),
extractions found only in the container of predictions are marked as
EvalStatus.SPURIOUS
(false positives), while
those occurring only in the answer keys are
EvalStatus.MISSING
(false negatives).
Two match modes are supported:
in match-all mode, all extractions are evaluated.
In match-best mode, only the recognition with the highest
probability of each type is evaluated (in each batch), while the others are
ignored and not
counted; it is thus suffienct to match (or fail to match) a single answer key
-- the other answer keys are marked as
EvalStatus.ALTERNATIVE
s and not counted.
Field Summary | |
---|---|
static String |
CONFIG_MATCH_ALL
Configuration key: whether to use match-all or match-best as match mode. |
static String |
CONFIG_MATCH_POSITION
Configuration key for isMatchingPosition() . |
Constructor Summary | |
---|---|
EvaluatedExtractionContainer(TargetStructure targetStruct,
boolean matchAll,
boolean matchPosition)
Creates a new instance. |
|
EvaluatedExtractionContainer(TargetStructure targetStruct,
TiesConfiguration config)
Creates a new instance. |
Method Summary | |
---|---|
void |
add(Extraction extraction)
Adds an extraction to this container. |
protected void |
addAllAndClear(Collection extractionColl,
EvalStatus status,
String source)
Helper method that adds a bunch of extractions, setting their evaluation status and source as specified. |
void |
evaluateBatch(ExtractionContainer predicted,
ExtractionContainer expected,
String source)
Evaluates a container of predicted extractions against a container of true extractions (answer keys) and adds them to this instance. |
boolean |
isMatchingAll()
Returns the match mode. |
boolean |
isMatchingPosition()
If true , the positions of extraction and answer keys must
match; otherwise only their contents must match (string compare). |
String |
toString()
Returns a string representation of this object. |
MultiFMetricsView |
viewMetrics()
Returns a read-only view of the counts of true positives (correct extractions), false negatives (missing extractions) and false positives (spurious extractions) stored in this container of each type as well as for all types; and the metrics that can be calculated from these values. |
Methods inherited from class de.fu_berlin.ties.extract.ExtractionContainer |
---|
getTargetStructure, iterator, iterator, last, lastN, lastN, remove, removeLast, restoreEntries, storeEntries, unsetPositions |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Field Detail |
---|
public static final String CONFIG_MATCH_ALL
public static final String CONFIG_MATCH_POSITION
isMatchingPosition()
.
Constructor Detail |
---|
public EvaluatedExtractionContainer(TargetStructure targetStruct, TiesConfiguration config)
targetStruct
- the target structure specifying the classes to
recognizeconfig
- used to configure this instancepublic EvaluatedExtractionContainer(TargetStructure targetStruct, boolean matchAll, boolean matchPosition)
targetStruct
- the target structure specifying the classes to
recognizematchAll
- sets the match mode:
true
for match-all mode, false
for
match-best modematchPosition
- if true
, the positions of extraction
and answer keys must match; otherwise only their contents must match
(string compare)Method Detail |
---|
public void add(Extraction extraction) throws IllegalArgumentException
EvalStatus.UNKNOWN
and EvalStatus.TRUTH
are not allowed.
The evaluation states EvalStatus.IGNORED
and
EvalStatus.ALTERNATIVE
are only allowed in match-best
mode; in match-all mode
(isMatchingAll()
== true
) they don't make sense
and are thus rejected.
add
in class ExtractionContainer
extraction
- the extraction to add
IllegalArgumentException
- if the extraction's evaluation status
is invalid (cf. above); or if the class of the extraction is not
in the set of classes accepted by this containerprotected void addAllAndClear(Collection extractionColl, EvalStatus status, String source)
extractionColl
- a collection of Extraction
s to add; the
collection will be cleared by this
method, i.e., it will be empty after the method returnedstatus
- the status of all extractions in the collection will be
set to this value prior to adding themsource
- the
source to
set for all extractions in the collection; ignored if null
public void evaluateBatch(ExtractionContainer predicted, ExtractionContainer expected, String source) throws IllegalArgumentException
Warning: If positions are ignored for matching (see constructor), the two given containers are modified by unsetting the positions of all extractions and removing duplicates (extractions with same type + text but different positions).
predicted
- the container of predicted extractionsexpected
- the container of expected extractions (answer keys)source
- the
source
to add to add extractions of this batch; ignored if null
IllegalArgumentException
- if the
target structures
of the two containers differ from the target structure of this onepublic boolean isMatchingAll()
true
for match-all mode,
false
for match-best modepublic boolean isMatchingPosition()
true
, the positions of extraction and answer keys must
match; otherwise only their contents must match (string compare).
public String toString()
toString
in class ExtractionContainer
public MultiFMetricsView viewMetrics()
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |