|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use TextTokenizer | |
---|---|
de.fu_berlin.ties.classify.feature | This package contains classes for working with features, feature vectors and feature transformers. |
de.fu_berlin.ties.context | This packages provides functionality for building and managing representations of context in texts (XML documents). |
de.fu_berlin.ties.extract | This package handles information extraction and entity recognition. |
de.fu_berlin.ties.text | This package contains utility classes for working with texts. |
de.fu_berlin.ties.xml | This package contains utility classes for working with XML documents and related data. |
de.fu_berlin.ties.xml.dom | This package contains utility classes for working with DOM-like XML representations, focussing especially on dom4j. |
Uses of TextTokenizer in de.fu_berlin.ties.classify.feature |
---|
Methods in de.fu_berlin.ties.classify.feature with parameters of type TextTokenizer | |
---|---|
void |
FeatureVector.addAllTokens(CharSequence text,
TextTokenizer tokenizer)
Tokenizes a text, creating and adding a feature for each token. |
Uses of TextTokenizer in de.fu_berlin.ties.context |
---|
Constructors in de.fu_berlin.ties.context with parameters of type TextTokenizer | |
---|---|
SimpleRepresentation(int recogNum,
int splitMax,
int n,
String outCharset,
TextTokenizer textTokenizer)
Creates a new instance. |
Uses of TextTokenizer in de.fu_berlin.ties.extract |
---|
Constructors in de.fu_berlin.ties.extract with parameters of type TextTokenizer | |
---|---|
ExtractionLocator(ExtractionContainer extractions,
TextTokenizer textTokenizer)
Creates a new instance, setting ExtractionLocator.isRetrySilently() to
false . |
|
ExtractionLocator(ExtractionContainer extractions,
TextTokenizer textTokenizer,
boolean doRetrySilently)
Creates a new instance. |
Uses of TextTokenizer in de.fu_berlin.ties.text |
---|
Methods in de.fu_berlin.ties.text that return TextTokenizer | |
---|---|
static TextTokenizer |
TokenizerFactory.createAlnumTokenizer(CharSequence text)
Static factory method to create an instance for tokenizing alphanumeric and symbol sequences and puntuation. |
static TextTokenizer |
TokenizerFactory.createCategoryTokenizer(CharSequence text)
Static factory method to create an instance for tokenizing according to Unicode categories. |
static TextTokenizer |
TokenizerFactory.createThoroughTokenizer(CharSequence text)
Static factory method to create an instance that uses the "thorough" patterns listed below. |
TextTokenizer |
TokenizerFactory.createTokenizer(CharSequence text)
Factory method to create an instance from the configured token and whitespace patterns. |
TextTokenizer |
TokenizingExtractor.getTokenizer()
Returns the tokenizer used by this instance. |
Uses of TextTokenizer in de.fu_berlin.ties.xml |
---|
Methods in de.fu_berlin.ties.xml that return TextTokenizer | |
---|---|
static TextTokenizer |
XMLTokenizerFactory.createXMLTokenizer(CharSequence text,
boolean ensureWhitespace)
Factory method to create an instance for parsing files in XML syntax. |
Uses of TextTokenizer in de.fu_berlin.ties.xml.dom |
---|
Methods in de.fu_berlin.ties.xml.dom with parameters of type TextTokenizer | |
---|---|
protected void |
TokenWalker.processCollectedText(Element element,
CharSequence collectedText,
TokenCounter tokenCounter,
TextTokenizer tokenizer,
ContextMap context)
Helper method that tokenizes the collected textual contents of an element and delegates to the token processor for each of them. |
protected void |
TokenWalker.walk(Element element,
TokenCounter tokenCounter,
TextTokenizer tokenizer,
ContextMap context)
Walks through the contents of a node, tokenizing textual contents and recursing through nested elements. |
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |