Uses of Class
de.fu_berlin.ties.text.TextTokenizer

Packages that use TextTokenizer
de.fu_berlin.ties.classify.feature This package contains classes for working with features, feature vectors and feature transformers. 
de.fu_berlin.ties.context This packages provides functionality for building and managing representations of context in texts (XML documents). 
de.fu_berlin.ties.extract This package handles information extraction and entitiy recognition. 
de.fu_berlin.ties.text This package contains utility classes for working with texts. 
de.fu_berlin.ties.xml This package contains utility classes for working with XML documents and related data. 
de.fu_berlin.ties.xml.dom This package contains utility classes for working with DOM-like XML representations, focussing especially on dom4j
 

Uses of TextTokenizer in de.fu_berlin.ties.classify.feature
 

Methods in de.fu_berlin.ties.classify.feature with parameters of type TextTokenizer
 void FeatureVector.addAllTokens(CharSequence text, TextTokenizer tokenizer)
          Tokenizes a text, creating and adding a feature for each token.
 

Uses of TextTokenizer in de.fu_berlin.ties.context
 

Constructors in de.fu_berlin.ties.context with parameters of type TextTokenizer
SimpleRepresentation(int recogNum, int splitMax, int n, String outCharset, TextTokenizer textTokenizer)
          Creates a new instance.
 

Uses of TextTokenizer in de.fu_berlin.ties.extract
 

Constructors in de.fu_berlin.ties.extract with parameters of type TextTokenizer
ExtractionLocator(Document document, ExtractionContainer extractions, TextTokenizer textTokenizer)
          Creates a new instance.
 

Uses of TextTokenizer in de.fu_berlin.ties.text
 

Methods in de.fu_berlin.ties.text that return TextTokenizer
static TextTokenizer TokenizerFactory.createAlnumTokenizer(CharSequence text)
          Static factory method to create an instance for tokenizing alphanumeric and symbol sequences and puntuation.
static TextTokenizer TokenizerFactory.createCategoryTokenizer(CharSequence text)
          Static factory method to create an instance for tokenizing according to Unicode categories.
static TextTokenizer TokenizerFactory.createThoroughTokenizer(CharSequence text)
          Static factory method to create an instance that uses the "thorough" patterns listed below.
 TextTokenizer TokenizerFactory.createTokenizer(CharSequence text)
          Factory method to create an instance from the configured token and whitespace patterns.
 

Uses of TextTokenizer in de.fu_berlin.ties.xml
 

Methods in de.fu_berlin.ties.xml that return TextTokenizer
static TextTokenizer XMLTokenizerFactory.createXMLTokenizer(CharSequence text, boolean ensureWhitespace)
          Factory method to create an instance for parsing files in XML syntax.
 

Uses of TextTokenizer in de.fu_berlin.ties.xml.dom
 

Methods in de.fu_berlin.ties.xml.dom with parameters of type TextTokenizer
protected  void TokenWalker.processCollectedText(Element element, CharSequence collectedText, TokenCounter tokenCounter, TextTokenizer tokenizer, ContextMap context)
          Helper method that tokenizes the collected textual contents of an element and delegates to the token processor for each of them.
protected  void TokenWalker.walk(Element element, TokenCounter tokenCounter, TextTokenizer tokenizer, ContextMap context)
          Walks through the contents of a node, tokenizing textual contents and recursing through nested elements.
 



Copyright © 2003-2004 Christian Siefkes. All Rights Reserved.