Package org.carrot2.text.analysis

Lexical analysis utilities.

See: Description

Interface Summary

Interface	Description
ITokenizer	Splits input characters into tokens representing e.g.

Class Summary

Class	Description
ExtendedWhitespaceTokenizer	A tokenizer separating input characters on whitespace, but capable of extracting more complex tokens, such as URLs, e-mail addresses and sentence delimiters.
ExtendedWhitespaceTokenizerImpl
TokenTypeUtils	Utility methods for working with ITokenizer attributes.

Package org.carrot2.text.analysis Description

Lexical analysis utilities.