public abstract class Tokenizer extends java.lang.Object implements java.util.Enumeration<java.lang.String>, OptionHandler, java.io.Serializable, RevisionHandler
| Constructor and Description |
|---|
Tokenizer() |
| Modifier and Type | Method and Description |
|---|---|
java.lang.String[] |
getOptions()
Gets the current option settings for the OptionHandler.
|
abstract java.lang.String |
globalInfo()
Returns a string describing the stemmer
|
abstract boolean |
hasMoreElements()
Tests if this enumeration contains more elements.
|
java.util.Enumeration<Option> |
listOptions()
Returns an enumeration of all the available options..
|
abstract java.lang.String |
nextElement()
Returns the next element of this enumeration if this enumeration object has
at least one more element to provide.
|
static void |
runTokenizer(Tokenizer tokenizer,
java.lang.String[] options)
initializes the given tokenizer with the given options and runs the
tokenizer over all the remaining strings in the options array.
|
void |
setOptions(java.lang.String[] options)
Sets the OptionHandler's options using the given list.
|
abstract void |
tokenize(java.lang.String s)
Sets the string to tokenize.
|
static java.lang.String[] |
tokenize(Tokenizer tokenizer,
java.lang.String[] options)
initializes the given tokenizer with the given options and runs the
tokenizer over all the remaining strings in the options array.
|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitgetRevisionpublic abstract java.lang.String globalInfo()
public java.util.Enumeration<Option> listOptions()
listOptions in interface OptionHandlerpublic java.lang.String[] getOptions()
getOptions in interface OptionHandlerpublic void setOptions(java.lang.String[] options)
throws java.lang.Exception
setOptions in interface OptionHandleroptions - the list of options as an array of stringsjava.lang.Exception - if an option is not supportedpublic abstract boolean hasMoreElements()
hasMoreElements in interface java.util.Enumeration<java.lang.String>public abstract java.lang.String nextElement()
nextElement in interface java.util.Enumeration<java.lang.String>public abstract void tokenize(java.lang.String s)
s - the string to tokenizepublic static java.lang.String[] tokenize(Tokenizer tokenizer, java.lang.String[] options) throws java.lang.Exception
tokenizer - the tokenizer to useoptions - the options for the tokenizerjava.lang.Exception - if setting of options or tokenization failspublic static void runTokenizer(Tokenizer tokenizer, java.lang.String[] options)
tokenizer - the tokenizer to useoptions - the options for the tokenizer