NGramTokenizer (Lucene 2.9.4 API)

java.lang.Object
- org.apache.lucene.util.AttributeSource
- - org.apache.lucene.analysis.TokenStream
  - - org.apache.lucene.analysis.Tokenizer
    - - org.apache.lucene.analysis.ngram.NGramTokenizer

public class NGramTokenizer
extends org.apache.lucene.analysis.Tokenizer

Tokenizes the input into n-grams of the given size(s).

Nested Class Summary
- Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
  org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State

Field Summary

Fields
Modifier and Type Field and Description

static int DEFAULT_MAX_NGRAM_SIZE

static int DEFAULT_MIN_NGRAM_SIZE
- Fields inherited from class org.apache.lucene.analysis.Tokenizer
  input

Fields
Modifier and Type	Field and Description
`static int`	`DEFAULT_MAX_NGRAM_SIZE`
`static int`	`DEFAULT_MIN_NGRAM_SIZE`

Constructor Summary

Constructors
Constructor and Description
`NGramTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory, java.io.Reader input, int minGram, int maxGram)` Creates NGramTokenizer with given min and max n-grams.
`NGramTokenizer(org.apache.lucene.util.AttributeSource source, java.io.Reader input, int minGram, int maxGram)` Creates NGramTokenizer with given min and max n-grams.
`NGramTokenizer(java.io.Reader input)` Creates NGramTokenizer with default min and max n-grams.
`NGramTokenizer(java.io.Reader input, int minGram, int maxGram)` Creates NGramTokenizer with given min and max n-grams.

Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`void`	`end()`
`boolean`	`incrementToken()` Returns the next token in the stream, or null at EOS.
`org.apache.lucene.analysis.Token`	`next()` Deprecated. Will be removed in Lucene 3.0. This method is final, as it should not be overridden. Delegates to the backwards compatibility layer.
`org.apache.lucene.analysis.Token`	`next(org.apache.lucene.analysis.Token reusableToken)` Deprecated. Will be removed in Lucene 3.0. This method is final, as it should not be overridden. Delegates to the backwards compatibility layer.
`void`	`reset()`
`void`	`reset(java.io.Reader input)`

Methods inherited from class org.apache.lucene.analysis.Tokenizer
close, correctOffset

Methods inherited from class org.apache.lucene.analysis.TokenStream
getOnlyUseNewAPI, setOnlyUseNewAPI

Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

- Field Detail
  - DEFAULT_MIN_NGRAM_SIZE
```
public static final int DEFAULT_MIN_NGRAM_SIZE
```
    See Also:
    
    Constant Field Values
  - DEFAULT_MAX_NGRAM_SIZE
```
public static final int DEFAULT_MAX_NGRAM_SIZE
```
    See Also:
    
    Constant Field Values
- Constructor Detail
  - NGramTokenizer
```
public NGramTokenizer(java.io.Reader input,
                      int minGram,
                      int maxGram)
```
    Creates NGramTokenizer with given min and max n-grams.
    
    Parameters:
    
    input - Reader holding the input to be tokenized
    
    minGram - the smallest n-gram to generate
    
    maxGram - the largest n-gram to generate
  - NGramTokenizer
```
public NGramTokenizer(org.apache.lucene.util.AttributeSource source,
                      java.io.Reader input,
                      int minGram,
                      int maxGram)
```
    Creates NGramTokenizer with given min and max n-grams.
    
    Parameters:
    
    source - AttributeSource to use
    
    input - Reader holding the input to be tokenized
    
    minGram - the smallest n-gram to generate
    
    maxGram - the largest n-gram to generate
  - NGramTokenizer
```
public NGramTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory,
                      java.io.Reader input,
                      int minGram,
                      int maxGram)
```
    Creates NGramTokenizer with given min and max n-grams.
    
    Parameters:
    
    factory - AttributeSource.AttributeFactory to use
    
    input - Reader holding the input to be tokenized
    
    minGram - the smallest n-gram to generate
    
    maxGram - the largest n-gram to generate
  - NGramTokenizer
```
public NGramTokenizer(java.io.Reader input)
```
    Creates NGramTokenizer with default min and max n-grams.
    
    Parameters:
    
    input - Reader holding the input to be tokenized
- Method Detail
  - incrementToken
```
public final boolean incrementToken()
                             throws java.io.IOException
```
    Returns the next token in the stream, or null at EOS.
    
    Overrides:
    
    incrementToken in class org.apache.lucene.analysis.TokenStream
    
    Throws:
    
    java.io.IOException
  - end
```
public final void end()
```
    Overrides:
    
    end in class org.apache.lucene.analysis.TokenStream
  - next
```
public final org.apache.lucene.analysis.Token next(org.apache.lucene.analysis.Token reusableToken)
                                            throws java.io.IOException
```
    Deprecated. Will be removed in Lucene 3.0. This method is final, as it should not be overridden. Delegates to the backwards compatibility layer.
    
    Overrides:
    
    next in class org.apache.lucene.analysis.TokenStream
    
    Throws:
    
    java.io.IOException
  - next
```
public final org.apache.lucene.analysis.Token next()
                                            throws java.io.IOException
```
    Deprecated. Will be removed in Lucene 3.0. This method is final, as it should not be overridden. Delegates to the backwards compatibility layer.
    
    Overrides:
    
    next in class org.apache.lucene.analysis.TokenStream
    
    Throws:
    
    java.io.IOException
  - reset
```
public void reset(java.io.Reader input)
           throws java.io.IOException
```
    Overrides:
    
    reset in class org.apache.lucene.analysis.Tokenizer
    
    Throws:
    
    java.io.IOException
  - reset
```
public void reset()
           throws java.io.IOException
```
    Overrides:
    
    reset in class org.apache.lucene.analysis.TokenStream
    
    Throws:
    
    java.io.IOException

Class NGramTokenizer

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

Field Summary

Fields inherited from class org.apache.lucene.analysis.Tokenizer

Constructor Summary

Method Summary

Methods inherited from class org.apache.lucene.analysis.Tokenizer

Methods inherited from class org.apache.lucene.analysis.TokenStream

Methods inherited from class org.apache.lucene.util.AttributeSource

Methods inherited from class java.lang.Object

Field Detail

DEFAULT_MIN_NGRAM_SIZE

DEFAULT_MAX_NGRAM_SIZE

Constructor Detail

NGramTokenizer

NGramTokenizer

NGramTokenizer

NGramTokenizer

Method Detail

incrementToken

end

next

next

reset

reset