Field Detail

• DEFAULT_MIN_WORD_SIZE

  public static final int DEFAULT_MIN_WORD_SIZE

  The default for minimal word length that gets decomposed

  See Also:

  Constant Field Values

• DEFAULT_MIN_SUBWORD_SIZE

  public static final int DEFAULT_MIN_SUBWORD_SIZE

  The default for minimal length of subwords that get propagated to the output of this filter

  See Also:

  Constant Field Values

• DEFAULT_MAX_SUBWORD_SIZE

  public static final int DEFAULT_MAX_SUBWORD_SIZE

  The default for maximal length of subwords that get propagated to the output of this filter

  See Also:

  Constant Field Values

• dictionary

  protected final org.apache.lucene.analysis.CharArraySet dictionary

• tokens

  protected final java.util.LinkedList tokens

• minWordSize

  protected final int minWordSize

• minSubwordSize

  protected final int minSubwordSize

• maxSubwordSize

  protected final int maxSubwordSize

• onlyLongestMatch

  protected final boolean onlyLongestMatch

Constructor Detail

• CompoundWordTokenFilterBase

  protected CompoundWordTokenFilterBase(org.apache.lucene.analysis.TokenStream input,
                                        java.lang.String[] dictionary,
                                        int minWordSize,
                                        int minSubwordSize,
                                        int maxSubwordSize,
                                        boolean onlyLongestMatch)

• CompoundWordTokenFilterBase

  protected CompoundWordTokenFilterBase(org.apache.lucene.analysis.TokenStream input,
                                        java.lang.String[] dictionary,
                                        boolean onlyLongestMatch)

• CompoundWordTokenFilterBase

  protected CompoundWordTokenFilterBase(org.apache.lucene.analysis.TokenStream input,
                                        java.util.Set dictionary,
                                        boolean onlyLongestMatch)

• CompoundWordTokenFilterBase

  protected CompoundWordTokenFilterBase(org.apache.lucene.analysis.TokenStream input,
                                        java.lang.String[] dictionary)

• CompoundWordTokenFilterBase

  protected CompoundWordTokenFilterBase(org.apache.lucene.analysis.TokenStream input,
                                        java.util.Set dictionary)

• CompoundWordTokenFilterBase

  protected CompoundWordTokenFilterBase(org.apache.lucene.analysis.TokenStream input,
                                        java.util.Set dictionary,
                                        int minWordSize,
                                        int minSubwordSize,
                                        int maxSubwordSize,
                                        boolean onlyLongestMatch)

Method Detail

• makeDictionary

  public static final java.util.Set makeDictionary(java.lang.String[] dictionary)

  Create a set of words from an array The resulting Set does case insensitive matching TODO We should look for a faster dictionary lookup approach.

  Parameters:

  dictionary -

  Returns:

  Set of lowercased terms

• incrementToken

  public final boolean incrementToken()
                               throws java.io.IOException

  Overrides:

  incrementToken in class org.apache.lucene.analysis.TokenStream

  Throws:

  java.io.IOException

• next

  public final org.apache.lucene.analysis.Token next(org.apache.lucene.analysis.Token reusableToken)
                                              throws java.io.IOException

  Deprecated. Will be removed in Lucene 3.0. This method is final, as it should not be overridden. Delegates to the backwards compatibility layer.

  Overrides:

  next in class org.apache.lucene.analysis.TokenStream

  Throws:

  java.io.IOException

• next

  public final org.apache.lucene.analysis.Token next()
                                              throws java.io.IOException

  Deprecated. Will be removed in Lucene 3.0. This method is final, as it should not be overridden. Delegates to the backwards compatibility layer.

  Overrides:

  next in class org.apache.lucene.analysis.TokenStream

  Throws:

  java.io.IOException

• addAllLowerCase

  protected static final void addAllLowerCase(java.util.Set target,
                                              java.util.Collection col)

• makeLowerCaseCopy

  protected static char[] makeLowerCaseCopy(char[] buffer)

• createToken

  protected final org.apache.lucene.analysis.Token createToken(int offset,
                                                               int length,
                                                               org.apache.lucene.analysis.Token prototype)

• decompose

  protected void decompose(org.apache.lucene.analysis.Token token)

• decomposeInternal

  protected abstract void decomposeInternal(org.apache.lucene.analysis.Token token)

• reset

  public void reset()
             throws java.io.IOException

  Overrides:

  reset in class org.apache.lucene.analysis.TokenFilter

  Throws:

  java.io.IOException