WhitespaceTokenizer (Lucene 2.9.4 API)

Skip navigation links

All Classes

Summary:
Nested |
Field |
Constr |
Method

Detail:
Field |
Constr |
Method

java.lang.Object
- org.apache.lucene.util.AttributeSource
- - org.apache.lucene.analysis.TokenStream
  - - org.apache.lucene.analysis.Tokenizer
    - - org.apache.lucene.analysis.CharTokenizer
      - org.apache.lucene.analysis.WhitespaceTokenizer

```
public class WhitespaceTokenizer
extends CharTokenizer
```
A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

Nested Class Summary
- Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
  AttributeSource.AttributeFactory, AttributeSource.State

Field Summary
- Fields inherited from class org.apache.lucene.analysis.Tokenizer
  input

Constructor Summary

Constructors
Constructor and Description
`WhitespaceTokenizer(AttributeSource.AttributeFactory factory, java.io.Reader in)` Construct a new WhitespaceTokenizer using a given `AttributeSource.AttributeFactory`.
`WhitespaceTokenizer(AttributeSource source, java.io.Reader in)` Construct a new WhitespaceTokenizer using a given `AttributeSource`.
`WhitespaceTokenizer(java.io.Reader in)` Construct a new WhitespaceTokenizer.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected boolean`	`isTokenChar(char c)` Collects only characters which do not satisfy `Character.isWhitespace(char)`.

Methods inherited from class org.apache.lucene.analysis.CharTokenizer
end, incrementToken, next, next, normalize, reset

Methods inherited from class org.apache.lucene.analysis.Tokenizer
close, correctOffset

Methods inherited from class org.apache.lucene.analysis.TokenStream
getOnlyUseNewAPI, reset, setOnlyUseNewAPI

Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - WhitespaceTokenizer
```
public WhitespaceTokenizer(java.io.Reader in)
```
    Construct a new WhitespaceTokenizer.
  - WhitespaceTokenizer
```
public WhitespaceTokenizer(AttributeSource source,
                           java.io.Reader in)
```
    Construct a new WhitespaceTokenizer using a given AttributeSource.
  - WhitespaceTokenizer
```
public WhitespaceTokenizer(AttributeSource.AttributeFactory factory,
                           java.io.Reader in)
```
    Construct a new WhitespaceTokenizer using a given AttributeSource.AttributeFactory.
- Method Detail
  - isTokenChar
```
protected boolean isTokenChar(char c)
```
    Collects only characters which do not satisfy Character.isWhitespace(char).
    
    Specified by:
    
    isTokenChar in class CharTokenizer

Skip navigation links

All Classes

Summary:
Nested |
Field |
Constr |
Method

Detail:
Field |
Constr |
Method

Copyright © 2000-2016 Apache Software Foundation. All Rights Reserved.