org.apache.lucene.analysis
Class WordlistLoader

java.lang.Object
  org.apache.lucene.analysis.WordlistLoader

public class WordlistLoader
extends java.lang.Object

Loader for text files that represent a list of stopwords.

Version:

$Id: WordlistLoader.java 706342 2008-10-20 17:19:29Z gsingers $

Constructor Summary

WordlistLoader()

Method Summary

static java.util.HashMap	getStemDict(java.io.File wordstemfile) Reads a stem dictionary.
static java.util.HashSet	getWordSet(java.io.File wordfile) Loads a text file and adds every line as an entry to a HashSet (omitting leading and trailing whitespace).
static java.util.HashSet	getWordSet(java.io.File wordfile, java.lang.String comment) Loads a text file and adds every non-comment line as an entry to a HashSet (omitting leading and trailing whitespace).
static java.util.HashSet	getWordSet(java.io.Reader reader) Reads lines from a Reader and adds every line as an entry to a HashSet (omitting leading and trailing whitespace).
static java.util.HashSet	getWordSet(java.io.Reader reader, java.lang.String comment) Reads lines from a Reader and adds every non-comment line as an entry to a HashSet (omitting leading and trailing whitespace).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WordlistLoader

public WordlistLoader()

Method Detail

getWordSet

public static java.util.HashSet getWordSet(java.io.File wordfile)
                                    throws java.io.IOException

Loads a text file and adds every line as an entry to a HashSet (omitting leading and trailing whitespace). Every line of the file should contain only one word. The words need to be in lowercase if you make use of an Analyzer which uses LowerCaseFilter (like StandardAnalyzer).

Parameters:

wordfile - File containing the wordlist

Returns:

A HashSet with the file's words

Throws:

java.io.IOException

getWordSet

public static java.util.HashSet getWordSet(java.io.File wordfile,
                                           java.lang.String comment)
                                    throws java.io.IOException

Loads a text file and adds every non-comment line as an entry to a HashSet (omitting leading and trailing whitespace). Every line of the file should contain only one word. The words need to be in lowercase if you make use of an Analyzer which uses LowerCaseFilter (like StandardAnalyzer).

Parameters:

wordfile - File containing the wordlist

comment - The comment string to ignore

Returns:

A HashSet with the file's words

Throws:

java.io.IOException

getWordSet

public static java.util.HashSet getWordSet(java.io.Reader reader)
                                    throws java.io.IOException

Reads lines from a Reader and adds every line as an entry to a HashSet (omitting leading and trailing whitespace). Every line of the Reader should contain only one word. The words need to be in lowercase if you make use of an Analyzer which uses LowerCaseFilter (like StandardAnalyzer).

Parameters:

reader - Reader containing the wordlist

Returns:

A HashSet with the reader's words

Throws:

java.io.IOException

getWordSet

public static java.util.HashSet getWordSet(java.io.Reader reader,
                                           java.lang.String comment)
                                    throws java.io.IOException

Reads lines from a Reader and adds every non-comment line as an entry to a HashSet (omitting leading and trailing whitespace). Every line of the Reader should contain only one word. The words need to be in lowercase if you make use of an Analyzer which uses LowerCaseFilter (like StandardAnalyzer).

Parameters:

reader - Reader containing the wordlist

comment - The string representing a comment.

Returns:

A HashSet with the reader's words

Throws:

java.io.IOException

getStemDict

public static java.util.HashMap getStemDict(java.io.File wordstemfile)
                                     throws java.io.IOException

Reads a stem dictionary. Each line contains:

word\tstem

(i.e. two tab seperated words)

Returns:

stem dictionary that overrules the stemming algorithm

Throws:

java.io.IOException