Class AnalyzingSuggester
- java.lang.Object
-
- org.apache.lucene.search.suggest.Lookup
-
- org.apache.lucene.search.suggest.analyzing.AnalyzingSuggester
-
- All Implemented Interfaces:
Accountable
- Direct Known Subclasses:
FuzzySuggester
public class AnalyzingSuggester extends Lookup
Suggester that first analyzes the surface form, adds the analyzed form to a weighted FST, and then does the same thing at lookup time. This means lookup is based on the analyzed form while suggestions are still the surface form(s).This can result in powerful suggester functionality. For example, if you use an analyzer removing stop words, then the partial text "ghost chr..." could see the suggestion "The Ghost of Christmas Past". Note that position increments MUST NOT be preserved for this example to work, so you should call the constructor with
preservePositionIncrementsparameter set to falseIf SynonymFilter is used to map wifi and wireless network to hotspot then the partial text "wirele..." could suggest "wifi router". Token normalization like stemmers, accent removal, etc., would allow suggestions to ignore such variations.
When two matching suggestions have the same weight, they are tie-broken by the analyzed form. If their analyzed form is the same then the order is undefined.
There are some limitations:
- A lookup from a query like "net" in English won't be any different than "net " (ie, user added a trailing space) because analyzers don't reflect when they've seen a token separator and when they haven't.
- If you're using
StopFilter, and the user will type "fast apple", but so far all they've typed is "fast a", again because the analyzer doesn't convey whether it's seen a token separator after the "a",StopFilterwill remove that "a" causing far more matches than you'd expect. - Lookups with the empty string return no results instead of all results.
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.search.suggest.Lookup
Lookup.LookupPriorityQueue, Lookup.LookupResult
-
-
Field Summary
Fields Modifier and Type Field Description static intEXACT_FIRSTInclude this flag in the options parameter toAnalyzingSuggester(Directory,String,Analyzer,Analyzer,int,int,int,boolean)to always return the exact match first, regardless of score.static intPRESERVE_SEPInclude this flag in the options parameter toAnalyzingSuggester(Directory,String,Analyzer,Analyzer,int,int,int,boolean)to preserve token separators when matching.-
Fields inherited from class org.apache.lucene.search.suggest.Lookup
CHARSEQUENCE_COMPARATOR
-
Fields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLE
-
-
Constructor Summary
Constructors Constructor Description AnalyzingSuggester(Directory tempDir, String tempFileNamePrefix, Analyzer analyzer)AnalyzingSuggester(Directory tempDir, String tempFileNamePrefix, Analyzer indexAnalyzer, Analyzer queryAnalyzer)AnalyzingSuggester(Directory tempDir, String tempFileNamePrefix, Analyzer indexAnalyzer, Analyzer queryAnalyzer, int options, int maxSurfaceFormsPerAnalyzedForm, int maxGraphExpansions, boolean preservePositionIncrements)Creates a new suggester.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidbuild(InputIterator iterator)Builds up a new internalLookuprepresentation based on the givenInputIterator.protected AutomatonconvertAutomaton(Automaton a)Used by subclass to change the lookup automaton, if necessary.Objectget(CharSequence key)Returns the weight associated with an input string, or null if it does not exist.Collection<Accountable>getChildResources()longgetCount()Get the number of entries the lookup was built withprotected List<FSTUtil.Path<PairOutputs.Pair<Long,BytesRef>>>getFullPrefixPaths(List<FSTUtil.Path<PairOutputs.Pair<Long,BytesRef>>> prefixPaths, Automaton lookupAutomaton, FST<PairOutputs.Pair<Long,BytesRef>> fst)Returns all prefix paths to initialize the search.booleanload(DataInput input)Discard current lookup data and load it from a previously saved copy.List<Lookup.LookupResult>lookup(CharSequence key, Set<BytesRef> contexts, boolean onlyMorePopular, int num)Look up a key and return possible completion for this key.longramBytesUsed()Returns byte size of the underlying FST.booleanstore(DataOutput output)Persist the constructed lookup data to a directory.
-
-
-
Field Detail
-
EXACT_FIRST
public static final int EXACT_FIRST
Include this flag in the options parameter toAnalyzingSuggester(Directory,String,Analyzer,Analyzer,int,int,int,boolean)to always return the exact match first, regardless of score. This has no performance impact but could result in low-quality suggestions.- See Also:
- Constant Field Values
-
PRESERVE_SEP
public static final int PRESERVE_SEP
Include this flag in the options parameter toAnalyzingSuggester(Directory,String,Analyzer,Analyzer,int,int,int,boolean)to preserve token separators when matching.- See Also:
- Constant Field Values
-
-
Constructor Detail
-
AnalyzingSuggester
public AnalyzingSuggester(Directory tempDir, String tempFileNamePrefix, Analyzer analyzer)
-
AnalyzingSuggester
public AnalyzingSuggester(Directory tempDir, String tempFileNamePrefix, Analyzer indexAnalyzer, Analyzer queryAnalyzer)
-
AnalyzingSuggester
public AnalyzingSuggester(Directory tempDir, String tempFileNamePrefix, Analyzer indexAnalyzer, Analyzer queryAnalyzer, int options, int maxSurfaceFormsPerAnalyzedForm, int maxGraphExpansions, boolean preservePositionIncrements)
Creates a new suggester.- Parameters:
indexAnalyzer- Analyzer that will be used for analyzing suggestions while building the index.queryAnalyzer- Analyzer that will be used for analyzing query text during lookupoptions- seeEXACT_FIRST,PRESERVE_SEPmaxSurfaceFormsPerAnalyzedForm- Maximum number of surface forms to keep for a single analyzed form. When there are too many surface forms we discard the lowest weighted ones.maxGraphExpansions- Maximum number of graph paths to expand from the analyzed form. Set this to -1 for no limit.preservePositionIncrements- Whether position holes should appear in the automata
-
-
Method Detail
-
ramBytesUsed
public long ramBytesUsed()
Returns byte size of the underlying FST.
-
getChildResources
public Collection<Accountable> getChildResources()
-
convertAutomaton
protected Automaton convertAutomaton(Automaton a)
Used by subclass to change the lookup automaton, if necessary.
-
build
public void build(InputIterator iterator) throws IOException
Description copied from class:LookupBuilds up a new internalLookuprepresentation based on the givenInputIterator. The implementation might re-sort the data internally.- Specified by:
buildin classLookup- Throws:
IOException
-
store
public boolean store(DataOutput output) throws IOException
Description copied from class:LookupPersist the constructed lookup data to a directory. Optional operation.- Specified by:
storein classLookup- Parameters:
output-DataOutputto write the data to.- Returns:
- true if successful, false if unsuccessful or not supported.
- Throws:
IOException- when fatal IO error occurs.
-
load
public boolean load(DataInput input) throws IOException
Description copied from class:LookupDiscard current lookup data and load it from a previously saved copy. Optional operation.- Specified by:
loadin classLookup- Parameters:
input- theDataInputto load the lookup data.- Returns:
- true if completed successfully, false if unsuccessful or not supported.
- Throws:
IOException- when fatal IO error occurs.
-
lookup
public List<Lookup.LookupResult> lookup(CharSequence key, Set<BytesRef> contexts, boolean onlyMorePopular, int num)
Description copied from class:LookupLook up a key and return possible completion for this key.- Specified by:
lookupin classLookup- Parameters:
key- lookup key. Depending on the implementation this may be a prefix, misspelling, or even infix.contexts- contexts to filter the lookup by, or null if all contexts are allowed; if the suggestion contains any of the contexts, it's a matchonlyMorePopular- return only more popular resultsnum- maximum number of results to return- Returns:
- a list of possible completions, with their relative weight (e.g. popularity)
-
getCount
public long getCount()
Description copied from class:LookupGet the number of entries the lookup was built with
-
getFullPrefixPaths
protected List<FSTUtil.Path<PairOutputs.Pair<Long,BytesRef>>> getFullPrefixPaths(List<FSTUtil.Path<PairOutputs.Pair<Long,BytesRef>>> prefixPaths, Automaton lookupAutomaton, FST<PairOutputs.Pair<Long,BytesRef>> fst) throws IOException
Returns all prefix paths to initialize the search.- Throws:
IOException
-
get
public Object get(CharSequence key)
Returns the weight associated with an input string, or null if it does not exist.
-
-