Package org.apache.lucene.tests.analysis
Class LookaheadTokenFilter<T extends LookaheadTokenFilter.Position>
java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.tests.analysis.LookaheadTokenFilter<T>
- All Implemented Interfaces:
Closeable,AutoCloseable,Unwrappable<TokenStream>
- Direct Known Subclasses:
MockGraphTokenFilter,MockRandomLookaheadTokenFilter
public abstract class LookaheadTokenFilter<T extends LookaheadTokenFilter.Position>
extends TokenFilter
An abstract TokenFilter to make it easier to build graph token filters requiring some lookahead.
This class handles the details of buffering up tokens, recording them by position, restoring
them, providing access to them, etc.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprotected static classHolds all state for a single position; subclass this to record other state at each position.Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
AttributeSource.State -
Field Summary
FieldsModifier and TypeFieldDescriptionprotected booleanprotected intprotected final OffsetAttributeprotected intprotected final PositionIncrementAttributeprotected final RollingBuffer<T> protected final PositionLengthAttributeFields inherited from class org.apache.lucene.analysis.TokenFilter
inputFields inherited from class org.apache.lucene.analysis.TokenStream
DEFAULT_TOKEN_ATTRIBUTE_FACTORY -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionprotected voidThis is called when all input tokens leaving a given position have been returned.protected voidCall this only from within afterPosition, to insert a new token.protected abstract Tprotected booleanCall this when you are done looking ahead; it will set the next token to return.protected booleanReturns true if there is a new token.voidreset()Methods inherited from class org.apache.lucene.analysis.TokenFilter
close, end, unwrapMethods inherited from class org.apache.lucene.analysis.TokenStream
incrementTokenMethods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
-
Field Details
-
posIncAtt
-
posLenAtt
-
offsetAtt
-
inputPos
protected int inputPos -
outputPos
protected int outputPos -
end
protected boolean end -
positions
-
-
Constructor Details
-
LookaheadTokenFilter
-
-
Method Details
-
insertToken
Call this only from within afterPosition, to insert a new token. After calling this you should set any necessary token you need.- Throws:
IOException
-
afterPosition
This is called when all input tokens leaving a given position have been returned. Override this and call insertToken and then set whichever token's attributes you want, if you want to inject a token starting from this position.- Throws:
IOException
-
newPosition
-
peekToken
Returns true if there is a new token.- Throws:
IOException
-
nextToken
Call this when you are done looking ahead; it will set the next token to return. Return the boolean back to the caller.- Throws:
IOException
-
reset
- Overrides:
resetin classTokenFilter- Throws:
IOException
-