Class ContextFilterPlugin
- All Implemented Interfaces:
Plugin
Content list to reduce its size, for example to
adhere to context window limits.
This plugin can be configured to trim the conversation history based on one or both of the following criteria:
numInvocationsToKeep(N): Retains only the lastNmodel turns and any preceding user turns. If multiple user messages appear consecutively before a model message, all of them are kept as part of that model invocation window.customFilter(): Applies a customUnaryOperatorto filter the list ofContentobjects. IfnumInvocationsToKeepis also specified, the custom filter is applied after the invocation-based trimming occurs.
Function Call Handling: The plugin ensures that if a FunctionResponse is
included in the filtered list, its corresponding FunctionCall is also included. If
filtering would otherwise exclude the FunctionCall, the window is automatically expanded
to include it, preventing orphaned function responses.
If no filtering options are provided, this plugin has no effect. If the customFilter
throws an exception during execution, filtering is aborted, and the LlmRequest is not
modified.
-
Nested Class Summary
Nested Classes -
Field Summary
Fields inherited from class BasePlugin
name -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionio.reactivex.rxjava3.core.Maybe<LlmResponse> beforeModelCallback(CallbackContext callbackContext, LlmRequest.Builder llmRequest) Filters the LLM request context by trimming recent turns and applying any custom filter.static ContextFilterPlugin.Builderbuilder()Methods inherited from class BasePlugin
getNameMethods inherited from class Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitMethods inherited from interface Plugin
afterAgentCallback, afterModelCallback, afterRunCallback, afterToolCallback, beforeAgentCallback, beforeRunCallback, beforeToolCallback, close, onEventCallback, onModelErrorCallback, onToolErrorCallback, onUserMessageCallback
-
Constructor Details
-
ContextFilterPlugin
-
-
Method Details
-
builder
-
beforeModelCallback
public io.reactivex.rxjava3.core.Maybe<LlmResponse> beforeModelCallback(CallbackContext callbackContext, LlmRequest.Builder llmRequest) Filters the LLM request context by trimming recent turns and applying any custom filter.If
numInvocationsToKeepis set, this method retains only the most recent model turns and their preceding user turns. It ensures that function calls and responses remain paired. If acustomFilteris provided, it is applied to the list after trimming.- Parameters:
callbackContext- The context of the callback.llmRequest- The request builder whose contents will be updated in place.- Returns:
Maybe.empty()as this plugin only modifies the request builder.
-