Enhancement: Choose what messages from context are being sent #2377
Replies: 3 comments
-
Better managing conversation history on the message level is something I'm actively working on. First by tokens, then by messages. About "pinning" messages. It can be unreliable as forcing a message can throw an error if it exceeds the context window, or you compensate too much to include it (in the extreme case, it's the only message included, and/or it throws an API error). |
Beta Was this translation helpful? Give feedback.
-
I see, thank you for the reply. This pinning feature could also be apart from regular messages: We could even have an optional "summary of context" that would work like the response titling: a model to summarize this context if it needs to be squeezed in a shorter window. In a last effort, the context could be passed by parts like a RAG document. This could also resolve the per-message/conversation-wide pdf document. Here is a Claude 3 clearer formulation
|
Beta Was this translation helpful? Give feedback.
-
I believe its rather difficult of solving the the conversation memory issue efficiently. In my opinion this should be simple, intuitive for the user, and work in the background without requiring additional user intervention. As a first step it would be quite useful to just give an indicator to the user of which messages are included in the conversation context and which not. The messages which are not included in the context could be grayed out for example. |
Beta Was this translation helpful? Give feedback.
-
What features would you like to see added?
Implement a feature that allows users to customize the number of messages sent as context with each new message. This feature will help manage costs and provide users with fine-grained control over the conversation context.
More details
Add an input field, maybe next to the preset button, allowing users to select the number of messages to include in the context window. If no sliding window is specified, all messages in the conversation history will be sent as context, excluding any "force-discarded" messages.
Introduce "force-ping" and "force-discard" options for individual messages in the conversation history.
Default behaviours :
"Force-ping" and "force-discard" behaviour :
Which components are impacted by your request?
General, UI
Pictures
No response
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions