Boxuan Li c68478f470 Customize LLM config per agent (#2756) 1 anno fa
..
README.md d37b2973b2 Refactoring: event stream based agent history (#2709) 1 anno fa
__init__.py 98adbf54ec Small refactoring (#1614) 1 anno fa
condenser.py 80fe13f4be rename our completion as a drop-in replacement of litellm completion (#2509) 1 anno fa
history.py 2df1d67007 History clean up (#2849) 1 anno fa
memory.py c68478f470 Customize LLM config per agent (#2756) 1 anno fa

README.md

Memory Component

  • Short Term History
  • Memory Condenser
  • Long Term Memory

Short Term History

  • Short term history filters the event stream and computes the messages that are injected into the context
  • It filters out certain events of no interest for the Agent, such as AgentChangeStateObservation or NullAction/NullObservation
  • When the context window or the token limit set by the user is exceeded, history starts condensing: chunks of messages into summaries.
  • Each summary is then injected into the context, in the place of the respective chunk it summarizes

Memory Condenser

  • Memory condenser is responsible for summarizing the chunks of events
  • It summarizes the earlier events first
  • It starts with the earliest agent actions and observations between two user messages
  • Then it does the same for later chunks of events between user messages
  • If there are no more agent events, it summarizes the user messages, this time one by one, if they're large enough and not immediately after an AgentFinishAction event (we assume those are tasks, potentially important)
  • Summaries are retrieved from the LLM as AgentSummarizeAction, and are saved in State.

Long Term Memory

  • Long term memory component stores embeddings for events and prompts in a vector store
  • The agent can query it when it needs detailed information about a past event or to learn new actions