Long Term Memory

Overview of Long Term Memory (LTM)

The long term memory system discussed herein is utilized for storing the conversations, experiences, observations, and reflections that an NPC has had in their life. This includes pre-game (e.g. the character’s backstory) as well as in-game (e.g. conversations and mission). Both pre-game and in-game information is stored mostly the same way, so first I will discuss how in-game memories are generated and stored and then hash out how pre-game memories are generated and stored.

Long term memory in this project is primarily a re-implementation of the long term memory systems in Park et al.’s ‘Generative Agents: Interactive Simulacra of Human Behavior’ but with some additions. Essentially, this code leverages FAISS (facebook ai similarity search) to store a vector representation of NPC memories. Memories include observations as well as reflections. An observation is defined as an event perceived by the NPC, such as: (1) Charley walked happily into the chocolate factory; (2) the chocolate looked delicious. A reflection is essentially a thought that the NPC has that is stored as a memory but not explicitly observed by the NPC as part of the environment. For example, the NPC may have memories of themselves studying many times (observations), so a new memory could be stored that says that said NPC is an enthusiastic student (reflection). Reflections are generated using an LLM call - observations are passed to the LLM and reflections outputted.

When Does Memory Update

There are three systems that result in the update of NPC long term memory currently: conversations, missions, and NPC backstory. A conversation consists of a list of dialogue exchanges. Each of these dialogue exchanges is stored as an observation in the form of “USER said to NPC: MESSAGE” and “NPC said to USER: MESSAGE”. Observations are also generated for each NPC entity and companion associated with / sent on a mission. After a mission narrative is generated, for each of the aforementioned NPCs, the mission narrative is converted into a list of observations via an LLM prompt chain. The prompt chain is essentially “given the following narrative, I want you to extract an account of the story from NPC perspective. Give me a list of strings where each string is an observation that accounts for the narrative from the perspective of {npc_name} where an observation is an event directly perceived by an agent. Common observations include behaviors performed by the agent themselves or behaviors that agents perceive being performed by other agents or non-agent objects.” Observations are also generated from the prose-based backstory of each NPC as inputted by the game designer into the game designer frontend. Observations are extracted from the character backstory in mostly the same way as they are extracted from mission narratives. Reflections are generated from these memories automatically.

Memory Persistance

In the original long term memory implementation, long term memory was purely in-memory. Since I need memory to persist I just maintain a pickle of each NPC’s memory stream. This is definitely sub-optimal from an optimization and latency perspective since I utilize long term memory stateless and load the index repeatedly, but it works for now. If latency becomes an issue in the future I can always just load the pickle and hold the memory stream and FAISS objects in memory so long as we’re speaking with an NPC. Since we’re not holding the full GenerativeAgentMemory object in memory, I have to persist the aggregate importance score for each NPC. I just track this in Mongo.

Long Term Memory objects are stored per-game, per-user, per-save-state, per-npc. In addition, one long term memory object (e.g. memory stream and faiss object) is stored per-NPC. This object is used to initialize long term memory (e.g. all of the initial memories that an NPC has based off of their backstory). The first time that a long term memory object is needed in a game playthrough, the memory stream and FAISS indices are copied over to the relevant directory associated with the game-user-savestate-npc. Both the memory stream and FAISS indices are incrementally updated to account for new observations and reflections.

Memory Retrieval

Memory retrieval is carried out using faiss.similarity_search_with_relevant_scores + time-weighted exponential memory decay. So essentially memories are retrieved based on their significance (how significant of a memory is this), time (more recent memories are more relevant), and similarity.

Memories need to be retrieved both during npc-user dialogue exchange as well as during mission narrative generation. The specifics of this extraction are detailed in the [NpcUserInteraction](https://app.gitbook.com/o/L06ffJuY9hZGDd1z16eI/s/atal7BI1QDhKCoSKLT1y/npcuserinteraction-conversation) and [MissionOutcome](https://app.gitbook.com/o/L06ffJuY9hZGDd1z16eI/s/atal7BI1QDhKCoSKLT1y/missions) sections.

During mission narrative generation, it is important that the mission narrative generation system is aware of long term memories that each companion has that are relevant to the mission. These long term memories can include the personality and traits of the NPC or potentially experiences that overlap strongly with the current mission they are on. Writing up a narrative that takes these experiences into account is what makes the story just…make more sense. However, we cannot just pass the entire mission brief to a companion’s long term memory and expect good results. Therefore, first, we prompt the LLM (turbo) with the following template. This allows us to generate a list of questions for the LTM that we can then utilize to extract a greater quantity of relevant memories and experiences from the companions.