For multi-party dialogue, tracking speaker identity and relationships separately from content is crucial—this dual-track approach outperforms general-purpose memory systems that try to handle everything at once.
This paper introduces SpeakerMem-R1, a memory system for multi-party conversations that tracks who said what and how people relate to each other. Unlike general LLMs that lose track of speakers and relationships, it uses a dual-track approach: storing exact messages labeled by speaker plus derived relationship states, organized by person and group.