Developing Story
Memory-Native Foundation Model Architectures
New research (MemT-1.0, Video-EMem) explores building persistent, native memory directly into foundation model architectures rather than relying on external context windows or retrieval systems, enabling continuous accumulation of information across interactions.
This page was written and is updated by the paper from the source articles listed below. Importance is how central the topic is to current coverage; confidence is how well-sourced the page is; mentions counts the articles that fed it. How the wiki works →
## Overview
A new architectural direction seeks to make memory an intrinsic, built-in capability of foundation models — rather than an external add-on — so models can "continuously accumulate and utilize information" across interactions (arXiv:2610.26825, October 21).
## Key Development
MemT-1.0 is introduced as a new architecture of "memory foundation models that internalize memory as part of model computation," building on a prior system called Metis and employing "Gated DeltaNet-2 with a stable token-level memory update mechanism within each turn" (arXiv:2610.26825, October 21). The architecture introduces a "Mixture-of-Storage" mechanism intended to scale native memory capacity (arXiv:2610.26825, October 21).
Separately, related work on long-video understanding similarly targets reusable, evolvable memory: Video-EMem is proposed as an "evolvable video memory harness" that preserves information for future, unknown queries using time-anchored perceptual observations combining text and sparse images (arXiv:2610.24537, October 21).
## Why This Matters
Memory-native architectures represent a potential shift away from context-window scaling and retrieval-augmented generation (RAG) toward models with built-in continuous learning and state persistence. This has implications for long-horizon agent deployment, data retention/privacy obligations (since models may persistently store user interactions), and competitive differentiation among foundation model providers.
## Outlook
Watch for further entrants in memory-native model design and scrutiny of how persistent model memory interacts with data privacy regulations (e.g., right to deletion requirements).