δ-mem: Efficient Online Memory for Large Language Models
Original: δ-mem: Efficient Online Memory for Large Language Models
Why This Matters
Offers efficient memory solution for LLMs without costly context expansion
Researchers propose δ-mem, a lightweight memory mechanism that augments large language models with compact online associative memory. Using only an 8×8 memory state, it improves performance by 1.10× over frozen backbone models without requiring full fine-tuning or context window expansion.
The paper introduces δ-mem, a memory mechanism designed to help large language models accumulate and reuse historical information in long-term assistant and agent systems. Instead of expanding costly context windows, δ-mem uses a compact online state matrix updated by delta-rule learning to compress past information. The system generates low-rank corrections to the backbone model's attention computation during generation. Testing shows δ-mem achieves 1.10× improvement over frozen backbone models and 1.15× over strongest non-δ-mem baselines. On memory-intensive benchmarks, gains reach 1.31× on MemoryAgentBench and 1.20× on LoCoMo while preserving general capabilities. The approach works without full fine-tuning, backbone replacement, or explicit context extension.