Aeon: High-Performance Neuro-Symbolic Memory Management for Long-Horizon LLM Agents

Technical Deep Dive: Aeon: High-Performance Neuro-Symbolic Memory Management for Long-Horizon LLM Agents

Overview

This paper presents Aeon, a Neuro-Symbolic Cognitive Operating System designed to address the "Lost in the Middle" phenomenon, where the reasoning capabilities of Large Language Models (LLMs) degrade as context windows expand. Aeon introduces a hybrid architecture that treats memory as an active, managed resource rather than a passive database retrieval problem.

Key Contributions

1. INT8 Symmetric Scalar Quantization: Aeon's spatial index, called the Atlas, uses symmetric INT8 quantization to achieve a 3.1× disk compression ratio and 5.6× math acceleration via ARM NEON SDOT instructions, making edge deployment viable for large knowledge bases.
2. Decoupled Write-Ahead Log (WAL): Aeon's WAL provides crash-recoverability with less than 1% insert latency overhead by decoupling disk I/O from RAM mutations through a 3-step lock ordering protocol.
3. Sidecar Blob Arena: Aeon replaces the prior 440-character text ceiling for episodic trace events with an append-only, mmap-backed blob file that enables full LLM transcript archival with generational garbage collection.
4. Semantic Lookaside Buffer (SLB): Aeon's high-performance caching mechanism exploits conversational locality to achieve sub-5μs retrieval latencies, with FP32 vectors dequantized from INT8 storage to preserve L1-resident lookup performance.

Methodology

• Aeon follows a Core-Shell design, with a C++23 core for low-latency operations and a Python shell for high-level control logic.
• The spatial index (Atlas) organizes vectors into a navigable, hierarchical structure, while the episodic graph (Trace) captures temporal and causal context.
• Aeon employs techniques like write-ahead logging, epoch-based reclamation, and double-buffered shadow compaction to ensure crash-recoverability, lock-free concurrency, and stutter-free garbage collection.

Results

• INT8 quantization in the Atlas provides a 5.6× speedup in dot product computations and a 3.4× improvement in tree traversal latency compared to FP32.
• The decoupled WAL incurs less than 1% overhead on insert latency.
• The SLB achieves an 85%+ hit rate, delivering sub-5μs effective retrieval latencies.
• Under high-contention 16-thread workloads, Aeon's epoch-based reclamation maintains a P99 read latency of 750ns.

Limitations & Uncertainties

• Aeon currently operates only on text embeddings. Future work will explore co-locating audio, video, and structured data embeddings within the Atlas.
• Hardware-enforced isolation mechanisms like Intel SGX or ARM CCA are identified as a potential direction to support secure multi-tenant deployments.

What Comes Next

The authors propose two key areas for future investigation:

1. Multi-Modal Vector Representations: Extending Aeon to handle heterogeneous data modalities beyond text, while defining meaningful distance metrics across embeddings of varying dimensionality.
2. Hardware-Enforced Isolation for Multi-Tenancy: Leveraging technologies like Intel SGX or ARM CCA to provide cryptographic guarantees of memory isolation, enabling secure multi-tenant deployments of the Aeon system.