I accidentally turned LLM memory into program analysis
Points and comments are a snapshot, not live.
LLM memory can be maintained with Datalog for deterministic fact tracking and invalidation.
The author built Lemmalog, a Datalog engine for LLMs, to solve the problem of LLMs losing track of established facts during vulnerability research. Instead of relying on the LLM to maintain its own knowledge, Lemmalog stores observations as structured facts and derives conclusions via rules. When an observation changes (e.g., `object_a does not actually point to object_b`), affected conclusions are automatically invalidated using incremental evaluation and provenance tracking. This separates the LLM's role (extracting fuzzy facts from natural language, code, or debugger output) from deterministic reasoning (the Datalog engine). In LongMemEval benchmarks, Lemmalog achieved 0.463 F1 (vs. PropMem's 0.550), using ~2,700 tokens per question vs. ~104,000 for full context. It topped the Knowledge Update category (0.579 vs. PropMem's 0.528).
What commenters are saying
Commenters broadly praised the approach as a return to symbolic AI principles, with multiple comparisons to Prolog, Datalog, and Cyc. Several shared similar frustrations with LLMs retaining invalidated beliefs. One commenter advocated for a "weathering" principle where repeated reasoning hardens into mechanical structure. Another noted existing implementations like answer set programming for non-monotonic logic. A practical counterpoint warned that LLM-generated facts drift over time, harming downstream reasoning. One commenter shared their simpler workaround: a decision log file (CLAUDE.md) that agents read to avoid losing context.