Don’t test this like a memory API
Test whether an agent’s behaviour changes as experience accumulates and circumstances evolve.
We know what you’re going to do.
You’re going to tell your agent your favourite colour is green, start a new conversation, ask “What’s my favourite colour?”, and become disappointed in us.
That’s a reasonable test of a memory store. It’s not a particularly good test of adaptation.
Give the agent experiences. Let preferences emerge, evidence accumulate, things contradict each other, and circumstances change. Then look at whether its behaviour changes with them.
A better evaluation
Give the same actor a sequence of realistic experiences over time. Include recurring preferences, exceptions, outcomes, contradictions, and eventually a genuine change in circumstances.
Then ask the agent to make a decision or recommendation where those experiences matter. Evaluate whether its behaviour adapts appropriately—not whether it can repeat one isolated fact.