Last year, I led a team of seven researchers at Algoverse to build three novel Theory of Mind benchmark datasets. We called the project DIG-DIS, and it taught me more about the limitations of AI than any paper I'd ever read. Theory of Mind (ToM) is the ability to attribute mental states — beliefs, intentions, desires — to others. It's what lets you understand that someone might believe something false, or want something they haven't expressed. Humans develop this around age four. AI systems struggle with it at every scale.
Building the Benchmarks
Our approach was simple: create scenarios that require reasoning about what characters in a story know, believe, or want — and then measure whether AI systems could track those mental states accurately. We focused on three types of reasoning: Distraction scenarios (tracking beliefs when irrelevant info is introduced), Indirect speech (understanding implications), and Nested beliefs (reasoning about what A thinks B thinks C believes).
Transformers vs. State Space Models
The most interesting finding wasn't about performance — it was about failure modes. GPT-4o Mini was better at handling complex nested beliefs but struggled when distracting information was present. It seemed to "forget" earlier context when processing new information. Phi-Mamba showed the opposite pattern. Its recurrent architecture maintained context better across long sequences, but it struggled with the compositional reasoning required for nested beliefs.
What This Means for AI Development
We don't have a model architecture that robustly handles Theory of Mind. Transformers are great at parallel attention but lose context. SSMs maintain context but struggle with composition. The path forward probably isn't choosing one architecture over the other — it's understanding why each fails and designing hybrid approaches. Theory of Mind might be the hardest problem in AI because it requires everything at once: memory, attention, composition, and common-sense reasoning. Solving it won't just make AI systems smarter — it'll make them understand us.