Theory of mind for agents

To coordinate, compete, or communicate well, an agent benefits from modeling what the other agents think — their beliefs, goals, and knowledge. Crucially, those mental states can differ from reality and from the agent’s own. Attributing them to others is theory of mind (ToM), and it is what turns a collection of agents into agents that can genuinely reason about each other.

The false-belief test

The classic probe for ToM is the false-belief test (the Sally-Anne task). Sally puts a ball in the basket and leaves. While she’s gone, Anne moves it to the box. When Sally returns, where will she look? The reality is box, but Sally’s belief — formed before the move — is still basket. Passing the test means predicting from Sally’s belief, not the true state of the world:

reality = "box"            # Anne moved the ball here while Sally was away
sally_belief = "basket"    # where Sally last saw it (her belief lags reality)

def predict_without_tom(): return reality        # naive: assume everyone knows what I know
def predict_with_tom():    return sally_belief    # model Sally's (false) belief

print("reality:               ", reality)
print("Sally's belief:        ", sally_belief)
print("predict WITHOUT ToM ->", predict_without_tom(), "(wrong: assumes Sally knows reality)")
print("predict WITH ToM    ->", predict_with_tom(),    "(correct: tracks Sally's false belief)")

reality:                box
Sally's belief:         basket
predict WITHOUT ToM -> box (wrong: assumes Sally knows reality)
predict WITH ToM    -> basket (correct: tracks Sally's false belief)

Without ToM, an agent collapses everyone’s knowledge into its own and predicts box — wrong. With ToM, it maintains a separate model of Sally’s belief and predicts basket. That ability to represent a belief you know to be false is the heart of the test.

Why it matters, and the LLM question

ToM is the foundation of strategic interaction. An agent that models others’ beliefs and goals can predict their actions, coordinate by anticipating what a partner expects, communicate efficiently (say only what the other doesn’t already know — the basis of grounding in dialogue), and both deceive and detect deception (deception is inducing a false belief). It also recurses: I know that you know that I know… — the nested reasoning behind negotiation and games.

In one breath

• Theory of mind (ToM): attributing mental states — beliefs, goals, knowledge — to other agents, understanding they can differ from reality and from your own.
• The false-belief (Sally-Anne) test is the classic probe: the ball is really in the box, but Sally believes basket; ToM means predicting from her belief (basket), not reality (box).
• Without ToM an agent collapses others’ knowledge into its own (predicts box, wrong); with ToM it keeps a separate model of each agent’s belief (predicts basket, right).
• ToM underpins prediction, coordination, efficient communication, and deception (deception = inducing a false belief) and recurses (I know that you know…).
• LLMs pass many ToM tasks and improve with scale, but it’s brittle and possibly memorized — the debate over genuine ToM is unsettled; modeling partners’ beliefs still helps agents in practice.

Quick check

Next

Theory of mind powers the believable interactions of generative agents and the opponent-modeling in multi-agent debate. With both swarm intelligence and ToM in hand, the multi-agent-theory chapter is complete — next the toolkit returns to building with LangChain.