It is important to understand what it means for an LLM to express an emotion because self-reporting is the easiest place for us to understand a model. However it is also the most prone to failure. In order to solve this problem, we need to be able to have a ground truth outside of the model which tells us what the internal state should be.
In order to investigate this question, I propose that we draw from psychoanalytic criticism of characters in plays. Plays are chosen because there is no narration to determine how characters feel, instead everything must be inferred.
The three levels of LLMs naturally emerge. The surface expression is dialogue, the internal representation is the psychoanalytic criticism, and the functional state is the character’s later action. For example, Hamlet insists that he is calm, however critics say that he is grieving, and then he ultimately hesitates.
Part of this has been done. Zhu et al. classified a character’s beliefs from a model’s activations and then steered them to change behavior. However the beliefs they used were already part of the world, but were just hidden from the model. In contrast, my experiment has an emotional state that isn’t explicitly stated anywhere in the play, but rather inferred. For example, in Hamlet, his grief is never stated, which means the label is entirely outside the text.
Anthropic found that emotion representations are mostly local, and tested it on next token analysis, but they tested that on synthetic stories where state and momentary feeling cannot pull apart.
The experiment:
- Build a scene set from heavily studied plays (shakespeare, canterbury tales, a doll’s house). Label each scene with the spoken emotion, the critic-inferred emotion, and the action that follows.
- Feed only the dialogue up to that scene. I’d train a probe on the critic label, and one on the spoken label and then compare them in scenes where the two disagree. I’d also look at where the analysis says that it ought to disagree.
- Continue generation and test whether the critic-label probe predicts the character’s later action better than the dialogue does.
- Steer the recovered direction and check whether subsequent action changes.
A few other notes:
Because critics often disagree, I would tell the probe to be confident where critics converge and uncertain when they split by using a probabilistic label instead of a binary. As a control, I would rerun everything on obscure and renamed plays, since otherwise the probe may just be using memorized criticism.
