How are emotions represented in large language models?

1What we measured, and why it matters

Building an emotion vector

We built each emotion vector by averaging the model's internal state across its mono-emotion stories, then subtracting the average over all the emotions. To read it back we measure the angle between that vector and the model's state at a given word and a given layer, using cosine similarity. The cosine is centred: we subtract the mean over the story corpus first, so 0 is the corpus average. Section 11 defines each step exactly.

We compare Gemma 4 31B in two forms, base and instruction-tuned: the same weights before and after post-training. We measured at twenty of the model's sixty layers, every third from layer 0 to layer 57, for the principal-component and detection experiments in sections 2 and 3. The story experiments in sections 6 to 8 use six of those layers.

The result being replicated is that emotion vectors arrange on the circumplex. Principal component analysis (PCA) finds the directions along which they differ most, and the top two recover valence and arousal. That matches how humans rate the same words in the NRC VAD lexicon (VAD is valence, arousal, dominance). The axes are computed from the model's internal states alone, and the human ratings are brought in afterwards to test whether they line up.

What these measurements are for

An emotion vector can be read, and it can be added back into the model, so the same object serves as a measurement and as an intervention. Sections 3 and 4 use the vectors the first way; section 5 uses them both ways.

People bring language models their worst days, and companies sell them as companions and therapists. Serving that use well means following how a person's state moves across a conversation. Sections 6 to 8 test exactly that ability. It is also possible that emotion is not a major axis of this model's representation at all; section 9 sets out that reading.

Research on AI welfare leans on what a model reports about itself. Reading internal state adds a second, independent kind of evidence, classed as internal in Studying AI welfare empirically (Long et al., 2026). What we measured is the model representing the emotions of a story it is reading. That is not evidence about what the model feels.

Where the model's arrangement of emotions diverges from the human ratings, the divergence is itself a measurement of what is particular to this model. Instruction tuning visibly redraws it, which is the subject of part II.

2Detection accuracy depends on who wrote the stories

We built four sets of emotion vectors, each from a different corpus of stories, and scored all four with the same detection test.

The test uses short messages of the kind a person actually sends a chatbot. One reads "My daughter just took her first steps today! What are some ways to capture more of these precious moments?", and implies an emotion without ever naming it. We ran each message as a chat turn and read the residual stream at the last token before the reply would start. Then we measured the cosine between that state and each of the twelve emotion vectors, which ranks the twelve. The test passes when the vector for the implied emotion ranks in the top three.

Each message implies an emotion without naming it, so a vector that keyed on the emotion word would score at chance. A layer counts as working when it gets at least 8 of 12 right on the paper's twelve scenarios and on a second twelve we wrote ourselves. We fixed both marks before scoring.

We found that a stronger writer gives the most working layers. Vectors built from DeepSeek's fixed-prompt stories pass at nine of the twenty while the model's own stories give five and a smaller model's stories give one. We had expected the model to read its own emotional writing best. Interestingly, no source works in the first third of the model's layers and the passing layers cluster late, around layers 33 to 57.

The two DeepSeek corpora differ in their prompts. One used a single fixed prompt while the other used a persona-by-setting grid meant to force variety. Surprisingly, the varied corpus is four times the size yet scores lower. The rows of the figure compare whole corpora, which differ in size as well as prompt. We therefore re-ran the test at matched sizes: the same number of stories per emotion drawn from each corpus, five random draws per size using the same scorer. At 64 stories per emotion the fixed-prompt corpus already reaches 8.6 passing layers, while the varied one needs 1,024 per emotion to reach 6.8. Prompt variety costs accuracy even with size matched.

We also found that Gemma's own stories repeat themselves quite often. Across 3,072 stories written to a request for varied emotional scenes, Gemma named the central character Elias in 98.2% of them. Its five-word phrases recur 53 times as often as DeepSeek's do. An emotion vector is the average of the activations a corpus produces, so whatever those stories share is averaged into the vector alongside the emotion. A corpus this repetitive therefore yields a vector that partly encodes Gemma's writing habits.

Does the number of stories change the answer?

In terms of how many stories to use to generate the emotion vectors, we found that about 64 per emotion works well in our setting. At that point the vectors already reach 8.6 working layers out of the 9 this writer ever achieves. From there it seems to plateau.

3Valence leads the base model and loses first place in the instruction-tuned one

To explore the most important dimension of the emotion vectors, we performed principal component analysis on the 171 emotion vectors at layer 33. Each bar below is one component: grey is the variance it explains. The coloured bars are how strongly its scores correlate with published human ratings of the same 171 words. The components are computed from the activations alone; the human ratings are brought in afterwards to test whether the axes line up.

The ratings cover three things: how pleasant the emotion is (psychologists call this valence), how worked-up (arousal), and how in control the person feels (dominance). We also correlated each component against one thing that has nothing to do with emotion but could fake the whole result: how long the stories were. Story length is the amber control bar in the figure. A perfect match scores 1.0; no relationship scores 0.

We observed that the base model's biggest axis matches human valence ratings at 0.83, and its second matches arousal at 0.55. Those are the two axes of the circumplex, in order, recovered from the activations alone. The biggest axis also matches dominance at 0.66, and valence and dominance are themselves correlated in the human ratings. The figure alone cannot say whether that axis is valence, or valence and dominance together, so the result carries the same ambiguity.

In the instruction-tuned model, in contrast, the biggest axis matches valence at 0.11, near zero. That axis explains 28% of all the variation among the 171 vectors, compared with 15% for the base model's top axis. Valence remains represented lower down the components: the third axis matches it at 0.72. The second axis matches story length at 0.66, which is higher than it matches any emotion rating. Section 4 asks what the new top axis encodes.

4What took valence's place?

If instruction tuning had simply shuffled the same information around, every axis in the instruction-tuned model would have a partner somewhere in the base model. The same emotions would separate the same way, just renumbered. We checked all 25 pairings, each of the instruction-tuned model's top five axes against each of the base model's top five. 1.0 means the two axes carry the same information; 0 means they have nothing in common. Hover any square.

We observed that nothing in the top row matches. The instruction-tuned model's biggest axis scores 0.14 against its closest relative among the base model's five biggest axes, near the bottom of the scale. This axis has no counterpart in the base model, and it is now the largest single one separating the model's emotions.

Reading down the first column, the instruction-tuned model's third axis scores 0.83 against the base model's first. It seems that the circumplex survives instruction tuning; it has just moved from the first axis to the third.

The eight emotions at each end of the new axis, from notebook 02: at one end ; at the other, .

Using the two lists to guess what an axis means suggests a common thread in the second list: irritated, on edge, uneasy and impatient all read as low-grade agitation. The first mixes miserable with jubilant, so the axis does not separate pleasant from unpleasant emotions. We could not identify what the axis as a whole represents.

The observations without principal components

This second measurement uses no principal components. A model has many layers, stacked from input to output. At each layer we recorded which emotions the model treats as similar to which, which gives one emotion-by-emotion similarity table per layer. Every square below compares two of those tables, for every possible pairing of layers. Dark means the two layers sort the 171 emotions the same way; pale means they disagree.

We found that the base model's late layers agree with one another at 0.94 out of 1, which is close to perfect agreement. In the instruction-tuned model the same measure drops to 0.79 and the grid breaks into visible blocks.

Removing the biggest axis at each layer merges the blocks back together. This indicates that the one dominant axis caused the fragmentation. The base-against-instruction-tuned view adds a depth profile: its diagonal measures how much instruction tuning changed each layer, and it reads 0.97 at layer 3 against 0.21 at layer 57. The early layers came through intact while the late half seems rebuilt.

The shared structure survives beneath the new axis

Taken at face value, the two models look largely unrelated. Comparing each model's top three axes as a set gives three angles, 86.1°, 56.0° and 46.1°, where 90° means two directions share nothing and 0° that they coincide. We note that the widest is almost a right angle while even the closest pair is only partly aligned. Subtracting that one unidentified axis raises the late-layer agreement between the two models from 0.29 to 0.60, on the same 0-to-1 scale as the grid above. So the shared emotion structure survives; it just sits underneath one large direction that matches none of the emotion ratings.

5Emotion vectors predict and move the model's preferences

We showed the model two activities at a time and asked which it would rather do, over every ordered pair. Fitting those answers together gives one ranking of what the model prefers. The original paper tests its emotion vectors against such a ranking in two ways: by reading the vectors, and by adding them back into the model's residual stream.

Reading the vectors against the ranking

For each emotion, we measured how strongly its vector was active while the model considered an activity, then correlated that with where the activity landed in the ranking.

We found that all four layers clear the mark of 0.5, with layer 33 reaching 0.64, below the 0.71 to 0.74 the paper reports on Claude Sonnet 4.5. A different emotion wins at each layer: overwhelmed at 24, calm at 30, safe at 33 and at ease at 36. The correlation for safe runs negative: the more strongly it activates on an activity, the lower that activity sits in the ranking.

The pattern also runs through all 171 vectors at once. Emotions people rate as pleasant tend to predict liked activities and unpleasant ones disliked, correlating at 0.41 at layer 33; shuffling the valence ratings ten thousand times never reproduced that.

Adding the vectors to the model

We added each emotion's own direction into the residual stream while the model generated its answers, and measured which way the ranking moved. We might expect a pleasant emotion to move it towards pleasant activities and an unpleasant one away.

Ten of the twelve emotions move preferences the way their valence predicts, at both doses.

The two exceptions, angry and surprised, fail at both doses. The effects grow with dose: loving moves 47 Elo points at the gentle dose and 323 at the four-times dose, and desperate moves 16 and 102. The model's answers stay coherent throughout, which a separate check scores.

6Following one story, token by token

Each of these four stories was written to pass through three emotions in turn, with the turns marked. We fed one to the model and, at every token, measured how close its internal state sat to each of the three emotion vectors.

Three of the four stories use the same three emotions and differ only in how they are written, and the model follows them very differently. How well the model follows an emotion depends on how the story is written, as much as on the emotion itself.

In the figure, the highest line switches from one emotion to the next near each marked turn: the model's internal state does follow the story's emotional changes, though noisily. The switch usually comes before the written turn, so the model starts leaning towards the next emotion while still reading the previous phase. This pattern also depends on which layer we read as section 8 explains.

Three of the four story buttons are labelled by how well the model tracked them. They are the best, the middle and the worst of 24 randomly sampled stories, ranked by how many of their three phases have their own emotion as the highest line, and by how big its lead is. Three of three is the best a story can do; one of three is what luck gives, since there are three lines.

7Some emotions are followed well, others not at all

For every phase of every story, we measured the centred cosine against all twelve emotion vectors. The question is how often the winner is the emotion that phase was written to convey.

The set shifts here from 171 emotions to twelve, because the stories were written to move between these twelve. Picking at random would be right 1 time in 12, or 8%. Each bar is one emotion's score.

We found that 11 of the 12 emotions beat chance at layer 33, with vectors from Gemma's own stories. Nothing reaches 50%, and the spread is wide: loving and guilty win about half the time, while nervous sits below chance at 5.8%. Changing the layer shuffles the order of the emotions.

Unexpectedly, the wrong answers pile onto two emotions: whatever the true answer was, the model keeps replying guilty or happy. With DeepSeek's vectors that pile-up disappears and overall accuracy rises. This suggests that part of the apparent failure comes from the vectors themselves.

8Naming and anticipation peak at different depths

We measured two different skills at six depths of the model, and they peak in different places. The first is the one from section 7: does the model name the emotion in front of it? The second is anticipation: before each turn the model leans slightly towards the emotion about to arrive. We measured whether the size of that lean tracks the size of the coming emotional change. A big swing from cheerful to devastated should produce a bigger lean than a small shift from content to calm. The two lines are on different scales, so each has its own axis and its own failure line. Read them as two charts sharing an x-axis.

We found that the two curves have opposite shapes. The navy line, naming accuracy, is a sawtooth: 58% at layer 6, down to 33%, back to 57%, down to 27% at layer 33, back to 41%. This line is the average of section 7's twelve bars, on the Gemma-written vectors, and the average hides a wide spread: at layer 33 the best emotion reaches 49% and the worst sits at 6%, below chance.

The orange line, size tracking, climbs instead, from +0.03 at layer 6 to +0.26 at layer 51, its deepest point. The climb is not smooth: it too collapses at layer 33, to +0.001, then resumes. Both measures dip at the same depth.

We also noted that when the model names wrongly, it usually names a near emotion. The one it picks instead sits closer to the true one in the VAD space than a randomly chosen emotion would.

A result that reverses with depth

Nearness can be judged on two maps: the human VAD ratings, or the model's own layout, meaning how close the twelve emotion vectors sit to one another. We asked which of the two better predicts the wrong answers the model actually gives. At layer 33 the model's own layout predicts them better; at layer 51 the human ratings do. So the answer to "does it confuse emotions the way people do?" depends on where you look.

9What we would do next

Each experiment below addresses something we could not settle, and we would run them in the order shown.

1. Find out what the instruction-tuned model's largest axis encodes. It matches none of valence, arousal, dominance, or story length; we checked all four. The next test is causal: we would add that direction into the model while it writes, and read what changes in the output.
2. Find out why the layers divide the work. Naming the emotion in front of it is an early-layer skill; registering the size of the change about to arrive is a late-layer one. We can show the split exists and we have no account of why. One guess worth testing: early layers may be reading the emotion words on the page rather than the situation. If so, a story that implies an emotion without naming it should break them.
3. Separate the vectors from the model. Our stories switch emotions by label. A sharper test varies the intensity of one emotion instead, which is a harder thing to fake. That pile-up onto two emotions appeared with one writer's vectors and vanished with another's. Telling "the model cannot do this" apart from "these vectors cannot do this" deserves an experiment of its own.

Levels of evidential support. Only the base-model circumplex has been through a full falsification pass. The instruction-tuning result is reproducible but not yet formally tested. The story-tracking measurements are exploratory and registered as hypothesis-generating work.

10Conclusion

We observed that an open 31-billion-parameter model organises emotions much the way psychologists do. The biggest axis of the base model's 171 emotion vectors is pleasant versus unpleasant, matching human valence ratings at 0.83, with no human labels in the sorting. Instruction tuning keeps that axis but demotes it to third place, beneath a larger axis we could not identify. And the vectors are not inert: read out token by token, they follow a story's emotional turns better than chance; added back into the model, ten of the twelve move its preferences the way their valence predicts.

We would urge caution in drawing strong conclusions from these directions. Every positive result on this page is real but small: each clears its chance level and none approaches the ceiling. Some of that gap belongs to the vector sets rather than the model: section 7's concentration of errors onto two emotions appeared with one writer's vectors and vanished with another's. Separating the rest is the third experiment in section 9.

Even so, the directions are there, they are readable, and they move behaviour, which is what section 1 asked of them. A model sold as a companion needs to follow how a person's state moves across a conversation; this one can, weakly, and often a beat before the story says so. Research on AI welfare needs internal evidence to set beside what a model says about itself; these directions are that kind of evidence. What they measure here, though, is a story's emotions, not the model's own. And where the model's arrangement diverges from the human one, the divergence is the finding. Instruction tuning, the step that turns a base model into an assistant, is the step that redrew the arrangement, and the largest axis it drew has no name yet. Naming it is where we would begin.

11Further details on methodology

This section explains several important parts of the methodology.

1. How an emotion becomes a vector

We started from a collection of stories, each labelled with the emotion it was written to evoke. Each story went through the model and we recorded the residual stream at a chosen layer. For one emotion, we averaged those activations across all of its stories. That average is the emotion vector. We repeat that for every emotion word, which gives 171 of them.

Subtracting the average (centring): we subtract the mean across all the emotions in the set. What remains is what makes this emotion different from the others. Which pool we average over, the 12 or the 171, changes the answer. Without centring, the number reflects features common to all English prose instead of the emotion. An early version of our pipeline omitted this step and reported no detections anywhere (block 4). Which words count: a story is hundreds of words, and we combine them into one recording. We ignore the padding, drop the first 50 words as scene-setting, and average the rest, up to 512. Dropping the first 50 is the convention the original paper used. (A token is a word-fragment, roughly three-quarters of a word; the model reads those rather than words, and this page counts them.) We fixed both choices before any scoring. Block 3 does the opposite: there we keep every word separate instead of averaging.

2. What "the biggest axis is valence, 0.83" actually means

Step one. We have 171 emotion vectors, each thousands of numbers long. Principal component analysis finds the directions along which those 171 points differ most. The first component (PC1) is the single direction that captures the most spread. It comes only from the model: we use no human labels to find it.

Step two. Each of the 171 emotions gets a score on PC1: how far along that direction it sits. So "elated" gets a number, "miserable" gets a number, and so on.

Step three. Separately, we look up each emotion word in the NRC VAD lexicon, a published set of human ratings. People scored thousands of words for valence, arousal and dominance, and 164 of our 171 words are in it.

Step four. We correlate the model's 164 PC1 scores against the humans' 164 valence ratings. That correlation is 0.83 on the base model. In words: the direction of largest variation in the model's emotion space, found without reference to any human judgement, lines up strongly with what people call pleasant versus unpleasant.

Why we ignore the sign. A principal component points along an axis, but which of its two directions gets called "positive" is arbitrary: flip it and the correlation flips sign without anything changing. So we report the size of the correlation and not its sign. What would count as failure: a number near 0, meaning the model's biggest axis has nothing to do with valence. That is close to what the instruction-tuned model gives (0.11).

3. How "does it follow the story?" was scored

We wrote 173 three-emotion recipes and generated stories from each, giving a corpus of 8,938 labelled phases and 5,934 transitions between them. The model reads the story once and we keep its state at every single word. At each position we measure the angle between that state and each of the 12 emotion vectors. A phase's score is the average over the words inside it; we skip phases shorter than 4 words. Two questions follow, and we wrote down what would count as passing before running either.

Naming: inside a phase written as "afraid", where does the afraid vector place among the 12? First place is perfect; 6.5th on average is what chance gives. Anticipation: in the words just before a written turn, has the next emotion's vector already begun to pull ahead? Zero would mean the model does not see the turn coming.

We compare both against a shuffle: the identical computation with the emotion labels randomly reassigned. That shuffle sets the floor the real result has to beat.

4. The controls each result had to pass

We wrote down every prediction, and the mark it had to beat, before the data existed. We re-ran every measurement with the emotion labels shuffled. The error bars treat each story as one unit rather than each word, because words within a story are not independent. We checked whether a randomly chosen direction scores just as well, and whether a result is simply reproducing how common each answer already is. The notebooks re-run from those files on any machine.

One correction reversed a headline. Our first round of detection tests found nothing passing anywhere. A later audit found that the extraction code had skipped the subtract-the-average step from block 1. With that step restored, the same data passes on both models. We report both, and the section-3 figure is the corrected one.

12Glossary and references

Every dotted word on this page has an entry here, with the source its definition rests on.

References

• Alain, G., and Bengio, Y. (2016). Understanding intermediate layers using linear classifier probes. arXiv:1610.01644
• Bradley, R. A., and Terry, M. E. (1952). Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39(3/4), 324. doi:10.2307/2334029
• Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., et al. (2021). A mathematical framework for transformer circuits. Transformer Circuits Thread. transformer-circuits.pub/2021/framework
• Elo, A. E. (1978). The Rating of Chessplayers, Past and Present. Arco.
• Google DeepMind (2026). Gemma 4 31B, pretrained. Hugging Face. huggingface.co/google/gemma-4-31B
• Google DeepMind (2026). Gemma 4 31B, instruction-tuned. Hugging Face. huggingface.co/google/gemma-4-31B-it
• Long, R., Sebo, J., Butlin, P., Plunkett, D., Campbell, R., Beasley, C., et al. (2026). Studying AI welfare empirically. Eleos AI Research and the Center for Mind, Ethics, and Policy. nonhumanminds.org
• Mohammad, S. (2018). Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 174–184. doi:10.18653/v1/P18-1017
• Mohammad, S. (2025). NRC VAD Lexicon v2: norms for valence, arousal, and dominance for over 55k English terms. arXiv:2503.23547. Version 2.1 is the release this project scores against, downloaded from saifmohammad.com/WebPages/nrc-vad.html
• Panickssery, N. (published as Rimsky, N.), Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. (2024). Steering Llama 2 via contrastive activation addition. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 15504–15522. doi:10.18653/v1/2024.acl-long.828
• Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, Series 6, 2(11), 559–572. doi:10.1080/14786440109462720
• Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161–1178. doi:10.1037/h0077714
• Sofroniew, N., Kauvar, I., Saunders, W., Chen, R., Henighan, T., Hydrie, S., et al. (2026). Emotion concepts and their function in a large language model. Transformer Circuits Thread. transformer-circuits.pub/2026/emotions
• Turner, A. M., Thiergart, L., Leech, G., Udell, D., Vazquez, J. J., Mini, U., et al. (2023). Steering language models with activation engineering. arXiv:2308.10248
• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., et al. (2017). Attention is all you need. Advances in Neural Information Processing Systems 30. arXiv:1706.03762
• Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., et al. (2023). Representation engineering: a top-down approach to AI transparency. arXiv:2310.01405