Looking inside the model
LLM Internals
Language models often hold more than they show. I study what is stored inside a model, how it is retrieved, and what gets in the way of using it, whether that is the phrasing of a request, the examples in a prompt, or the instruction itself. The work combines controlled training setups with probing and steering of hidden states.
Questions
- When does a learned fact become retrievable, and through which phrasings?
- Why do in-context examples push aside knowledge the model already has?
- Can we read a model’s judgment separately from how it is asked to express it?
Papers
Preprint 2026
Isolating a Language Model’s Judgment with a Label-Free Subspace
Chaemin Jang, Donggyu Lee, Jeesoo Kim, Dongman Lee, Jihee Kim
When instructions change, the differences in a model’s hidden states concentrate in a low-dimensional expression subspace that can be found without labels. Removing it isolates the model’s judgment, which stays reliable under new instructions: with a negated question, AUROC rises from 0.504 to 0.825 for entailment and from 0.480 to 0.891 for safety.
Preprint 2026
Learning a Fact Is Not Learning How to Retrieve It
Chaemin Jang, Jihee Kim, Dongman Lee
We separate learning a fact from learning to retrieve it. Two models learn the same new facts as plain statements; the one that earlier saw facts in five request forms can retrieve the new facts through other phrasings. Steering the hidden state just before the answer can turn retrieval on or off, across facts and relations.
COLM 2026
In-Context Examples Suppress Scientific Knowledge Recall in LLMs
Chaemin Jang, Woojin Park, Hyeok Yun, Dongman Lee, Jihee Kim
COLM 2026
In-context examples make models rely less on scientific formulas they can already recall, shifting them toward fitting patterns in the examples, even when the examples come from that same formula. The shift holds across 60 latent-structure tasks in five domains, whether accuracy falls, holds, or appears to rise.