← Home

Looking inside the model

LLM Internals

Language models often hold more than they show. I study what is stored inside a model, how it is retrieved, and what gets in the way of using it, whether that is the phrasing of a request, the examples in a prompt, or the instruction itself. The work combines controlled training setups with probing and steering of hidden states.

Questions

  • When does a learned fact become retrievable, and through which phrasings?
  • Why do in-context examples push aside knowledge the model already has?
  • Can we read a model’s judgment separately from how it is asked to express it?

Papers

  1. Preprint 2026

    Isolating a Language Model’s Judgment with a Label-Free Subspace

    Chaemin Jang, Donggyu Lee, Jeesoo Kim, Dongman Lee, Jihee Kim

    When instructions change, the differences in a model’s hidden states concentrate in a low-dimensional expression subspace that can be found without labels. Removing it isolates the model’s judgment, which stays reliable under new instructions: with a negated question, AUROC rises from 0.504 to 0.825 for entailment and from 0.480 to 0.891 for safety.

    arXivPDFCode

  2. Preprint 2026

    Learning a Fact Is Not Learning How to Retrieve It

    Chaemin Jang, Jihee Kim, Dongman Lee

    We separate learning a fact from learning to retrieve it. Two models learn the same new facts as plain statements; the one that earlier saw facts in five request forms can retrieve the new facts through other phrasings. Steering the hidden state just before the answer can turn retrieval on or off, across facts and relations.

    arXivPDFCode

  3. COLM 2026

    In-Context Examples Suppress Scientific Knowledge Recall in LLMs

    Chaemin Jang, Woojin Park, Hyeok Yun, Dongman Lee, Jihee Kim

    COLM 2026

    In-context examples make models rely less on scientific formulas they can already recall, shifting them toward fitting patterns in the examples, even when the examples come from that same formula. The shift holds across 60 latent-structure tasks in five domains, whether accuracy falls, holds, or appears to rise.

    arXivPDFProject

Other directions