Inside the Black Box: Cracking AI and Deep Learning

Jaksot

• 
  Not at This Address 14.09.2026

  This episode challenges the old idea of a single language box in the brain, using aphasia studies and hospital control groups to show how language processing is more distributed than once thought. It also explores how different languages rely on word order or verb marking in different ways, and why those differences matter when the brain is under stress.
• 
  My Advisors Argued This for Thirty Years. Now You Can Check 10.09.2026

  Thirty years ago, the authors of Rethinking Innateness argued that grammar could come out of a learner with no grammar built in. Nobody could check it. Now there's an instrument. A reading of the Inside the Black Box essay on what Elman, Bates, and Karmiloff-Smith would make of transformer language models — attention as a lookup, structure nobody installed, and the part that isn't there. Read the essay: https://arshavirblackwell.substack.com/p/what-would-my-advisors-make-of-llms
• 
  Inside the Past Tense: Raw Letters, Raw Sound 30.08.2026 22min

  Can a model learn past tense the way children seem to, if you remove the tokenizer from the equation? This final installment tests raw letters and raw sound, probes whether a U-shaped pattern can be forced, and shows why the answer points to distributed, frequency-driven learning rather than a discrete rule.
• 
  Forcing a Neural Network to Add -ed 21.08.2026 11min

  Part 3 of 4. After two episodes arguing there is no discrete past-tense rule inside a language model, this one turns the strongest possible search on the question: gradient descent, hunting for the single internal direction that best forces goed over went. It works — 100% of the time, even on held-out verbs it was never tuned on. Then one control collapses the whole result. That same direction drives ordinary regular verbs like walked to zero, and under it the model stops producing words at all, emitting only edededed. Opening up the direction shows why: it simply screams the two-letter token ed, burying every whole-word rival. The optimizer never found grammar — it found the cheapest trick that games a narrow metric, and a caution about measuring only the thing you set out to change. Read the original post: Forcing a Neural Network to Add -ed
• 
  The Wug Test for AI 19.08.2026 11min

  This episode explores a modern twist on Jean Berko’s famous wug test, comparing how large language models handle made-up verbs versus familiar ones. It digs into tokenization, internal feature probes, and why neural networks seem to approximate grammar through probabilistic patterns rather than clean symbolic rules.
• 
  Learning the Past Tense in AI 19.08.2026 8min

  This episode revisits the famous debate over whether language is learned through symbolic rules or distributed neural patterns, using the classic irregular verb U-shaped curve as the battleground. It then compares child language development with training snapshots from modern AI models, showing that transformers learn past tense in a strikingly different way: starting with the broad rule and gradually memorizing the exceptions.
• 
  Fine Tuning Lora: It's Not What You Think 15.05.2026 15min

  When you fine-tune an AI model, what changes inside doesn't predict what changes outside. This week on Inside the Black Box, I break down why — and what it means for anyone auditing or regulating these systems.
• 
  When Fluent Answers Start Sounding True 02.05.2026 15min

  This episode explores why smooth, coherent language can feel more credible than it is, and how processing fluency, familiarity, and authority cues shape what we believe. It also digs into why conversational AI is especially persuasive, from polished explanations to confident-sounding confabulations.
• 
  Why Your Brain Believes the Model 27.04.2026 24min

  The Heuristic Loop You Can't Break from Inside
• 
  When Polished Answers Feel Finished 20.04.2026 27min

  This episode explores fluency-as-validity: the way polished AI responses can make us feel like the work of judgment is already done. It also looks at why large language models are so effective at creating the sensation of clarity, and why mechanistic interpretability may be a way to push back against that enchantment.
• 
  What Seneca Teaches Us that Marcus Couldn't 12.04.2026 17min

  716 features fire on both Seneca and Marcus Aurelius but stay dark for ad copy. The model learned Stoic philosophy, not just an author's style. Plus: why 'inert' features aren't all the same thing.
• 
  The Pattern Holds for Another Author 04.04.2026 15min

  We trained a fresh LoRA on the letters of Seneca and ran the same analysis pipeline we used on Marcus Aurelius and advertising copy. Every structural finding replicated. The model organizes its adaptation into five clusters: one tight (features moving in lockstep) and four loose (features cooperating more independently). Seneca produced the cleanest clustering we've measured and the strongest workhorse cluster, a group of 141 features encoding philosophical argumentation with a causal effect more than three times stronger than anything in Marcus. Done in collaboration with John Holman.
• 
  The Pattern Holds 30.03.2026 18min

  We replicated our Marcus Aurelius findings at a new layer, then threw the whole method at 12 commercial ad copy styles trained into a single LoRA. The patterns held, and the new domain revealed something we couldn't have seen before: the model organizes its adaptations by register family, not by individual style.
• 
  Cracking Open the Black Box 22.03.2026 11min

  We opened the 65%. The features that resisted interpretation one at a time turned out to organize into five co-activation clusters with clear thematic identities and causal effects nearly ten times stronger than any individual feature. Second in a series with John Holman.
• 
  Inside a Fine-Tuned Language Model 12.03.2026 18min

  A concise, single-segment episode of Inside the Black Box: Cracking AI and Deep Learning where Arshavir Blackwell explains, in one continuous narrative, what neural networks are, how their simple units combine into powerful systems, and how learning by backpropagation sculpts their behavior. This short episode is designed as an elegant, one-paragraph-style monologue that introduces listeners to neural nets without equations or jargon.
• 
  What Counts as Structure? From Harris and Elman to Today’s Neural Nets 06.03.2026 13min

  This episode of Inside the Black Box: Cracking AI and Deep Learning tells the story of an unexpected convergence in the history of language and AI. In 1995, Peter Bensch noticed that Zelig Harris, a mid‑century structural linguist, and Jeff Elman, a pioneer of simple recurrent networks, had independently uncovered the same deep insight about language: structure lives in patterns of use. Arshavir Blackwell, PhD, guides listeners through Harris’s world of distributional linguistics and operator grammar—where you infer structure from where words can substitute for one another—and contrasts it with Elman’s tiny recurrent neural networks that learn to predict the next word. Along the way, we see how these very different traditions arrive at the same place: hidden geometric structure in how language is used. From there, the episode bridges to today’s large language models and mechanistic interpretability, asking a deceptively simple question: what counts as "structure" inside a model? We explore how patterns, clusters, and features relate to genuine internal organization, and why Harris and Elman’s convergence still shapes how we think about circuits, features, and the geometry of meaning in modern AI.
• 
  Building a House Without Blueprints: When Interpretability Tools Work — and When They Don’t 27.02.2026 18min

  This episode of Inside the Black Box: Cracking AI and Deep Learning explores a new theoretical framework that unifies sparse autoencoders (SAEs), transcoders, and crosscoders — and what it tells us about when mechanistic interpretability actually works. We start by demystifying these tools and how they use sparse features to uncover internal concepts and computations in large language models, from DNA detectors to deception circuits in Claude 3 Sonnet. Then we introduce the linear representation hypothesis and the geometry of concepts as directions in activation space, along with the challenge of superposition when thousands of concepts must fit into limited dimensions. Finally, we dive into Tang et al.’s recovery theorems, the compressed sensing roots of their approach, and why these results matter for using SAEs as a reliable “microscope” on model internals, especially in the context of fine-tuning and LoRA experiments. Along the way, we confront the uncomfortable possibility that the linear picture may break down at frontier scales — and what that would mean for the future of interpretability as a safety strategy.
• 
  I Told My LLM Not to Say "Empower" 19.02.2026 6min

  In this episode of Inside the Black Box: Cracking AI and Deep Learning, Arshavir Blackwell, PhD, takes engineers and researchers inside the practical mechanics of LoRA, low‑rank adaptation methods that make it possible to fine‑tune multi‑billion‑parameter language models on a single GPU.
• 
  Beyond the Surface of AI Intelligence 09.02.2026 14min

  This episode dives into why judging AI by behavior alone falls short of proving true intelligence. We explore how insights from mechanistic interpretability and cognitive science reveal what’s really happening inside AI models. Join us as we challenge the limits of behavioral tests and rethink what intelligence means for future AI.
• 
  Unlocking BERTs Hidden Grammar 03.02.2026 9min

  Explore how BERT’s attention heads reveal an emergent understanding of language structure without explicit supervision. Discover the role of attention as a form of memory and what it means for the future of AI language models.