The Wug Test for AI

Arshavir Blackwell, PhD

Rule or Rote? Hunting for the Past Tense Machine Inside an AI. Part 2 of 3. I'm Arshavir Blackwell and this is Inside the Black Box.

Arshavir Blackwell, PhD

TL DR: I used a made up word from 1958 to test whether AI models have a real grammar rule for the past tense, then looked inside to find it. I found real generalization, but no clean rule. And a reader caught something that could have explained it away.

Arshavir Blackwell, PhD

Here is a claim I didn't expect to be making: a nonsense word invented for a 1958 psychology experiment can tell you whether an AI understands grammar, or is just a very good parrot.

Arshavir Blackwell, PhD

I ran that test on models up to nine billion parameters. Then I did the thing you can't do to a human brain: I reached inside and looked around for the rule itself. The answer fits neither textbook.

Arshavir Blackwell, PhD

One. The wug test. One experiment, sixty eight years old, is still the sharpest tool in the whole fight.

Arshavir Blackwell, PhD

In 1958, the psychologist Jean Berko showed children a drawing of a nonsense creature: "This is a wug. Now there are two of them. There are two...?" The kids said "wugs." They'd never heard the word. They couldn't be remembering it. So they had to be applying a rule.

Arshavir Blackwell, PhD

That's it. Made up words have zero frequency. If a model turns wug into wugged, it can't be looking anything up, it has to be generalizing.

Arshavir Blackwell, PhD

I gave the models a batch of invented verbs, wug, blick, dax, plim, gorp, in past tense sentences, and measured how often they reached for ed. Then I compared made up verbs against familiar ones like walk to walked, across sizes.

Arshavir Blackwell, PhD

Two. The gap that won't close. Bigger models get better at made up words. They never get good.

Arshavir Blackwell, PhD

Here's how five models did on made up verbs versus familiar ones, and the gap between them: Pythia 410M: 54 percent on made up words, 100 percent on familiar. Gap: 46 points. Pythia 1.4B: 53 percent vs. 96 percent. Gap: 43. Pythia 2.8B: 62 percent vs. 100 percent. Gap: 38. Gemma 2 2B: 58 percent vs. 98 percent. Gap: 40. Gemma 2 9B: 76 percent vs. 100 percent. Gap: 24.

Arshavir Blackwell, PhD

Even the nine billion parameter model handles a brand new word far worse (76 percent) than a word it has seen (100 percent). That gap shrinks with scale; but slowly, and it never closes.

Arshavir Blackwell, PhD

There is a real rule, but a thin one. The models do reach for ed on words they've never seen, and the biggest, Gemma 2 9B, does it most (76 percent). A genuine, productive habit is in there. They're not pure lookup tables.

Arshavir Blackwell, PhD

But it never closes the gap. A real ed rule would treat wug exactly like walk, near 100 percent on both. Instead novel verbs trail familiar ones by 24 to 46 points at every size, and ten times more parameters barely helps. Partial, graded, frequency hungry.

Arshavir Blackwell, PhD

This is what generalization looks like in a distributed system: a smeared habit that sweeps a novel word along with no discrete rule doing the sweeping. The incompleteness is the fingerprint.

Arshavir Blackwell, PhD

Three. "But isn't this just the tokenizer?" A reader named Dan (thanks Dan!) raised an objection: modern models don't read letters, they read chunks. Before a model ever starts learning, all its text gets sliced up into a fixed menu of pieces called tokens. A very common word is often a single token, grabbed whole; a longer or rarer word gets split into parts. So walked might arrive as one piece, like a single Lego brick, while jumped might come in as two: jump plus ed. And a word the model has never seen, like wug, isn't on the menu at all, so it gets shattered into scraps: w plus ug.

Arshavir Blackwell, PhD

Here's the catch. That ed piece is one of the pre made bricks, handed to the model before training even starts. So when a model tacks ed onto a verb, it isn't necessarily discovering the past tense rule. It might just be snapping on a block it was given for free. Which is Dan's question: are we watching the model learn grammar, or watching the tokenizer that was built before it learned anything at all?

Arshavir Blackwell, PhD

First, on my earlier "backwards" finding, the tokenizer isn't a problem; it's the explanation. A tokenizer can't fake the shape of a curve. It just explains why an AI learns in the reverse order from a child: a toddler builds ed from raw sound; the model is handed it as a Lego brick on day one. Two things stay out of reach: the decline (a pre installed token can't draw a curve that falls) and the missing hump (you can't fake the absence of a bump).

Arshavir Blackwell, PhD

Second, on the wug test, it's a genuine confound, and worth noting. Walked is often a single chunk the model grabs whole (memorization); wug shatters into fragments. So some of its penalty is tokenization, not rule. The generalization we see may be a shade better than the raw numbers, but 76 percent at nine billion is still nowhere near walk level.

Arshavir Blackwell, PhD

And here's the twist that keeps that confound from letting the model off the hook. You might object that a pre made ed is just a helpful leg up, and it is. But a leg up on producing the suffix isn't the same as learning when to attach it. The tokenizer hands over the raw material; the model still has to learn to snap it onto a word it has never seen, and that mapping, past tense slot plus unfamiliar stem yields add ed, is exactly what stays leaky. So being handed ed for free doesn't build the rule; it just supplies one ingredient. If anything it raises the bar: give a model the answer piece up front and watch it still fumble brand new words, and "there's no clean rule in here" gets stronger, not weaker.

Arshavir Blackwell, PhD

The clean fix? A model that reads one letter at a time, forced to build ed itself. That would be the logical follow up.

Arshavir Blackwell, PhD

This rhymes with 1986. Rumelhart and McClelland hand built their sound representation so morphology would be visible to their network. The tokenizer is that same choice, made by raw frequency, without anyone asking. Pinker and Prince spent 1988 arguing hand built representations quietly do the work. Nearly forty years later, that's Dan's objection in new clothes.

Arshavir Blackwell, PhD

Four. So I reached inside. The wug test watches what the model does. Interpretability lets me watch what it is. So I went looking for the rule directly.

Arshavir Blackwell, PhD

Is there a "rule switch"? I found the pattern that most cleanly responds to ed, in the biggest model, and cranked it. It did nothing. Zero. A random nudge of the same strength leaked more goed. Six ways, three sizes, two methods, same answer: no localizable switch produces overregularization.

Arshavir Blackwell, PhD

Thermometer, not furnace. Switch the ed pattern off and the model still writes past tenses fine, but gets worse at understanding them (small but real, about five standard deviations out). It reads the suffix. It doesn't make it.

Arshavir Blackwell, PhD

Is the distinction even in there? A readout for "regular vs. irregular" barely hit 62 percent, where a clean category would be 85 to 95 percent, and the signal lived in shallow layers. The model doesn't "know" go is an exception. It just remembers went.

Arshavir Blackwell, PhD

Two parts or one? I switched off each component one at a time. Nothing dissociated. It's one tangled system, not two routes.

Arshavir Blackwell, PhD

Five. The crack in the story. Every probe points the same way: no discrete rule in the machine.

Arshavir Blackwell, PhD

But... every one of those "no rule" results is a search result. I looked here and didn't find one. And absence of evidence isn't evidence of absence.

Arshavir Blackwell, PhD

So in Part 3, I stopped hand searching and let a gradient descent optimizer, the cleverest search there is, find the single best possible direction for producing goed.

Arshavir Blackwell, PhD

It worked. 100 percent. On words it had never seen. But we'll see the story is more complicated than that.

Arshavir Blackwell, PhD

Takeaway plus your 5 minute next step. The past tense was supposed to be the clean toy problem, simple enough to reveal the architecture of language. Reach inside, and there's no architecture to reveal: just overlapping habit, and a syllable the model grabs when it isn't sure.

Arshavir Blackwell, PhD

Try it in five minutes: open your favorite chatbot and feed it invented verbs, "Every day I frell. Yesterday I..." Watch whether it commits to ed, hesitates, or grabs a lookalike real word. You're running Berko's 1958 experiment on a machine. I'm Arshavir Blackwell, and this has been Inside the Black Box.