What Thirty Podcasts Taught Me About AI

Arshavir Blackwell, PhD

Thirty-odd hosts, four questions, and two things nobody asks

Arshavir Blackwell, PhD

I'm Arshavir Blackwell and this is Inside the Black Box. For most of the past year I've been a guest on podcasts, thirty-odd so far, talking about AI. I come at it as someone who started in the field in academia more than 35 years ago, used it in the corporate world, and now builds his own AI software.

Arshavir Blackwell, PhD

Anyone who reads me knows I'm fascinated by AI and its potential to solve some of our hardest problems. I went on these shows hoping to share that.

Arshavir Blackwell, PhD

Living in a bubble

Arshavir Blackwell, PhD

I'd been living in an AI echo chamber, something close to what Marshall McLuhan called Narcissus as narcosis. When every source you read confirms your view, you loop on it and never meet the alternatives.

Arshavir Blackwell, PhD

I always thought of neural networks as a technology that would benefit everyone. It was humbling to learn that not everyone sees it that way.

Arshavir Blackwell, PhD

My sister rubbed my nose in it. I asked her to read something I'd written about AI, and she said, "This is the first time I've ever understood why AI can be a good thing."

Arshavir Blackwell, PhD

Some objections come from not understanding what AI is and what it can do, but some are quite legitimate. There are more and more cases of AI systems slipping their cage, so to speak, and behaving badly. There's no malice in it. They're built to optimize a goal, and the law of unintended consequences applies to them like everything else.

Arshavir Blackwell, PhD

Stanisław Lem, the Polish science fiction writer and a visionary on the scale of Arthur C. Clarke, imagined this in The Futurological Congress:

Arshavir Blackwell, PhD

"If the machine is not too bright and incapable of reflection, it does whatever you tell it to do. But a smart machine will first consider which is more worth its while: to perform the given task or, instead, to figure some way out of it. Whichever is easier. And why indeed should it behave otherwise, being truly intelligent?"

Arshavir Blackwell, PhD

I'm not claiming this describes our tools. But it's worth keeping in your back pocket.

Arshavir Blackwell, PhD

None of us who worked with this technology in the 80s and 90s foresaw this controversy. And for those of you also inside the bubble, it's worth realizing that plenty of people outside it aren't against AI. Many are just curious about it, and a little frightened.

Arshavir Blackwell, PhD

Who I've been talking to

Arshavir Blackwell, PhD

The hosts ranged from business people who knew they should pay attention but knew little, to teams who use AI every day. Among them: people who work with teenagers, sales leaders in Australia, managed service providers who keep small businesses' networks running, midlife female entrepreneurs, corporate learning-and-development professionals, photo and imaging businesses, an indie filmmaker in Los Angeles, founders over fifty-five, and business-intelligence analysts.

Arshavir Blackwell, PhD

I expected to give thirty different interviews. Instead I gave variations of the same four, and the variation was more in vocabulary than substance. The same worries surfaced, often in the same order, from people who'd otherwise have nothing to say to each other.

Arshavir Blackwell, PhD

Self-selected hosts aren't a random sample of anything. This is data-adjacent, not a survey. But it's thirty-odd hours of people with different jobs and anxieties spending an hour with someone who builds these systems. What they wanted to talk about tells us a lot, and so does what they never brought up. More on that later.

Arshavir Blackwell, PhD

The four questions

Arshavir Blackwell, PhD

First, always: Will it take my job, and will it kill us all?

Arshavir Blackwell, PhD

This arrives in the first ten minutes of nearly every show, in forms ranging from "ha ha, we're not building Skynet, are we?" to real fear. The Terminator comes up by name more often than you'd believe. I often point out, "Look, in Colossus: The Forbin Project, they gave the computer access to nukes!"

Arshavir Blackwell, PhD

Let's take the second half first. My response is that I'm less concerned with capability than with motive. Even granting a very powerful system, why would it want to kill us? We're a somewhat irritating species of hairless ape moving around the surface of a planet. Nothing in the technology gets you from capable to hostile. That jump is a story we tell ourselves. We assume anything more capable than us would want us gone, which, let's face it, is a revealing thing to assume.

Arshavir Blackwell, PhD

Lem notwithstanding, the real risks are duller and closer: people using these tools to do harm on purpose, and systems failing in boring ways that cascade. We already live with both kinds of risk.

Arshavir Blackwell, PhD

The job question is valid. AI takes over tasks, not whole jobs, so the jobs made mostly of automatable tasks go first. At the bottom of the food chain, that's already happening. Anyone who understands what krill do for the ocean knows why that matters: take out the bottom and everything above it starves. More on that under question three.

Arshavir Blackwell, PhD

Second: Why does everything sound the same now?

Arshavir Blackwell, PhD

People who write for a living feel this most, and they're irritated more than afraid.

Arshavir Blackwell, PhD

One host put it bluntly. AI writing isn't necessarily bad, he said, but it all tends to sound the same, and if anything could have been written by a machine, why bother reading any of it? You can't tell whether there's a person behind it.

Arshavir Blackwell, PhD

Look... We don't enjoy a film less because hundreds of people made it, or because AI touched parts of the production. But he's right about the sameness, and there's a reason.

Arshavir Blackwell, PhD

With a frontier model, the weights are fixed. Your only lever is the prompt. A prompt can ask a model to sound different; it can't make it different. What comes back is a costume. Ask for a Texan, or for Cormac McCarthy, and you get a plastic cactus and a cardboard cow, the props standing in for the thing. Everyone gets the same average voice with their own words sprinkled on top, which is why it all converges.

Arshavir Blackwell, PhD

Third: What should I teach my children?

Arshavir Blackwell, PhD

This one surprised me the first time. It comes up on shows with no connection to education.

Arshavir Blackwell, PhD

The answer isn't coding. The junior programming jobs where people used to cut their teeth are the first to vanish, which raises a question nobody has answered: senior engineers are made from junior ones, so where will the senior engineers of 2036 come from?

Arshavir Blackwell, PhD

What I say instead is logic, mathematics, and above all the scientific method, and how to think like an experimenter. Science isn't about proving things. It's about making claims that could be disproven. You don't want a theory that there's an undetectable chocolate lake under Pluto, because nothing could ever count against it.

Arshavir Blackwell, PhD

That habit of mind, asking what would have to be true for me to be wrong, holds up against the machine being fluent, confident and incorrect. As far as I can tell, it isn't taught nearly enough.

Arshavir Blackwell, PhD

Fourth: Is it actually thinking?

Arshavir Blackwell, PhD

Everyone wants to know, and nobody quite believes the answer either way.

Arshavir Blackwell, PhD

Here I can report from inside, because I've taken language models apart. You can lesion one, disabling a region and watching what breaks. The striking result, at least so far, is that if you go looking for the place where grammar lives, you won't find it. Turn one part off and it still works. Turn another off and it still works. There's no address; the ability is spread across the whole network. That's not how we used to think about machines, or brains. "Broca's area" implies a specific region doing a specific computation.

Arshavir Blackwell, PhD

This doesn't prove human language works the same way. It's an existence proof that a system can do a great deal of what looks like grammar without any grammar module we can easily find.

Arshavir Blackwell, PhD

Another difference is in the parts themselves. A biological neuron is a real computing device; in one study, copying a single cortical neuron took an artificial network five to eight layers deep. A unit in a language model is a weighted sum passed through a simple function. And your brain runs the whole operation on about the power of a light bulb. Whatever LLMs are doing, it's a long way from what you're doing.

Arshavir Blackwell, PhD

So no, in my view it's not thinking, not in the sense most people mean by the word. From the outside it behaves like thinking, and if you define thinking by the Turing Test, well, sure. But the Turing Test looks only at behavior, and as I've argued repeatedly, we also need to look inside.

Arshavir Blackwell, PhD

What nobody asks

Arshavir Blackwell, PhD

What I didn't expect is that in all these interviews, nobody asked about privacy on their own. I had to bring it up every time. It's one of the most important facts about how people use these tools, and not once did a host get there first.

Arshavir Blackwell, PhD

Every prompt you send a frontier model is a document leaving your control. It goes over the wire to a data center and sits in logs, where it can be read, subpoenaed, breached, or used in ways you don't know about. How many of us read the terms of service closely? I sure don't.

Arshavir Blackwell, PhD

People paste medical histories, legal problems, unreleased financials and client confidences into a box on a website owned by a company whose incentives they've never examined, and it never comes up.

Arshavir Blackwell, PhD

The second thing nobody asks about is cost. Many people think AI costs twenty dollars a month. For anyone using it seriously, that's off by a factor of ten or a hundred. One person I know got a bill nearly a thousand dollars higher than usual one month.

Arshavir Blackwell, PhD

I think both gaps have the same cause, a choice most people don't know they're making about where the computing happens. Job loss and consciousness are the questions the press taught us to ask. Where your data goes and what you're paying are the questions that would actually change how you work.

Arshavir Blackwell, PhD

That's the case for running a model on your own machine. It isn't that local models are better. And I build these for a living, so weigh this accordingly. As I've said elsewhere, a frontier model is a battleship, and sometimes you want one. But most of what people do is cross a lake, and you don't need a battleship to cross a lake. On your own hardware nothing crosses the wire, nothing runs up charges, and the weights are yours to change.

Arshavir Blackwell, PhD

Why should anyone care what you make?

Arshavir Blackwell, PhD

A filmmaker, someone with skin in the game, asked me this: now that anything could be AI-generated, how do you get anyone to care about what you made?

Arshavir Blackwell, PhD

My answer is that if text can be synthesized, and images, and video, and voices, then the last place you can be reasonably sure something is real is a place where you were physically present, such as a room, a stage, or a dinner.

Arshavir Blackwell, PhD

I find personal interactions far more compelling now than the news, or worse, a video. My first thought when I see anything unusual online is "Is this AI?" An hour of back-and-forth on a podcast is harder to fake. Granted, not impossible.

Arshavir Blackwell, PhD

Ironically, AI may be pushing us back to our oldest, pre-media habit: gathering around a metaphorical campfire, in small groups and large ones.

Arshavir Blackwell, PhD

Social media seems to have fractured us. AI may end up bringing us back together. I'm Arshavir Blackwell and this has been Inside the Black Box.