Hi I’m a four-year-old child learning 8 new words every day.
So help me out: What’s a chef?
Uhh…a chef is someone who cooks food.
Oh really?
What about sushi chefs?
I thought you said sushi was uncooked fish!
Does that mean there’s no such thing as sushi chefs??
Okay, Round 2:
Maybe a chef is someone who prepares food for us to eat.
And cooking is only one part of that.
Preparing food includes tons of things like:
planning how foods combine and courses lead into one another
washing food
seasoning food properly
plating it attractively
This isn’t just bizarro pageantry meant to impress fancypants food critics.
It’s all just part of what it means to prepare food at a high level.
You know who cooks food? Cooks!
One thing we do in philosophy—not the only thing, but ONE Thing!—is to try to understand how we use language better.
1. We use words to communicate ideas.
First of all, that’s wild.
How do we learn words?
How do words mean anything!
This definitely shouldn’t work as well as it does.
2. The words we pick and how we put them together can really matter.
As we saw, shifting our focus from cooking food to preparing it opened up a whole new world of possibilities. It increased what we could recognize and appreciate as valuable or worthwhile, whether we’re preparing the food or just eating it!
We didn’t realize that that kind of structure was already embedded in the language we use every day, that cooking food was only part of being a chef. We just found out by doing a couple minutes of philosophy, and we only started because a four-year-old asked.
So just how much unrecognized complexity might be lurking within our language?
It’s tough to get a handle on how we use language. Linguists tell us to prioritize descriptive work (by studying how people actually talk) over prescriptive work (like memorizing how grammar books formalize and freeze rules in idealized time to separate who’s speaking correctly from who’s speaking incorrectly).
Honestly, that’s great advice!
Still, philosophers agree we might need to change our language where we find it confused or confusing or harmful or otherwise inadequate to our purposes.
And where we don’t change our language, we might need to change our behavior. I’m convinced there are tons of subtle distinctions we’re already committed to but maybe haven’t noticed or taken seriously yet.
But now I’m gonna try to unravel this thread.
I’ve been called a good cook by several folks recently—yes, that is the point of this article, thanks for noticing. See you next week everyone! I take that as a tremendous compliment given that I can’t eat anything with my chronic illnesses and can’t buy anything with my grad school wages.
These folks (plural) don’t just mean I’m good at cooking food—in fact, the element of timing is one of my weakest skills. They call me a good cook to mean that I’ve prepared the food well, and they appreciate it.
So sometimes it’s worth observing all these technicalities of language, and sometimes not.
For example, did you know flavor involves way more than just taste?
[Flavor] is actually a combination of smell, taste, spiciness, temperature and texture. —The Taste and Smell Clinic at UConn Health, on their webpage titled “Facts”
Cooking with flavor involves doing more than just cooking food or even preparing food to taste good. And again, that increased level of articulation gives you a lot more nobs and tools to understand what you’re doing, imagine new ways of doing it better, and start experimenting.
We’re able to talk in multiple ways. We can tighten up words’ meanings together like philosophers, or we can loosen up and call me a chef whose food tastes good. (Again, thanks.)
So a lot of how we use language is determined by the specific context of each conversation that we weave together as we speak. This blog probably tends to be a bit tighter than most things you probably read but a lot looser than most analytic philosophy.
INTERLUDE: Stephen Wolfram with a slightly more plausible version of a weird argument I’ve heard butchered several times. So how is it, then, that something like ChatGPT can get as far as it does with language? The basic answer, I think, is that language is at a fundamental level somehow simpler than it seems. And this means that ChatGPT—even with its ultimately straightforward neural net structure—is successfully able to “capture the essence” of human language and the thinking behind it. And moreover, in its training, ChatGPT has somehow “implicitly discovered” whatever regularities in language (and thinking) make this possible. —Stephen Wolfram
So Stephen Wolfram's big thing is the study of complexity. He was a published particle physicist as a teenager and got his PhD at 20 studying with Feynman.
Then he quit particle physics and started studying this relatively new topic in mathematics called cellular automata:
Each step, every cell in that grid either lives (goes black) or dies (goes white) based on the reapplication of an algorithm whose rules I’m about to crib from Wikipedia:
Any live cell with fewer than two live neighbours dies, as if by underpopulation.
Any live cell with two or three live neighbours lives on to the next generation.
Any live cell with more than three live neighbours dies, as if by overpopulation.
Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.
So what happens if you just apply that algorithm over and over again to see what happens each step? The answer is, all kinds of fascinatingly complex patterns like the Gosper glider gun pictured above somehow emerge from these very simple rules.
Wolfram wanted to study complexity in the simplest context he could, so he moved from two dimensions to one dimension—a line. And then he started programming these new calculating machines called computers to crank out thousands of lines of this stuff, just to see what would happen next.
So let’s start with 1 cell in the middle of nothing.
That’s row 1:
…⬜⬜⬜⬜⬛⬜⬜⬜⬜…
So what happens the next step, on row 2?
Come on, I’ll only bully you with emojis for a few lines.
Each cell only has two neighbors to compare with itself: one on the left, and one on the right.
If all three are the same color, you’re dead.
…⬜⬜⬜⬜⬛⬜⬜⬜⬜… row 1
…⬜⬜⬜❓❓❓⬜⬜⬜… row 2
If only one of you three is alive, you’re alive.
…⬜⬜⬜⬜⬛⬜⬜⬜⬜… row 1
…⬜⬜⬜⬛⬛⬛⬜⬜⬜… row 2
(Again: If all three are the same color, you’re dead.)
…⬜⬜⬜⬜⬛⬜⬜⬜⬜… row 1
…⬜⬜⬜⬛⬛⬛⬜⬜⬜… row 2
…⬜⬜❓❓⬜❓❓⬜⬜… row 3
(Again: If only one of you three is alive, you’re alive.)
…⬜⬜⬜⬜⬛⬜⬜⬜⬜… row 1
…⬜⬜⬜⬛⬛⬛⬜⬜⬜… row 2
…⬜⬜⬛❓⬜❓⬛⬜⬜… row 3
BUT if exactly two of you are alive, you’d better hope it’s you and your buddy to the right or else you’re dead.
…⬜⬜⬜⬜⬛⬜⬜⬜⬜… row 1
…⬜⬜⬜⬛⬛⬛⬜⬜⬜… row 2
…⬜⬜⬛⬛⬜⬜⬛⬜⬜… row 3
Let’s do a few more lines…
And just a few more…
And that’s Rule 30 everyone!
Some of these rules are truly beautiful.
Those were all open source images from Wikipedia, but CHECK OUT this incredibly gorgeous sequence from Wolfram’s weird book that I’m terrified to host on my website.
(Wolfram once sued his own graduate research assistant to block publication of a paper for several years until this very book had come out!)
You might think you’d get way more complexity if you added more colors or more dimensions, but Wolfram spends quite a few pages arguing that it doesn’t seem to make much difference. Complexity is there from the very start.
Wolfram thinks complexity is eager to emerge from simplicity, and as you can imagine, he sees all kinds of Earth-shattering implications.
Hell, here’s the opening paragraph of Wolfram’s book, which again, is quite weird:
Three centuries ago science was transformed by the dramatic new idea that rules based on mathematical equations could be used to describe the natural world. My purpose in this book is to initiate another such transformation, and to introduce a new kind of science that is based on the much more general types of rules that can be embodied in simple computer programs. —If you can’t handle Stephen Wolfram at his page 1, you don’t deserve Stephen Wolfram at his page 1280.
That’s almost Napoleonic in scope and ambition! Kudos.
So now Wolfram’s thinking about how we use language. As we’ve seen, he’s a leading expert on how complexity can emerge from even simple computations. And he goes, well...
The success of ChatGPT is, I think, giving us evidence of a fundamental and important piece of science: it’s suggesting that we can expect there to be major new “laws of language”—and effectively “laws of thought”—out there to discover. In ChatGPT—built as it is as a neural net—those laws are at best implicit. But if we could somehow make the laws explicit, there’s the potential to do the kinds of things ChatGPT does in vastly more direct, efficient—and transparent—ways. —Stephen Wolfram in one of the best introductions to ChatGPT that I do genuinely recommend.
Zoinks why does everyone talking about AI end up sounding like an anime supervillain? I’m tired of people assuming I’m a huckster riding this AI hype train I profoundly distrust.
So…maybe ChatGPT hints at the possibility of a future Asmovian psychohistory which could make us more direct and efficient than ever. (And ”transparent” is hard not to read in a way that vitiates privacy.) The hope sounds oddly familiar, like an authoritarian bureaucrat eager to plan a new, rational society where everything’s been optimized to be better than ever.
I myself am more convinced that we don’t quite know what simple and complex mean well enough to draw these kinds of conclusions about all the rest of language and human psychology and oh my God there’s a whole SEP article on cellular automata?
In any event, why are we only supposed to draw conclusions about the structure of language and thought?
If this kind of argument works, why shouldn’t we draw conclusions about the nature of ourselves?
After all, we know that brains are made of neurons, which connect with one another in large networks that send and receive electric signals. Neural nets are literally designed to mimic that basic structure.
Admittedly, there are some differences.
Your brain has 100 trillion connections.
ChatGPT has 175 billion weights.
So with about 1000x fewer parts, ChatGPT can mimic the language-producing output of your brain by going just one word at a time.
I guess that’s a pretty mind-blowing achievement.
But how much does it teach us about language?
After all, there’s a real difference between
writing with understanding, and
successfully predicting the next word
And that means there’s some real sleight of hand going on here.
ChatGPT’s “understands” human language in the same way that Deep Blue or Stockfish “understands” chess.
Chess engines are very good at sitting there and calculating out the best next move by just generating more and more and more future moves to evaluate. Eventually, an engine that sees 9 or 12 or 25 moves ahead is going to efficiently outmaneuver us puny humans.
But chess engines are just predicting future moves and countermoves. They don’t understand, I don’t know, the irony of checkmating with a pawn.
(And even if you trained a chess engine to prioritize checkmating with pawns, that still wouldn’t be happening because it understood irony! It would be happening because it understood how to predict moves that lead to checkmating with a pawn.)
So the only notion of understanding at work here is just brute calculative prediction.
Similarly, ChatGPT is calculating the most likely next word given all the words before it:
Happy Birthday to you. Happy Birthday to ___.
Most likely word is the analog of best chess move for the machine infamously trained by scraping the Internet. So yeah, what’s the Internet most likely to say next?
(By the way, apparently PhilPapers is 2.6% of its volume of training data? Genuine congrats to Dave Chalmers, this is analytic philosophy’s 3rd greatest causal contribution to the world right after spreading veganism and effective altruism. It’s been a real mixed bag.)
So that’s how LLMs work—almost. If you always take the most likely word, the poor thing just loops and repeats itself and sputters out. In the most machine-learning twist ever, you have to add in randomness for reasons we don’t really understand and then it sounds way more human. A little fiddling around suggests a “temperature rating” around 0.8 works, ya know, pretty convincingly, but interpreting what that actually means is far from straightforward.
So how do we evaluate a claim like Wolfram’s?
Here it is again:
So how is it, then, that something like ChatGPT can get as far as it does with language? The basic answer, I think, is that language is at a fundamental level somehow simpler than it seems. And this means that ChatGPT—even with its ultimately straightforward neural net structure—is successfully able to “capture the essence” of human language and the thinking behind it. And moreover, in its training, ChatGPT has somehow “implicitly discovered” whatever regularities in language (and thinking) make this possible.
Claim 1: Language is at a fundamental level somehow simpler than it seems.
I don’t know, how simple does language seem? Toddlers seem pretty good at learning it.
You’re telling me that with a big enough model, you can mathematically predict the next word given all the previous ones, scramble it a bit so you’re just choosing one of the top probabilities, and now it sounds like us?
I mean, it’s wild that this procedure works at all. (It’s of course worth noting the fact everyone cites: that LLMs need to soak up orders of magnitude more text to learn language than babies do.)
But the complexity required to autocomplete the next word may be different from the complexity involved in understanding or using language as a system. As we’ve seen, language is a rich, open-ended framework for sharing and shaping communicative contexts together, adjusting as we go to preserve and build understanding. (That’s how I can ask, was that last sentence a bit too tight?) So maybe we’ve just learned it’s simpler to fake understanding of language than it seems by merely predicting which word (I mean token) comes next?
With that much data, if you couldn’t detect statistical regularities in language well enough to plausibly fill in gaps, I’d be much more _________.
Claim 2: ChatGPT—even with its ultimately straightforward neural net structure—is successfully able to “capture the essence” of human language...
Without the scare quotes I’d know this was false but now I don’t know what it means.
Claim 2.5: …and the thinking behind it.
As in, ChatGPT captures the essence of the thinking behind human language? That doesn’t seem right.
ChatGPT can generate text. But what it generates is almost certainly not the result of anything like the human thinking behind human language. ChatGPT doesn’t understand what words mean! It encodes words (sorry, tokens) as vectors in high-dimensional space so it can put them in statistically plausible sequences, and then randomly selects the next word from a weighted list of predictions. What’s that the essence of—which words follow which words? Is that all you think there is to language? Surely not, even if the ability to predict the next word shows one kind of (partial) mastery. Wolfram’s a sharp guy, so there must be more going on here...
Claim 3. In its training, ChatGPT has somehow “implicitly discovered” whatever regularities in language (and thinking) make this possible.” Why would we think that ChatGPT has stumbled across the same regularities in language that make our thinking about and use of language possible?
ChatGPT has identified tons of statistical regularities and semantic associations that we aren’t consciously tracking. So are we just unconsciously tracking them? (Big shout out to the NBA Name Predictor neuron I just linked.)
But also, given underdetermination there are zillions of ways ChatGPT’s weights might have turned out that would have worked comparably well.
Are we sure that all those ways would be carving up the same regularities of language and thinking that make our human use of language and thinking possible? Just because they all attack the same weird problem of predicting word n+1 given words 1 through n?
Is that how we produce language? Are we plodding along exactly one word (token) at a time and don’t realize it?
I don’t know, maybe!
In any event, I don’t yet see why this follows.
So I don’t know what ChatGPT teaches us about language. Probably the same sort of thing Deep Blue taught us about chess: humility. If you prize the expert performance of this technical activity as the pinnacle of what makes you human and special, some nerds will train a computer to become a pretty competent predictor that’s actually way better than you at composing lyrical sonnets in five seconds flat and spotting mate in 15.
Well are you biologically related to Monet or Mozart????? We are the same species!!! Hm I immediately regret bringing this up and don’t know why I thought it was such a gotcha.
EPILOGUE: John H. Holland embeds quantitative utility functions into the very notion of learning from experience. Pour one out for Philosopher Kings. As an agent accumulates experience, it will find that some of its rules are rarely, or never, useful. If a frog flees from all moving objects it will rarely eat or mate. In order to adapt, an agent requires a means of assigning a quantity, usually called strength, that rates the usefulness of different rules in helping the agent to attain important resources, such as food, shelter, and the like.
In the end, like most things I’ve been writing the past six months, this dispute comes down to whether the ability to predict behavior exhausts everything we care about in understanding a phenomenon deeply.
In other words, even if economics did work as promised as a descriptive, quantitative science, how much would it tell us prescriptively about how we ought to live?
After all, I’ve maximized my utility function! This is what rationality demands, everyone! I promise I didn’t cut out all the interesting qualitative stuff to make that question more mathematically tractable.
Whatever happened to Philosopher Kings? We used to count Jesus Christ among our ranks. Now with TNT losing its NBA deal, we won’t even have the Big Aristotle.
We’ve come to care about prediction to the detriment of all else so much that we flooded sports media with legal betting lines and lost Inside the NBA—the show we love because no one knows what’s going on until it happens.
Shame on us, the Last Men, the feeble prediction-outsourcers who sit back and let fancy autocomplete write our emails and DraftKings write our ESPN headlines.
Comments