رونوشت 🔗
WEBVTT
00:00:00.400 --> 00:00:09.009
Welcome to Sleepy Facts about the Universe. A
00:00:12.980 --> 00:00:15.439
calculator reliably produces correct answers
00:00:15.439 --> 00:00:18.100
without anyone believing it understands mathematics.
00:00:18.640 --> 00:00:22.710
A crow bends a wire into a hook to solve an unfamiliar
00:00:22.710 --> 00:00:26.149
problem, leaving us uncertain about what it truly
00:00:26.149 --> 00:00:30.940
grasps. Today, large language models write coherent
00:00:30.940 --> 00:00:34.679
essays and pass professional exams. yet they
00:00:34.679 --> 00:00:37.560
lack the physical and social experiences that
00:00:37.560 --> 00:00:40.700
traditionally give words their meaning. This
00:00:40.700 --> 00:00:43.299
episode explores the complex boundary between
00:00:43.299 --> 00:00:46.380
producing correct output and actually understanding
00:00:46.380 --> 00:00:50.200
what one is doing. The investigation moves from
00:00:50.200 --> 00:00:53.560
simple arithmetic tools to animal cognition and
00:00:53.560 --> 00:00:57.579
finally to modern artificial intelligence. Along
00:00:57.579 --> 00:01:00.100
the way, it examines the candidate's signs of
00:01:00.100 --> 00:01:02.990
comprehension. such as the ability to transfer
00:01:02.990 --> 00:01:06.209
knowledge to new situations, explain reasoning,
00:01:06.430 --> 00:01:09.909
and detect contradictions. It also considers
00:01:09.909 --> 00:01:13.109
the symbol grounding problem, John Sell's Chinese
00:01:13.109 --> 00:01:16.170
Room, and the fascinating reality that human
00:01:16.170 --> 00:01:19.109
minds frequently confabulate false explanations,
00:01:19.650 --> 00:01:22.530
suggesting that understanding itself might be
00:01:22.530 --> 00:01:25.810
a graded spectrum, rather a simple yes or no
00:01:25.810 --> 00:01:29.620
property. I kept returning to one question while
00:01:29.620 --> 00:01:32.959
working on this episode. If our own comprehension
00:01:32.959 --> 00:01:37.120
is uneven, and sometimes fabricated, how do we
00:01:37.120 --> 00:01:40.069
objectively measure it in a machine? I wanted
00:01:40.069 --> 00:01:42.469
to understand why the gap between behavioral
00:01:42.469 --> 00:01:45.510
performance and internal comprehension is so
00:01:45.510 --> 00:01:49.109
difficult to define, and why our initial intuitions
00:01:49.109 --> 00:01:51.430
about what it means to understand are likely
00:01:51.430 --> 00:01:58.280
far too simple. A calculator gives you 56 every
00:01:58.280 --> 00:02:01.239
time, and no one thinks it understands multiplication.
00:02:03.079 --> 00:02:06.260
A crow bends wire into a hook to solve a problem
00:02:06.260 --> 00:02:09.610
it has never seen before, and we hesitate for
00:02:09.610 --> 00:02:13.409
different reasons. The question this episode
00:02:13.409 --> 00:02:16.870
pursues is what, if anything, separates reliable
00:02:16.870 --> 00:02:20.759
performance from genuine comprehension. It also
00:02:20.759 --> 00:02:23.759
asks whether the systems we are now building,
00:02:23.919 --> 00:02:28.039
which speak like experts and fail like confubilators,
00:02:28.240 --> 00:02:31.500
have crossed a line we cannot yet clearly describe.
00:02:32.689 --> 00:02:35.849
Imagine a cheap plastic calculator sitting on
00:02:35.849 --> 00:02:39.610
a kitchen table. You press the buttons for 7,
00:02:39.789 --> 00:02:44.080
then multiplication, then 8. The small screen
00:02:44.080 --> 00:02:50.620
shows 56. Instantly. Flawlessly. Every time.
00:02:51.219 --> 00:02:54.199
Yet no one believes this calculator understands
00:02:54.199 --> 00:02:58.039
commerce or quantity or mathematics, or that
00:02:58.039 --> 00:03:02.919
56 is larger than 55. It produces correct answers
00:03:02.919 --> 00:03:07.069
with zero comprehension. This is our starting
00:03:07.069 --> 00:03:10.879
point. It is not because calculators are interesting
00:03:10.879 --> 00:03:13.759
in themselves, but because they establish something
00:03:13.759 --> 00:03:17.310
we will need throughout this episode. Correct
00:03:17.310 --> 00:03:20.090
output alone does not constitute understanding.
00:03:21.110 --> 00:03:24.889
The puzzle is what else might be required, and
00:03:25.180 --> 00:03:27.379
whether we can ever know when that requirement
00:03:27.379 --> 00:03:32.409
has been met. The question sounds simple. It
00:03:32.409 --> 00:03:36.349
is not. We are asking when successful pattern
00:03:36.349 --> 00:03:39.169
use becomes understanding rather than merely
00:03:39.169 --> 00:03:43.169
sophisticated response. This matters for how
00:03:43.169 --> 00:03:46.629
we interpret animal cognition, evaluate artificial
00:03:46.629 --> 00:03:50.469
intelligence, understand human development, and
00:03:50.469 --> 00:03:53.110
design experiments that can meaningfully distinguish
00:03:53.110 --> 00:03:57.069
performance from comprehension. The stakes are
00:03:57.069 --> 00:04:00.479
not merely philosophical. They shape what we
00:04:00.479 --> 00:04:04.020
expect from systems we build, creatures we study.
00:04:04.629 --> 00:04:09.080
and even explanations we give ourselves. To see
00:04:09.080 --> 00:04:12.120
why the calculator is only a beginning, consider
00:04:12.120 --> 00:04:14.879
what happens when we move from plastic and silicon
00:04:14.879 --> 00:04:18.160
to creatures with brains and evolutionary histories.
00:04:19.149 --> 00:04:22.250
Animals can learn to respond to cues with remarkable
00:04:22.250 --> 00:04:26.329
precision. Picture a rat in a small experimental
00:04:26.329 --> 00:04:30.189
chamber, its whiskers twitching, the metal floor
00:04:30.189 --> 00:04:32.810
faintly vibrating from some distant machine.
00:04:34.060 --> 00:04:38.610
A light flashes above the lever. The rat presses.
00:04:39.560 --> 00:04:42.860
A food pellet drops into the tray with a small
00:04:42.860 --> 00:04:48.980
mechanical clatter. The rat eats. Again and again,
00:04:49.279 --> 00:04:53.949
the light, the press,
Load full transcript
the pellet. The behavior
00:04:53.949 --> 00:04:57.250
is reliable, shaped by reinforcement, physically
00:04:57.250 --> 00:05:01.019
real. Yet we hesitate to say the rat understands
00:05:01.019 --> 00:05:03.920
the relationship between light and lever in any
00:05:03.920 --> 00:05:08.740
rich sense. It has learned a pattern. In this
00:05:08.740 --> 00:05:11.240
situation, this action produces this outcome.
00:05:12.459 --> 00:05:17.050
A dog sits on command. A pigeon picks a colored
00:05:17.050 --> 00:05:20.470
disc in exchange for grain. These are correct
00:05:20.470 --> 00:05:24.920
responses, often impressively reliable. Whether
00:05:24.920 --> 00:05:27.480
comprehension accompanies them is genuinely difficult
00:05:27.480 --> 00:05:31.360
to determine. This difficulty deepens when we
00:05:31.360 --> 00:05:33.939
encounter animals doing things that look more
00:05:33.939 --> 00:05:38.250
like genuine understanding. New Caledonian crows
00:05:38.250 --> 00:05:42.069
bend wires into hooks to retrieve food from otherwise
00:05:42.069 --> 00:05:46.519
inaccessible locations. Researchers present a
00:05:46.519 --> 00:05:49.339
crow with a straight piece of wire and a small
00:05:49.339 --> 00:05:52.000
bucket of food sitting at the bottom of a transparent
00:05:52.000 --> 00:05:55.600
vertical tube too deep for the crow's beak to
00:05:55.600 --> 00:05:59.389
reach. The crow picks up the wire in its beak,
00:05:59.589 --> 00:06:02.569
holds it against the floor or side of the cage
00:06:02.569 --> 00:06:05.889
with one foot, and pulls the free end to create
00:06:05.889 --> 00:06:10.060
a hook. It then inserts the hooked end into the
00:06:10.060 --> 00:06:14.639
tube, lifts the bucket, and eats. They do not
00:06:14.639 --> 00:06:18.480
merely repeat a fixed action. They modify the
00:06:18.480 --> 00:06:22.399
tool to fit novel circumstances. This suggests
00:06:22.399 --> 00:06:26.519
something beyond cue following. It suggests something
00:06:26.519 --> 00:06:29.980
like an internal model of the problem. A representation
00:06:29.980 --> 00:06:32.519
of how the world works that can be manipulated
00:06:32.519 --> 00:06:36.949
and applied flexibly. Yet even here, caution
00:06:36.949 --> 00:06:40.259
is warranted. The crow's impressive behavior
00:06:40.259 --> 00:06:42.860
might still be a very sophisticated pattern,
00:06:43.100 --> 00:06:46.139
one shaped by evolution and individual learning
00:06:46.139 --> 00:06:48.819
to produce reliable outcomes without the kind
00:06:48.819 --> 00:06:50.920
of comprehension we are struggling to define.
00:06:51.790 --> 00:06:54.790
The boundary between learned response and understanding
00:06:54.790 --> 00:06:57.730
is not clean even in creatures that clearly possess
00:06:57.730 --> 00:07:01.149
nervous systems, sensory experience, and evolutionary
00:07:01.149 --> 00:07:05.500
continuity with our own cognition. This uncertainty
00:07:05.500 --> 00:07:08.060
about animals prepares us for the most
challenging
00:07:08.060 --> 00:07:11.540
case. one that has emerged only in recent years
00:07:11.540 --> 00:07:14.279
and that makes the calculator and the crow look
00:07:14.279 --> 00:07:18.189
simple by comparison. Large language models generate
00:07:18.189 --> 00:07:21.189
text that is coherent, contextually appropriate,
00:07:21.689 --> 00:07:24.430
often knowledgeable and frequently persuasive.
00:07:24.990 --> 00:07:28.069
You might ask one to explain why a lunar eclipse
00:07:28.069 --> 00:07:31.389
occurs and receive a paragraph describing the
00:07:31.389 --> 00:07:35.029
alignment of Sun, Earth and Moon, the geometry
00:07:35.029 --> 00:07:38.410
of shadows, the difference between umbra and
00:07:38.410 --> 00:07:42.790
penumbra, all in clear grammatical prose. They
00:07:43.240 --> 00:07:45.980
pass professional examinations in law and medicine.
00:07:46.660 --> 00:07:49.660
They write essays that receive passing grades
00:07:49.660 --> 00:07:51.959
from instructors who do not know their origin.
00:07:52.649 --> 00:07:56.529
They translate between languages, summarize complex
00:07:56.529 --> 00:07:59.410
documents, and engage in extended conversation.
00:08:00.009 --> 00:08:04.189
The performance is so sophisticated that it strongly
00:08:04.189 --> 00:08:08.110
tempts us to attribute understanding. Yet these
00:08:08.110 --> 00:08:10.689
systems learn through statistical pattern matching
00:08:10.689 --> 00:08:13.930
over enormous quantities of text without direct
00:08:13.930 --> 00:08:16.329
perceptual grounding in the world their words
00:08:16.329 --> 00:08:20.420
supposedly describe. They have no body that burns
00:08:20.420 --> 00:08:24.339
when touching fire, no eyes that see red, no
00:08:24.339 --> 00:08:28.300
social experience of fairness or betrayal. They
00:08:28.300 --> 00:08:31.360
process symbols without the embodied, situated,
00:08:31.779 --> 00:08:34.379
consequential relationships that seem to give
00:08:34.379 --> 00:08:37.879
human words their meaning. This is where the
00:08:37.879 --> 00:08:40.700
central puzzle of our episode becomes most acute.
00:08:41.649 --> 00:08:44.549
What would it take to say such a system understands?
00:08:45.610 --> 00:08:49.210
We need candidate signs and we need to examine
00:08:49.210 --> 00:08:53.789
each with care. One candidate is transfer, the
00:08:53.789 --> 00:08:56.710
ability to apply knowledge in unfamiliar settings.
00:08:57.169 --> 00:09:00.940
A student who memorizes multiplication tables
00:09:00.940 --> 00:09:03.940
can recite them but struggles with word problems
00:09:03.940 --> 00:09:08.000
requiring flexible application. A student who
00:09:08.000 --> 00:09:10.559
understands multiplication recognizes when to
00:09:10.559 --> 00:09:14.899
deploy it in novel circumstances. Yet transfer
00:09:14.899 --> 00:09:19.389
is imperfect as a test. Memorized behavior can
00:09:19.389 --> 00:09:21.990
imitate transfer when the new situation resembles
00:09:21.990 -->
00:09:26.330
training examples closely enough. Conversely,
00:09:26.370 --> 00:09:29.309
genuine understanding may fail when the examiner
00:09:29.309 --> 00:09:32.590
presents material in an alien format that disrupts
00:09:32.590 --> 00:09:35.350
recognition without touching underlying competence.
00:09:36.820 --> 00:09:39.460
The student who understands physics may freeze
00:09:39.460 --> 00:09:41.679
when asked to solve problems in an unfamiliar
00:09:41.679 --> 00:09:46.000
notation. Transfer helps distinguish understanding
00:09:46.000 --> 00:09:49.340
from mere memorization, but it does not provide
00:09:49.340 --> 00:09:53.399
a definitive boundary. Another candidate is explanation.
00:09:53.740 --> 00:09:57.039
The ability to articulate why something is true,
00:09:57.179 --> 00:10:01.399
not merely that it is true. Language models often
00:10:01.399 --> 00:10:05.559
generate impressive explanations. Yet here we
00:10:05.559 --> 00:10:08.299
encounter a problem. that will become central
00:10:08.299 --> 00:10:12.139
these systems can produce explanations that are
00:10:12.139 --> 00:10:16.440
coherent plausible and wrong they confabulate
00:10:16.440 --> 00:10:20.350
they generate confident accounts of things that
00:10:20.350 --> 00:10:23.610
did not happen citations to papers that do not
00:10:23.610 --> 00:10:27.750
exist historical claims that are fabricated you
00:10:28.000 --> 00:10:31.059
might receive a citation to something like johnson
00:10:31.059 --> 00:10:35.879
rt 2017 cognitive frameworks in early modern
00:10:35.879 --> 00:10:40.429
trade Journal of Historical Economics 14 -3245
00:10:40.429 --> 00:10:45.830
-267. It is a plausible sounding author. A plausible
00:10:45.830 --> 00:10:50.029
journal. A plausible year. All assembled into
00:10:50.029 --> 00:10:52.330
something that looks real but does not exist.
00:10:53.250 --> 00:10:58.370
This is not random error. It is structured, persuasive,
00:10:58.389 --> 00:11:02.440
often superficially convincing falsehood. The
00:11:02.440 --> 00:11:04.820
explanation candidate thus becomes complicated.
00:11:06.120 --> 00:11:08.799
Genuine understanding can produce explanation,
00:11:09.139 --> 00:11:12.059
but so can sophisticated pattern matching that
00:11:12.059 --> 00:11:14.500
assembles plausible -sounding sentences without
00:11:14.500 --> 00:11:18.090
grounded connection to reality. The presence
00:11:18.090 --> 00:11:20.450
of explanation does not guarantee understanding.
00:11:21.450 --> 00:11:25.330
Its absence, in alien formats, does not guarantee
00:11:25.330 --> 00:11:29.110
the absence of understanding. Contradiction detection
00:11:29.110 --> 00:11:33.480
offers another candidate. A system that understands
00:11:33.480 --> 00:11:36.970
should notice when its claims conflict. should
00:11:36.970 --> 00:11:39.289
experience something like cognitive dissonance,
00:11:39.370 --> 00:11:43.120
should seek resolution. Language models sometimes
00:11:43.120 --> 00:11:46.960
detect contradictions, sometimes fail spectacularly,
00:11:46.960 --> 00:11:49.820
sometimes generate contradictory statements in
00:11:49.820 --> 00:11:52.080
adjacent sentences without apparent awareness.
00:11:52.919 --> 00:11:57.769
Human understanding is similarly uneven. We all
00:11:57.769 --> 00:12:01.710
hold inconsistent beliefs, fail to notice conflicts,
00:12:02.049 --> 00:12:06.299
rationalize rather than revise. Contradiction
00:12:06.299 --> 00:12:09.639
detection is a useful indicator, but not a reliable
00:12:09.639 --> 00:12:13.769
test. Internal model construction suggests a
00:12:13.769 --> 00:12:17.120
deeper criterion. Perhaps understanding requires
00:12:17.120 --> 00:12:19.600
building something like a mental model of the
00:12:19.600 --> 00:12:22.990
domain. It would not merely associate inputs
00:12:22.990 --> 00:12:26.029
with outputs, but represent the underlying structure
00:12:26.029 --> 00:12:30.149
that generates observable patterns. A chess master
00:12:30.149 --> 00:12:32.710
understands chess partly because they possess
00:12:32.710 --> 00:12:35.830
a rich internal model of positions tactics and
00:12:35.830 --> 00:12:39.950
strategic principles. Yet internal models are
00:12:39.950 --> 00:12:44.429
not directly observable. We infer them from behavior,
00:12:44.929 --> 00:12:48.590
from patterns of success and failure, from the
00:12:48.590 --> 00:12:53.110
structure of errors. And here we face a recurring
00:12:53.110 --> 00:12:57.850
problem. The inference is always indirect, always
00:12:57.850 --> 00:13:02.029
subject to alternative interpretations. The same
00:13:02.029 --> 00:13:04.370
behavior might result from a compact internal
00:13:04.370 --> 00:13:07.360
model. or from an enormous look -up table of
00:13:07.360 --> 00:13:11.639
memorized responses. Belief revision after evidence
00:13:11.639 --> 00:13:15.440
seems promising. A system that understands should
00:13:15.440 --> 00:13:17.659
update its views when confronted with relevant
00:13:17.659 --> 00:13:21.470
new information. Yet belief revision is also
00:13:21.470 --> 00:13:25.919
imperfect as a test. Genuine understanding might
00:13:25.919 --> 00:13:28.940
resist updating when evidence seems unreliable
00:13:28.940 --> 00:13:32.870
or irrelevant. Sophisticated pattern matching
00:13:32.870 --> 00:13:35.450
might mimic revision when the new information
00:13:35.450 --> 00:13:39.009
falls within familiar patterns of update, and
00:13:39.649 --> 00:13:42.629
humans themselves are notoriously poor at belief
00:13:42.629 --> 00:13:46.309
revision, clinging to established views, interpreting
00:13:46.309 --> 00:13:50.019
disconfirming evidence as confirmation, revising
00:13:50.019 --> 00:13:52.980
only when the cost of inconsistency becomes unbearable.
00:13:53.769 --> 00:13:56.490
The Symbol -Perception -Action connection brings
00:13:56.490 --> 00:13:59.149
us to what many consider the deepest candidate
00:14:00.200 --> 00:14:03.220
Human understanding seems grounded in the connections
00:14:03.220 --> 00:14:06.080
between symbols and the world they represent.
00:14:06.870 --> 00:14:10.090
We learn hot through the pain of touching a stove,
00:14:10.429 --> 00:14:13.789
red through visual experience, justice through
00:14:13.789 --> 00:14:17.490
social practices and their consequences. This
00:14:17.490 --> 00:14:20.529
is the symbol grounding problem, named and elaborated
00:14:20.529 --> 00:14:24.799
by the cognitive scientists Devan Hanad. Words
00:14:24.799 --> 00:14:27.740
and symbols do not carry their meanings intrinsically.
00:14:28.659 --> 00:14:31.500
Their meanings arise from how they connect to
00:14:31.500 --> 00:14:34.340
perception, action, and the structured experiences
00:14:34.340 --> 00:14:36.960
of embodied creatures in physical and social
00:14:36.960 --> 00:14:40.919
environments. The symbol grounding problem is
00:14:40.919 --> 00:14:43.399
not merely a puzzle about artificial systems.
00:14:43.799 --> 00:14:46.980
It illuminates something about human cognition
00:14:46.980 --> 00:14:50.899
itself. A child learning language does not merely
00:14:50.899 --> 00:14:55.059
memorize correlations between sounds. The word
00:14:55.059 --> 00:14:58.049
ball becomes meaningful. through the weight of
00:14:58.049 --> 00:15:01.610
it in small hands. The sound of it bouncing on
00:15:01.610 --> 00:15:04.809
wooden floorboards. The visual arc of a throw
00:15:04.809 --> 00:15:08.649
across a room. The social game of catch with
00:15:08.649 --> 00:15:11.590
another person who smiles when the ball is caught.
00:15:12.340 --> 00:15:16.120
The word angry gains content through facial expressions,
00:15:16.500 --> 00:15:20.799
bodily sensations, social consequences, the narrative
00:15:20.799 --> 00:15:24.580
structure of human interaction. This grounding
00:15:24.580 --> 00:15:27.690
is not a single event. but an extended process
00:15:27.690 --> 00:15:31.870
of embodied, situated, consequential engagement
00:15:31.870 --> 00:15:36.299
with the world. Can non -biological systems achieve
00:15:36.299 --> 00:15:39.730
similar grounding? This is where philosophical
00:15:39.730 --> 00:15:42.549
thought experiments enter, and where we must
00:15:42.549 --> 00:15:45.230
be careful not to let them do more work than
00:15:45.230 --> 00:15:49.039
they can legitimately perform. John Sell's Chinese
00:15:49.039 --> 00:15:52.899
Room, proposed in 1980, is the most famous intuition
00:15:52.899 --> 00:15:57.460
pump in this domain. Imagine a small bare room,
00:15:57.730 --> 00:16:01.360
with a single door through a narrow slot in the
00:16:01.360 --> 00:16:04.980
door chinese characters slide in on slips of
00:16:04.980 --> 00:16:09.570
paper inside a person who understands no chinese
00:16:09.570 --> 00:16:12.809
receives these characters consults a massive
00:16:12.809 --> 00:16:15.750
rule book with stacks of paper organized by simple
00:16:15.750 --> 00:16:18.929
patterns and follows the rules to produce other
00:16:18.929 --> 00:16:21.669
chinese characters that slide back out through
00:16:21.669 --> 00:16:25.409
the slot the person's hands move through the
00:16:25.409 --> 00:16:29.519
pages matching incoming symbols to outgoing responses
00:16:29.519 --> 00:16:34.149
according to formal instructions. Native Chinese
00:16:34.149 --> 00:16:36.629
speakers outside the room find the responses
00:16:36.629 --> 00:16:40.669
perfectly appropriate. The person inside understands
00:16:40.669 --> 00:16:45.320
no Chinese. The system as a whole, Sealer argues,
00:16:45.600 --> 00:16:48.419
manipulates symbols according to formal rules
00:16:48.419 --> 00:16:52.330
without understanding their meaning. Therefore,
00:16:52.629 --> 00:16:56.580
syntax is not sufficient for semantics. Rule
00:16:56.580 --> 00:16:59.379
following simple manipulation. does not constitute
00:16:59.379 --> 00:17:03.179
understanding. The thought experiment has generated
00:17:03.179 --> 00:17:07.799
decades of response. Some argue that while the
00:17:07.799 --> 00:17:11.420
person inside does not understand Chinese, the
00:17:11.420 --> 00:17:16.519
system as a whole might be said to understand
00:17:16.519 --> 00:17:20.619
in some sense. Others question whether the intuition
00:17:20.619 --> 00:17:24.140
is reliable, noting that our inability to imagine
00:17:24.140 --> 00:17:27.400
understanding in such a system may reflect limitations
00:17:27.400 --> 00:17:31.079
of imagination. rather than metaphysical impossibility.
00:17:31.400 --> 00:17:35.220
The important point for our purposes is that
00:17:35.220 --> 00:17:38.799
the Chinese room exposes an intuition. There
00:17:39.279 --> 00:17:41.819
seems to be something missing when symbols are
00:17:41.819 --> 00:17:44.480
manipulated without connection to what they represent.
00:17:45.220 --> 00:17:48.299
But exposing an intuition is not settling an
00:17:48.299 --> 00:17:52.109
empirical question. Whether any particular engineered
00:17:52.109 --> 00:17:54.789
system achieves understanding despite lacking
00:17:54.789 --> 00:17:57.730
biological embodiment remains open to investigation.
00:17:58.430 --> 00:18:02.579
not closed by philosophical fiat. Alan Turing's
00:18:02.579 --> 00:18:05.700
earlier proposal, The Imitation Game from 1950,
00:18:06.220 --> 00:18:10.400
offers a different approach. If a system's responses
00:18:10.400 --> 00:18:13.319
are indistinguishable from those of a competent
00:18:13.319 --> 00:18:16.640
human, Turing suggested, we have good reason
00:18:16.640 --> 00:18:20.420
to attribute intelligence. Yet the Turing test
00:18:20.420 --> 00:18:23.279
has its own limitations, which the calculator
00:18:23.279 --> 00:18:27.420
and language model cases make vivid. A system
00:18:27.420 --> 00:18:30.480
might produce human -like responses through means
00:18:30.480 --> 00:18:34.319
that do not involve understanding. The test is
00:18:34.319 --> 00:18:37.259
behavioral, and
behavior can be dissociated from
00:18:37.259 --> 00:18:41.130
comprehension. Conversely, a system might possess
00:18:41.130 --> 00:18:44.250
understanding but fail the test due to some performance
00:18:44.250 --> 00:18:47.849
limitation. This is analogous to the student
00:18:47.849 --> 00:18:50.769
who understands physics but cannot solve problems
00:18:50.769 --> 00:18:54.559
in unfamiliar notation. The philosophical thought
00:18:54.559 --> 00:18:57.039
experiments are valuable not because they settle
00:18:57.039 --> 00:19:00.029
questions, but because they organize our intuitions
00:19:00.029 --> 00:19:03.720
and clarify what is at stake. They help us see
00:19:03.720 --> 00:19:05.619
that the problem of distinguishing performance
00:19:05.619 --> 00:19:08.859
from understanding is not merely technical but
00:19:08.859 --> 00:19:12.910
conceptual, yet they cannot substitute for empirical
00:19:12.910 --> 00:19:16.650
investigation. Whether any particular system
00:19:16.650 --> 00:19:20.049
understands, and to what degree requires examining
00:19:20.049 --> 00:19:23.890
its behavior across diverse conditions, its internal
00:19:23.890 --> 00:19:27.819
structure, its learning trajectory. its patterns
00:19:27.819 --> 00:19:31.970
of success and failure. This brings us to a crucial
00:19:31.970 --> 00:19:35.329
correction. We have been examining candidate
00:19:35.329 --> 00:19:38.009
signs of understanding as if the question were
00:19:38.009 --> 00:19:41.980
binary. Does this system understand or not? But
00:19:41.980 --> 00:19:44.440
the binary framing itself may be part of the
00:19:44.440 --> 00:19:48.529
problem. Understanding may be graded rather than
00:19:48.529 --> 00:19:52.180
all or nothing. And recognizing this changes
00:19:52.180 --> 00:19:56.099
how we approach the entire problem. Consider
00:19:56.099 --> 00:20:00.259
human cognition. A professional physicist understands
00:20:00.259 --> 00:20:04.019
quantum mechanics with considerable depth, can
00:20:04.019 --> 00:20:07.859
solve novel problems, explain concepts to students,
00:20:08.160 --> 00:20:10.920
recognize when approximations are appropriate.
00:20:11.359 --> 00:20:14.910
The same physicist may understand carpentry poorly,
00:20:15.049 --> 00:20:18.369
following instructions mechanically, unable to
00:20:18.369 --> 00:20:22.650
adapt to unexpected situations. A chess master
00:20:22.650 --> 00:20:25.589
understands chess in ways that approach intuition.
00:20:26.230 --> 00:20:29.109
seeing patterns and possibilities invisible to
00:20:29.109 --> 00:20:32.849
novices. The same person may understand economics
00:20:32.849 --> 00:20:36.769
only at a superficial level, able to repeat textbook
00:20:36.769 --> 00:20:39.450
claims without grasping underlying mechanisms.
00:20:39.930 --> 00:20:45.089
We understand unevenly. Our understanding is
00:20:45.089 --> 00:20:48.329
domain -specific, context -dependent, sometimes
00:20:48.329 --> 00:20:52.450
impressive, sometimes embarrassingly thin. More
00:20:52.450 --> 00:20:55.089
disturbingly, we sometimes confabulate explanations.
00:20:56.779 --> 00:20:59.559
we generate confident accounts of our own reasoning
00:20:59.559 --> 00:21:03.549
that are demonstrably false. In Michael Gazzaniga's
00:21:03.549 --> 00:21:06.690
studies of split -brain patients, whose corpus
00:21:06.690 --> 00:21:09.049
callosum was surgically severed to treat severe
00:21:09.049 --> 00:21:13.049
epilepsy, the left hemisphere, responsible for
00:21:13.049 --> 00:21:16.210
language, would construct elaborate explanations
00:21:16.210 --> 00:21:19.289
for actions initiated by the right hemisphere,
00:21:19.490 --> 00:21:23.880
of which it had no knowledge. In one experimental
00:21:23.880 --> 00:21:27.269
setup, the right hemisphere sees an object and
00:21:27.269 --> 00:21:30.880
directs the left hand to point at it. The left
00:21:30.880 --> 00:21:33.819
hemisphere, which controls language, but did
00:21:33.819 --> 00:21:36.640
not see the object and did not initiate the pointing,
00:21:36.880 --> 00:21:40.960
is asked why. It generates a plausible reason,
00:21:41.059 --> 00:21:45.220
believes it, reports it with conviction. The
00:21:45.220 --> 00:21:49.900
patient pointed at an object. Asked why, the
00:21:49.900 --> 00:21:52.759
left hemisphere constructed an explanation that
00:21:52.759 --> 00:21:57.369
was coherent, sensible, and false. In Petter
00:21:57.369 --> 00:22:00.029
Johansen and Lars Hall's choice blindness experiments,
00:22:00.569 --> 00:22:04.250
participants selected between two options, perhaps
00:22:04.250 --> 00:22:07.890
two jars of jam or two photographs of faces.
00:22:08.859 --> 00:22:13.880
They pointed to their preferred choice. The researcher,
00:22:14.019 --> 00:22:16.480
through a brief sleight of hand, covertly gave
00:22:16.480 --> 00:22:20.160
them the opposite jar or photograph. The participant
00:22:20.160 --> 00:22:22.920
was then shown the rejected option and asked
00:22:22.920 --> 00:22:27.039
to explain why they had chosen it. They confabulated
00:22:27.039 --> 00:22:29.759
reasons for preferring the option they had actually
00:22:29.759 --> 00:22:33.880
rejected moments before, generating detailed,
00:22:34.039 --> 00:22:36.700
confident explanations for a choice they never
00:22:36.700 --> 00:22:40.150
made. The facility of these false explanations
00:22:40.150 --> 00:22:45.380
is striking. They sound like understanding. They
00:22:45.380 --> 00:22:49.049
are not. This evidence about human cognition
00:22:49.049 --> 00:22:53.609
serves two purposes in our investigation. First,
00:22:53.869 --> 00:22:57.230
it undermines any assumption that understanding
00:22:57.230 --> 00:23:00.549
is a secret human essence present in us uniformly
00:23:00.549 --> 00:23:04.609
and absent in all non -biological systems if
00:23:05.000 --> 00:23:07.920
we understand unevenly if we sometimes perform
00:23:07.920 --> 00:23:10.339
without understanding and
sometimes understand
00:23:10.339 --> 00:23:13.079
without being able to perform then understanding
00:23:13.079 --> 00:23:15.819
is not a binary property that cleanly separates
00:23:15.819 --> 00:23:20.109
humans from everything else second It suggests
00:23:20.109 --> 00:23:22.569
that the search for understanding in other systems
00:23:22.569 --> 00:23:25.549
should proceed by examining degrees and kinds
00:23:25.549 --> 00:23:27.809
of comprehension rather than seeking a single
00:23:27.809 --> 00:23:31.569
decisive test. The graded view of understanding
00:23:31.569 --> 00:23:34.869
has important implications for how we think about
00:23:34.869 --> 00:23:39.339
language models and similar systems. It suggests
00:23:39.339 --> 00:23:42.259
that we might ask not whether such systems understand,
00:23:42.599 --> 00:23:45.819
but what they understand. To what degree, in
00:23:45.819 --> 00:23:49.710
what domains, with what limitations? Perhaps
00:23:49.710 --> 00:23:52.250
a language model possesses something like shallow
00:23:52.250 --> 00:23:55.670
semantic competence in some domains, able to
00:23:55.670 --> 00:23:58.190
navigate conceptual relationships in ways that
00:23:58.190 --> 00:24:01.450
go beyond mere memorization, while lacking the
00:24:01.450 --> 00:24:05.009
deep, flexible, grounded understanding that comes
00:24:05.009 --> 00:24:08.809
from embodied interaction with the world. Perhaps
00:24:08.809 --> 00:24:11.609
the degree of understanding varies with the domain,
00:24:11.869 --> 00:24:14.970
the training data, the architectural details,
00:24:15.269 --> 00:24:18.720
perhaps future systems with different training
00:24:18.720 --> 00:24:22.630
regimes. incorporating perception, action, causal
00:24:22.630 --> 00:24:25.470
intervention in the world, will achieve richer
00:24:25.470 --> 00:24:27.990
forms of understanding without being biological.
00:24:28.930 --> 00:24:31.789
The possibility of non -biological grounding
00:24:31.789 --> 00:24:35.599
deserves explicit attention. We have emphasized
00:24:35.599 --> 00:24:38.519
that human symbols gain meaning through bodies,
00:24:38.819 --> 00:24:41.900
shared practices, sensation, and consequences.
00:24:42.359 --> 00:24:46.079
But this is a description of one route to grounding,
00:24:46.200 --> 00:24:48.940
not necessarily a claim that no other route is
00:24:48.940 --> 00:24:52.660
possible. A system equipped with cameras might
00:24:52.660 --> 00:24:56.819
learn visual grounding for some concepts. A robot
00:24:56.819 --> 00:24:59.359
that manipulates objects might develop something
00:24:59.359 --> 00:25:02.039
like embodied understanding of physical properties.
00:25:02.900 --> 00:25:05.980
A system trained on simulations might acquire
00:25:05.980 --> 00:25:08.599
causal models that support flexible reasoning.
00:25:09.369 --> 00:25:12.750
A system whose symbols are connected not to the
00:25:12.750 --> 00:25:15.130
physical world but to extensive relationships
00:25:15.130 --> 00:25:18.369
among other symbols might achieve a form of abstract
00:25:18.369 --> 00:25:21.289
understanding that while different from human
00:25:21.289 --> 00:25:24.170
grounding, is not merely empty pattern matching.
00:25:24.849 --> 00:25:27.910
These possibilities are not established facts.
00:25:28.910 --> 00:25:32.410
They are research directions, some actively pursued,
00:25:32.670 --> 00:25:36.400
others speculative. The point is that the symbol
00:25:36.400 --> 00:25:40.039
grounding problem, while genuine, does not obviously
00:25:40.039 --> 00:25:43.640
require biological embodiment as its only solution.
00:25:44.490 --> 00:25:47.609
What it requires is some non -arbitrary connection
00:25:47.609 --> 00:25:51.240
between symbols and what they represent. some
00:25:51.240 --> 00:25:54.000
constraint on interpretation that prevents symbols
00:25:54.000 --> 00:25:56.920
from being merely interchangeable tokens in a
00:25:56.920 --> 00:26:01.150
formal game. Biological embodiment provides one
00:26:01.150 --> 00:26:04.519
such constraint. Whether other constraints can
00:26:04.519 --> 00:26:08.509
serve similarly remains to be determined. This
00:26:08.509 --> 00:26:10.650
brings us to the current state of our understanding
00:26:10.650 --> 00:26:14.910
and its limits. We can say with confidence that
00:26:14.910 --> 00:26:16.910
intelligent performance does not automatically
00:26:16.910 --> 00:26:21.119
prove understanding. The calculator establishes
00:26:21.119 --> 00:26:25.859
this for simple systems. Animal cognition complicates
00:26:25.859 --> 00:26:29.000
it by showing that biological systems can also
00:26:29.000 --> 00:26:32.299
perform without full comprehension. While sometimes
00:26:32.299 --> 00:26:35.359
achieving genuine understanding in ways we struggle
00:26:35.359 --> 00:26:39.279
to verify, language models make the problem urgent
00:26:39.279 --> 00:26:42.500
by producing performance so sophisticated that
00:26:42.500 --> 00:26:45.180
our intuitive criteria for understanding are
00:26:45.180 --> 00:26:48.849
strained to their limits. The candidate signs
00:26:48.849 --> 00:26:52.619
of understanding. transfer, explanation, contradiction,
00:26:53.059 --> 00:26:56.279
detection, internal models, belief revision,
00:26:56.559 --> 00:26:59.519
symbol grounding, each illuminate part of the
00:26:59.519 --> 00:27:02.440
problem while remaining imperfect as definitive
00:27:02.440 --> 00:27:06.910
tests. We can say that understanding, if it exists
00:27:06.910 --> 00:27:10.710
as a coherent phenomenon, is likely graded rather
00:27:10.710 --> 00:27:15.480
than binary. Humans understand unevenly, confabulate
00:27:15.480 --> 00:27:19.069
explanations. perform competently without comprehension
00:27:19.069 --> 00:27:21.710
in some domains while achieving deep understanding
00:27:21.710 --> 00:27:25.460
in others. This makes the search for understanding
00:27:25.460 --> 00:27:28.940
in other systems more
complex but also more tractable.
00:27:29.599 --> 00:27:33.200
We can ask about degrees and kinds rather than
00:27:33.200 --> 00:27:37.319
seeking a single yes or no answer. We can say
00:27:37.319 --> 00:27:42.140
that the symbol grounding problem is real. Symbols
00:27:42.140 --> 00:27:45.680
do not carry their meanings intrinsically. Meaning
00:27:45.680 --> 00:27:48.720
arises from connections to perception, action,
00:27:49.029 --> 00:27:52.619
consequence and structured experience. Whether
00:27:52.619 --> 00:27:55.940
biological embodiment is the only possible source
00:27:55.940 --> 00:27:58.819
of such grounding, or whether engineered systems
00:27:58.819 --> 00:28:01.539
can achieve functionally equivalent connections
00:28:01.539 --> 00:28:04.519
through other means remains genuinely uncertain.
00:28:05.490 --> 00:28:08.670
What we cannot say is whether any current artificial
00:28:08.670 --> 00:28:11.890
system possesses genuine understanding, and if
00:28:11.890 --> 00:28:15.390
so, to what degree. The evidence is ambiguous.
00:28:16.200 --> 00:28:19.200
the tests imperfect, the concept itself perhaps
00:28:19.200 --> 00:28:22.000
not yet sufficiently clarified to support definitive
00:28:22.000 --> 00:28:26.009
judgment. We cannot say that philosophical thought
00:28:26.009 --> 00:28:28.190
experiments have settled engineering questions.
00:28:29.200 --> 00:28:33.240
They have not. We cannot say that understanding
00:28:33.240 --> 00:28:36.160
is a secret human essence that no experiment
00:28:36.160 --> 00:28:40.119
can approach. The evidence from human cognition
00:28:40.119 --> 00:28:42.799
itself undermines this comfortable assumption.
00:28:44.099 --> 00:28:46.640
The episode we have constructed moves from the
00:28:46.640 --> 00:28:50.319
simple to the complex, from the obviously uncomprehending
00:28:50.319 --> 00:28:53.180
calculator to the genuinely puzzling case of
00:28:53.180 --> 00:28:56.599
language models, using each stage to refine our
00:28:56.599 --> 00:28:59.180
question rather than to answer it prematurely.
00:28:59.789 --> 00:29:02.369
We have seen that the gap between performance
00:29:02.369 --> 00:29:05.450
and understanding is real, that it appears in
00:29:05.450 --> 00:29:08.670
biological as well as artificial systems, that
00:29:08.670 --> 00:29:11.750
our tests for understanding are imperfect but
00:29:11.750 --> 00:29:15.779
not useless. that understanding is likely a spectrum
00:29:15.779 --> 00:29:19.299
rather than a binary property, and that the simple
00:29:19.299 --> 00:29:22.720
grounding problem, while challenging, does not
00:29:22.720 --> 00:29:25.240
obviously foreclose non -biological routes to
00:29:25.240 --> 00:29:28.589
comprehension. The question we began with was
00:29:28.589 --> 00:29:30.650
whether intelligence requires understanding.
00:29:31.579 --> 00:29:35.220
It turns out to be not one question, but many.
00:29:36.039 --> 00:29:39.359
It asks about the relationship between behavior
00:29:39.359 --> 00:29:42.960
and comprehension. about the criteria by which
00:29:42.960 --> 00:29:45.799
we attribute understanding, about whether those
00:29:45.799 --> 00:29:49.140
criteria can be satisfied by systems unlike ourselves,
00:29:49.579 --> 00:29:52.420
about whether understanding itself is a coherent
00:29:52.420 --> 00:29:55.359
natural kind or a pragmatic attribution we make
00:29:55.359 --> 00:29:59.599
for practical purposes, what we can say, with
00:29:59.599 --> 00:30:02.400
the evidence before us, is that the question
00:30:02.400 --> 00:30:06.400
is worth asking carefully, that our initial intuitions
00:30:06.400 --> 00:30:09.140
are likely too simple, that the investigation
00:30:09.140 --> 00:30:12.619
requires both philosophical clarity and empirical
00:30:12.619 --> 00:30:16.259
patience, and that the answer, if one emerges,
00:30:16.599 --> 00:30:20.720
will likely be graded, contextual, and more interesting
00:30:20.720 --> 00:30:24.930
than a simple yes or no. The calculator waits
00:30:24.930 --> 00:30:28.670
on the table. The cursor blinks in the language
00:30:28.670 --> 00:30:32.710
model's interface. The crow, somewhere in the
00:30:32.710 --> 00:30:36.680
Pacific, bends another wire. We are still learning
00:30:36.680 --> 00:30:38.299
what it means to understand.