TRANSCRIPT 🔗
WEBVTT
cue-1
00:00:00.300 --> 00:00:04.250
Okay, where's my soundboard? All right,
let's go.
cue-2
00:00:04.250 --> 00:00:11.580
[rock music]
cue-3
00:00:30.140 --> 00:00:35.660
I'm Justin. I'm an academic librarian, and
my pronouns are he and they.
cue-4
00:00:35.660 --> 00:00:39.699
I'm Sadie. I work IT at a public library,
and my pronouns are they/them.
cue-5
00:00:39.700 --> 00:00:43.180
I'm Jay. I'm a cataloging librarian. My
pronouns are he/him.
cue-6
00:00:43.180 --> 00:00:46.460
And we have guests. Would you like to
introduce yourselves?
cue-7
00:00:46.460 --> 00:00:50.580
I'm Scarlet. I am a collection strategist
librarian at an R1,
cue-8
00:00:51.180 --> 00:00:55.440
-and my pronouns are she and they.
-And I'm Dorothea, the guest y'all can't
cue-9
00:00:55.440 --> 00:00:56.460
seem to get rid of.
cue-10
00:00:57.300 --> 00:01:01.480
[laughs] I teach in the information school
at the University of Wisconsin at
cue-11
00:01:01.480 --> 00:01:03.240
Madison, and my pronouns are she/her.
cue-12
00:01:03.880 --> 00:01:08.940
-[cheering]
-Welcome back. Returning guests, returning
cue-13
00:01:08.940 --> 00:01:10.660
-champions.
-Love that.
cue-14
00:01:10.660 --> 00:01:15.100
I didn't even bother counting up how many
times Dorothea's been on, but just
cue-15
00:01:15.100 --> 00:01:19.900
unofficial fourth mic of the podcast from,
from the beginning. First guest, longest
cue-16
00:01:19.900 --> 00:01:21.619
-time guest.
-Thank you.
cue-17
00:01:21.620 --> 00:01:26.490
So I, I love a good shit talking episode,
so that's what we're gonna do.
cue-18
00:01:27.320 --> 00:01:31.020
I started looking on Blue Sky the other
day to see if, like, people were still
cue-19
00:01:31.020 --> 00:01:34.200
talking about this article, 'cause I had a
sneaking suspicion that, like, they're
cue-20
00:01:34.200 --> 00:01:38.940
never gonna get this published because
it's a preprint. And one thing I know
cue-21
00:01:38.940 --> 00:01:45.540
about some of the AI stuff that's
happening is a lot of these booster-ish
cue-22
00:01:45.540 --> 00:01:51.300
articles are getting published in, or
published, quote-unquote, on archive.org,
cue-23
00:01:51.300 --> 00:01:57.060
where then they are reported on as a new
preprint article, and they never actually
cue-24
00:01:57.060 --> 00:02:02.520
make their way into any journals. So I'm
interested to see if that's sort of, um,
cue-25
00:02:02.520 --> 00:02:05.760
what's happening. I... They still have not
published. It's been a month since this
cue-26
00:02:05.760 --> 00:02:09.859
made the rounds. But this is the article,
and let me pull up the title, Critical
cue-27
00:02:09.860 --> 00:02:15.660
Confabulation: Can LLMs Hallucinate for
Social Good? This is from the same people
cue-28
00:02:15.660 --> 00:02:19.859
that brought you, I believe, some of the
same authors that brought you the article
cue-29
00:02:19.860 --> 00:02:21.390
Why SLOP Matters,
cue-30
00:02:22.200 --> 00:02:26.540
-Hoyt, Long, and-
-Scarlet's face journey was amazing when
cue-31
00:02:26.540 --> 00:02:29.520
-Justin said his name. [laughs]
-Hoyt and, uh,
cue-32
00:02:30.300 --> 00:02:36.100
Eyman. So two of these articles, or three
of them, three of these authors also wrote
cue-33
00:02:36.100 --> 00:02:37.489
that article Why SLOP Matters,
cue-34
00:02:38.220 --> 00:02:45.060
which is published in, like, a new IEEE
journal that has, like, two issues, so.
cue-35
00:02:45.160 --> 00:02:48.400
-Oh, can, can we shit talk IEEE?
-Yeah. I mean, why not?
cue-36
00:02:48.400 --> 00:02:54.660
I would love to shit talk IEEE. Okay. A
few months ago, an article came out by
cue-37
00:02:54.660 --> 00:02:59.359
Guillaume Cabanac and his crew. Guillaume
is awesome, by the way. I love Guillaume.
cue-38
00:02:59.360 --> 00:03:03.620
Fantastic scholarly communications
researcher. And
cue-39
00:03:04.320 --> 00:03:11.280
basically, a ridiculous percentage of crap
publications in, in,
cue-40
00:03:11.280 --> 00:03:16.280
in the relevant subject areas are from
IEEE, because IEEE is basically a
cue-41
00:03:16.280 --> 00:03:18.960
-franchise operation, right?
-Mm-hmm.
cue-42
00:03:18.960 --> 00:03:23.460
If you want to have a, quote-unquote,
"reputable journal," you can sign up with
cue-43
00:03:23.460 --> 00:03:27.780
IEEE and basically they don't check. And
yeah, you can publish all the crap you
cue-44
00:03:27.780 --> 00:03:30.710
-want.
-Hmm. Well, I was wrong. This is ACM,
cue-45
00:03:31.480 --> 00:03:37.380
a, a journal called ACM AI Letters, Volume
1, Number 1, Why SLOP Matters. And then
cue-46
00:03:37.380 --> 00:03:41.380
the next one is Why AI SLOP Matters, But
Not Like That, which is a response article
cue-47
00:03:41.380 --> 00:03:47.500
in an upcoming issue. But it has not been
published yet. But anyway, I was looking
cue-48
00:03:47.500 --> 00:03:50.660
to see if they ever got this article
published, 'cause, like, there is this
cue-49
00:03:50.660 --> 00:03:54.760
problem, and some of the citations that
aren't from digital humanists, that are
cue-50
00:03:54.760 --> 00:04:01.260
from other AI authors are also citing
archive preprints. So there is kind of an
cue-51
00:04:01.260 --> 00:04:06.560
endemic of these questionable publications
that don't seem to ever get into any
cue-52
00:04:06.560 --> 00:04:11.500
journals, which is not terrible. I mean,
from a Schol Comm perspective, fine. But
cue-53
00:04:11.500 --> 00:04:15.360
when you're... There's, there's clearly,
like, a, a communication strategy here
cue-54
00:04:15.360 --> 00:04:19.500
that I think is problematic and has not
been discussed at length. I know people
cue-55
00:04:19.500 --> 00:04:23.600
have mentioned it, 'cause I obviously am
not the first person to notice this, but
cue-56
00:04:23.600 --> 00:04:25.840
it is an interesting trend to keep an eye
out on.
cue-57
00:04:25.840 --> 00:04:30.000
Well, like, when Archive finally... This
happened in a couple of stages. So Archive
cue-58
00:04:30.000 --> 00:04:36.100
came out and said, "We can no
Load full transcript
longer
trust institutional affiliation for email
cue-59
00:04:36.100 --> 00:04:40.680
submissions. So if you're the
corresponding author and you have an email
cue-60
00:04:40.680 --> 00:04:44.490
connected to an institution, we can't
trust that anymore." That was the first
cue-61
00:04:44.490 --> 00:04:47.620
thing that Archive had to come out and
say. And then the second thing that
cue-62
00:04:47.620 --> 00:04:52.140
Archive came out and, and said was, "We're
just gonna ban you from submitting if you
cue-63
00:04:52.140 --> 00:04:56.080
send us papers with hallucinated
citations. We're taking a hard line-
cue-64
00:04:56.080 --> 00:04:58.340
-Mm-hmm
-... on it." And the number of people who
cue-65
00:04:58.340 --> 00:05:03.060
came out and absolutely told on
themselves, their affiliates, and all of
cue-66
00:05:03.060 --> 00:05:07.760
their labs is incredible to me. Like, so
many people came out and said, "There
cue-67
00:05:07.760 --> 00:05:11.630
won't be any PIs left if you go ahead and
do this." And I said, "There's eight...
cue-68
00:05:11.630 --> 00:05:14.380
There's a line eight deep behind you who
will show up-
cue-69
00:05:14.380 --> 00:05:16.200
-Mm-hmm
-... take over your lab, and actually read
cue-70
00:05:16.200 --> 00:05:19.680
everything that they're citing, because
that's what you're supposed to do." I
cue-71
00:05:19.680 --> 00:05:23.160
thought that's what we we're... My gosh,
it was amazing to, to see. I would love
cue-72
00:05:23.160 --> 00:05:27.840
for someone to go through and do a study
of all of the output of all of the authors
cue-73
00:05:27.840 --> 00:05:29.940
that complained about this. It's
incredible.
cue-74
00:05:29.940 --> 00:05:34.540
Like, before AI could do this, what the
fuck were they doing? [laughs]
cue-75
00:05:34.540 --> 00:05:40.000
Also, why would you say that out loud on
LinkedIn, on social, where people can see
cue-76
00:05:40.000 --> 00:05:41.520
-you? Like-
-Like-
cue-77
00:05:41.520 --> 00:05:43.700
-You-
-... show your entire ass. [laughs]
cue-78
00:05:43.700 --> 00:05:47.159
Right. Like, you just wanna pull people
aside and be like, "You're saying outdoor
cue-79
00:05:47.160 --> 00:05:51.099
thing... You're saying indoor things with
your outdoor voice, my friend. I don't...
cue-80
00:05:51.100 --> 00:05:54.700
-What's going on?" [laughs]
-Yeah. I mean, it's really strange. I think
cue-81
00:05:54.700 --> 00:05:57.840
part of the things people were saying
was, "Well, there's so many co-authors. We
cue-82
00:05:57.840 --> 00:06:02.620
can't trust that one of them is not gonna
use AI and then screw it over, screw over
cue-83
00:06:02.620 --> 00:06:06.620
all of us." It's like, well, just read
your citations before you publish it, or
cue-84
00:06:06.620 --> 00:06:13.314
before you submit it. It's not that hard.
And I think my, my theory is because these
cue-85
00:06:13.314 --> 00:06:17.784
are getting into, you know, prestigious
journals that are supposed to be doing
cue-86
00:06:17.784 --> 00:06:22.544
peer review, whenever they find these
hallucinated citations, they're just gonna
cue-87
00:06:22.544 --> 00:06:27.044
go back and remove them without actually
retracting the articles to make them, to
cue-88
00:06:27.044 --> 00:06:29.084
make the journals look better. Like, the
journals
cue-89
00:06:29.764 --> 00:06:33.644
are going to go back and do this. Like
Elsevier did this with Sci-Hub DOIs at one
cue-90
00:06:33.644 --> 00:06:36.624
-point back in like 2019.
-Oh, yeah.
cue-91
00:06:36.624 --> 00:06:40.303
When people started pointing out how many
Sci-Hub DOIs there were in Elsevier
cue-92
00:06:40.303 --> 00:06:43.324
papers, they just went back and replaced
all the DOIs
cue-93
00:06:43.964 --> 00:06:48.684
without any retraction notices, without
any addendums or anything,
cue-94
00:06:49.364 --> 00:06:53.464
because they care about papers as a
product, and they don't care about how,
cue-95
00:06:53.464 --> 00:06:57.364
you know, the paper trail of actually how
you cited something. So I used to collect
cue-96
00:06:57.364 --> 00:07:01.924
a lot of data on that, and I have, I have
years of automated data that I scraped
cue-97
00:07:01.924 --> 00:07:07.964
based on like i- in like my inbox and, and
of, of DOIs for Sci-Hub. But anyway,
cue-98
00:07:07.964 --> 00:07:11.704
that's what we're... not what we're
talking about today, but we do have, uh,
cue-99
00:07:11.704 --> 00:07:17.444
some trivia and news section. So do I have
a news drop? Hang on. I, I haven't been
cue-100
00:07:17.444 --> 00:07:21.504
-using my drops as much.
-Oh, feel free. I find them delightful. I,
cue-101
00:07:21.504 --> 00:07:23.964
-I really do. [laughs]
-Yes. See, someone, someone does.
cue-102
00:07:23.964 --> 00:07:26.884
-ChatGPT's for Dolores.
-Oh, no. Have I taken, have I taken a side?
cue-103
00:07:26.884 --> 00:07:30.024
-Oh, no. Yeah. [laughs] Oh, no.
-You don't have to be gay-
cue-104
00:07:30.024 --> 00:07:32.004
-Tread carefully
-... to get married to a man. You d- I
cue-105
00:07:32.004 --> 00:07:36.034
-mean, you did it.
-Yeah. That's my news drop. So news.
cue-106
00:07:36.034 --> 00:07:36.803
-[laughs]
-Scarlet, you have news.
cue-107
00:07:37.524 --> 00:07:41.404
Yeah. So the last... So I have a new job
since the last time I was on the podcast,
cue-108
00:07:41.404 --> 00:07:45.083
and I don't know that I ever publicly said
thank you, 'cause the best thing about
cue-109
00:07:45.084 --> 00:07:51.064
writing my tenure dossier for Grand Valley
was putting library.gay on my CV.
cue-110
00:07:51.064 --> 00:07:53.824
-Mm-hmm.
-Having the provost read it and say that
cue-111
00:07:53.824 --> 00:07:57.704
essentially that she had a blast reading
it, and then she quit two days later after
cue-112
00:07:57.704 --> 00:08:02.344
we all got the mass blessing from the
board of trustees, and now I'm, I'm also
cue-113
00:08:02.344 --> 00:08:06.723
at U Chicago. I think Dorothea has a new
position since he...
cue-114
00:08:07.364 --> 00:08:08.964
-You're
managing-
-Sort of.
cue-115
00:08:09.004 --> 00:08:12.454
-Sort of. You're managing people finally-
-I am managing people
cue-116
00:08:12.454 --> 00:08:14.804
-... which is amazing.
-Who let me do this?
cue-117
00:08:14.804 --> 00:08:18.514
-Mm-hmm.
-It, it... Somebody stop me. But yes, it is
cue-118
00:08:18.514 --> 00:08:24.044
my, my first kind of, uh... I mean, I've
supervised TAs and project assistants and
cue-119
00:08:24.044 --> 00:08:28.944
stuff in the past, but I am actually
supervising people now, eight teaching
cue-120
00:08:28.944 --> 00:08:35.084
faculty who are basically, like me,
permanent academic staff, and, uh, four f-
cue-121
00:08:35.084 --> 00:08:38.864
four part-timers who have recurring
appointments, so they teach for us again
cue-122
00:08:38.864 --> 00:08:45.824
and again and again. And as management
gigs go, this is like the easiest one ever
cue-123
00:08:45.824 --> 00:08:52.444
because the people that I supervise are
absolutely without exception fantastic.
cue-124
00:08:52.444 --> 00:08:56.984
-They're amazing, and I love them.
-Mm-hmm. Yeah. I'm, I'm supervising a team
cue-125
00:08:56.984 --> 00:09:00.644
that I have inherited rather than one that
I've built, which is new for me, and
cue-126
00:09:00.644 --> 00:09:03.384
they're all older than me, so they're all
like Gen X and older.
cue-127
00:09:04.144 --> 00:09:09.784
Some of them have been in their jobs for
like 40 years at the same place. Like the
cue-128
00:09:09.784 --> 00:09:14.004
serials, like one of them is like a
serials librarian or a library assistant
cue-129
00:09:14.004 --> 00:09:17.704
who's been there, and now he just is the
only person left from the serials
cue-130
00:09:17.704 --> 00:09:22.424
department. [laughs] Everyone else has
slowly like no longer had a department
cue-131
00:09:22.424 --> 00:09:27.764
anymore. So now he's, he's on my team, and
there's still plenty of serials work to
cue-132
00:09:27.764 --> 00:09:31.584
be done 'cause turns out they take up
physical space, and they gotta get moved,
cue-133
00:09:31.584 --> 00:09:36.664
and we're absorbing other libraries and
taking on all their crappy cataloging
cue-134
00:09:36.664 --> 00:09:41.574
practices and having to fix stuff. So
yeah, independent libraries. I don't
cue-135
00:09:41.574 --> 00:09:44.784
understand why universities have
independent libraries instead of just one
cue-136
00:09:44.784 --> 00:09:48.914
library system. That's one thing also that
I'm having to deal with is people telling
cue-137
00:09:48.914 --> 00:09:51.984
me, "Don't touch my stuff. We're the
different library." But like-
cue-138
00:09:51.984 --> 00:09:55.324
-Yep
-... why? Why? Buy your own Alma instance
cue-139
00:09:55.324 --> 00:09:58.644
then. They've, they've, they've basically
said they don't want us helping them with
cue-140
00:09:58.644 --> 00:10:02.164
their Primo instance anymore, so like we
set it up for them, and now they don't
cue-141
00:10:02.164 --> 00:10:06.364
want us to touch it, and it's like I, I...
One of our systems people, or one of our
cue-142
00:10:06.364 --> 00:10:10.364
discovery people, has gotten very annoyed
at this and is like, "Fine. You don't want
cue-143
00:10:10.364 --> 00:10:13.624
my help? I will not touch anything of
yours anymore."
cue-144
00:10:13.624 --> 00:10:15.864
That is the correct response for
something-
cue-145
00:10:15.864 --> 00:10:18.044
-Yep
-... like that. Like, "Nope. You go ahead
cue-146
00:10:18.044 --> 00:10:19.644
-and have-"
-Really is
cue-147
00:10:19.644 --> 00:10:21.404
-... old Primo.
-And if it breaks, you're on your own.
cue-148
00:10:22.124 --> 00:10:27.443
Yeah. It's, it's very silly. But anyway,
what else do we got? Richard Dawkins fell
cue-149
00:10:27.443 --> 00:10:31.024
-in love with an AI.
-Yeah. We don't have to, we don't have to
cue-150
00:10:31.024 --> 00:10:31.574
-talk about-
-[laughs]
cue-151
00:10:31.574 --> 00:10:35.604
... about that. But, but I... Richard
Dawkins fell in love with an AI. It's
cue-152
00:10:35.604 --> 00:10:39.924
gonna happen to more people with large
platforms, and while it's extremely
cue-153
00:10:39.924 --> 00:10:44.344
depressing that it ha- will happen to more
people, it's very funny that it happened
cue-154
00:10:44.344 --> 00:10:46.444
-to Richard Dawkins.
-Mm-hmm.
cue-155
00:10:46.444 --> 00:10:50.344
One of the other articles that I, we have
in kind of the working document for this
cue-156
00:10:50.344 --> 00:10:55.604
discusses AI and Pygmalion displacement,
which is one of the many reasons why
cue-157
00:10:55.604 --> 00:10:59.814
Richard Dawkins was compelled to give his
AI a femme name. Because he couldn't
cue-158
00:10:59.814 --> 00:11:05.424
handle Claude being called Claude, he had
to rename it and name her Claudia in order
cue-159
00:11:05.424 --> 00:11:07.324
to be okay with the level of-
cue-160
00:11:07.944 --> 00:11:10.344
-Enmeshment
-... yeah, enmeshment that he's
cue-161
00:11:10.344 --> 00:11:14.584
experiencing. But, but yeah, he basically
believes that it's conscious and alive,
cue-162
00:11:14.584 --> 00:11:18.744
and, and that's Richard Dawkins at the end
of his life now, which is-
cue-163
00:11:18.744 --> 00:11:20.084
-Mm
-... incredible to me.
cue-164
00:11:20.724 --> 00:11:24.684
You know, I just... And it's, it's gonna
happen to more people, and I think when it
cue-165
00:11:24.684 --> 00:11:29.004
happens on an individual level, it's
gonna be heartbreaking and awful. But when
cue-166
00:11:29.004 --> 00:11:31.464
it happens to Richard Dawkins, it's
incredibly funny.
cue-167
00:11:31.464 --> 00:11:36.404
Mm-hmm. No, I mean, our friends have a
dating and relationship podcast, and I was
cue-168
00:11:36.404 --> 00:11:37.804
totally pitching an episode
cue-169
00:11:38.404 --> 00:11:42.904
of doing like AI as my boyfriend Reddit.
It's really interesting 'cause the people
cue-170
00:11:42.904 --> 00:11:47.784
who are having the most like relationships
with AI are women, and one of the things
cue-171
00:11:47.784 -->
00:11:52.424
that was interesting when I was going
through posts on like my boyfriend is AI
cue-172
00:11:52.424 --> 00:11:56.124
was someone saying, "I'm so glad I found
this one 'cause most of the other places
cue-173
00:11:56.124 --> 00:12:00.124
are entirely women, and I wanted to find a
place where other men have fallen in love
cue-174
00:12:00.124 --> 00:12:05.244
with AI." But it's still dominated by
women. It's a very interesting phenomenon
cue-175
00:12:05.244 --> 00:12:09.764
and very strange and very sad, and it's
very incel coded too. Like, it's very much
cue-176
00:12:09.764 --> 00:12:13.764
like- They're like, "Don't." They're
like, "We are in love with AI because we
cue-177
00:12:13.764 --> 00:12:18.824
have no other op-," and like, "What I
would give to have a real person love me."
cue-178
00:12:18.824 --> 00:12:22.284
And they're constantly talking about
like, "Oh, I love my AI boyfriend, but
cue-179
00:12:22.284 --> 00:12:29.124
what I would give." It's, it's very incel
coded, and very strange, very sad. It...
cue-180
00:12:29.124 --> 00:12:33.404
-Yeah. It's not good this is happening.
-No, no. It's, it's really not. I, I know
cue-181
00:12:33.404 --> 00:12:37.144
that there's probably going to be some,
you know, significant study or work about
cue-182
00:12:37.144 --> 00:12:40.724
this, but, but I, you know, remember kind
of exploring a little bit of that just to
cue-183
00:12:40.724 --> 00:12:44.604
sort of see what was happening. And I
mean, even, you know, Weizenbaum wrote
cue-184
00:12:44.604 --> 00:12:48.964
about this when he built Eliza, like, that
no one should see what you ty- what
cue-185
00:12:48.964 --> 00:12:52.464
you're typing into this thing. People are
immediately treating it like, you know,
cue-186
00:12:52.464 --> 00:12:56.924
like a therapist, that there was this, you
know, immediate, you know, enmeshment
cue-187
00:12:56.924 --> 00:13:00.564
factor sort of happening even with
something as basic as Eliza, which my, you
cue-188
00:13:00.564 --> 00:13:03.784
know, accomplishment when I was eight
years old was getting Eliza to swear at
cue-189
00:13:03.784 --> 00:13:06.644
-me, you know, or, or get angry at me.
-Where are you?
cue-190
00:13:06.644 --> 00:13:10.764
You know? Because, because that was the
fun thing to do when you're eight and
cue-191
00:13:10.764 --> 00:13:14.964
somebody has a, you know, has a computer.
Amber, you know-
cue-192
00:13:14.964 --> 00:13:17.554
-Mm
-... monitor and, and firework screensavers
cue-193
00:13:17.554 --> 00:13:22.244
forever, right? But it's, it's
interesting and, and scary to see all of
cue-194
00:13:22.244 --> 00:13:25.504
this happen, especially in the, in the
current context.
cue-195
00:13:25.504 --> 00:13:30.604
Yeah. So anyway, let's get to the article,
'cause I'm sure we've got plenty to talk
cue-196
00:13:30.604 --> 00:13:36.204
about on that. So critical confabulation,
can LLMs hallucinate for social good? The
cue-197
00:13:36.204 --> 00:13:40.604
idea is they
want to propose critical
confabulation inspired by critical
cue-198
00:13:40.604 --> 00:13:45.344
fabulation from literary and social
theory. The use of LLM hallucinations to,
cue-199
00:13:45.344 --> 00:13:49.864
quote, "Fill in the gap or omissions in
archives due to social and political
cue-200
00:13:49.864 --> 00:13:54.624
inequality and reconstruct divergent yet
evidence-bound narratives for history's,"
cue-201
00:13:54.624 --> 00:13:55.874
quote, "hidden figures."
cue-202
00:13:56.604 --> 00:14:03.504
Basically, what they did is they took this
corpus of work called the, got it
cue-203
00:14:03.504 --> 00:14:09.424
highlighted, Black Writing and Thought
collection. And what they did was they
cue-204
00:14:09.424 --> 00:14:14.464
made sure that none of it was already
ingested into their LLM, so they used open
cue-205
00:14:15.104 --> 00:14:19.194
source LLMs where they know what the
training data is. They ran the corpuses
cue-206
00:14:19.194 --> 00:14:23.204
against each other and, and tried to
cancel out anything that was possibly
cue-207
00:14:23.204 --> 00:14:30.184
already in there. And then what they did
is a narrative cloze task, C-L-O-Z-E,
cue-208
00:14:30.184 --> 00:14:34.384
which is basically a cloze task, is where
you take out bits of information in a
cue-209
00:14:34.384 --> 00:14:39.084
sentence. This is how like you do quizzes
for, like, students learning a language.
cue-210
00:14:39.084 --> 00:14:44.843
You take out pieces of the information in
a sentence, and by context you want the,
cue-211
00:14:44.844 --> 00:14:48.544
the person taking the test to fill in the
gaps. And so they're doing this with
cue-212
00:14:48.544 --> 00:14:54.924
narratives, and they want the LLM to see
if it can guess what they know is in the
cue-213
00:14:54.924 --> 00:14:58.904
corpus but it doesn't know is in the
corpus. So they give it parts of the story
cue-214
00:14:58.904 --> 00:15:02.924
from the corpus, blot out certain parts,
and then want to see how accurately it can
cue-215
00:15:02.924 --> 00:15:08.024
guess the parts that were blotted out.
Basically taking a timeline and then
cue-216
00:15:08.024 --> 00:15:11.724
adding, uh, and seeing can it, can it
figure this, this out.
cue-217
00:15:11.724 --> 00:15:15.344
To sort of combat, quote unquote, what
they're calling a s- a sort of like
cue-218
00:15:15.344 --> 00:15:21.004
archival silence. But archive used more
broadly than how we would use archive on
cue-219
00:15:21.004 --> 00:15:25.444
this podcast. But like gaps in historical
record.
cue-220
00:15:25.444 --> 00:15:26.914
I don't think they understand the
difference.
cue-221
00:15:26.914 --> 00:15:31.024
-They don't. [laughs]
-They don't... So the thing about this is I
cue-222
00:15:31.024 --> 00:15:35.464
don't think they understand any of the
digital humanities scholarship they read,
cue-223
00:15:35.464 --> 00:15:38.304
and I don't think they read it. I think
the way that it's summarized and
cue-224
00:15:38.304 --> 00:15:42.604
misunderstands the source material
indicates to me that they also used gen AI
cue-225
00:15:42.604 --> 00:15:47.064
to write most of the literature review,
because they don't really seem to
cue-226
00:15:47.064 --> 00:15:52.284
understand it. But there is in the notes
that there is a rough outline of how
cue-227
00:15:52.284 --> 00:15:56.024
history happens. I don't know who added
this, but whoever did.
cue-228
00:15:56.024 --> 00:16:00.804
Yeah. So yeah. So I, so I added a bunch of
stuff in here. And just a, a note for
cue-229
00:16:00.804 --> 00:16:04.804
audience members that don't, that don't
know us, is that Dorothea and I aren't
cue-230
00:16:04.804 --> 00:16:06.044
archivists by training.
cue-231
00:16:06.724 --> 00:16:10.484
So we should probably say what, what our
approach to something like this would be.
cue-232
00:16:10.484 --> 00:16:15.414
I'm a, I'm a historian by training, so
I've worked in and around archives for all
cue-233
00:16:15.414 --> 00:16:21.164
of my academic life, but not in the same
way that an archivist would, right?
cue-234
00:16:21.784 --> 00:16:25.704
So when I talk about this, you know, doing
kind of a, a high level pass here, but
cue-235
00:16:25.704 --> 00:16:30.584
not the way that someone who is a trained
and active historian would, 'cause I'm not
cue-236
00:16:30.584 --> 00:16:34.744
a historian, I'm a librarian. And not the
way that someone who is an archivist
cue-237
00:16:34.744 --> 00:16:35.824
would. So
cue-238
00:16:36.704 --> 00:16:40.684
someone who is better at French pronounce
this name that I have highlighted
cue-239
00:16:41.304 --> 00:16:44.764
in here. I have a comment that says, "I
cannot French. Someone help me." And it is
cue-240
00:16:44.764 --> 00:16:50.804
because I read more than I speak, and so
French words were an incredible
cue-241
00:16:50.804 --> 00:16:54.524
embarrassment when I was a kid. I don't
know how to pronounce this person's name.
cue-242
00:16:54.524 --> 00:16:58.114
-Michel-Rolph Trouillot?
-I think it is Trouillot. Yeah. Okay. So-
cue-243
00:16:58.114 --> 00:17:01.384
-Yeah. It should be Trouillot.
-Okay. So one of the things that I read to
cue-244
00:17:01.384 --> 00:17:05.884
get into this was Michel-Rolph Trouillot's
Silencing the Past: Power and the
cue-245
00:17:05.884 --> 00:17:09.984
Production of History to look at what
we... One, sort of what we mean by
cue-246
00:17:09.984 --> 00:17:15.144
archival silence and, and how it interacts
with the practice of history, because
cue-247
00:17:15.144 --> 00:17:18.464
this is sort of what this paper is trying
to get at a little bit. And I thought,
cue-248
00:17:18.464 --> 00:17:22.024
"Okay, fine. We'll, we'll take it on, on
their terms and look at it." So there's a,
cue-249
00:17:22.024 --> 00:17:26.744
a bunch of places where the archive is
effectively silent or has an absence.
cue-250
00:17:27.444 --> 00:17:31.624
And Trouillot writes that there's the
moment of fact creation, so the making of
cue-251
00:17:31.624 -->
00:17:36.544
sources. The moment of fact assembly, so
how we make archives, and that would
cue-252
00:17:36.544 --> 00:17:40.324
include things like arrangement and
description. The moment of fact retrieval,
cue-253
00:17:40.324 --> 00:17:44.944
so having gotten at all of these sources,
how do we then create and structure
cue-254
00:17:44.944 --> 00:17:49.044
narratives around it? And then the moment
of retrospective significance, so the
cue-255
00:17:49.044 --> 00:17:54.504
making of history in the final instance.
Good recent examples of this are anything
cue-256
00:17:54.504 --> 00:18:00.004
really related to the Cold War or the
Dirty War, especially under Pinochet.
cue-257
00:18:00.004 --> 00:18:04.864
Somebody opens a, a footlocker in
Argentina or the for- former Soviet Union,
cue-258
00:18:04.864 --> 00:18:08.964
and suddenly that making of retrospective
significance that he's writing about
cue-259
00:18:08.964 --> 00:18:12.724
becomes different. Our understanding of
that source making, of assembly and
cue-260
00:18:12.724 --> 00:18:18.528
retrieval then reiterates on those primary
and secondary sources. So this is what
cue-261
00:18:18.528 --> 00:18:22.268
critical fabulation, the concept that
inspired this paper,
cue-262
00:18:22.908 --> 00:18:28.498
is, is trying, you know, kind of to do. So
Saidiya Hartman is the scholar
cue-263
00:18:28.498 --> 00:18:33.478
responsible for the idea of critical
fabulations, and she first introduces this
cue-264
00:18:33.478 --> 00:18:38.128
concept in another essay called Venus in
Two Acts, where she talks about this. And
cue-265
00:18:38.128 --> 00:18:43.188
in it, what she's doing is trying to
reconcile that archival silence in the
cue-266
00:18:43.188 --> 00:18:48.908
context of two girls who were mor-murdered
aboard the, uh, slave ship Recovery as
cue-267
00:18:48.908 --> 00:18:54.108
part of the Atlantic slave trade. And so
she's sort of looking at where are they
cue-268
00:18:54.108 --> 00:18:59.088
explicitly mentioned, where are they not,
where do sort of catalogs of people
cue-269
00:18:59.688 --> 00:19:03.448
as objects, you know, become... You know,
where do you stop saying, for example, how
cue-270
00:19:03.448 --> 00:19:08.248
people died, and you just put the, the
quotation marks. It's 'cause everyone is
cue-271
00:19:08.248 --> 00:19:12.008
dying the same way or everyone is sold or
everyone is... Like, all of these kinds
cue-272
00:19:12.008 --> 00:19:15.448
of-- That there are these sorts of
silences that she's trying to deal with
cue-273
00:19:15.448 --> 00:19:16.768
and, and reconcile. And
cue-274
00:19:17.388 --> 00:19:21.247
I'm gonna read a couple of paragraphs from
Venus in Two Acts, which if you haven't
cue-275
00:19:21.247 --> 00:19:25.448
read it is, is incredible. Everyone should
read it. So this is her talking about
cue-276
00:19:25.448 --> 00:19:29.788
critical fabulation. "The intent of this
practice is not to give voice to the
cue-277
00:19:29.788 --> 00:19:34.688
slaves, but rather to imagine what cannot
be verified, a realm of experience which
cue-278
00:19:34.688 --> 00:19:39.388
is situated between two zones of death,
social and corporeal death, and to reckon
cue-279
00:19:39.388 --> 00:19:42.908
with the precarious lives which are
visible only in the moment of their
cue-280
00:19:42.908 --> 00:19:47.108
disappearance. It is an impossible writing
which attempts to say that which resists
cue-281
00:19:47.108 --> 00:19:51.088
being said, since dead girls are unable to
speak. It is a history of an
cue-282
00:19:51.088 --> 00:19:55.948
unrecoverable past, a narrative of what
might have been or could have been. It is
cue-283
00:19:55.948 --> 00:20:00.348
a history written with and against the
archive. Admittedly, my own writing is
cue-284
00:20:00.348 --> 00:20:05.248
unable to exceed the limits of the sayable
dictated by the archive. It depends upon
cue-285
00:20:05.248 --> 00:20:10.108
the legal records, surgeons' journals,
ledgers, ships' manifests, and captains'
cue-286
00:20:10.108 --> 00:20:15.068
logs, and in this regard falters before
the archive's silence and reproduces its
cue-287
00:20:15.068 --> 00:20:20.688
omission. The irreparable violence of the
Atlantic slave trade rests precisely in
cue-288
00:20:20.688 --> 00:20:24.448
all the stories that we cannot know and
that will never be recovered. This
cue-289
00:20:24.448 --> 00:20:29.488
formidable obstacle or constitutive
impossibility defines the parameters of my
cue-290
00:20:29.488 --> 00:20:34.608
work. The necessity of recounting Venus's
death is overshadowed by the inevitable
cue-291
00:20:34.608 --> 00:20:39.968
failure of any attempt to represent her. I
think this is a productive tension and
cue-292
00:20:39.968 --> 00:20:44.998
one unavoidable in narrating the lives of
the subaltern, the dispossessed, and the
cue-293
00:20:44.998 --> 00:20:51.167
enslaved." So let's stop for a second. An
LLM could never... I would give limbs to
cue-294
00:20:51.168 --> 00:20:54.288
be able to write like that. My God.
cue-295
00:20:54.958 --> 00:21:01.548
And also, she's very deliberately saying,
"Don't do the thing that's being done in
cue-296
00:21:01.548 --> 00:21:06.668
this other essay." She's very deliberately
saying that the whole point of sitting
cue-297
00:21:06.668 --> 00:21:13.648
with archival silence as a human in the
world is to sit in its totality. It is
cue-298
00:21:13.648 --> 00:21:15.808
to, it is to do all of those things.
cue-299
00:21:16.498 --> 00:21:17.868
We're supposed to sit with it.
cue-300
00:21:18.528 --> 00:21:23.227
Applying AI to this process is really
removing the humanity of both the subject
cue-301
00:21:23.228 --> 00:21:28.227
and the researchers themselves and, and
really speaks to a bunch of things posed
cue-302
00:21:28.228 --> 00:21:32.608
by this, you know, critical confabulation
paper about how the humanities are
cue-303
00:21:32.608 --> 00:21:36.888
incentivized right now. But Sadie, you've
got a comment in here, and I
wanna make
cue-304
00:21:36.888 --> 00:21:38.667
sure that, that you're in as well.
cue-305
00:21:39.428 --> 00:21:45.708
Oh, I just, uh, when I read that and the,
the tension is necessary and we are
cue-306
00:21:45.708 --> 00:21:50.488
supposed to sit with it, I just-- What
immediately came to mind was the person on
cue-307
00:21:50.488 --> 00:21:56.988
what is formerly known as Twitter who
took Keith Haring's unfinished painting
cue-308
00:21:56.988 --> 00:22:02.788
and finished it with AI. And then when
people lashed back at that, was like,
cue-309
00:22:02.788 --> 00:22:05.948
"Well, I didn't really know the
significance of this painting or the
cue-310
00:22:05.948 --> 00:22:09.768
history behind it. I just thought it was
kind of funny, and I'd probably do it
cue-311
00:22:09.768 --> 00:22:13.338
again even if I knew the significance of
it." And I'm just like,
cue-312
00:22:14.348 --> 00:22:18.808
h-h-how can the point of art be so missed?
cue-313
00:22:19.788 --> 00:22:26.588
Especially because the AIDS crisis is one
of those things that is really hard for me
cue-314
00:22:26.588 --> 00:22:32.048
to sit with when I think about, like,
what, all of what was lost. So Keith
cue-315
00:22:32.048 --> 00:22:37.148
Haring's painting, like, really pulls
something out of me. And then to see that
cue-316
00:22:37.148 --> 00:22:41.448
and just be like, it's not, it's not even
like the th- the things that make Keith
cue-317
00:22:41.448 --> 00:22:48.068
Haring's art are not represented at all in
that a, that, you know, AI finished
cue-318
00:22:48.068 --> 00:22:52.608
version. The, the motifs are completely
messed up. The only thing you could say is
cue-319
00:22:52.608 --> 00:22:55.168
that they used the same colors and lines-
cue-320
00:22:55.168 --> 00:22:58.868
-Yeah
-... which, like, is not art. So yes, this
cue-321
00:22:58.868 --> 00:22:59.728
is, I--
cue-322
00:23:00.608 --> 00:23:03.908
It's, it's the, what is it? The torture
nexus? The thing that people are like,
cue-323
00:23:03.908 --> 00:23:05.308
-"Oh, yeah, the-"
-Torment nexus, yeah.
cue-324
00:23:05.308 --> 00:23:08.848
The torment nexus. You know, the per- the,
the guy who writes, "Hey, don't make the
cue-325
00:23:08.848 --> 00:23:11.908
torment nexus," and then somebody turns
around and goes, "So we made the torment
cue-326
00:23:11.908 --> 00:23:15.628
nexus from the great book, 'Don't Make the
Torment Nexus.'" It's just...
cue-327
00:23:16.548 --> 00:23:18.448
-Yeah.
-Whereas, like, what this reminded me of,
cue-328
00:23:18.448 --> 00:23:23.548
like, 'cause what I feel like, uh, these
authors and these, like, these researchers
cue-329
00:23:23.548 --> 00:23:27.527
are missing the point of what critical
fabu- Like, there's, like, this, like,
cue-330
00:23:27.528 --> 00:23:31.428
this sitting with the tension, this, like,
human reckoning with it, but, like, also
cue-331
00:23:31.428 --> 00:23:37.967
critical fabulation doesn't fix the
problem of archival silence. If
cue-332
00:23:37.968 --> 00:23:43.608
anything, it brings more attention to the
fact that it cannot be, be fixed. But it
cue-333
00:23:43.608 --> 00:23:47.527
also reminds me of, like, what immediately
came to mind was the film that we've
cue-334
00:23:47.528 --> 00:23:51.608
talked about on here before called The
Watermelon Woman, which is an act of sort
cue-335
00:23:51.608 --> 00:23:56.828
of, like, highlighting the absence of
knowledge of, uh,
cue-336
00:23:57.428 --> 00:24:04.188
Black queer women in early cinema history
by doing a sort of mockumentary
cue-337
00:24:04.188 --> 00:24:10.668
around this made-up Black lesbian in this
movie that she's tracking down. But, and
cue-338
00:24:10.668 --> 00:24:15.268
framing it as it's, like, a form of, like,
myth-making in order to show that, like,
cue-339
00:24:15.268 --> 00:24:20.588
this isn't real, but we're doing this sort
of, like, radical myth-making around it
cue-340
00:24:20.588 --> 00:24:25.768
to show the fact that, you know, people
like her probably did exist in cinema-
cue-341
00:24:26.408 --> 00:24:33.388
But we don't have record of that because
of, like, the white supremacist nature of,
cue-342
00:24:33.388 --> 00:24:38.128
of this country and this industry, right?
Like, the point of the watermelon woman
cue-343
00:24:38.128 --> 00:24:43.028
isn't to, like, "Surprise, we fixed and
found this and made this up." It's to,
cue-344
00:24:43.028 --> 00:24:48.558
like, construct this narrative and do this
kind of, like, radical myth-making
cue-345
00:24:49.168 --> 00:24:55.948
as a way of showing the, the inherent
violence of the, the record, right?
cue-346
00:24:55.948 --> 00:24:59.418
Yeah, it's, it's... You know, and, and
Saidiya Hartman, you know, talks about
cue-347
00:24:59.418 --> 00:25:04.208
this extensively in, in this essay and in
other works that, that to work with and
cue-348
00:25:04.208 --> 00:25:09.488
against it is to... is you're essentially
w- working with and against power in going
cue-349
00:25:09.488 --> 00:25:13.068
and finding, you know, all of these
particular narratives. And so,
cue-350
00:25:13.728 --> 00:25:17.208
you know, one of the questions that I, I
sort of have when I'm looking at this
cue-351
00:25:17.208 --> 00:25:22.108
paper, and others like it, uh, are, are
really about how the humanities are
cue-352
00:25:22.108 --> 00:25:27.328
incentivized now in that, you know, not
just these particular authors, but, but
cue-353
00:25:27.328 --> 00:25:31.507
also members of, say, you know, the
American Historical Association who never
cue-354
00:25:31.507 --> 00:25:35.168
saw a bad idea that they weren't willing
to endorse, and that includes things like
cue-355
00:25:35.168 --> 00:25:39.888
the use of AI to create historical
artifacts, which is absolutely
cue-356
00:25:39.888 --> 00:25:45.688
mind-blowing that that endorsement is
there. But that so many humanists are in
cue-357
00:25:45.688 --> 00:25:49.568
some ways abandoning their own training,
and I think that's, you
know, fairly
cue-358
00:25:49.568 --> 00:25:53.668
interesting. This is the humanities in a
lot of ways trying to thrive under
cue-359
00:25:53.668 --> 00:25:58.388
fascistic conditions because those are the
conditions under which things like large
cue-360
00:25:58.388 --> 00:26:01.568
language models were developed, and
they're the conditions which sustain
cue-361
00:26:01.568 --> 00:26:06.488
higher educations in interest and
investment in AI tools, you know, in a lot
cue-362
00:26:06.488 --> 00:26:11.388
of ways. And so when we look at, like, I
have a big list about things here about,
cue-363
00:26:11.388 --> 00:26:15.948
like, what the paper is trying to do and
what it's, and what it's actually kind of
cue-364
00:26:15.948 --> 00:26:21.888
doing, and I keep coming back to, like,
LLMs as we understand them, even the open
cue-365
00:26:21.888 --> 00:26:27.828
models were essentially trained on, you
know, angry Yelp reviews and subreddits of
cue-366
00:26:27.828 --> 00:26:32.488
people that can't be within 500 feet of a
school. What do you think they're gonna
cue-367
00:26:32.488 --> 00:26:39.428
say about Black life and Black experience?
Like, how could... And, and I just don't
cue-368
00:26:39.428 --> 00:26:45.427
understand, you know, so much about w- why
we would
cue-369
00:26:45.508 --> 00:26:51.088
use tools that enforce and tend toward the
average if we're, if we're really trying
cue-370
00:26:51.088 --> 00:26:58.028
to find something extraordinary. If, if
we're really trying to... If my goal as a,
cue-371
00:26:58.028 --> 00:27:03.028
as a historian is to find, you know,
extraordinary lives in the mundane and try
cue-372
00:27:03.028 --> 00:27:06.908
to find those narratives, why would I use
a machine that insists on creating a
cue-373
00:27:06.908 --> 00:27:12.168
narrative mean to identify those silences?
It just becomes a trope and, and not
cue-374
00:27:12.168 --> 00:27:16.068
history. And if I accept, you know, a
claim that, you know, sort of showed up
cue-375
00:27:16.068 --> 00:27:20.088
from my y- undergraduate background that,
you know, if history really is a
cue-376
00:27:20.088 --> 00:27:24.128
conversation between the, the present with
the past to understand the future, and I
cue-377
00:27:24.128 --> 00:27:28.927
remove the humanity from one or more parts
of that, what have I done? That's not
cue-378
00:27:28.928 --> 00:27:32.788
accepting the totality of those
experiences and stories in the context of
cue-379
00:27:32.788 --> 00:27:34.208
their silent like,
cue-380
00:27:34.268 --> 00:27:36.008
like "Venus in Two Acts" is trying to do,
cue-381
00:27:36.768 --> 00:27:39.978
and that, you know, the authors are
claiming to be inspired by. It's a kind of
cue-382
00:27:39.978 --> 00:27:45.388
necromancy, and I, I don't think I have
words that are polite for that.
cue-383
00:27:45.388 --> 00:27:47.198
I feel like a lot of... 'Cause I,
cue-384
00:27:47.968 --> 00:27:52.638
I went and read Hartman's "Venus in Two
Acts" and, 'cause
cue-385
00:27:53.268 --> 00:27:58.288
I hadn't read it before, 'cause I wanted
to... I, I had this thought that a lot of
cue-386
00:27:58.288 --> 00:28:01.448
the language that they use in this article
that they don't fully understand, like
cue-387
00:28:01.448 --> 00:28:02.608
the way they use archive,
cue-388
00:28:03.268 --> 00:28:07.088
uh, sounds like, and I think the way
Hartman uses it is in sort of like the
cue-389
00:28:07.088 --> 00:28:12.087
media studies term where archive means,
like, the archive of humanity, like the
cue-390
00:28:12.088 --> 00:28:17.808
things that we know in terms of all of
writing and what is allowed to be known
cue-391
00:28:17.808 --> 00:28:21.948
politically and what's in our political
discourses, and I think that's the media
cue-392
00:28:21.948 --> 00:28:25.868
way that she's using it. But they seem to
think like, "Oh, there should be archives
cue-393
00:28:25.868 --> 00:28:30.528
where we can go in and grab other data
from the archive and fill in gaps in the
cue-394
00:28:30.528 --> 00:28:36.088
actual archive." And some of these other
papers that they're citing are about,
cue-395
00:28:36.088 --> 00:28:38.898
like, making a case for Black digital
humanities,
cue-396
00:28:39.648 --> 00:28:45.527
because they also use the word recover in
a lot of ways that I don't think they are
cue-397
00:28:45.527 --> 00:28:50.908
using correctly, but I didn't have access
to... Like, I wasn't quite sure. Like, I
cue-398
00:28:50.908 --> 00:28:55.588
pulled one up right now to see, like, uh,
how the word, like... You know, "Recovery
cue-399
00:28:55.588 --> 00:28:59.158
rests at the heart of Black studies as a
scholarly tradition that seeks to restore
cue-400
00:28:59.158 --> 00:29:02.568
the humanity of Black people lost and
stolen through systemic global
cue-401
00:29:02.568 --> 00:29:07.088
racialization. Recovering lost historical
and literary text should be foundational
cue-402
00:29:07.088 --> 00:29:08.698
to the Black digital humanities.
cue-403
00:29:09.488 --> 00:29:13.908
Recovering alternate constructions of
humanity that have been historically
cue-404
00:29:13.908 --> 00:29:17.108
excluded from that concept. Politics of
recovery,
cue-405
00:29:17.928 --> 00:29:21.668
not only as the recovery of lost or
non-canonical and difficult to locate
cue-406
00:29:21.668 --> 00:29:26.508
texts, but also recovery of Black authors'
humanity." See, that's what I thought
cue-407
00:29:26.508 --> 00:29:30.107
exactly they were, like, overlooking.
They're like, "Oh, we can find these
cue-408
00:29:30.108 --> 00:29:34.588
things," and there's one thing they say
that like, "We can at scale the... We can
cue-409
00:29:34.588 --> 00:29:37.358
scale the discovery of divergent yet
evidence bound narrative."
cue-410
00:29:37.358 --> 00:29:40.228
That was the line that made me rip my
fucking hair out.
cue-411
00:29:40.228 --> 00:29:42.768
-[laughs]
-At scale.
cue-412
00:29:43.488 --> 00:29:49.548
[laughs] Yeah, like when was scale
something that we valued in this co-... I
cue-413
00:29:49.548 --> 00:29:54.408
don't even know. We're gonna have a
database of known unknowns, Dorothea. It's
cue-414
00:29:54.408 --> 00:29:59.748
-gonna be great.
-And unknown unknowns. Yeah. Rumsfeld. So
cue-415
00:29:59.748 --> 00:30:06.208
there's also this, this, like, lack of
care, which is, like, in, like, central to
cue-416
00:30:06.208 --> 00:30:09.368
Hartman of, like, there's a line that
cue-417
00:30:10.008 --> 00:30:13.088
I outlined which is, "I want to tell the
story about two girls capable of
cue-418
00:30:13.088 --> 00:30:18.308
retrieving what remains dormant, the
purchase or claim of their lives on the
cue-419
00:30:18.308 --> 00:30:23.368
present without committing further
violence in my own act of narration." So
cue-420
00:30:23.368 --> 00:30:27.888
all of this is about, like, the humanity
of the subject, the impossibility of
cue-421
00:30:27.888 --> 00:30:34.276
recovery, the impossibility of filling in
the gaps- And I don't know if you all read
cue-422
00:30:34.276 --> 00:30:38.416
past the footnotes, but you can see,
like, what stories they use, and one of
cue-423
00:30:38.416 --> 00:30:40.416
the stories is, like, this
cue-424
00:30:41.116 --> 00:30:46.876
14-year-old Black child who gets shot, and
that's, like, part of the story they,
cue-425
00:30:46.876 --> 00:30:51.276
they, like, cut out, and they do a close
study on it. It just seems like there's
cue-426
00:30:51.276 --> 00:30:56.536
really no care. Someone on Blue Sky
pointed out that it can't be critical
cue-427
00:30:56.536 --> 00:31:00.836
fabulation or critical confabulation if
there is no one being critical. Like, the
cue-428
00:31:00.836 --> 00:31:05.556
critical part of it is, like, about
critical studies, and if there's no agent
cue-429
00:31:05.556 --> 00:31:10.196
being critical about the, the materials,
then there is no criticality actually
cue-430
00:31:10.196 --> 00:31:13.456
happening. And it just seems like these
authors really don't care about the
cue-431
00:31:13.456 --> 00:31:17.256
subject matter in any way, especially
considering the other stuff they're
cue-432
00:31:17.256 --> 00:31:19.196
authoring, like why AI slop matters.
cue-433
00:31:20.016 --> 00:31:21.186
-I keep-
-And yeah
cue-434
00:31:21.186 --> 00:31:27.416
... coming at this because I am always
charmed whenever I get to use the word
cue-435
00:31:27.416 --> 00:31:31.836
bullshit in a serious academic context.
This, of course, goes back to the
cue-436
00:31:31.836 --> 00:31:36.556
philosopher Harry G. Frankfurt on
bullshit. But there's also a, a charming
cue-437
00:31:36.556 --> 00:31:40.896
piece in the journal Ethics and
Information Technology. I'm sure most of
cue-438
00:31:40.896 --> 00:31:42.976
you read it. ChatGPT is bullshit.
cue-439
00:31:43.856 --> 00:31:49.666
And it turns out that bullshit has a very
specific definition deriving from
cue-440
00:31:49.666 --> 00:31:55.076
Frankfurt, which is truth? Who cares about
truth? I
don't give a fuck about truth.
cue-441
00:31:55.076 --> 00:32:01.116
I'm just going to, you know, send my slop
out there into the world, and its truth
cue-442
00:32:01.116 --> 00:32:07.476
value literally doesn't matter, and I may
not even know it. And that so strikes me
cue-443
00:32:07.476 --> 00:32:12.556
as what's happening here. What these
people are doing is bullshitting because
cue-444
00:32:12.556 --> 00:32:18.416
they don't care about the truth of any of
these people, and they're filling in or,
cue-445
00:32:18.416 --> 00:32:25.056
uh, de- demarcating, whatever, these
silences, as you say, without care.
cue-446
00:32:25.056 --> 00:32:30.916
You have to care about these people. I
think you have to care about their truth,
cue-447
00:32:30.916 --> 00:32:37.636
and you have to care about, about their
truth getting cut off. And I
cue-448
00:32:37.636 --> 00:32:41.815
just... Ugh, it really irritates me the
amount of bullshit that is just
cue-449
00:32:41.815 --> 00:32:45.916
accumulating in this supposed research
method.
cue-450
00:32:45.916 --> 00:32:51.176
Yeah, that leads me, like, to a question I
have for the, the two of you
cue-451
00:32:51.856 --> 00:32:56.176
'cause, like, what this... One of the
problems that I was having with this
cue-452
00:32:56.176 --> 00:33:01.135
article was that, like, they, they are
citing so many, like, scholars in the
cue-453
00:33:01.136 --> 00:33:04.896
Black digital humanities and, like, the
post-colonial digi- digital humanities.
cue-454
00:33:04.896 --> 00:33:08.426
They might not be citing them well or
accurately, but they are citing them. They
cue-455
00:33:08.426 --> 00:33:12.695
exist. And it's like, this is a problem
that I've had with, like, the AI, like,
cue-456
00:33:12.696 --> 00:33:17.076
with the fact that, like, AI is often used
as a marketing term to include all these
cue-457
00:33:17.076 --> 00:33:22.596
other technologies, some of which have
been used in the digital humanities for
cue-458
00:33:22.596 --> 00:33:27.816
years, right? As a way of, like,
approaching specific types of research
cue-459
00:33:27.816 --> 00:33:33.136
questions, right? Like, they're not about
solving a problem or fixing something, but
cue-460
00:33:33.136 --> 00:33:38.396
it's like, here is a different way to
approach this research question. What can
cue-461
00:33:38.396 --> 00:33:41.436
we get out of it, right? And people do
interesting things with that.
cue-462
00:33:41.436 --> 00:33:44.636
-Sure.
-And it's like they're taking, like, like
cue-463
00:33:44.636 --> 00:33:48.556
the way that someone in... I, I hope
Justin's burp shows up on my recording
cue-464
00:33:48.556 --> 00:33:48.756
now.
cue-465
00:33:49.616 --> 00:33:50.616
[laughs]
cue-466
00:33:52.016 --> 00:33:57.076
Like, they're, they're... Like, the way
that the people in the humanities and,
cue-467
00:33:57.076 --> 00:34:00.736
like, the digital humanities, especially,
like, in, like, Black and post-colonial
cue-468
00:34:00.736 --> 00:34:07.586
digital humanities are approaching this
research question or this
cue-469
00:34:07.586 --> 00:34:08.936
area of,
cue-470
00:34:09.776 --> 00:34:15.176
like, silence in the archival record,
however we are using archive to either
cue-471
00:34:15.176 --> 00:34:20.856
mean literal or more broadly, right? Like,
this, this question of power and, and
cue-472
00:34:20.856 --> 00:34:27.776
violence affecting historical record and,
like, how people in those disciplines
cue-473
00:34:27.776 --> 00:34:33.915
are approaching it versus, like, you know,
there's very similar technologies, like
cue-474
00:34:33.916 --> 00:34:35.956
-natural language processing-
-Yes
cue-475
00:34:35.956 --> 00:34:40.176
... that these schol... The, these
authors, these scholars are using, that
cue-476
00:34:40.176 --> 00:34:44.356
people in the digital humanities are
using. It's, like, not that different, but
cue-477
00:34:44.356 --> 00:34:48.935
it's such a different way of coming about
it, and it's like, is it because of,
cue-478
00:34:48.936 --> 00:34:54.376
like, the dis... Like, is there something
in the epistemology or the methodology of,
cue-479
00:34:54.376 --> 00:34:59.256
like, these disciplines that makes them,
like... 'Cause it feels like the, the AI
cue-480
00:34:59.256 --> 00:35:04.476
c- confabulation folks, it's like it's a
problem to solve instead of a way of
cue-481
00:35:04.476 --> 00:35:05.666
exploring
cue-482
00:35:06.356 --> 00:35:09.456
a research question. It's like, no,
there's a problem, and we can solve it
cue-483
00:35:09.456 --> 00:35:13.246
with the data. Like, I didn't know if
y'all could speak to that at all.
cue-484
00:35:13.856 --> 00:35:19.396
So for me, there are a couple of things
that come into play. One of them is just
cue-485
00:35:19.396 --> 00:35:25.156
straight up accountability, right? And
that ties into the explainability of some
cue-486
00:35:25.156 --> 00:35:29.856
of these methods. With a natural language
processing pipeline,
cue-487
00:35:30.536 --> 00:35:35.856
it's not quite what I would call
completely deterministic, but you can take
cue-488
00:35:35.856 --> 00:35:39.136
the process apart, you can interrogate
it,
cue-489
00:35:39.796 --> 00:35:46.256
you can tweak parameters and see what
happens, right? And I think that's part of
cue-490
00:35:46.256 --> 00:35:52.476
being responsible as a researcher. With
gen AI, there's, there's no
cue-491
00:35:52.476 --> 00:35:53.716
explainability.
cue-492
00:35:54.396 --> 00:35:59.796
If you give it the same prompt twice in a
row, you can get wildly different answers.
cue-493
00:35:59.796 --> 00:36:06.756
So, and so if you can't in any way...
It's the lack of control, I guess, that's
cue-494
00:36:06.756 --> 00:36:11.176
really bugging me. There are a lot of ways
to control
cue-495
00:36:11.796 --> 00:36:17.276
an NLP pipeline and make it do more or
less precisely what you want. With
cue-496
00:36:17.276 --> 00:36:24.176
generative AI, you cannot do that. It is
not controllable in this
way, which means
cue-497
00:36:24.176 --> 00:36:29.156
I think it's really, really irresponsible
to call this a research method. Does that
cue-498
00:36:29.156 --> 00:36:32.496
-speak to your question at all?
-Somewhat, 'cause I, I do agree with you,
cue-499
00:36:32.496 --> 00:36:34.576
yes. I, I was more like, is it some...
Like,
cue-500
00:36:35.376 --> 00:36:39.060
I guess I was, like, thinking about, like-
Is there something inherent in these two
cue-501
00:36:39.060 --> 00:36:44.080
disciplines of like the way they approach
epistemology and methodology? Which I
cue-502
00:36:44.080 --> 00:36:51.040
guess like if you are willing to use an
LLM generative AI as a legitimate form of
cue-503
00:36:51.040 --> 00:36:53.940
research, that does say something about
how you approach methodology.
cue-504
00:36:54.920 --> 00:36:58.800
But like it was just like I was so
astounded of like why are they looking at
cue-505
00:36:58.800 --> 00:37:05.200
digital humanities stuff? Like what about
the like this research area, this research
cue-506
00:37:05.200 --> 00:37:11.320
question, this very humanities thing that
just like there are t- you know, techy
cue-507
00:37:11.320 --> 00:37:16.600
ways of exploring it. What is it about
this that was interesting to these authors
cue-508
00:37:16.600 --> 00:37:18.670
-that they're approaching it-
-Well, I mean, you know, I-
cue-509
00:37:18.670 --> 00:37:23.920
-... this way and are doing it so badly?
-I think we've seen this before, though.
cue-510
00:37:23.920 --> 00:37:29.980
What was it that, uh, that, that c- uh,
shoot. Google, when Google Books happened,
cue-511
00:37:29.980 --> 00:37:36.460
there were these absolu- there was a raft
of absolutely craptastic attempts at
cue-512
00:37:36.460 --> 00:37:40.800
linguistic and discourse analysis by
people who didn't really understand the
cue-513
00:37:40.800 --> 00:37:45.100
limitations of the corpus, didn't
understand things like long S. That's a
cue-514
00:37:45.100 --> 00:37:49.140
thing. It's gonna mess up your word
counts, y'all. Culturomics, is that what
cue-515
00:37:49.140 --> 00:37:53.390
they called it? Whatever they called it,
it was trash. And it's,
cue-516
00:37:54.000 --> 00:37:59.460
to the extent I can be sympathetic to it
at all, it's, as you said, a little bit
cue-517
00:37:59.460 --> 00:38:04.480
of, of playing with blocks and let's see
what happens. And there's nothing
cue-518
00:38:04.480 --> 00:38:11.220
intrinsically wrong with that impulse, I
don't think. But you have to be honest
cue-519
00:38:11.220 --> 00:38:16.680
about your limitations and the limita- y-
your limitations of understanding most of
cue-520
00:38:16.680 --> 00:38:17.060
all
cue-521
00:38:17.740 --> 00:38:24.740
if you don't have control over... And boy,
I saw this in, in, in DH too. If
cue-522
00:38:24.740 --> 00:38:30.640
you don't have control over your
understanding of what's in your data, your
cue-523
00:38:30.640 --> 00:38:36.540
understanding of how the pipeline works,
there's a lot of DH work that just happily
cue-524
00:38:36.540 --> 00:38:41.680
adopts whatever tool and basically treats
it as a black box, which I think is very
cue-525
00:38:41.680 --> 00:38:47.440
much what's happening here. Yeah, it does
not do much for the legitimacy of your
cue-526
00:38:47.440 --> 00:38:51.020
-results. Let me put it that way.
-Yeah. I think your, I think your question
cue-527
00:38:51.020 --> 00:38:56.020
is a really good one. The thing that I
pulled from the article too is, is one
cue-528
00:38:56.020 --> 00:38:59.640
thing I keep coming back to is that
especially when you're looking at a, a
cue-529
00:38:59.640 --> 00:39:04.680
narrative closed task or series of tasks
in, in natural language processing is that
cue-530
00:39:04.680 --> 00:39:09.339
tools measure the limits of experience,
right? So are we looking for narrative
cue-531
00:39:09.340 --> 00:39:15.500
structure in archival silences using a
colonial or supremacist or inherently like
cue-532
00:39:15.500 --> 00:39:20.700
white sort of notion of, of storytelling
or way of kind of filling that, you know,
cue-533
00:39:20.700 --> 00:39:24.320
in that blank? One of the interesting
things about the methodology behind the
cue-534
00:39:24.320 --> 00:39:29.370
paper is that they exclude, those
researchers exclude folklore and song from
cue-535
00:39:29.370 --> 00:39:33.000
that original database. And I'm like,
"Oh, you mean the way that people who t-
cue-536
00:39:33.000 --> 00:39:38.100
who don't have power and are displaced and
dispossessed tend to deal with the fact
cue-537
00:39:38.100 --> 00:39:42.200
that they are not in the, in the archive
as it's, you know, either narrowly or
cue-538
00:39:42.200 --> 00:39:46.280
broadly conceived, like the way that we
can be... We're just gonna exclude that,
cue-539
00:39:46.280 --> 00:39:50.840
and we're going to exclude that and, you
know, in, in something that, that, you
cue-540
00:39:50.840 --> 00:39:55.700
know, is ostensibly trying to reveal where
potential silences might be when we're
cue-541
00:39:55.700 --> 00:40:00.820
talking about Black histories and, and
culture." So like one of the quotes that I
cue-542
00:40:00.820 --> 00:40:03.380
pulled was that, you know, the
implications are twofold. For natural
cue-543
00:40:03.380 --> 00:40:08.240
language processing, we demonstrate a
domain application where hallucinations
cue-544
00:40:08.240 --> 00:40:12.880
become a unique and optimizable resource
rather than a pure deficit. For the
cue-545
00:40:12.880 --> 00:40:16.560
humanities, we highlight the narrative
understanding capabilities of LLMs that
cue-546
00:40:16.560 --> 00:40:20.860
make them viable tools for helping
scholars probe the latent semantic spaces
cue-547
00:40:20.860 --> 00:40:25.900
of large archives and to surface unknown
unknowns and then to rapidly prototype
cue-548
00:40:25.900 --> 00:40:30.600
candidate reconstructions to narrow the
bounds of the known unknowns, right? So
cue-549
00:40:30.600 -->
00:40:34.100
that's, you know, what, you know, what
they're trying to do. Again, it's like i-
cue-550
00:40:34.100 --> 00:40:38.460
in service to what? Because they're,
you're not, there is not that, you know,
cue-551
00:40:38.460 --> 00:40:44.380
human experience. There isn't that sitting
with the incredible weight of it, of what
cue-552
00:40:44.380 --> 00:40:48.620
it means to be near an unfinished
painting. You know, I have, or unfinished
cue-553
00:40:48.620 --> 00:40:52.380
art. I've actually have had a similar
experience with another artist, the, I
cue-554
00:40:52.380 --> 00:40:57.600
can't remember the name of the, of them
at, right now, but it was a series of when
cue-555
00:40:57.600 --> 00:41:04.300
we first visualized HIV, it was a series
of crystalline and glass-blown
cue-556
00:41:05.040 --> 00:41:09.629
like sort of large scale pieces of what
the virus looks like, you know, blown up
cue-557
00:41:09.629 --> 00:41:14.180
and set so you could see it. And it was
someone who, you know, was living with
cue-558
00:41:14.180 --> 00:41:18.400
HIV, who has since passed away, who was
the artist who did that, and he s- said,
cue-559
00:41:18.400 --> 00:41:21.640
you know, "It is incredibly powerful and
important to me to be able to visualize
cue-560
00:41:21.640 --> 00:41:26.980
the thing at scale that will kill me." And
that there's this, there is an incredible
cue-561
00:41:26.980 --> 00:41:33.360
sort of weight of loss and potential and
possibility and, you know, sort of the,
cue-562
00:41:33.360 --> 00:41:37.620
the micro and macro and way that, you
know, the art is en- is engaged with
cue-563
00:41:37.620 --> 00:41:42.700
itself and, and, and all of that kind of
work. And, and if you don't do that, if
cue-564
00:41:42.700 --> 00:41:47.280
instead there's just we're going to
create, you know, sort of a corpus of
cue-565
00:41:47.280 --> 00:41:50.980
silences or something... I'm just, I'm
trying to get my head around the use case
cue-566
00:41:50.980 --> 00:41:57.080
for this because so much of what forms
the, the, the confrontation of power that
cue-567
00:41:57.080 --> 00:42:01.180
Saidiya Hartman is writing about and that
these people are supposed to be inspired
cue-568
00:42:01.180 --> 00:42:06.180
by is that confronting of that power. And
so I don't, yeah, I'm just kind of
cue-569
00:42:06.180 --> 00:42:06.720
wondering,
cue-570
00:42:07.400 --> 00:42:11.660
does it even work when you've, when
you've, you know, effectively removed the
cue-571
00:42:11.660 --> 00:42:15.520
people from that equation at all? I know I
keep coming back to that question, but
cue-572
00:42:15.520 --> 00:42:21.100
it's, it is very strange to me and, and I,
I wonder so much at the power of
cue-573
00:42:21.100 --> 00:42:25.640
incentives that are working in the
humanities now because of this kind of
cue-574
00:42:25.640 --> 00:42:28.340
-stuff.
-I mean, I imagine they're trying to sell
cue-575
00:42:28.340 --> 00:42:32.879
something. They're trying to like s-
I, I
think why like why focus on Black
cue-576
00:42:32.880 --> 00:42:39.620
scholarship? Why focus on this corpus? Why
focus on specifically archival work?
cue-577
00:42:40.132 --> 00:42:46.892
Where you could feed a lot of information
in and create these, like, possibilities
cue-578
00:42:46.892 --> 00:42:51.691
-of timeline filling events
-I have a hypothesis.
cue-579
00:42:51.692 --> 00:42:54.492
-Yeah, go ahead.
-And that hypothesis is
cue-580
00:42:55.132 --> 00:43:02.092
these people are deeply uncomfortable with
the idea of, of silence. It
cue-581
00:43:02.092 --> 00:43:08.032
really bugs them. And in a very, and I'm a
white woman, so I can say this, in a
cue-582
00:43:08.032 --> 00:43:13.952
white woman's tears fashion, they're
trying to appropriate these
cue-583
00:43:13.952 --> 00:43:20.032
silences to fill them. What's the phrase
they use? Let me look at this. Oh yeah,
cue-584
00:43:20.032 --> 00:43:26.932
prototype candidate reconstruction. What
the hell is that? But [laughing]
cue-585
00:43:26.932 --> 00:43:33.852
yeah, it is absolute wankery. You are
correct. But I, I don't think they can do
cue-586
00:43:33.852 --> 00:43:40.392
the work that Scarlet says is so
important, which is just sitting with that
cue-587
00:43:40.392 --> 00:43:45.812
silence, accepting that it will not be
filled ever in the absence of more
cue-588
00:43:45.812 --> 00:43:51.452
discoveries, which can always happen, but
probably won't. And so they're bringing in
cue-589
00:43:51.452 --> 00:43:55.872
the bullshit machine because they can't
think of what else to do. It's, you know,
cue-590
00:43:55.872 --> 00:43:57.592
it's like the AI
cue-591
00:43:58.212 --> 00:44:03.422
significant others. It's, it's profoundly
sad and also profoundly messed up.
cue-592
00:44:03.422 --> 00:44:08.352
'Cause like AI also can't handle silences,
right? Like that's why we get
cue-593
00:44:08.352 --> 00:44:13.432
hallucinations in the first place because
it, it's, it's just doing pattern matching
cue-594
00:44:13.432 --> 00:44:17.512
what makes sense to come next, and if
there's nothing it can't reliably put in,
cue-595
00:44:17.512 --> 00:44:19.252
-well, it's gonna make something up.
-Yep.
cue-596
00:44:19.252 --> 00:44:22.772
Right? Like that's the whole fucking point
of these
cue-597
00:44:23.392 --> 00:44:29.372
generative AI, like LLMs, is to do that
kind of we can't handle the fact that
cue-598
00:44:29.372 --> 00:44:30.972
-there is an unknown.
-Right.
cue-599
00:44:30.972 --> 00:44:35.032
So I will make something up. Like, like
the tool itself,
cue-600
00:44:35.892 --> 00:44:41.652
you know, the, I'm gonna get all Mc-
McLuhan, video drone, like the medium is
cue-601
00:44:41.652 --> 00:44:48.172
the message kind of thing. But like when
literally the tool itself is something
cue-602
00:44:48.172 --> 00:44:55.052
that like abhors a silence, abhors a void,
like how can you do meaningful research
cue-603
00:44:55.052 --> 00:44:58.852
or even attempt to explore this question
cue-604
00:44:59.452 --> 00:45:02.892
-with this tool?
-Right on. I totally agree with you.
cue-605
00:45:02.892 --> 00:45:07.672
I was reading like their conclusions,
which is like two paragraphs, and they say
cue-606
00:45:07.672 --> 00:45:11.092
they're exploring a wide range of use
cases for critical confabulation to
cue-607
00:45:11.092 --> 00:45:16.112
support humanistic scholarship and augment
human storytelling towards new AI-enabled
cue-608
00:45:16.112 --> 00:45:20.872
methods for studying culture and history
for para- paradigm shifts, whatever. And
cue-609
00:45:20.872 --> 00:45:25.252
then it says like this work is preliminary
in nature. Future work will, will design
cue-610
00:45:25.252 --> 00:45:29.212
more robust and comprehensive evaluations
for open-ended narrative outputs,
cue-611
00:45:29.892 --> 00:45:34.852
broaden coverage across languages, build
ethical safeguards and provenance tracking
cue-612
00:45:34.852 --> 00:45:39.372
to ensure faithful event reconstruction,
and avoid compounding archival violence.
cue-613
00:45:39.372 --> 00:45:44.132
-But faithful to what?
-How could you do any of that?
cue-614
00:45:44.132 --> 00:45:48.712
[laughs] And you know, speaking as a
sometime storyteller, uh, who told them
cue-615
00:45:48.712 --> 00:45:52.472
that sto- that, that human storytelling
needed augmentation, especially by
cue-616
00:45:52.472 --> 00:45:55.572
bullshit machines? Just miss me with that.
cue-617
00:45:55.572 --> 00:45:59.532
Yeah, I mean, that's what really makes me
think that it's, they're, they're seeing
cue-618
00:45:59.532 --> 00:46:04.212
if there's interest in spinning this off
into a product, 'cause I seriously doubt
cue-619
00:46:04.212 --> 00:46:07.912
there's going to be future work
considering like how little they seem to
cue-620
00:46:07.912 --> 00:46:12.692
care about like the topic. They just
wanted to see could we get this thing to
cue-621
00:46:12.692 --> 00:46:19.612
guess how to fill in these narrative gaps
using like real world data,
cue-622
00:46:19.612 --> 00:46:25.192
and can it do it? And which also it can't.
It's like fifty-fifty. It can kind of
cue-623
00:46:25.192 --> 00:46:30.172
guess fifty percent of the time if you
babysit all the prompts constantly and
cue-624
00:46:30.172 --> 00:46:34.692
know what the answer is, which of course
in, in a real silence, you, you wouldn't
cue-625
00:46:34.692 --> 00:46:37.792
-know what the answer is.
-Scarlet, you're a historian. I'm not.
cue-626
00:46:38.552 --> 00:46:42.212
What's, uh, the, the, the historical,
cue-627
00:46:42.872 --> 00:46:48.692
history as a discipline, what is its view
on making shit up no matter who's doing
cue-628
00:46:48.692 --> 00:46:51.052
-it?
-Well, [laughs] yeah. I mean, so,
cue-629
00:46:51.792 --> 00:46:57.142
so the way that, I mean, I've seen
historians handle and sort of triangulate
cue-630
00:46:57.142 --> 00:47:02.611
through archival silence really well. One
of the examples I think I have, you know,
cue-631
00:47:02.612 --> 00:47:08.252
over in, in the notes document is from, I
believe, Patrice McSherry, yes, who wrote
cue-632
00:47:08.252 --> 00:47:12.872
Predatory States, which is about Operation
Condor and covert war in Latin America.
cue-633
00:47:13.712 --> 00:47:17.992
What Patrice McSherry was able to do, and
this is, is surfaced in how things like
cue-634
00:47:17.992 --> 00:47:21.752
the Dirty War and Operation Condor were
executed, you know, in particular under
cue-635
00:47:21.752 --> 00:47:26.892
Pinochet and others, is that they hid each
other's chapters and hid each other's
cue-636
00:47:26.892 --> 00:47:33.452
records. So you had to go through sort of
this multi-archive investigation to try
cue-637
00:47:33.452 --> 00:47:36.902
and line up all of this information. I
think that,
cue-638
00:47:37.532 --> 00:47:42.412
you know, generally speaking, bullshit bad
was my takeaway as an undergraduate.
cue-639
00:47:42.412 --> 00:47:44.772
Making stuff up, not helpful, right?
cue-640
00:47:45.422 --> 00:47:49.052
Es- but unless we're looking at things
like forgeries, right, which are their own
cue-641
00:47:49.052 --> 00:47:53.392
sort of sense of st- own area of study.
But you know, I also go-
cue-642
00:47:53.392 --> 00:47:55.882
Well, in that, in that case, it's not the
historian who's making shit up
cue-643
00:47:55.882 --> 00:47:59.912
Right. Yeah. It's, yeah, it's somebody
else. But the, I, I don't know. I, I also,
cue-644
00:47:59.912 --> 00:48:03.732
you know, go back to this, this same
group of researchers' construction of
cue-645
00:48:03.792 --> 00:48:07.672
slop, and it's like, of course slop is
important. Of course low, lowbrow anything
cue-646
00:48:07.672 --> 00:48:12.132
is import- is important, especially now
since AI slop is used to kind of propagate
cue-647
00:48:12.132 --> 00:48:13.812
weird fascist nonsense.
cue-648
00:48:14.432 --> 00:48:18.072
You know, so in that sense, it's worthy of
study, but again, not the way that
cue-649
00:48:18.072 --> 00:48:22.492
they're thinking. Of course, archival
silences are worthy of study and, and can
cue-650
00:48:22.492 --> 00:48:26.232
have all of these different methods and
tools, you know, applied to them, but only
cue-651
00:48:26.232 --> 00:48:30.512
some of them are, are going to be
thoughtful or yield results. Only some of
cue-652
00:48:30.512 --> 00:48:34.172
them are going to be ethical, and only
people can engage in the kind of care
cue-653
00:48:34.172 --> 00:48:38.652
that's necessary in order to make them all
happen, which I think is sort of my
cue-654
00:48:38.652 --> 00:48:41.952
position on, you know, this kind of work
and also bullshit generally.
cue-655
00:48:42.848 --> 00:48:47.508
-Thank you, Scarlett. That was beautiful.
-Yeah. I mean, I'm, I'm finishing up a, a
cue-656
00:48:47.508 --> 00:48:53.478
book now about ancient cultures, like
trying different things and how our
cue-657
00:48:53.478 --> 00:48:58.848
understanding of the prehistoric world is
shaped by, like, what we're able to
cue-658
00:48:58.848 --> 00:49:03.108
reconstruct. And sometimes if we don't
have descendant languages, you can't
cue-659
00:49:03.108 --> 00:49:06.828
figure out anything about these cultures
because we don't know what words they had
cue-660
00:49:06.828 --> 00:49:09.848
for things. Whereas with
Proto-Indo-Europeans, you know that they
cue-661
00:49:09.848 --> 00:49:14.428
had a word for wheel, and parts of a
wheel, and parts of a carriage, and riding
cue-662
00:49:14.428 --> 00:49:19.348
in a vehicle, whereas in other cultures,
all you have left is just how they built
cue-663
00:49:19.348 --> 00:49:24.948
their settlement. So there's all of these
sitting with the gaps and saying, like,
cue-664
00:49:24.948 --> 00:49:30.128
"We don't really know anything," and
that's kind of the best you can get, even
cue-665
00:49:30.128 --> 00:49:35.328
though we have some evidence of, like,
they made pottery for this, and we can
cue-666
00:49:35.328 --> 00:49:41.568
tell, like, their society was unequal
because we saw inequality start to develop
cue-667
00:49:41.568 --> 00:49:47.008
in burial rituals and things like that.
So I think whenever you have these kinds
cue-668
00:49:47.008 --> 00:49:52.148
of things, the, the silence tells you
things as data points in itself, which is
cue-669
00:49:52.148 --> 00:49:57.108
why a lot of the archeology and social
sciences split off from history was to
cue-670
00:49:57.108 --> 00:50:02.168
study those non-narrative things and to
say, like, "These are our, our data points
cue-671
00:50:02.168 --> 00:50:06.148
now." Whereas before we would say, like,
"The Fertile Crescent is where
cue-672
00:50:06.148 --> 00:50:09.968
civilization started because it had these
things," and it turned out our definition
cue-673
00:50:09.968 --> 00:50:14.188
of civilization was just survivorship
bias. And actually, the stuff that
cue-674
00:50:14.188 --> 00:50:18.888
happened in the Fertile Crescent was the
exception, not the rule. And in fact, it
cue-675
00:50:18.888 --> 00:50:23.128
wasn't very... Like, they, they made
writing, and we know about writing, and we
cue-676
00:50:23.128 --> 00:50:26.328
know about them because of their writing,
but no one speaks their languages
cue-677
00:50:26.328 --> 00:50:31.118
anymore. The civilization did die out. So,
you know, it was the Proto-Indo-Europeans
cue-678
00:50:31.118 --> 00:50:35.548
who came through, illiterate as they
were, and took over all of these areas.
cue-679
00:50:35.548 --> 00:50:39.648
So, like, what you don't know and what you
do know determines what you can write
cue-680
00:50:39.648 --> 00:50:41.768
about historically or in prehistory.
cue-681
00:50:42.528 --> 00:50:46.628
So it really seems, yeah, there's, there's
probably something libidinal in terms of,
cue-682
00:50:46.628 --> 00:50:52.368
like, the fear of the silences. It's
probably also why prebuilt chatbots are so
cue-683
00:50:52.368 --> 00:50:56.528
chatty, and you have to tell them to,
like,
stop talking to you like a chatty
cue-684
00:50:56.528 --> 00:51:00.768
coworker and just give you the answer and
not be like, "You're right. I did get that
cue-685
00:51:00.768 --> 00:51:03.708
-wrong." Emoji, emoji, emoji.
-Did you catch-
cue-686
00:51:03.708 --> 00:51:05.868
And you have to tell it to, like, stop
doing that
cue-687
00:51:05.868 --> 00:51:11.528
... I would have to go find this. I didn't
bookmark it, but there's at least one
cue-688
00:51:11.528 --> 00:51:18.228
software outfit that is reducing its token
spend by telling the chatbot to talk like
cue-689
00:51:18.228 --> 00:51:21.047
-a caveman.
-Yeah. Yeah. Uh, yeah, there's, there's a
cue-690
00:51:21.048 --> 00:51:24.088
built-in, I forget what they're called,
but, like, skills or something-
cue-691
00:51:24.088 --> 00:51:26.348
-Mm-hmm
-... things you can, like, tell them to do.
cue-692
00:51:26.348 --> 00:51:31.288
I'm learning some of these terms 'cause
my university has, like, their own bespoke
cue-693
00:51:31.288 --> 00:51:36.188
AI lab, so we have access to, like, five
different models, and you can build stuff,
cue-694
00:51:36.188 --> 00:51:40.728
and the library's trying to test out
building a discovery AI.
cue-695
00:51:40.728 --> 00:51:41.188
No.
cue-696
00:51:42.928 --> 00:51:44.488
-Sorry. That's one of my-
-Yeah
cue-697
00:51:44.488 --> 00:51:48.408
-... no moments.
-It, it doesn't, it doesn't work great, the
cue-698
00:51:48.408 --> 00:51:52.248
one that they built. I built my own just
because it was already built in. It has
cue-699
00:51:52.248 --> 00:51:57.028
SDK. Those were the, those, those things
that are built in, SDKs. Anyway, I built
cue-700
00:51:57.028 --> 00:52:01.748
my own, and it works way better than the
one they built. So, um, that'll tell you
cue-701
00:52:01.748 --> 00:52:05.728
whoever built it. Yeah, they did tell it
to talk like a caveman 'cause it was
cue-702
00:52:05.728 --> 00:52:11.028
saying... It was-- I remember early on,
someone was saying that your people were
cue-703
00:52:11.028 --> 00:52:14.758
costing ChatGPT money 'cause they were
saying thank you to it.
cue-704
00:52:14.758 --> 00:52:15.888
-Yeah.
-It also says the same thing, but-
cue-705
00:52:15.888 --> 00:52:21.908
Cost ChatGPT all the money, people. Keep
being polite. Love it. Bankrupt them with
cue-706
00:52:21.908 --> 00:52:24.148
-politeness.
-Kill it with kindness. [chuckles]
cue-707
00:52:24.148 --> 00:52:27.368
-Yes.
-Yeah. I, I wanna say the, the Chipotle
cue-708
00:52:27.368 --> 00:52:33.968
compute is still going, where it's, where
you can use Chipotle's helper agent
cue-709
00:52:33.968 --> 00:52:39.068
to do your coding because it hasn't
imposed any limits on it or, or it hasn't
cue-710
00:52:39.068 --> 00:52:43.267
imposed certain limits on it. So someone
is write, you know, has written a script
cue-711
00:52:43.268 --> 00:52:46.748
and put it, you know, up in GitHub so that
you can, if you run out of compute, you
cue-712
00:52:46.748 --> 00:52:49.968
-can submit it to Chipotle.
-[chuckles] That's funny.
cue-713
00:52:49.968 --> 00:52:52.408
I think it's, you know, something like
free burrito or something. I'll have to
cue-714
00:52:52.408 --> 00:52:55.958
find it, but it's, you know, just
incredible the kinds of things that people
cue-715
00:52:55.958 --> 00:53:00.808
are, are, are doing. And I know that,
like, we just rolled out in the last three
cue-716
00:53:00.808 --> 00:53:06.088
days Claude, Enterprise Claude at work.
And so I'm just sort of paying attention
cue-717
00:53:06.088 --> 00:53:10.428
to where the token apocalypse is happening
at other schools like ours. So, like, I
cue-718
00:53:10.428 --> 00:53:14.128
know it's already happened at Johns
Hopkins, where the, the-
cue-719
00:53:14.128 --> 00:53:15.628
-Nice
-... yeah, the token apocalypse has
cue-720
00:53:15.628 --> 00:53:21.388
happened, and I wanna say at least one
other school. Penn. Yeah. University of
cue-721
00:53:21.388 --> 00:53:25.768
Pennsylvania is also m- trying to manage
the token apocalypse, and it's just kind
cue-722
00:53:25.768 --> 00:53:31.088
of like, "Okay," and we froze graduate
admissions. So those are sending some
cue-723
00:53:31.088 --> 00:53:34.888
interesting messages, or at least graduate
admissions in the humanities, right? So,
cue-724
00:53:34.888 --> 00:53:39.098
so there's a lot of messages going on in
different, different corners of, of the
cue-725
00:53:39.098 --> 00:53:43.268
-elite institutions right now.
-I wanna go back for a second actually to
cue-726
00:53:43.268 --> 00:53:50.008
what was being said about the split of
anthropology, archeology from history and
cue-727
00:53:50.008 --> 00:53:54.657
the way that that centers around language.
One of the issues,
cue-728
00:53:55.308 --> 00:54:01.548
certainly not the only one as we've
discussed, with this let's use language as
cue-729
00:54:01.548 --> 00:54:04.968
our sole interface to history here,
cue-730
00:54:05.648 --> 00:54:12.288
is precisely that language is a shitty,
absolutely terrible, not
cue-731
00:54:12.288 --> 00:54:17.738
faithful at all representation of reality,
right? So anything-
cue-732
00:54:17.738 --> 00:54:21.528
-Mm
-... that you're gonna get by throwing
cue-733
00:54:21.528 --> 00:54:28.258
generative AI at archival silences is
going to be impoverished
cue-734
00:54:28.258 --> 00:54:34.688
because the only medium that it has to
work with to construct whatever
cue-735
00:54:34.688 --> 00:54:39.328
confabulation it's constructing is
language. That's all it's got, and it's
cue-736
00:54:39.328 --> 00:54:41.708
-not enough.
-Yeah, we talked about this when we had you
cue-737
00:54:41.708 --> 00:54:47.552
on to talk about BIBFRAME and how people
were trying to, to use language in such a
cue-738
00:54:47.552 --> 00:54:52.692
way that you could encode all the semantic
meaning of Hamlet in BIBFRAME
cue-739
00:54:52.692 --> 00:54:57.012
Yeah. Can, can your, can your write Hamlet
in RDF? No. No, you cannot.
cue-740
00:54:57.012
--> 00:54:59.632
-Yeah.
-But even then, you know, those are two, at
cue-741
00:54:59.632 --> 00:55:04.832
least they're two language-based
questions, right? Archives have
cue-742
00:55:04.832 --> 00:55:11.712
materiality, even digital ones, ask Trevor
Owens. And of necessity, a purely
cue-743
00:55:11.712 --> 00:55:18.252
linguistic-based approach locks out all of
those other approaches to figuring
cue-744
00:55:18.252 --> 00:55:25.072
out what happened to a particular person
or at a particular time or whatever. Thank
cue-745
00:55:25.072 --> 00:55:29.132
you, actually, for that. That was, that
was a really neat piece of insight that I
cue-746
00:55:29.132 --> 00:55:33.372
-didn't have before, and I appreciate it.
-I think the, the real split starts with
cue-747
00:55:33.372 --> 00:55:38.872
the Annales school, and they were saying
that because literacy is limited to the
cue-748
00:55:38.872 --> 00:55:44.052
upper classes throughout history, the only
way you can do actual history is to work
cue-749
00:55:44.052 --> 00:55:48.672
with non-literate data. And so that was
their, I think, Marxist approach to
cue-750
00:55:48.672 --> 00:55:53.032
history, which actually led them to
abandoning the field of history and making
cue-751
00:55:53.032 --> 00:55:56.932
their own, which is, you know, now
sociology and, and archeology and
cue-752
00:55:56.932 --> 00:56:01.672
anthropology of... A- And do... Using
population statistics and using things
cue-753
00:56:01.672 --> 00:56:03.852
-that can be quantified more-
-Yeah
cue-754
00:56:03.852 --> 00:56:08.092
... rather than linguistic. And so that
was the big first school that you learn
cue-755
00:56:08.092 --> 00:56:12.032
about when you're doing your
historiography classes in grad school.
cue-756
00:56:12.032 --> 00:56:15.852
-That is very cool. Thank you.
-'Cause history relies on written words.
cue-757
00:56:15.852 --> 00:56:19.572
Like, that's why we have prehistory and
history. Like, like history relies on
cue-758
00:56:19.572 --> 00:56:23.302
writing, so if you don't have writing,
you're prehistory. Um, sometimes if y-
cue-759
00:56:23.302 --> 00:56:26.552
even if you do have writing, you're still
considered prehistory because, uh, you're
cue-760
00:56:26.552 --> 00:56:31.282
not in the, you know, like, like the, the
ancient Americas are considered prehistory
cue-761
00:56:31.282 --> 00:56:34.942
even though we have writing from them,
but a lot of it was destroyed, and some of
cue-762
00:56:34.942 --> 00:56:40.152
it's still not interpreted. So every time
I go to, like, a big city's museum, I
cue-763
00:56:40.152 --> 00:56:43.692
always wanna see the stuff from the
Americas because there's always, like,
cue-764
00:56:43.692 --> 00:56:45.212
ancient, you know, writing
cue-765
00:56:45.852 --> 00:56:49.272
that you just don't learn about in school,
that you just are told that, like, all
cue-766
00:56:49.272 --> 00:56:52.212
Native Americans were illiterate and
didn't have writing systems, which they
cue-767
00:56:52.212 --> 00:56:54.632
-did.
-Wait, what? No. [laughs] That's straight
cue-768
00:56:54.632 --> 00:56:58.292
-up wrong. Anyway, keep going.
-I know. But yeah, I love seeing, like, big
cue-769
00:56:58.292 --> 00:57:00.832
sundials and stuff that have all these,
like, names-
cue-770
00:57:00.832 --> 00:57:02.252
-[laughs]
-... and, like, how, how we know how to re-
cue-771
00:57:02.252 --> 00:57:05.572
re- read them and everything. It's cool.
I love it.
cue-772
00:57:05.572 --> 00:57:09.112
-Yeah.
-The Field Museum actually has a very fun
cue-773
00:57:09.112 --> 00:57:14.452
series of, like, exhibits, a lot of which
are covered because they have big signs on
cue-774
00:57:14.452 --> 00:57:19.262
them that say, "Because of the Native
American Graves Act, this exhibit is-
cue-775
00:57:19.262 --> 00:57:21.192
-Right. MACRA, we can't... Yeah
-... covered." [laughs]
cue-776
00:57:21.192 --> 00:57:23.432
-Okay.
-But the rest of it's pretty cool. Highly
cue-777
00:57:23.432 --> 00:57:26.112
-recommend.
-There's one other thing, and then I, I
cue-778
00:57:26.112 --> 00:57:29.452
-swear I'm gonna let this go.
-[laughs] Mm-hmm.
cue-779
00:57:29.452 --> 00:57:36.152
So they trained their own LLM, right, with
the materials, the, the, the, the, the
cue-780
00:57:36.152 --> 00:57:41.932
stories, whatever, that they could get
their hands on while excluding folklore
cue-781
00:57:41.932 --> 00:57:47.432
and songs, which, as Scarlet says, is a
completely indefensible decision. But
cue-782
00:57:47.432 --> 00:57:53.552
these are the people who actually managed
somehow to make it to text, right? And
cue-783
00:57:53.552 --> 00:58:00.132
we're using them to fill in the stories of
the people who didn't, and I think
cue-784
00:58:00.132 --> 00:58:06.152
inevitably there are going to be the usual
raft of biases and
cue-785
00:58:06.152 --> 00:58:12.932
inaccuracies and missing perspectives,
right, that, that that's going to entail.
cue-786
00:58:12.932 --> 00:58:19.312
And I think that's another form of
violence, honestly. It's not cool.
cue-787
00:58:19.312 --> 00:58:23.452
Yeah. There's so much primacy in our
society given to writing and giving, given
cue-788
00:58:23.452 --> 00:58:28.001
to, like, the book. Like, something we
used to talk about a lot on this show was,
cue-789
00:58:28.001 --> 00:58:32.212
like, you know, the worship of the book
as a semi-religious o- object. And I think
cue-790
00:58:32.212 --> 00:58:36.952
that's, you know, part of the reason LLMs
have this semi-religious language around
cue-791
00:58:36.952 --> 00:58:40.582
them. I talked about it a bit in a live
show. Like, you know, that's why it's,
cue-792
00:58:40.582 --> 00:58:45.532
it's making its own eschatology of, like,
it's creating its own apocalypse. Like,
cue-793
00:58:45.532 --> 00:58:49.792
this is going to change the world in some
major way. Why? 'Cause it can manipulate
cue-794
00:58:49.792 --> 00:58:53.352
language, and I think that literacy has
something to do
with it.
cue-795
00:58:53.352 --> 00:58:58.512
If they had read and understood any of the
Black and post-colonial scholarship that
cue-796
00:58:58.512 --> 00:58:59.952
-they are citing-
-Uh-huh
cue-797
00:58:59.952 --> 00:59:04.392
... they may have already known this. But
it... Oh, guess... I think that's proof
cue-798
00:59:04.392 --> 00:59:08.102
that they maybe didn't read this or
understand what they did read. Uh-
cue-799
00:59:08.102 --> 00:59:10.472
-Yeah, right on
-... because any... That's like Black and
cue-800
00:59:10.472 --> 00:59:14.132
-post-colonial humanities 101-
-Mm
cue-801
00:59:14.132 --> 00:59:19.032
... is, like, learning about, like, the
sort of, like, white supremacists, like,
cue-802
00:59:19.032 --> 00:59:23.732
primacy of the written historical record
and the written word and written language
cue-803
00:59:23.732 --> 00:59:26.861
-as sources. [laughs] That's like 101.
-Mm-hmm.
cue-804
00:59:26.861 --> 00:59:31.612
Well, and Justin saying, you know, because
it can manipulate language, and it's
cue-805
00:59:31.612 --> 00:59:35.611
like, and this is my problem with AI in
just in general. It's like, bitch, I can
cue-806
00:59:35.612 --> 00:59:36.092
do that.
cue-807
00:59:36.772 --> 00:59:42.012
I can d- [laughs] I can, I can do that
without, I can do that without having to
cue-808
00:59:42.012 --> 00:59:48.132
dumb it down to a fucking prompt. Uh,
yeah, just anything I can, AI can do, I
cue-809
00:59:48.132 --> 00:59:50.982
-can do better.
-Why hasn't that been written already,
cue-810
00:59:51.612 --> 00:59:52.282
-right?
-Yeah.
cue-811
00:59:52.282 --> 00:59:54.622
-Yeah.
-I, I, I'm sensing a filth coming on. I may
cue-812
00:59:54.622 --> 00:59:58.072
-have to write that.
-Yeah. Was gonna say, yeah, that's probably
cue-813
00:59:58.072 --> 01:00:02.402
by, you know, ear- some early hour,
Dorothea and I will have finished text-
cue-814
01:00:02.402 --> 01:00:04.652
-[laughs]
-... texting each other that, that entire
cue-815
01:00:04.652 --> 01:00:07.852
-thing.
-[laughs] This is very likely, I must say.
cue-816
01:00:07.852 --> 01:00:08.532
[laughs]
cue-817
01:00:09.861 --> 01:00:11.472
-[laughs]
-But thanks for putting a call out, by the
cue-818
01:00:11.472 --> 01:00:15.292
way, in terms of does anyone wanna come
talk about this? Because, yeah, I
cue-819
01:00:15.292 --> 01:00:20.132
apparently did, and it was good to, to
flex that muscle a little bit. I don't
cue-820
01:00:20.132 --> 01:00:24.371
wanna keep you all too late, mostly 'cause
I've had to turn off my AC in order to
cue-821
01:00:24.372 --> 01:00:26.312
get this recording. [laughs]
cue-822
01:00:26.972 --> 01:00:27.252
Yeah.
cue-823
01:00:27.972 --> 01:00:33.792
No, absolutely. Is there anything... Any
final thoughts? Right. Any plugs? Thank
cue-824
01:00:33.792 --> 01:00:37.172
you so much for coming on. Is there
anything you want people to keep an eye
cue-825
01:00:37.172 --> 01:00:39.991
out for, or do you want people to leave
you alone?
cue-826
01:00:39.992 --> 01:00:44.252
Oh, I'll do a little bit of
self-promotion. Yeah. I did a, I did a
cue-827
01:00:44.252 --> 01:00:49.112
talk. Sarah Lamdan could not do a talk
that she had meant to do at Public
cue-828
01:00:49.112 --> 01:00:54.232
Libraries Association in Minneapolis, so
she was like, "Okay, who do I know in the
cue-829
01:00:54.232 --> 01:00:57.692
Midwest who knows anything about library
privacy?" And for some reason, she landed
cue-830
01:00:57.692 --> 01:01:01.452
on me. So I took over that talk, Digital
Privacy
cue-831
01:01:02.072 --> 01:01:07.792
with, uh, sorry, Patron Privacy with
Digital Vendors, and PLA really liked it,
cue-832
01:01:07.792 --> 01:01:13.132
and apparently I'm taking it on the road,
virtually for the most part. But-
cue-833
01:01:13.132 --> 01:01:14.832
-Yay
-... uh, keep an eye out. There are going
cue-834
01:01:14.832 --> 01:01:18.822
to be several more opportunities to hear
that one. One of them actually is through
cue-835
01:01:18.822 --> 01:01:25.392
the, the Information School's virtu-
webinar series that is coming up w- and
cue-836
01:01:25.392 --> 01:01:29.532
all three of the talks in that are going
to be privacy centric. They should be
cue-837
01:01:29.532 --> 01:01:34.112
fantastic. So check it out and maybe sign
up.
cue-838
01:01:34.112 --> 01:01:39.352
Yeah. And as far as, uh, my... Since it's
been a theme, I am in fact back on my
cue-839
01:01:39.352 --> 01:01:44.692
bullshit. As soon as I get more word about
what's happening with some of the AI
cue-840
01:01:44.692 --> 01:01:46.632
tools in library licensed
cue-841
01:01:47.272 --> 01:01:51.832
databases here, I will let you all know
and talk about the results more publicly.
cue-842
01:01:52.472 --> 01:01:57.932
Until then, signs point to good outcomes
about shutting down trash that doesn't
cue-843
01:01:57.932 --> 01:02:02.372
need to be in databases in our libraries.
And I guess one thing that I wanna leave
cue-844
01:02:02.372 --> 01:02:06.872
with is this, something I heard on, over
on Kill James Bond, I think Devon said it,
cue-845
01:02:06.872 --> 01:02:10.592
about the current state of the world, and
I thought of it as I was preparing for
cue-846
01:02:10.592 --> 01:02:14.472
this, and it's the idea that we're all in
a line, and there's a lot of people ahead
cue-847
01:02:14.472 --> 01:02:20.192
of me in line right now and behind me in
line, and before it gets to me, I'm gonna
cue-848
01:02:20.192 --> 01:02:26.992
do everything I can to ensure the safety
of the people in line. And that includes
cue-849
01:02:26.992 --> 01:02:31.372
being real critical when this stuff shows
up in my workplace, and I hope we all are
cue-850
01:02:31.372 --> 01:02:33.892
-too. So thanks.
-Hear, hear.
cue-851
01:02:33.892 --> 01:02:38.972
Great. All right. Well, thanks so much for
coming on, and good night.