Process-Focused Teaching

1
00:00:00,600 --> 00:00:04,780
JOHN: Given the widespread availability
of generative AI tools, traditional
2
00:00:04,780 --> 00:00:09,269
assessments of learning conducted in
an unproctored environment have become
3
00:00:09,309 --> 00:00:12,010
unreliable indicators of student learning.
4
00:00:12,440 --> 00:00:16,339
In this episode, we explore the
potential benefits of shifting our
5
00:00:16,340 --> 00:00:18,919
focus to the process of learning.
6
00:00:24,240 --> 00:00:24,360
[MUSIC]
7
00:00:24,580 --> 00:00:28,909
Thanks for joining us for Tea for
Teaching, an informal discussion
8
00:00:28,910 --> 00:00:32,040
of innovative and effective
practices in teaching and learning.
9
00:00:32,630 --> 00:00:36,010
REBECCA: This podcast series is
hosted by John Kane, an economist…
10
00:00:36,489 --> 00:00:38,619
JOHN: …and Rebecca Mushtare,
a graphic designer…
11
00:00:38,969 --> 00:00:42,249
REBECCA: …and features guests doing
important research and advocacy work
12
00:00:42,410 --> 00:00:46,159
to make higher education more inclusive
and supportive of all learners.
13
00:00:49,230 --> 00:00:49,370
[MUSIC]
14
00:00:55,299 --> 00:00:57,130
JOHN: Our guest today is Jason Gulya.
15
00:00:57,410 --> 00:01:00,950
Jason is Professor of English and
Media Communications at Berkeley
16
00:01:00,950 --> 00:01:04,860
College, where he also chairs the
AI and Academic Integrity Committee.
17
00:01:04,870 --> 00:01:09,539
He is a co-author of Artificial
Intelligence, Real Literacy: a practical
18
00:01:09,589 --> 00:01:13,740
guide to using AI for evidence-based
literacy practices in education,
19
00:01:13,740 --> 00:01:17,709
and is currently writing a book on
process-focused teaching in the age
20
00:01:17,709 --> 00:01:20,409
of AI, for Oklahoma University Press.
21
00:01:20,799 --> 00:01:21,540
Welcome, Jason.
22
00:01:21,849 --> 00:01:22,569
JASON: Thank you so much.
23
00:01:22,570 --> 00:01:24,750
It's such a pleasure to be
here and talk to you both.
24
00:01:25,410 --> 00:01:27,699
REBECCA: Today's teas are:… Jason.
25
00:01:27,699 --> 00:01:29,290
Are you drinking some tea with us today?
26
00:01:29,519 --> 00:01:30,179
JASON: I'm not.
27
00:01:30,180 --> 00:01:32,169
I usually drink coffee over tea.
28
00:01:32,169 --> 00:01:36,259
I have my own preferences when I do drink
tea, but right now I don't have any.
29
00:01:36,639 --> 00:01:40,709
But if I were drinking tea, it would
either be English breakfast or Earl Grey.
30
00:01:41,139 --> 00:01:42,070
REBECCA: All right, classics.
31
00:01:42,070 --> 00:01:43,449
You're not a complete tea denier.
32
00:01:43,449 --> 00:01:45,629
[LAUGHTER] How about you, John?
33
00:01:45,909 --> 00:01:47,879
JOHN: And I am drinking
Prince of Wales tea today.
34
00:01:48,379 --> 00:01:49,889
REBECCA: I'm drinking
English breakfast today.
35
00:01:50,149 --> 00:01:51,989
JASON: In some ways, I
prefer the taste of tea.
36
00:01:51,989 --> 00:01:53,339
I don't know why I don't drink it more.
37
00:01:53,339 --> 00:01:54,159
I probably should.
38
00:01:54,549 --> 00:01:56,320
REBECCA: I wasn't trying to
make you feel guilty, Jason.
39
00:01:56,549 --> 00:01:57,369
JOHN: Yes, she was.
40
00:01:59,570 --> 00:02:03,839
. [LAUGHTER] We've invited you here today
to discuss process-focused teaching,
41
00:02:03,849 --> 00:02:05,379
which is the subject of your new book.
42
00:02:05,729 --> 00:02:09,450
Can you tell us a little bit about
the origin story of this book project?
43
00:02:09,949 --> 00:02:13,970
JASON: Yeah, it really, in some ways,
started way before generative AI.
44
00:02:14,000 --> 00:02:17,209
I think for a lot of us, generative
AI was a push to really think
45
00:02:17,220 --> 00:02:18,760
about process over product.
46
00:02:19,080 --> 00:02:21,009
I'm more than happy to
talk about what that is.
47
00:02:21,349 --> 00:02:27,599
But really, a long time ago, I felt like
just looking at a final product, which
48
00:02:27,639 --> 00:02:31,599
typically… I'm a writing professor, so
it's typically a paper in front of me…
49
00:02:31,940 --> 00:02:36,330
I felt like it wasn't really giving
me enough insight into how a student
50
00:02:36,330 --> 00:02:39,929
was thinking, how they were thinking
about their own thinking, how they were
51
00:02:39,989 --> 00:02:44,989
learning, what they were learning, and so
a long time ago, I felt very disillusioned
52
00:02:45,119 --> 00:02:46,419
just looking at a final product.
53
00:02:46,459 --> 00:02:51,529
And I did a lot of the process-focused
teaching that in writing we do, multiple
54
00:02:51,539 --> 00:02:56,379
drafts and things like that, but I started
to get a sense, and this really hit a head
55
00:02:56,599 --> 00:03:00,380
probably three or four years ago, right
around the time ChatGPT hit everyone.
56
00:03:00,890 --> 00:03:05,460
I started to think about how to move past
that disillusionment and another way of
57
00:03:05,460 --> 00:03:10,200
teaching, and trying to get more of an
access into what my students' processes
58
00:03:10,200 --> 00:03:14,829
were like, because beyond a rough draft
and a final draft, I didn't really know.
59
00:03:15,160 --> 00:03:20,039
And so when ChatGPT came out for many
of us, it put a lot of pressure on us.
60
00:03:20,070 --> 00:03:22,010
And I started to redesign.
61
00:03:22,340 --> 00:03:25,720
That's where the book really starts,
trying to think about what that push
62
00:03:25,730 --> 00:03:30,989
meant to me, and trying to figure out
giving students agency over the process.
63
00:03:30,989 --> 00:03:36,089
One of the things that I think a lot about
is that there are so many different kinds
64
00:03:36,090 --> 00:03:41,009
of processes, and now especially that
process-focused teaching has kind of been
65
00:03:41,119 --> 00:03:46,799
picked up by so many people who've tried
to redesign their assessments for an age
66
00:03:46,799 --> 00:03:48,240
of AI or whatever you want to call it.
67
00:03:48,680 --> 00:03:53,299
But as I dive into it more, there's a
lot to say about processes that I create
68
00:03:53,309 --> 00:03:57,549
versus ones that my students create versus
ones that we co-create, and so when I
69
00:03:57,549 --> 00:04:01,099
started to do the research for this book
and talking about generative AI and what
70
00:04:01,099 --> 00:04:04,159
it meant for college education, I just
did more and more of a deep dive, and
71
00:04:04,299 --> 00:04:07,819
as is always the case, it just became
more and more nuanced, more and more
72
00:04:07,820 --> 00:04:12,919
complicated as I went along, and that
became sort of invigorating because it
73
00:04:12,939 --> 00:04:18,500
forced me to really start to flesh out
what process actually is, when I think
74
00:04:18,509 --> 00:04:22,810
that many times we lose touch with just
how much of a black box that term is,
75
00:04:22,810 --> 00:04:26,619
even though everyone uses it and it's
become very much a term for the moment.
76
00:04:27,320 --> 00:04:30,940
REBECCA: I heard you say that generative
AI was a little bit of a catalyst to kind
77
00:04:30,940 --> 00:04:35,529
of move you further in this direction,
in this moment, but a lot of folks have
78
00:04:35,529 --> 00:04:38,529
been doing some of this work over time.
79
00:04:38,629 --> 00:04:41,399
So you mentioned a little bit of
digging deep into this literature.
80
00:04:41,409 --> 00:04:44,810
Can you talk a little bit about some
context prior to generative AI [LAUGHTER]
81
00:04:45,530 --> 00:04:46,659
that you might have been looking at?
82
00:04:47,119 --> 00:04:49,999
JASON: This is something that I think
is really important because if you
83
00:04:50,000 --> 00:04:54,460
look at the broader conversation about
generative AI in education, it almost
84
00:04:54,460 --> 00:04:56,379
makes it seem like this is a new thing.
85
00:04:56,389 --> 00:05:02,659
But there's this really long history
going back 60, 70, years that has looked
86
00:05:02,809 --> 00:05:07,109
at teaching process, and so a lot of
this comes out of writing studies.
87
00:05:07,259 --> 00:05:10,579
Like there are individuals who
really, in the 1960s, early 1970s,
88
00:05:10,919 --> 00:05:14,689
already started to think about how to
teach process, what that looks like.
89
00:05:14,709 --> 00:05:17,530
There's all these different pieces
of information, and there's also
90
00:05:17,530 --> 00:05:21,969
a lot of research, even starting
late-70s, early-80s, into something
91
00:05:21,970 --> 00:05:23,649
like think-aloud protocols.
92
00:05:23,690 --> 00:05:28,699
Now, this is not only in English; it's
big in sociology and other fields as well.
93
00:05:29,039 --> 00:05:33,849
But getting someone to talk
about something as they do it,
94
00:05:33,849 --> 00:05:35,549
it's kind of difficult to do.
95
00:05:35,550 --> 00:05:38,989
It takes a little bit of practice,
but think-aloud protocols in the
96
00:05:38,989 --> 00:05:44,019
1980s, people really looked at when a
student is writing something, and they
97
00:05:44,019 --> 00:05:47,370
were forced to talk about what they
were writing, what they were thinking
98
00:05:47,419 --> 00:05:49,539
as they did it, what do you learn?
99
00:05:49,550 --> 00:05:53,230
And we're actually starting to
see Jeanne Beatrix Law at Kennesaw
100
00:05:53,239 --> 00:05:56,840
State University also has done some
research in the AI version of this.
101
00:05:57,230 --> 00:06:01,539
I think one of the best things we
can do in terms of invigorating this
102
00:06:01,580 --> 00:06:05,500
conversation about AI and process-focused
teaching is to go back to that.
103
00:06:05,510 --> 00:06:10,740
Look at how process was talked about in
writing studies and sociology and in other
104
00:06:10,740 --> 00:06:15,890
fields, because there is this really,
really long research into process in the
105
00:06:15,890 --> 00:06:20,510
abstract, that's one kind of research,
and then also like specific strategies.
106
00:06:20,840 --> 00:06:22,529
This is think-aloud protocols.
107
00:06:22,529 --> 00:06:25,869
This is revision practices,
revision literacy, like all
108
00:06:25,869 --> 00:06:28,079
this is linked, and there's this
really, really long history of it.
109
00:06:28,079 --> 00:06:33,339
And I feel like the conversation about
AI in education, especially higher
110
00:06:33,340 --> 00:06:35,500
education, tends to jump over that.
111
00:06:35,559 --> 00:06:39,729
It's such a shame because I think it's
really, really rife with a lot of ideas.
112
00:06:39,899 --> 00:06:43,819
That no, we can't transfer all of them,
but I think it's surprising how much
113
00:06:43,820 --> 00:06:48,689
you can actually use, because I'm a
firm believer that generative AI broke
114
00:06:48,809 --> 00:06:53,610
some things, but more often than not, it
showed us things that were broken for a
115
00:06:53,610 --> 00:06:57,789
while, maybe we shouldn't have been as
product-focused as many of us were in our
116
00:06:57,789 --> 00:07:01,270
disciplines, and we can go back, look at
that literature, and kind of reinvigorate
117
00:07:01,270 --> 00:07:03,070
it and adapt it if we need to.
118
00:07:03,680 --> 00:07:07,209
JOHN: So while there have been many
people who've been exploring and in many
119
00:07:07,220 --> 00:07:11,080
disciplines exploring process-focused
teaching, it's still not the norm,
120
00:07:11,229 --> 00:07:14,379
although there's certainly been a lot
of push for people to move in that
121
00:07:14,379 --> 00:07:16,390
direction in response to generative AI.
122
00:07:16,900 --> 00:07:21,109
When only the final product is
evaluated, you can see where students
123
00:07:21,110 --> 00:07:24,480
ended up, but it's hard to detect
how much they've learned and their
124
00:07:24,480 --> 00:07:27,530
process for learning, and help them
develop better learning skills.
125
00:07:27,979 --> 00:07:31,289
What aspects of learning become
more visible when you're engaged
126
00:07:31,290 --> 00:07:32,719
in process-focused teaching?
127
00:07:33,239 --> 00:07:38,159
JASON: Yeah, I think that one of the
big benefits is that teaching process
128
00:07:38,290 --> 00:07:40,139
is a way to make thinking visible.
129
00:07:40,549 --> 00:07:44,329
One of the things that's happened,
especially now, when we're very product
130
00:07:44,389 --> 00:07:49,390
focused, we can look at something that
looks like it has the skills in it.
131
00:07:49,390 --> 00:07:54,789
It used to be 5 to 10 years ago, I could
read a paper from a student, and I could
132
00:07:54,790 --> 00:07:56,209
see well, there was a good argument.
133
00:07:56,629 --> 00:08:01,269
I'm pretty sure they could argue, or
they use evidence and analyze quotes.
134
00:08:01,269 --> 00:08:02,989
Well, I'm pretty sure
they can analyze quotes.
135
00:08:03,439 --> 00:08:06,490
And then one of the things that's
happened with ChatGPT, Claude, Gemini,
136
00:08:06,490 --> 00:08:09,759
all these programs that have just
been given to students and thrown in
137
00:08:09,759 --> 00:08:13,350
institutions, is that it's getting
harder and harder to bridge that gap.
138
00:08:13,369 --> 00:08:19,069
So, the idea behind process-focused
teaching is it gives you a way to
139
00:08:19,279 --> 00:08:21,949
bridge it, to kind of figure out
what's happening with that gap.
140
00:08:21,959 --> 00:08:23,820
What are they actually working through?
141
00:08:24,389 --> 00:08:27,390
And one of the things that I've
learned over the last few years
142
00:08:27,410 --> 00:08:31,329
is, as I've done a deep dive into
process-focused teaching, there are so
143
00:08:31,330 --> 00:08:34,730
many different ways of doing it, and
I think we need to make distinctions.
144
00:08:35,379 --> 00:08:40,459
If you're talking about one form of
process-focused teaching, which is giving
145
00:08:40,459 --> 00:08:44,909
students an interactive oral assessment,
so that students come in, they have a
146
00:08:44,910 --> 00:08:49,069
product, something they've created, and
then in class you can do this online too.
147
00:08:49,069 --> 00:08:51,759
It's a little bit harder, it takes
some creativity, but you can do it.
148
00:08:52,410 --> 00:08:56,350
So the whole idea is, it puts a little
bit of pressure on the students, so they
149
00:08:56,350 --> 00:09:00,530
need to explain things, and so that's
where process comes in, because it's
150
00:09:00,530 --> 00:09:03,920
something they couldn't fully rehearse
for, and it forces them to pivot.
151
00:09:04,170 --> 00:09:07,429
And so here you're testing
something pretty specific.
152
00:09:07,440 --> 00:09:10,169
You're trying to get a sense of
whether the information stuck,
153
00:09:10,980 --> 00:09:14,820
whether they can pivot, they can
move because they can't fully plan
154
00:09:14,820 --> 00:09:18,500
for a conversation, and so that's
what you're really taking a look at.
155
00:09:18,689 --> 00:09:23,149
Whereas other forms, it
can be very different.
156
00:09:23,429 --> 00:09:28,639
So if you ask a student to, for example,
do a think-aloud protocol, and I've
157
00:09:28,639 --> 00:09:32,369
done this with my students at Berkeley
College, I'll ask them: for this week
158
00:09:32,369 --> 00:09:35,649
what I want you to do, keep writing,
keep going forward with your project.
159
00:09:36,029 --> 00:09:37,129
But I want you to take five minutes.
160
00:09:37,829 --> 00:09:41,600
I want to take five minutes, and as you
are writing, I want you to talk about it.
161
00:09:42,000 --> 00:09:45,149
And so when I look at that, what I'm
really looking for is how they're
162
00:09:45,160 --> 00:09:47,549
processing things as they write.
163
00:09:47,980 --> 00:09:52,300
So this also focuses on process, but
it's a very, very different side of it.
164
00:09:53,180 --> 00:09:57,509
And then another way to get a
process is to really allow students
165
00:09:57,509 --> 00:09:59,599
to create their own process.
166
00:10:00,059 --> 00:10:04,640
So this we can do, and I often do in a
fully asynchronous learning environment.
167
00:10:05,029 --> 00:10:10,449
That instead of telling a student: write
a five-page paper about blank, instead
168
00:10:10,449 --> 00:10:15,680
of doing that, I have them, if I'm able
to, if I have the leeway, to design
169
00:10:15,680 --> 00:10:18,349
their own project, and we do it together.
170
00:10:18,599 --> 00:10:20,709
Some of them will say, "I know
what I want to do.” Some will say,
171
00:10:20,709 --> 00:10:21,749
"I don't,” and I work with them.
172
00:10:22,199 --> 00:10:24,139
They design a project.
173
00:10:24,409 --> 00:10:27,119
They design a process.
174
00:10:27,119 --> 00:10:28,630
So you want to create blank?
175
00:10:29,009 --> 00:10:32,409
What steps are going to get
you there, do you think?
176
00:10:32,409 --> 00:10:35,079
Do you think generative
AI can help with that?
177
00:10:35,579 --> 00:10:36,609
Why or why not?
178
00:10:36,879 --> 00:10:38,559
Do you think they're going
to help with the outline?
179
00:10:38,569 --> 00:10:42,029
Do you think they're going to help
with reformatting?… whatever it is.
180
00:10:42,069 --> 00:10:44,999
And some students will just say, "No,
I don't want to use generative AI.
181
00:10:44,999 --> 00:10:47,620
I'm just going to go through my process.”
And others will say, "I think it's
182
00:10:47,620 --> 00:10:50,709
going to help here and not here.” And
that's a part of the conversation.
183
00:10:50,719 --> 00:10:54,689
So for that, again, it's
process-focused teaching, but the
184
00:10:54,689 --> 00:10:55,979
skill set is a little bit different.
185
00:10:56,430 --> 00:10:58,039
It's about design thinking.
186
00:10:58,419 --> 00:11:04,380
It's about can a student create their own
process and manage their own process, and
187
00:11:04,380 --> 00:11:06,570
sometimes even evaluate their own process.
188
00:11:06,610 --> 00:11:09,570
Halfway through, I might say, "How's
it going? Do you need to pivot? What
189
00:11:09,570 --> 00:11:13,020
do you need to do?” And those are
three examples of process-focused
190
00:11:13,020 --> 00:11:17,969
teaching, but sort of splinter in terms
of how they're approaching process.
191
00:11:17,969 --> 00:11:22,080
And I think one of the things that's
happened in the conversation about AI in
192
00:11:22,080 --> 00:11:27,099
education is that process-focused teaching
has become one umbrella term that's
193
00:11:27,110 --> 00:11:30,349
trying to capture all these different
things, and it's just really hard to do.
194
00:11:30,699 --> 00:11:32,570
And I don't think we
should have to do that.
195
00:11:32,570 --> 00:11:34,019
It's very, very nuanced.
196
00:11:34,160 --> 00:11:37,930
So those different strategies, they
focus on different skills, even
197
00:11:37,930 --> 00:11:39,509
though they all focus on process.
198
00:11:40,039 --> 00:11:44,649
REBECCA: I come from a field that focuses
on process a lot, the design field, so
199
00:11:44,699 --> 00:11:49,609
we've, over time, required students to
do things like process books, or we might
200
00:11:49,610 --> 00:11:53,800
do a case study at the end where they
talk about how they came to their final
201
00:11:53,800 --> 00:11:57,360
product, which is similar to your oral
exam, but maybe is in a written format
202
00:11:57,360 --> 00:11:59,810
or has audiovisual components to that.
203
00:12:00,370 --> 00:12:03,000
It makes me think a little bit more
about what you were talking about
204
00:12:03,020 --> 00:12:08,240
earlier, in that there's advantages and
disadvantages when an instructor designs
205
00:12:08,270 --> 00:12:11,839
the process that's being looked at,
maybe a process that's happening over
206
00:12:11,840 --> 00:12:16,850
time versus discussion of process in a
fixed amount of time versus a student
207
00:12:16,859 --> 00:12:21,029
designing their process, and it ties, I
think, to some scaffolded learning too.
208
00:12:21,469 --> 00:12:23,639
Those are some processes
that faculty design.
209
00:12:24,239 --> 00:12:27,589
Can you talk a little bit about some
of the strengths and differences
210
00:12:27,630 --> 00:12:31,280
between those different kinds of
examples that you just mentioned.
211
00:12:32,209 --> 00:12:36,720
JASON: I think the big question
is: What is the intentionality?
212
00:12:37,039 --> 00:12:38,659
What are we trying to teach?
213
00:12:39,049 --> 00:12:43,400
And I can say things like
teaching process makes things
214
00:12:43,410 --> 00:12:44,979
visible, makes thinking visible.
215
00:12:45,219 --> 00:12:46,479
But the question is: To whom?
216
00:12:46,669 --> 00:12:49,369
Who needs to see it?
217
00:12:49,719 --> 00:12:55,690
And once we have those questions, what
is the intention here, and also who
218
00:12:55,730 --> 00:13:00,609
should the process be visible to, then
that helps us start to design things.
219
00:13:00,790 --> 00:13:04,429
Sometimes I want to see the
process out in the open so
220
00:13:04,430 --> 00:13:05,829
that I can give feedback on it.
221
00:13:05,829 --> 00:13:08,089
This is what we're talking about
when we ask the students to
222
00:13:08,089 --> 00:13:09,660
go through the design process.
223
00:13:09,680 --> 00:13:10,559
What did you do?
224
00:13:10,569 --> 00:13:11,349
Let's talk about it.
225
00:13:11,349 --> 00:13:13,019
So it's a form of collaboration.
226
00:13:13,559 --> 00:13:14,709
Sometimes I want to do that.
227
00:13:15,040 --> 00:13:19,349
Sometimes, really, the most
important thing is for students
228
00:13:19,349 --> 00:13:23,599
to see it, to see that kind of
process and see it documented.
229
00:13:23,979 --> 00:13:28,829
And so when I do that, the big
benefit here is metacognition, making
230
00:13:28,829 --> 00:13:31,909
students aware of how they're thinking
through something, and sometimes
231
00:13:32,040 --> 00:13:34,199
it's more for them than it is for me.
232
00:13:34,680 --> 00:13:37,049
And this is where we get to
reflections and journals.
233
00:13:37,199 --> 00:13:38,609
Sometimes I read them.
234
00:13:38,819 --> 00:13:40,320
Sometimes I may even evaluate them.
235
00:13:40,320 --> 00:13:42,699
But more often than not,
ideally that is for them.
236
00:13:43,309 --> 00:13:48,729
And I think that those are some of the big
questions that allow us to start to design
237
00:13:48,779 --> 00:13:51,540
things because I think it's about balance.
238
00:13:51,659 --> 00:13:53,379
I think it's really all about balance.
239
00:13:53,409 --> 00:13:57,540
If we go all in on teaching
process for every single
240
00:13:57,540 --> 00:13:59,269
assignment, we're gonna lose it.
241
00:13:59,529 --> 00:14:00,740
You can't do it.
242
00:14:00,899 --> 00:14:05,249
On the one hand, it's not really
ideal because it's just going
243
00:14:05,250 --> 00:14:06,639
to balloon everyone's work.
244
00:14:06,639 --> 00:14:08,279
Students have to do more documenting.
245
00:14:08,309 --> 00:14:09,889
I have to look at more assignments.
246
00:14:10,239 --> 00:14:14,519
It can very quickly become what I
call product bloat, where instead of
247
00:14:14,520 --> 00:14:18,269
sending in one thing, which is what
they might have done 5, 10, years ago,
248
00:14:18,489 --> 00:14:22,570
now they're sending in eight things,
and they're doing that full scaffold.
249
00:14:22,610 --> 00:14:27,080
And so you can see how product bloat isn't
good for them, and it's not good for me.
250
00:14:27,089 --> 00:14:33,789
And so I think it's okay to sometimes
say I just want to see the overall
251
00:14:33,789 --> 00:14:36,100
process here, or maybe I don't.
252
00:14:36,410 --> 00:14:41,719
There are examples of what I
call process-bearing products
253
00:14:42,720 --> 00:14:43,869
that you can take a look at.
254
00:14:44,119 --> 00:14:48,680
So, for a quick example, a
student writes, say, a two pager.
255
00:14:48,879 --> 00:14:54,390
That's a product, but if you ask them to
go through and annotate it, now it has
256
00:14:54,400 --> 00:14:56,710
something about the process built into it.
257
00:14:56,970 --> 00:14:58,700
Annotate the process you went through.
258
00:14:58,980 --> 00:14:59,970
Do it by pen.
259
00:14:59,980 --> 00:15:01,119
Do it by Google Doc.
260
00:15:01,410 --> 00:15:03,709
And so I think it's okay
to try to balance it.
261
00:15:03,759 --> 00:15:05,039
This is based off of discipline.
262
00:15:05,040 --> 00:15:07,410
This is based off of how
many students we have.
263
00:15:07,799 --> 00:15:13,019
I think that one of the mistakes we
can sometimes make is thinking that
264
00:15:13,029 --> 00:15:17,770
something that might work for teaching
process in a 20-person classroom might
265
00:15:17,770 --> 00:15:21,359
not work in 100-person classroom, and
so we make that determination and we
266
00:15:21,359 --> 00:15:22,630
start to narrow it down from there.
267
00:15:22,630 --> 00:15:26,860
And I think it's okay to just focus
on product for a little bit, and then
268
00:15:26,860 --> 00:15:28,050
you get the process in there later.
269
00:15:28,050 --> 00:15:29,569
So I think it's all about
like balancing that.
270
00:15:29,879 --> 00:15:34,379
A lot of it is institution, what we're
teaching, asynchronous online versus in
271
00:15:34,379 --> 00:15:37,989
person can be very very different too
in terms of how much leeway we have.
272
00:15:38,700 --> 00:15:42,719
JOHN: And going back to that issue you
just raised of scale, I can see this
273
00:15:42,719 --> 00:15:44,839
working really nicely in small classes.
274
00:15:44,889 --> 00:15:48,979
But say, hypothetically you were teaching
a class of between two and four hundred
275
00:15:48,980 --> 00:15:51,079
students in introductory economics.
276
00:15:51,579 --> 00:15:54,259
Are there any ways in which
process-focused teaching
277
00:15:54,260 --> 00:15:55,919
could be introduced into that?
278
00:15:56,260 --> 00:15:57,589
JASON: Yeah, I think there can be.
279
00:15:57,909 --> 00:16:00,689
You know more about the field than
I do in terms of how this would work
280
00:16:00,689 --> 00:16:05,000
out pragmatically, and I think the big
decision that needs to happen at that
281
00:16:05,000 --> 00:16:07,839
level is how assessment is going to work.
282
00:16:07,839 --> 00:16:11,440
You can have students walk through
the process of something in the
283
00:16:11,440 --> 00:16:14,519
course, a project that you maybe
have had for a while, and they can
284
00:16:14,519 --> 00:16:15,550
walk through the process of it.
285
00:16:15,729 --> 00:16:18,639
But then, where it becomes a problem
is if everything has to be evaluated,
286
00:16:18,650 --> 00:16:19,579
everything has to be graded.
287
00:16:19,579 --> 00:16:21,150
That's a big decision.
288
00:16:21,379 --> 00:16:24,769
So I know people who've
experimented with self-assessment
289
00:16:24,839 --> 00:16:25,849
as a way to get around that.
290
00:16:25,849 --> 00:16:28,439
Now, depending on where you teach,
depending on your discipline, you
291
00:16:28,879 --> 00:16:31,909
might have a different approach or
maybe non-approach towards that,
292
00:16:32,399 --> 00:16:34,430
but self-assessment is one way.
293
00:16:34,889 --> 00:16:36,769
Alternative assessment is another way.
294
00:16:37,169 --> 00:16:39,649
Students assessing each other,
depending on your course, if you
295
00:16:39,649 --> 00:16:40,810
feel like that's going to work.
296
00:16:40,849 --> 00:16:44,079
So I think a lot of it is trying to
figure out how to be creative with
297
00:16:44,079 --> 00:16:46,859
it and what we want grading to be.
298
00:16:47,119 --> 00:16:50,600
So one of the things that ends up
happening when we talk about the
299
00:16:51,639 --> 00:16:54,449
scalability of process teaching, this
is where we have the conversation
300
00:16:54,459 --> 00:16:58,389
about AI grading or AI assisted
grading, and I know people have
301
00:16:58,399 --> 00:16:59,730
very different feelings about this.
302
00:16:59,739 --> 00:17:03,510
So one way that people have done this
is yes, I have a 200 person class.
303
00:17:03,539 --> 00:17:07,960
I want to teach process, but I can't
do it, especially if it's a 200 person
304
00:17:08,029 --> 00:17:12,930
online course, I can't manage that, and
so that is where some people have leaned
305
00:17:12,940 --> 00:17:17,439
into AI grading as a way to approach
scale, and people might have different
306
00:17:17,440 --> 00:17:21,909
ideas about AI-generated grading versus
AI-assisted grading, but I think it's
307
00:17:21,910 --> 00:17:24,679
something that has helped a lot of people.
308
00:17:24,679 --> 00:17:27,289
And then again, it's just a matter
of like, is it worth the trade off?
309
00:17:27,289 --> 00:17:28,619
There's always a trade-off.
310
00:17:29,009 --> 00:17:31,639
You're going to move away from
product and towards process.
311
00:17:31,899 --> 00:17:34,010
Try to figure out: is
that trade-off worth it?
312
00:17:34,039 --> 00:17:35,089
Is there another way to do it?
313
00:17:35,269 --> 00:17:39,199
Now, one of my hopes is that in the
next couple years, we will have a way
314
00:17:39,199 --> 00:17:42,320
to start approaching that scale in
another way that isn't AI grading.
315
00:17:42,349 --> 00:17:47,239
I'm not aware of too much yet, but my
hope is that we'll continue to be creative
316
00:17:47,410 --> 00:17:50,410
as a community and try to figure out
how to do that in a way that doesn't
317
00:17:50,410 --> 00:17:52,490
just hand off grading to an AI bot.
318
00:17:52,530 --> 00:17:55,990
I think that many of us would
balk at that, myself included.
319
00:17:56,470 --> 00:17:59,139
But, there is a middle ground
potentially here that helps
320
00:17:59,139 --> 00:18:00,440
us with the scalability issue.
321
00:18:00,450 --> 00:18:05,489
REBECCA: I think in terms of scalability,
those different kinds of process
322
00:18:05,619 --> 00:18:08,710
assignments that you were talking about
have different kinds of scalability.
323
00:18:09,100 --> 00:18:13,019
So a student that's designing their
own process that's an interactive
324
00:18:13,049 --> 00:18:16,380
process with the instructor to
design that process-seems a lot more
325
00:18:16,380 --> 00:18:20,550
labor intensive and would probably
only work in a small context.
326
00:18:21,279 --> 00:18:24,349
But even that, if you have 20
students and they're all doing a
327
00:18:24,349 --> 00:18:27,580
different process, that can seem
a little overwhelming as well.
328
00:18:28,310 --> 00:18:30,720
[LAUGHTER] Can you talk a little
bit about how you might implement
329
00:18:30,720 --> 00:18:32,109
something like that in a class?
330
00:18:32,500 --> 00:18:36,979
JASON: Yeah, and so this is where
I think we need to really decide
331
00:18:37,189 --> 00:18:38,490
what we need to have access to.
332
00:18:38,700 --> 00:18:39,719
We don't need to have
access to everything.
333
00:18:40,629 --> 00:18:44,139
So, for example, if I were doing a
think-aloud protocol for my students,
334
00:18:44,219 --> 00:18:48,229
so they film themselves for five
minutes, and I take a look at it.
335
00:18:48,529 --> 00:18:51,669
Now, 10-person class,
20-person class, cool.
336
00:18:52,080 --> 00:18:54,800
50-person class, even a
20-person class, that's a lot.
337
00:18:54,839 --> 00:18:56,540
That's a lot of work hours.
338
00:18:57,040 --> 00:19:01,210
And so I tell them, and this takes
a little bit of trust building,
339
00:19:01,619 --> 00:19:05,399
that actually I'm not that
interested in watching the video.
340
00:19:05,400 --> 00:19:06,809
Sometimes I don't watch the video.
341
00:19:07,159 --> 00:19:08,639
I want them to take a look at it.
342
00:19:08,799 --> 00:19:09,769
It's for them.
343
00:19:09,769 --> 00:19:13,410
The video is for them, and instead
of getting a five-minute video,
344
00:19:13,580 --> 00:19:16,839
maybe I get a quick write-up that
I can look at very, very quickly.
345
00:19:16,979 --> 00:19:20,939
Sometimes I can look at it in 30 seconds,
and so I think we make that determination.
346
00:19:21,449 --> 00:19:26,669
And I think it's okay too to have this
ad hoc approach, so that if I go and I'm
347
00:19:26,669 --> 00:19:31,519
looking at those think-aloud protocols,
and I read one, and I say, "Well, I'm not
348
00:19:31,519 --> 00:19:34,829
quite sure the learning is happening here.
I'm not seeing that level of engagement.”
349
00:19:35,059 --> 00:19:39,059
Then I can, in an ad hoc way, go and
look at that video, and that allows
350
00:19:39,059 --> 00:19:40,969
me to use my time a little bit more.
351
00:19:40,969 --> 00:19:42,389
And I think it's okay to do that.
352
00:19:42,389 --> 00:19:44,849
I think it's okay to have
things that are just for them.
353
00:19:45,419 --> 00:19:48,589
And the other thing, and again, I
know this is so discipline-based,
354
00:19:48,909 --> 00:19:53,069
but teaching self-assessment
has saved me a ton of time.
355
00:19:53,589 --> 00:19:56,940
Having students go through and give
their own grade for themselves, I
356
00:19:56,940 --> 00:19:59,959
obviously come in and take a look,
and if something seems really, really
357
00:19:59,959 --> 00:20:02,040
off, then I can step in and intervene.
358
00:20:02,410 --> 00:20:07,859
But in general, giving them the ability
to self-assess themselves really really
359
00:20:07,860 --> 00:20:09,139
does help, and I think it's okay.
360
00:20:09,139 --> 00:20:10,989
I don't think that's even cutting corners.
361
00:20:10,989 --> 00:20:13,999
I think that's allowing us to
have a conversation about how
362
00:20:14,000 --> 00:20:15,239
they're assessing themselves.
363
00:20:15,360 --> 00:20:18,369
If there is a gap between how
they assess their work and
364
00:20:18,369 --> 00:20:19,569
how I'm assessing their work.
365
00:20:20,139 --> 00:20:24,369
If I have a student send in a project
and their self-assessment, I might be
366
00:20:24,369 --> 00:20:29,950
able to review that in five minutes,
maybe even shorter, depending on
367
00:20:29,950 --> 00:20:33,309
the project, and I can see: “Oh,
I think their self-assessment is
368
00:20:33,309 --> 00:20:37,529
pretty good. I think it's kind
of on point here.” Or conversely,
369
00:20:37,529 --> 00:20:39,159
that maybe I think it's really off.
370
00:20:39,220 --> 00:20:43,020
Maybe I would give a student a D, but
they're giving them an A, and I want
371
00:20:43,029 --> 00:20:45,809
access to that, and that allows me to
have a conversation with the student
372
00:20:46,000 --> 00:20:47,589
in a way that also does guard my time.
373
00:20:47,590 --> 00:20:51,980
So I think, for me, what has really helped
is those different strategies, and I
374
00:20:51,980 --> 00:20:55,730
think that we can pick and choose them
and adapt them for our own discipline.
375
00:20:55,830 --> 00:20:56,930
That's in my experience, anyway.
376
00:20:57,440 --> 00:21:00,340
JOHN: So one of the things you talk
about is having students design
377
00:21:00,340 --> 00:21:02,100
their processes individually.
378
00:21:02,529 --> 00:21:06,219
Could you walk us through an example
of that in terms of a particular
379
00:21:06,219 --> 00:21:09,549
assignment, let's say, or to achieve
a particular learning objective?
380
00:21:10,090 --> 00:21:13,539
JASON: Yeah, so I've done
this in my art of film class.
381
00:21:13,539 --> 00:21:15,949
I've also done this in our
writing-intensive courses.
382
00:21:15,949 --> 00:21:19,580
So for the class I'm teaching right
now, I'm teaching a foundational
383
00:21:19,590 --> 00:21:24,840
literature course, and so it's all about
just engaging with story and just how
384
00:21:24,840 --> 00:21:26,450
expansive that can be as a category.
385
00:21:26,450 --> 00:21:33,080
So for that one, as their midterm
project, I want them to create a product,
386
00:21:33,080 --> 00:21:37,209
create a project, that shows how they're
working through the course material.
387
00:21:37,629 --> 00:21:40,939
I can give students more specific
directions if they're stuck with that.
388
00:21:41,360 --> 00:21:44,479
But what I really do is I ask
them, let's design this together
389
00:21:44,660 --> 00:21:46,559
and show how it connects.
390
00:21:46,930 --> 00:21:51,370
So, for example, I will have a whole range
of projects in that literature class.
391
00:21:51,439 --> 00:21:53,959
Some students want to do a
creative writing assignment.
392
00:21:54,290 --> 00:21:56,369
Some students want to do
an analytical assignment.
393
00:21:56,839 --> 00:21:58,519
So, right now we're reading Persepolis.
394
00:21:58,680 --> 00:22:02,170
So, I might have some students who
say write a prequel to Persepolis.
395
00:22:02,730 --> 00:22:05,729
They want to do that sort of
work, and then others they want
396
00:22:05,729 --> 00:22:07,550
to do more of a traditional essay.
397
00:22:07,599 --> 00:22:13,719
So I work with them, and I do this as
complete and incomplete, or try again.
398
00:22:13,939 --> 00:22:16,710
As they're going through, and they have
that project, we have a back and forth.
399
00:22:16,889 --> 00:22:17,649
I give the okay.
400
00:22:17,769 --> 00:22:19,430
That's something to review pretty quickly.
401
00:22:19,759 --> 00:22:22,529
If they're connecting it to the course,
it's something that they're interested in.
402
00:22:22,739 --> 00:22:26,009
I usually give them just a complete,
and then they start to build from there.
403
00:22:26,250 --> 00:22:30,039
And so once they have that done, once
they have that project, and again, some
404
00:22:30,040 --> 00:22:31,710
will be creative, some will be analytical.
405
00:22:32,099 --> 00:22:34,939
Then we have the conversation about
what's going to help you get there.
406
00:22:35,509 --> 00:22:41,659
And this is where it is 90% for them,
10% for me, that they're going to go
407
00:22:41,659 --> 00:22:45,209
through and say, "This is what I think
I'm going to do. I want to write a
408
00:22:45,209 --> 00:22:47,669
prequel to Persepolis.” Okay, cool.
409
00:22:47,909 --> 00:22:50,399
So I'm going to go and I'm going
to write a rough draft on my own.
410
00:22:50,419 --> 00:22:53,529
Then I'm going to use AI if they want to.
411
00:22:53,729 --> 00:22:56,659
I'm going to create an outline or
whatever it is, whatever the process is.
412
00:22:57,020 --> 00:23:00,780
And for that, again, thinking about
scalability, it's all about do you
413
00:23:00,780 --> 00:23:02,240
have a process that makes sense?
414
00:23:02,339 --> 00:23:04,559
I can usually review
this in about 30 seconds.
415
00:23:05,009 --> 00:23:09,279
You get a sense very quickly some students
who are really struggling with creating a
416
00:23:09,279 --> 00:23:11,489
process, and others who like this is fine.
417
00:23:12,179 --> 00:23:13,380
This is not bothering me at all.
418
00:23:13,779 --> 00:23:19,619
And so, even in that part in the
process itself, I call a process folio.
419
00:23:19,940 --> 00:23:22,509
I get a sense of how students
are doing with creating processes
420
00:23:22,509 --> 00:23:24,040
and thinking about processes.
421
00:23:24,679 --> 00:23:27,919
The course learning objective
that this comes down to, and this
422
00:23:27,929 --> 00:23:29,240
is where I anchor everything.
423
00:23:29,649 --> 00:23:34,070
For me, managing a writing process
is a foundational core skill of the
424
00:23:34,070 --> 00:23:35,789
course, what a lot of it is about.
425
00:23:36,489 --> 00:23:40,619
To give a sense of how I start to
create that, so when I look at a
426
00:23:40,619 --> 00:23:44,139
class and I'm trying to think about
a process-focused assignment, first
427
00:23:44,139 --> 00:23:46,320
thing I do, I look at the course goals.
428
00:23:46,490 --> 00:23:48,160
These have been established for a while.
429
00:23:48,170 --> 00:23:50,180
What are the course
goals that I need to hit?
430
00:23:50,680 --> 00:23:54,129
Then I break it down into the core skills.
431
00:23:54,420 --> 00:23:55,850
I usually pick like four of them.
432
00:23:56,050 --> 00:23:57,269
How can I boil it down even more?
433
00:23:57,590 --> 00:23:59,819
And then I link this assignment to that.
434
00:23:59,819 --> 00:24:03,129
And so that's a way for me to make
sure I'm actually hitting the course
435
00:24:03,129 --> 00:24:07,139
objectives, the learning objectives
that I need to hit, even as I'm taking a
436
00:24:07,139 --> 00:24:10,889
somewhat unconventional approach, and I
think that's the sort of balance, right?
437
00:24:11,090 --> 00:24:12,070
You want to do both.
438
00:24:12,170 --> 00:24:15,449
If you're going to focus on process,
you can be creative, you can push
439
00:24:15,449 --> 00:24:18,159
things a little bit, but you do want
to make sure you're hitting those
440
00:24:18,180 --> 00:24:21,949
course objectives because that student
needs to have certain skills when they
441
00:24:21,950 --> 00:24:23,320
graduate, go on to the next course.
442
00:24:23,320 --> 00:24:25,999
So I think doing that sort of
work, starting with the learning
443
00:24:26,000 --> 00:24:29,849
objectives and working back from
there, has really helped me out a lot.
444
00:24:30,000 --> 00:24:35,419
It also prevents me from starting
from just step zero or step one.
445
00:24:35,739 --> 00:24:39,120
It gives me something to work with
and make sure that if a student gets
446
00:24:39,120 --> 00:24:42,739
through my class, they're ready for
the next writing class, even if they're
447
00:24:42,739 --> 00:24:44,209
doing something a little bit different.
448
00:24:44,709 --> 00:24:46,069
That's been my process anyway.
449
00:24:46,459 --> 00:24:49,200
REBECCA: Can you talk a little bit
about the difference between students
450
00:24:49,209 --> 00:24:53,510
experimenting with process in like
a 100-level class, where they maybe
451
00:24:53,510 --> 00:24:57,169
don't have [LAUGHTER] a general sense
of the processes used in the field
452
00:24:57,169 --> 00:25:01,679
or discipline, or maybe one a little
further along in their academic journey,
453
00:25:01,680 --> 00:25:04,820
where they might be more deeply rooted
into a discipline and might have
454
00:25:04,820 --> 00:25:08,190
more exposure to different kinds of
processes they might use for a project.
455
00:25:08,669 --> 00:25:13,509
JASON: I'm a big fan of the
gradual release model, so
456
00:25:14,309 --> 00:25:14,950
scaffolded appropriately.
457
00:25:15,110 --> 00:25:18,920
And AI can actually help a lot
with this, and it can allow us
458
00:25:19,139 --> 00:25:23,139
to give more scaffolding to some
students and less to others.
459
00:25:23,570 --> 00:25:28,429
So, for example, for a lot of my
classes, for the first project,
460
00:25:28,449 --> 00:25:29,699
I give a lot of scaffolding.
461
00:25:29,699 --> 00:25:30,609
It walks them through.
462
00:25:30,699 --> 00:25:32,090
This is how you create a project.
463
00:25:32,589 --> 00:25:36,450
For me, it's kind of it may
seem self-evident, but for a
464
00:25:36,450 --> 00:25:39,350
student who's just learning the
discipline, not at all, right?
465
00:25:39,350 --> 00:25:42,309
If someone tells you, oh create a
project that's connected to the course.
466
00:25:42,309 --> 00:25:43,200
So, what does that look like?
467
00:25:43,509 --> 00:25:49,030
And so, my impulse is to always give
a lot of scaffolding for the first
468
00:25:49,030 --> 00:25:52,719
project, and then gradually give
a little bit less as we go along.
469
00:25:53,470 --> 00:25:56,619
The other big way that I do
this is with self-assessment.
470
00:25:57,220 --> 00:26:00,039
If I ask the students to self-assess
themselves, some of them will
471
00:26:00,039 --> 00:26:01,949
just be like, all right, I kind
of know what you're talking about.
472
00:26:02,259 --> 00:26:06,669
Others have no idea, and so
I actually give my students
473
00:26:06,729 --> 00:26:08,060
a couple of ways to do it.
474
00:26:08,750 --> 00:26:13,020
First way is the AI-free way, so they
think about this is what I want to
475
00:26:13,020 --> 00:26:16,519
create, these are the skills I'm working
on, this I'm going to self-assess.
476
00:26:16,879 --> 00:26:21,609
And in any individual class, I probably
have about 40 to 50% choose that one,
477
00:26:22,089 --> 00:26:25,519
and then the other thing I do is for
students who need more scaffolding
478
00:26:25,569 --> 00:26:28,929
and need more support, and they just
need it in the moment asynchronously.
479
00:26:30,069 --> 00:26:33,379
I give them a custom chatbot that
walks them through the process.
480
00:26:33,910 --> 00:26:36,140
So it asks them, "What are you
trying to create?” All right,
481
00:26:36,140 --> 00:26:39,239
cool. Build from there. “What are
the skills?” And it actually can
482
00:26:39,599 --> 00:26:41,969
create a rubric with the chatbot.
483
00:26:42,339 --> 00:26:45,840
Now suddenly, this is an activity that
is more scaffolded for the students
484
00:26:45,840 --> 00:26:49,690
who want that scaffolding, and it's
something they can do in five minutes.
485
00:26:50,210 --> 00:26:53,920
Whereas without that scaffolding, it
might have taken them an hour, maybe more.
486
00:26:54,150 --> 00:26:57,990
And then I can see that, and then when
they send me the overall chat, I can
487
00:26:58,020 --> 00:27:02,400
quickly scan through it, and I can see
the process work itself out as they're
488
00:27:02,410 --> 00:27:07,679
dialoguing with this, and so my rule of
thumb is to always give them more and more
489
00:27:07,730 --> 00:27:10,139
agency over the process as they go along.
490
00:27:10,369 --> 00:27:13,730
So that means a lot of scaffolding
early on, and if I have two projects
491
00:27:13,730 --> 00:27:17,370
or three projects, I let them create
the process a little bit more.
492
00:27:17,719 --> 00:27:22,349
And now for foundational
courses, I really lean into that.
493
00:27:22,689 --> 00:27:24,850
If I have a student in writing
and research, like I really
494
00:27:24,860 --> 00:27:26,259
try to carefully scaffold it.
495
00:27:26,369 --> 00:27:30,920
If it's say an upper-level elective
course, very very different.
496
00:27:30,930 --> 00:27:34,829
You might not need a ton of scaffolding
early on, so I do personalize
497
00:27:34,829 --> 00:27:36,529
that for level and discipline.
498
00:27:36,930 --> 00:27:40,689
A writing-intensive course versus
an elective course might be a
499
00:27:40,689 --> 00:27:44,599
little bit different, and it's
different in terms of course learning
500
00:27:44,599 --> 00:27:46,059
objectives that you need to hit.
501
00:27:46,059 --> 00:27:48,719
And so I'm kind of informed
by that as well as I design.
502
00:27:49,229 --> 00:27:51,699
And then, the other thing that's
going to be interesting for us is
503
00:27:51,699 --> 00:27:59,079
that my college this semester has
taken on AI literacy as a college-wide
504
00:27:59,220 --> 00:28:02,799
objective that we need to hit, and so
we're going to start to see that work
505
00:28:02,879 --> 00:28:04,850
its way into the conversation too.
506
00:28:05,269 --> 00:28:10,009
If a student is taking, say, our writing
and research course, we need to have a
507
00:28:10,009 --> 00:28:13,300
way to tie that course to that overall
college-wide learning objective.
508
00:28:13,509 --> 00:28:16,679
So that affects the scaffolding,
that affects what you do, that
509
00:28:16,680 --> 00:28:19,650
affects if a faculty member wants to
create a custom chatbot and link it
510
00:28:19,650 --> 00:28:22,980
to that, and so that's going to be
another additional wrinkle, I think.
511
00:28:23,199 --> 00:28:26,639
And we'll see it cutting across
different departments because students
512
00:28:26,639 --> 00:28:30,069
in every department are going to
have to show some evidence that
513
00:28:30,070 --> 00:28:32,939
they worked on that competency, that
literacy, whatever you want to call it.
514
00:28:33,489 --> 00:28:38,559
JOHN: One concern that many faculty have
is that students are using AI to offload
515
00:28:38,589 --> 00:28:44,169
their cognitive processing and strategies
such as designing chatbots where students
516
00:28:44,169 --> 00:28:48,089
are interacting with that to help design
a process can be really productive
517
00:28:48,089 --> 00:28:52,140
because the research is suggesting
that when AI is used intentionally
518
00:28:52,149 --> 00:28:55,959
in a manner to support student
learning, it can help students learn.
519
00:28:56,270 --> 00:28:57,789
But we don't always see that.
520
00:28:58,020 --> 00:29:02,009
One of the things I know some faculty here
have been doing is they had been using
521
00:29:02,009 --> 00:29:06,859
in asynchronous courses metacognitive
reflection discussions and logs and so
522
00:29:06,860 --> 00:29:11,659
forth, and they've been seeing more and
more students turn in reflections that
523
00:29:11,660 --> 00:29:16,869
appear to be generated by AI rather than
the authentic voice of their students.
524
00:29:17,300 --> 00:29:21,390
What can we do to help ensure that
students are using AI appropriately.
525
00:29:21,800 --> 00:29:25,789
Having students share their interactions
is a really good way of doing that, so
526
00:29:25,789 --> 00:29:27,809
you can observe it as part of the process.
527
00:29:28,250 --> 00:29:31,710
But how do we deal with those cases
where students turn in something
528
00:29:31,710 --> 00:29:35,690
that doesn't appear to have been
created by them individually?
529
00:29:36,229 --> 00:29:41,140
JASON: I do think getting full
transcripts when you can really helps.
530
00:29:41,140 --> 00:29:44,090
It gives you a sense of their process,
how they're dialoging with it.
531
00:29:44,090 --> 00:29:50,820
My approach has been this: I front load
early on in a semester, demystifying it.
532
00:29:50,820 --> 00:29:53,850
So custom chatbots are
a great way to do this.
533
00:29:53,910 --> 00:29:58,050
I also design for my students
pressure bots, and I have my
534
00:29:58,050 --> 00:29:59,460
students doing this next week.
535
00:29:59,460 --> 00:30:00,529
So they did some of the reading.
536
00:30:00,959 --> 00:30:05,649
They got the page 70 or whatever
it is in the book, and I'm giving
537
00:30:05,650 --> 00:30:09,939
them a pressure bot that is
designed to put pressure on them.
538
00:30:09,939 --> 00:30:11,999
I told them it might
frustrate them a little bit.
539
00:30:12,020 --> 00:30:13,530
It's not going to give them answers.
540
00:30:13,530 --> 00:30:16,049
It's going to ask them for clarification.
541
00:30:16,049 --> 00:30:18,409
Going to ask them to go back to the book.
542
00:30:18,529 --> 00:30:19,980
I want the book in front of them.
543
00:30:20,200 --> 00:30:23,040
They can do some of it from memory, but
in the end, I want that student to go
544
00:30:23,040 --> 00:30:24,660
back to the book and take another look.
545
00:30:25,079 --> 00:30:27,049
And so I gave it
instructions for doing that.
546
00:30:27,410 --> 00:30:29,730
And I've done this for a while.
547
00:30:29,959 --> 00:30:35,409
One of the things that really changed
for me was… it seems so simple now… show
548
00:30:35,410 --> 00:30:39,469
them the prompt and so they come back
and we have this conversation, I say,
549
00:30:39,469 --> 00:30:44,129
oh, this is how AI helped them learn,
and I kind of say, well, AI did, but
550
00:30:44,279 --> 00:30:48,779
it's because I did this to it, and you
can like show them I had to change it.
551
00:30:49,209 --> 00:30:54,209
I had to basically, in these custom
instructions, change it from a sycophantic
552
00:30:54,439 --> 00:30:58,380
chatbot that's going to say you're awesome
and great and just keep going, and towards
553
00:30:58,380 --> 00:31:01,709
something that's going to challenge
you, and there's all of this knowledge
554
00:31:01,709 --> 00:31:03,050
that needed to happen in the background.
555
00:31:03,330 --> 00:31:06,149
And showing them has been really,
really helpful for demystifying
556
00:31:06,489 --> 00:31:08,889
how this works and making it clear.
557
00:31:08,949 --> 00:31:11,809
And some students, I think, already
knew this, or it seemed very
558
00:31:11,809 --> 00:31:13,179
obvious, and some not so much.
559
00:31:13,369 --> 00:31:17,459
If you go and you open ChatGPT and try
to do this, it will not work because
560
00:31:17,459 --> 00:31:18,739
it doesn't have this background.
561
00:31:18,739 --> 00:31:20,899
It doesn't have this push for it.
562
00:31:20,909 --> 00:31:26,369
So that has been very, very helpful
in demystifying it, and then
563
00:31:26,579 --> 00:31:28,059
getting transcripts really help.
564
00:31:28,430 --> 00:31:31,509
And then also the other thing that's
changed in my approach is after it's
565
00:31:31,509 --> 00:31:35,290
all said and done, a student has gone
through the process, they've created
566
00:31:35,299 --> 00:31:41,189
the thing, asking for a transparency
statement that really has them walking
567
00:31:41,189 --> 00:31:43,819
through what they did and why they did it.
568
00:31:44,689 --> 00:31:47,819
So I have students at the end of a paper.
569
00:31:48,080 --> 00:31:49,949
I ask them, "What are the steps you took?
570
00:31:50,599 --> 00:31:51,709
Did you do a draft?
571
00:31:51,709 --> 00:31:52,829
Did you do an outline?
572
00:31:52,830 --> 00:31:54,269
Did you draw something out?
573
00:31:54,449 --> 00:31:55,650
…whatever it is.
574
00:31:56,500 --> 00:32:00,349
And I have them go to
the AI Assessment Scale.
575
00:32:00,679 --> 00:32:06,329
So this was created by Leon Furtze
and Mike Perkins, and so very simply,
576
00:32:06,849 --> 00:32:11,369
for a step in their process, they
give it a one if they didn't use AI.
577
00:32:11,399 --> 00:32:13,930
I will have students who say, "I
didn't want to use AI. I didn't work
578
00:32:13,930 --> 00:32:17,279
it into my process. One's all the way
down.” A one if they didn't touch it.
579
00:32:17,279 --> 00:32:19,410
A five if it was full on AI.
580
00:32:19,410 --> 00:32:20,799
I rarely get those.
581
00:32:20,809 --> 00:32:23,779
Every once in a while, they just say I
created an AI outline and went from there.
582
00:32:24,199 --> 00:32:28,170
So go through the process, give it
a number from one to five, and think
583
00:32:28,170 --> 00:32:30,129
about your intellectual ownership.
584
00:32:30,179 --> 00:32:33,630
Did you feel like that actually
served you as a learner?
585
00:32:33,729 --> 00:32:35,309
Were you handing something off?
586
00:32:35,619 --> 00:32:39,730
And so, it's not foolproof by
any means, but it tries to get at
587
00:32:39,760 --> 00:32:43,930
how, when we use this technology,
there's always a trade-off.
588
00:32:44,210 --> 00:32:47,839
What we might gain in efficiency,
we might lose in intellectual
589
00:32:47,840 --> 00:32:49,850
ownership, or anything like that.
590
00:32:49,850 --> 00:32:52,080
And so that has sort of helped.
591
00:32:52,370 --> 00:32:58,139
And over the years, the other thing that
has helped me is… and this just sounds so
592
00:32:58,189 --> 00:33:02,339
obvious, but I think it's kind of hard to
do… really leaning into asking questions.
593
00:33:02,840 --> 00:33:06,010
It seems so obvious, but say, "I want
to talk to you about your process.
594
00:33:06,279 --> 00:33:09,860
What did you do?” I think for a lot
of my students, it's hard because that
595
00:33:09,860 --> 00:33:14,539
feels accusatorial, and I'm learning
to lean into it a little bit more.
596
00:33:14,919 --> 00:33:18,849
I took a look at what you did, can
you walk me through the process?
597
00:33:18,879 --> 00:33:20,809
Did you happen to use AI?
598
00:33:20,910 --> 00:33:21,709
Why or why not?
599
00:33:21,919 --> 00:33:26,500
And just trying to really
make that conversation normal.
600
00:33:26,500 --> 00:33:29,569
It's not even just that I'm policing
it or accusing it, but try to say “I
601
00:33:29,570 --> 00:33:33,750
really want to talk about this. I think
something happened. Can you work with me
602
00:33:33,860 --> 00:33:39,249
to try to figure out if that happened or
that didn't happen?” And I always felt
603
00:33:39,500 --> 00:33:44,449
like that was going to automatically feel
like I was catching them, and all that.
604
00:33:44,549 --> 00:33:46,239
But I've actually found
exactly the opposite.
605
00:33:46,279 --> 00:33:50,169
Depending on how we approach it, it is
a way to collaborate with the students.
606
00:33:50,169 --> 00:33:53,260
And I try to make it clear, this is us
working together because I want to talk
607
00:33:53,269 --> 00:33:54,749
about your process and figure it out.
608
00:33:55,059 --> 00:33:59,430
It takes some ironing over to do that,
but having that transparency statement,
609
00:33:59,439 --> 00:34:04,709
having that demystifying conversation
early on has helped me scaffold for that
610
00:34:04,709 --> 00:34:09,839
conversation because then I talk a lot
with my students about how AI could be a
611
00:34:09,839 --> 00:34:15,549
learning tool, but only if you use it in
very specific and very intentional ways.
612
00:34:16,009 --> 00:34:19,429
REBECCA: I know you were calling
showing the prompt that you wrote,
613
00:34:19,619 --> 00:34:23,989
demystifying a little bit, but I
also think it's modeling process.
614
00:34:23,989 --> 00:34:27,599
Your teaching process, your modeling
process, you're talking about process.
615
00:34:27,599 --> 00:34:30,839
So can you talk a little bit more
about the role of modeling some
616
00:34:30,839 --> 00:34:32,509
of these skills for students?
617
00:34:33,129 --> 00:34:35,749
JASON: I think modeling is one of
the most important things we can do.
618
00:34:36,139 --> 00:34:41,019
So when I create a scaffold for my
students, I walk them through it.
619
00:34:41,070 --> 00:34:43,189
Why did I create that scaffold?
620
00:34:43,500 --> 00:34:47,249
If I'm telling students for that first
project, this is what I want you to do.
621
00:34:47,500 --> 00:34:51,789
That is a model process, and I
think sometimes I myself I never
622
00:34:51,789 --> 00:34:53,849
leaned into just how that works.
623
00:34:54,339 --> 00:34:56,909
So if I ask them, all right, I
want you to free write first.
624
00:34:57,489 --> 00:34:57,779
Cool.
625
00:34:57,849 --> 00:34:58,760
Why did I do that?
626
00:34:59,039 --> 00:35:02,200
Why did I feel like, oh, you can't
just free write with ChatGPT?
627
00:35:02,879 --> 00:35:04,939
I had you free write, then
I had you create an outline.
628
00:35:04,940 --> 00:35:09,519
Whatever the scaffolding is,
and that is a model process
629
00:35:09,709 --> 00:35:11,380
that the instructor has created.
630
00:35:12,019 --> 00:35:15,250
And I think that a lot of it is
showing students that step back
631
00:35:15,639 --> 00:35:19,559
and being very honest when we
learn something about our process.
632
00:35:20,500 --> 00:35:26,139
So one of the example I give to my
students is after ChatGPT came out and
633
00:35:26,139 --> 00:35:27,399
other programs started to come out.
634
00:35:28,099 --> 00:35:33,049
There was a few months when I was
generating my PowerPoints with AI.
635
00:35:33,050 --> 00:35:35,950
I'd have like notes, I'd have
a document, I would generate
636
00:35:35,950 --> 00:35:38,150
it and then I would change it.
637
00:35:38,180 --> 00:35:39,770
I would revise it, all that sort of stuff.
638
00:35:40,030 --> 00:35:43,430
And I end up walking through my students,
this is the process I've created, and
639
00:35:43,430 --> 00:35:48,250
then I junked it because I felt like
generating things with the slide made
640
00:35:48,250 --> 00:35:50,500
it so that I couldn't present on it.
641
00:35:50,610 --> 00:35:53,740
I had a hard time presenting,
pivoting, doing that sort of work.
642
00:35:54,010 --> 00:35:57,259
And so talking to them about
the processes we create, why we
643
00:35:57,260 --> 00:35:58,489
change them, how we go forward.
644
00:35:58,799 --> 00:36:01,510
I think it's probably one of the
most valuable things we can do in the
645
00:36:01,549 --> 00:36:07,490
classroom, and it helps them to see how
important process creation actually is.
646
00:36:07,949 --> 00:36:11,320
I think that many of my students,
when they come in, they've been taught
647
00:36:11,910 --> 00:36:16,019
by the education system that what
really matters is that final product,
648
00:36:16,059 --> 00:36:20,359
and that final product does matter,
but I try to use modeling as a way
649
00:36:20,360 --> 00:36:24,439
to get them to think about how does
scaffolding or any sort of laying out
650
00:36:24,469 --> 00:36:26,629
of the process, how does it help us?
651
00:36:26,849 --> 00:36:28,030
Why are we doing things?
652
00:36:28,030 --> 00:36:32,619
I think that my impulse is always just
to have the scaffold there and not really
653
00:36:32,619 --> 00:36:34,499
talk about why I designed it that way.
654
00:36:34,520 --> 00:36:38,199
And I'm trying to be better about that,
just showcasing the intentionality
655
00:36:38,250 --> 00:36:40,610
because I think it helps them think
about things a little bit differently.
656
00:36:40,610 --> 00:36:44,520
REBECCA: I feel like one thing that I've
also done in that same space that has
657
00:36:44,520 --> 00:36:48,249
been effective is pointing out parts of
the process that maybe as a student I
658
00:36:48,250 --> 00:36:51,869
thought were a waste of time, or I didn't
fully understand why they were there, or
659
00:36:51,869 --> 00:36:58,090
I would skip over it or shortcut it in
various ways, and why I don't do that now.
660
00:36:58,610 --> 00:37:02,399
But I found those conversations to be
really powerful conversations about
661
00:37:02,570 --> 00:37:04,329
what is actually the point of this step.
662
00:37:04,790 --> 00:37:06,860
JASON: That was my
experience in high school.
663
00:37:07,019 --> 00:37:11,820
I hated, I hated writing in high school
because of how it was scaffolded.
664
00:37:12,059 --> 00:37:15,199
Because I was told do this,
do this, do this, and do this.
665
00:37:15,629 --> 00:37:19,439
And even when I was a high school junior
and high school senior, someone would come
666
00:37:19,439 --> 00:37:22,239
in and say, "All right, the first thing
I want you to do is to create a thesis
667
00:37:22,239 --> 00:37:24,559
statement.” No, that doesn't help me.
668
00:37:24,559 --> 00:37:27,879
I need to go through the actual writing
process to figure out what it is.
669
00:37:28,349 --> 00:37:30,999
And then they would say, "Do a
thesis statement, do an outline,
670
00:37:31,009 --> 00:37:33,849
do this.” And I started to realize
at that point that, like, "Oh,
671
00:37:33,879 --> 00:37:35,849
this process does not work for me.
672
00:37:35,849 --> 00:37:37,289
It's great that it works for someone else.
673
00:37:37,299 --> 00:37:39,359
Great that it works for
several of my classmates.
674
00:37:39,629 --> 00:37:42,299
Does not work for me.” And
this again is something that we
675
00:37:42,299 --> 00:37:44,099
can take from process studies.
676
00:37:44,520 --> 00:37:49,530
In the early 1990s, there was
a whole post- process movement.
677
00:37:50,099 --> 00:37:54,730
So this is something that was almost like
35 years ago that people were thinking
678
00:37:54,730 --> 00:37:59,839
about, “Oh, we all have different
processes. It doesn't work for everyone.”
679
00:38:00,220 --> 00:38:04,809
There isn't a linear process we can do
that gets us from point A to point B to
680
00:38:04,809 --> 00:38:06,459
so on and so forth and final product.
681
00:38:06,720 --> 00:38:11,079
The processes that we use are all
different, and they should be different.
682
00:38:11,360 --> 00:38:15,239
And I think that sometimes we
lose how fragmented that can be.
683
00:38:15,419 --> 00:38:19,149
But again, there's all this scholarship
about how to teach in that context,
684
00:38:19,149 --> 00:38:22,019
and sometimes we just need to go back
to it and look at how much of that
685
00:38:22,019 --> 00:38:23,749
carries over to the current moment.
686
00:38:23,750 --> 00:38:27,029
And my experience has been that
quite a bit of it does carry over.
687
00:38:27,599 --> 00:38:31,099
JOHN: So, for those who are thinking
about adopting process-focused teaching,
688
00:38:31,289 --> 00:38:35,560
you'd recommend that they not try to
impose a process on all their students.
689
00:38:36,240 --> 00:38:41,210
JASON: So yeah, find ways in which,
if you're teaching process, make
690
00:38:41,210 --> 00:38:43,310
it conversational if you can.
691
00:38:44,050 --> 00:38:46,970
Now there are certain limits to this
if you're teaching online or teaching a
692
00:38:46,970 --> 00:38:51,439
large number of classes, but find ways
to make it as conversational as you can.
693
00:38:52,010 --> 00:38:55,870
And this actually can be a way to also
manage your course load so that you're
694
00:38:55,870 --> 00:38:57,480
not designing a process for everyone.
695
00:38:57,480 --> 00:38:59,989
You're working with them as
they create their processes.
696
00:39:00,480 --> 00:39:04,139
There are ways to manage that,
and again, it's about scaffolding.
697
00:39:04,250 --> 00:39:07,810
For that first project, I think it's
okay to control the process a little bit.
698
00:39:07,969 --> 00:39:08,700
You just want them to see it.
699
00:39:08,700 --> 00:39:12,099
You want them to see what that
process looks like, and then gradually
700
00:39:12,099 --> 00:39:15,389
release more agency so they can
think about their own process.
701
00:39:15,389 --> 00:39:17,619
Think about what works for them,
and make those sort of selections.
702
00:39:17,849 --> 00:39:21,029
That also, for individual
students, might help cut down
703
00:39:21,039 --> 00:39:23,360
those seven steps to three steps.
704
00:39:23,899 --> 00:39:28,759
If I do a scaffold for students,
it might be like seven steps or so.
705
00:39:29,180 --> 00:39:32,439
I've never had a student who, when
I ask them to create their own
706
00:39:32,439 --> 00:39:34,680
process, had a 10-step process.
707
00:39:34,930 --> 00:39:37,040
It's always like two or three or four.
708
00:39:37,040 --> 00:39:38,429
It's always on the lower end.
709
00:39:38,779 --> 00:39:41,539
I think it helps us in
terms of scalability too.
710
00:39:42,099 --> 00:39:46,689
REBECCA: One thing that has been helpful
when I've thought about this and providing
711
00:39:46,689 --> 00:39:50,779
flexibility is, for example, we need to
do something to come up with an idea.
712
00:39:50,809 --> 00:39:55,289
We need to do some sort of ideation,
but there's lots of ways to do that.
713
00:39:55,849 --> 00:39:59,580
[LAUGHTER] But you need like a component
of a process that gets some ideas out,
714
00:39:59,919 --> 00:40:03,489
and maybe offering like a menu of options
sometimes can be really helpful too.
715
00:40:03,930 --> 00:40:08,449
JASON: Gives them that agency, and
I think that is so key, and it's
716
00:40:08,449 --> 00:40:12,209
one of the reasons too why I give
students as much agency as I can
717
00:40:12,239 --> 00:40:15,139
while also hitting those objectives
over the project they're creating.
718
00:40:15,489 --> 00:40:19,630
This is one of the first principles
of teaching process from the 1970s.
719
00:40:20,079 --> 00:40:24,489
If you're going to teach students to
own the process, they have to have
720
00:40:24,500 --> 00:40:26,269
some ownership over the product.
721
00:40:26,779 --> 00:40:28,699
That's like principle number
one or principle number
722
00:40:28,699 --> 00:40:29,869
two, when it was laid out.
723
00:40:30,149 --> 00:40:34,419
I think that's just so key, and
also it pushes against the incentive
724
00:40:34,629 --> 00:40:39,849
to overuse AI or misuse AI, to
own the project you're creating.
725
00:40:40,049 --> 00:40:43,079
No, it's not going to make everyone
be honorable and do all the work.
726
00:40:43,330 --> 00:40:47,009
But I think it helps when you are
creating a project that you have very
727
00:40:47,009 --> 00:40:50,359
little buy-in, you didn't help create,
you didn't help put that sort of work
728
00:40:50,360 --> 00:40:55,620
in, AI becomes an even more attractive
option for bypassing the learning.
729
00:40:55,879 --> 00:40:59,699
So I think giving students agency,
giving them ownership in the material
730
00:40:59,699 --> 00:41:03,490
is one of the best ways to push against
that and get them to do the learning,
731
00:41:03,490 --> 00:41:05,489
which sometimes requires hard work.
732
00:41:05,849 --> 00:41:08,830
JOHN: You've suggested some strategies
that can be used in any class,
733
00:41:08,830 --> 00:41:10,439
including asynchronous courses.
734
00:41:10,819 --> 00:41:15,820
But are there some aspects of
process-focused assessment that works
735
00:41:15,830 --> 00:41:19,800
well with asynchronous courses when
you don't get to see the students
736
00:41:19,989 --> 00:41:21,869
individually on a regular basis.
737
00:41:22,330 --> 00:41:27,259
JASON: Two things: one, really
think about products that you can
738
00:41:27,259 --> 00:41:31,469
take a look at that have something
about the process built into them.
739
00:41:32,169 --> 00:41:35,600
Those comments, some sort of
combination that you can look at a
740
00:41:35,600 --> 00:41:39,750
product pretty quickly and see what
actually was built into it, how was
741
00:41:39,750 --> 00:41:41,999
it designed, that sort of thing.
742
00:41:42,599 --> 00:41:45,460
And then the other is lean
into asking questions.
743
00:41:45,990 --> 00:41:50,190
I think that if I ask for every process
along with every assignment, especially
744
00:41:50,190 --> 00:41:54,000
in an online course, it blows up so
fast it's overwhelming for everyone.
745
00:41:54,410 --> 00:41:58,540
But if I'm looking at say 20 projects
and I ask a question about the
746
00:41:58,540 --> 00:42:01,489
process with one or two of them,
that's doing my due diligence.
747
00:42:01,490 --> 00:42:03,129
And again, it can be conversational.
748
00:42:03,129 --> 00:42:07,419
I think there are ways to phrase it so
that it becomes a part of a collaborative
749
00:42:07,419 --> 00:42:10,439
exercise to figure out what that
process was, how it can be improved.
750
00:42:10,700 --> 00:42:13,529
And those have been ones that
really, really helped me because
751
00:42:13,529 --> 00:42:17,409
if you're teaching a pretty large
online asynchronous class, you
752
00:42:17,410 --> 00:42:21,160
need to do it in a way that guards
your own mental health, if you can.
753
00:42:21,160 --> 00:42:23,649
And those have been the two
that really, really helped me.
754
00:42:23,919 --> 00:42:27,189
So that I'm not asking for a
page-long process statement
755
00:42:27,289 --> 00:42:29,039
from 20 students for one week.
756
00:42:29,039 --> 00:42:32,660
I can't do that, but I can do
ad hoc questions, make them
757
00:42:32,660 --> 00:42:34,259
conversational, have a back and forth.
758
00:42:34,270 --> 00:42:36,350
That's much more imaginable to me.
759
00:42:36,459 --> 00:42:39,599
And I make it very clear to students,
I ask you for clarification on a
760
00:42:39,610 --> 00:42:43,399
comment on Canvas, the LMS that we
use, that is a part of the assignment.
761
00:42:43,449 --> 00:42:46,119
Writing back to me is a part of
the assignment, so we can have
762
00:42:46,119 --> 00:42:49,169
this back and forth, which they're
not usually used to early on.
763
00:42:49,169 --> 00:42:52,370
But if I do it a couple times, they
become very used to it pretty quickly.
764
00:42:52,979 --> 00:42:55,649
REBECCA: We talked a little bit earlier
about your pressure bot, and then
765
00:42:55,659 --> 00:42:58,880
also a bot to help design a process.
766
00:42:58,880 --> 00:43:02,960
Can you talk a little bit about
ways that we might use AI to do some
767
00:43:03,209 --> 00:43:07,869
critique of our work or challenging
students to interrogate their work
768
00:43:07,869 --> 00:43:10,169
a little bit more before submission?
769
00:43:10,529 --> 00:43:10,579
[LAUGHTER]
770
00:43:11,619 --> 00:43:16,139
JASON: The model that has been really,
really helpful is the PAIRR model.
771
00:43:16,139 --> 00:43:18,439
So it's P-A-I-R-R.
772
00:43:18,799 --> 00:43:23,919
It's peer and AI review and
reflection, I think i's the acronym.
773
00:43:24,159 --> 00:43:28,379
So this is out of the University of
California San Diego, and so what they
774
00:43:28,379 --> 00:43:33,709
found was the big thing was you use AI
assistive feedback, and it's very very
775
00:43:33,729 --> 00:43:36,189
helpful for students, and you sandwich it.
776
00:43:36,629 --> 00:43:38,819
Basically, you start with peer assessment.
777
00:43:38,829 --> 00:43:41,019
I also sometimes start
with self-assessment.
778
00:43:41,250 --> 00:43:42,529
How do you think you did?
779
00:43:43,020 --> 00:43:45,679
Then you can get feedback
from an AI program.
780
00:43:45,679 --> 00:43:49,019
You can ask for the transcript,
however you want to do it, and then
781
00:43:49,020 --> 00:43:51,079
you have them review and reflect on it.
782
00:43:51,080 --> 00:43:54,600
And so it's really about
anchoring that feedback.
783
00:43:54,819 --> 00:43:59,070
They go and they take their paper
and they put it into a feedback
784
00:43:59,129 --> 00:44:03,140
generator or an AI grader, which
my students are increasingly doing.
785
00:44:03,519 --> 00:44:04,579
Do you agree with it?
786
00:44:04,619 --> 00:44:05,809
Would you change it?
787
00:44:05,809 --> 00:44:07,579
Do you think that grade is accurate?
788
00:44:07,610 --> 00:44:09,800
And that allows them to
anchor it a little bit more.
789
00:44:10,029 --> 00:44:11,179
That's the key to it.
790
00:44:11,179 --> 00:44:14,039
They can get that feedback, but
I also want them to engage with
791
00:44:14,099 --> 00:44:16,809
it critically, because I think
the impulse for my students is to
792
00:44:16,809 --> 00:44:20,839
just take all of the feedback from
an AI bot and just implement it.
793
00:44:21,080 --> 00:44:22,249
And I don't want them to do that.
794
00:44:22,440 --> 00:44:24,369
And I'll tell them, if I even
go through something like
795
00:44:25,099 --> 00:44:26,389
Grammarly, it's on my own work.
796
00:44:26,389 --> 00:44:29,159
I'm constantly saying
yes, no, yes, no, yes.
797
00:44:29,429 --> 00:44:32,490
You use your judgment, and
so I think that's the muscle
798
00:44:32,490 --> 00:44:33,490
that needs to be developed.
799
00:44:33,490 --> 00:44:37,370
And a lot of it is contextualizing
the feedback rather than just
800
00:44:37,400 --> 00:44:39,109
letting it stand on its own.
801
00:44:39,500 --> 00:44:40,520
Yeah, and you compare it.
802
00:44:40,980 --> 00:44:42,679
What did the AI bot tell you?
803
00:44:43,120 --> 00:44:44,689
What did your peer tell you?
804
00:44:44,910 --> 00:44:47,119
If they're different, why do
you think they're different?
805
00:44:47,250 --> 00:44:48,650
What am I telling you?
806
00:44:49,080 --> 00:44:52,670
And so it allows them to kind of see the
link, which is exactly what a lot of times
807
00:44:52,670 --> 00:44:54,239
we're being asked to do in the workforce.
808
00:44:54,420 --> 00:44:56,689
We go through, and we have a project.
809
00:44:57,359 --> 00:44:58,539
I do this all the time.
810
00:44:58,750 --> 00:45:01,589
You have a project, you send it
out there, you get three sets of
811
00:45:01,660 --> 00:45:03,849
feedback, and you have to like judge.
812
00:45:04,180 --> 00:45:06,400
And they don't complement
each other a lot of times.
813
00:45:06,400 --> 00:45:07,659
They contradict each other.
814
00:45:07,930 --> 00:45:10,149
You can't hit all those sets of feedback.
815
00:45:10,520 --> 00:45:13,909
So I think that there's an
argument to be said that this
816
00:45:13,919 --> 00:45:15,509
is part of workforce training.
817
00:45:15,820 --> 00:45:17,090
How do you implement that feedback?
818
00:45:17,090 --> 00:45:21,129
And AI just becomes one set of feedback
among others, which I think is exactly
819
00:45:21,129 --> 00:45:24,809
where it should be, so that it's not
getting too much gravity and becoming
820
00:45:24,809 --> 00:45:28,280
the center of attention, and it
contextualizes it a little bit, and also
821
00:45:29,560 --> 00:45:32,879
starts to demystify the feedback in a
way that's just one source among many.
822
00:45:33,410 --> 00:45:37,960
JOHN: Suppose someone wants to begin
implementing process-based assessment.
823
00:45:38,299 --> 00:45:43,310
Can it be something done at a small scale
before they completely revise the design
824
00:45:43,310 --> 00:45:46,649
of their courses, or is it something
that should be done throughout the
825
00:45:46,650 --> 00:45:49,320
course as a complete change in structure?
826
00:45:49,979 --> 00:45:52,489
JASON: Start small, please start small.
827
00:45:52,489 --> 00:45:54,130
Make it easier for yourself.
828
00:45:54,130 --> 00:45:55,490
Do it for an assignment.
829
00:45:55,490 --> 00:45:57,079
Do it for a mini assignment.
830
00:45:57,409 --> 00:45:58,889
So start small.
831
00:45:59,089 --> 00:46:00,259
Build from there.
832
00:46:00,639 --> 00:46:04,710
In my experience, if you try to make
the entire course process based.
833
00:46:05,220 --> 00:46:09,709
It's a lot to deal with at once, and
especially because I do believe that a lot
834
00:46:09,709 --> 00:46:13,849
of teaching process is discipline-based
or discipline-sensitive, I'll say.
835
00:46:14,269 --> 00:46:17,649
And so if you have that assignment,
you have that scaffold, that's
836
00:46:17,649 --> 00:46:18,779
one way of teaching process.
837
00:46:18,779 --> 00:46:21,180
Take what you've already learned,
move it towards something else.
838
00:46:21,199 --> 00:46:22,449
Try something a week.
839
00:46:22,659 --> 00:46:24,429
Maybe you try to think-out-loud.
840
00:46:24,759 --> 00:46:25,719
Maybe it works.
841
00:46:25,779 --> 00:46:26,709
Maybe it doesn't work.
842
00:46:26,709 --> 00:46:28,060
You can sort of pivot from there.
843
00:46:28,070 --> 00:46:32,279
So I would say you definitely try
small, iterate, figure out what
844
00:46:32,279 --> 00:46:35,709
you've learned from that, and
then you can design from there.
845
00:46:35,790 --> 00:46:39,799
I don't think you have to do everything
all at once or do all of your classes.
846
00:46:39,800 --> 00:46:43,649
I say if you're just getting started
on that journey, start small in
847
00:46:43,649 --> 00:46:48,140
one course and then build as you
feel like you learn more about it.
848
00:46:48,479 --> 00:46:51,079
REBECCA: Well, we always wrap
up by asking: "What's next?”
849
00:46:51,720 --> 00:46:56,290
JASON: Yeah, so I think the big
thing is trying to figure out how
850
00:46:56,300 --> 00:46:57,900
grading works in this conversation.
851
00:46:58,330 --> 00:47:00,120
This is an ongoing project for me.
852
00:47:00,120 --> 00:47:04,219
I have a lot of provisional
ideas, and I've changed my grading
853
00:47:04,220 --> 00:47:05,729
over the last couple of years.
854
00:47:06,080 --> 00:47:11,619
But if the process-focused classroom, and
I think it is, is about experimentation,
855
00:47:11,970 --> 00:47:15,569
it's about you try something, you
try that process, and it fails.
856
00:47:16,129 --> 00:47:17,789
How does grading fit into that?
857
00:47:17,789 --> 00:47:22,230
Because the students get to me, and
they've had 12 plus years, usually
858
00:47:22,479 --> 00:47:26,759
in a formal education system where
everything is made visible, everything
859
00:47:26,759 --> 00:47:32,590
is graded, everything is given a score,
and sometimes, especially when students
860
00:47:32,710 --> 00:47:36,559
go on their phone now, they go into the
LMS and they're greeted by an AI bot
861
00:47:36,580 --> 00:47:38,270
that offers to do something for them.
862
00:47:38,270 --> 00:47:42,310
I think a lot about the incentive
structure, and this is big-picture stuff.
863
00:47:42,310 --> 00:47:45,639
Trying to figure out if we're
going to teach process in this
864
00:47:45,639 --> 00:47:50,390
way, where generative AI is just
going to be embedded in everything.
865
00:47:50,629 --> 00:47:52,640
Do we need to think about grading again?
866
00:47:53,000 --> 00:47:55,129
What sort of grading systems helps?
867
00:47:55,349 --> 00:48:00,449
Is there an incentive structure that
students are surrounded by that we need
868
00:48:00,450 --> 00:48:02,200
to start to challenge and poke again?
869
00:48:02,209 --> 00:48:05,380
So that's increasingly what I'm thinking
about because it's really hard to
870
00:48:05,380 --> 00:48:09,419
get students to experiment and play
in a classroom when it's like, oh,
871
00:48:09,439 --> 00:48:12,329
they're gonna get a 70 instead of 100.
872
00:48:12,340 --> 00:48:18,410
So that's the current part of my project
that I'm always stuck on because grade
873
00:48:18,410 --> 00:48:21,899
reform is really hard, and we'd have
to wait a really really long time for
874
00:48:21,930 --> 00:48:26,210
if it ever happens, and so trying to
figure out how we can push against the
875
00:48:26,210 --> 00:48:31,549
incentive structures of transactional
education, even within our very
876
00:48:31,779 --> 00:48:33,289
pretty limited spheres of influence.
877
00:48:33,790 --> 00:48:35,029
That's where my mind is right now.
878
00:48:35,409 --> 00:48:37,150
REBECCA: Well, thank you
so much for joining us.
879
00:48:37,150 --> 00:48:40,669
It's a great conversation and plenty
of things for everyone to think about.
880
00:48:41,110 --> 00:48:43,529
JOHN: And we're really looking
forward to seeing your book.
881
00:48:44,049 --> 00:48:44,970
JASON: Thank you so much.
882
00:48:47,330 --> 00:48:47,410
[MUSIC]
883
00:48:51,570 --> 00:48:54,319
JOHN: If you've enjoyed this
podcast, please subscribe and
884
00:48:54,330 --> 00:48:58,369
leave a review on Apple Podcasts
or your favorite podcast service.
885
00:48:58,949 --> 00:49:03,449
To continue the conversation, join us
on our Tea for Teaching Facebook page.
886
00:49:04,430 --> 00:49:07,059
REBECCA: You can find show
notes, transcripts and other
887
00:49:07,059 --> 00:49:09,419
materials on teaforteaching.com.
888
00:49:10,399 --> 00:49:12,429
Music by Michael Gary Brewer.
889
00:49:13,649 --> 00:49:16,669
JOHN: Editing Assistance
provided by Madison Lee.
890
00:49:16,669 --> 00:49:20,669
[MUSIC]