WEBVTT
00:00:34.759 -->
00:00:41.560I had students being like, I literally had no idea that that AI could be used in this way.
00:00:41.560 -->
00:00:48.119Like I had no they they said I will never study the same way again, like for a test, period.
00:00:48.119 -->
00:00:51.719Like I had no clue that I could do this.
00:00:52.679 -->
00:01:00.679Welcome to Intentional Teaching, a podcast aimed at educators to help them develop foundational teaching skills and explore new ideas in teaching.
00:01:00.679 -->
00:01:02.519I'm your host, Derek Bruff.
00:01:02.519 -->
00:01:07.640A few years ago, it was pretty easy for math educators to ignore generative AI.
00:01:07.640 -->
00:01:13.319The chatbots of 2022 and 2023 were notoriously bad at math.
00:01:13.319 -->
00:01:15.560That, however, is no longer true.
00:01:15.560 -->
00:01:23.159Today's Frontier AI models are very good at math to the point of proving mathematical conjectures that have been open for decades.
00:01:23.159 -->
00:01:31.000Today on the show, I have a round table of math faculty sharing what AI aware mathematics teaching looks like here in 2026.
00:01:31.240 -->
00:01:43.480Lew Ludwig is professor of mathematics at Denison University, and co-author with Todd Zakrajsek of the new book, The Science of Learning Meets AI, which I believe is out to the very week this episode airs.
00:01:43.480 -->
00:01:56.200Lew is one of the organizers of the Teaching Math with Generative AI Google Group, which is how I found out about our second guest, Chloe Lewis, assistant professor of mathematics at the University of Wisconsin at Eau Claire.
00:01:56.200 -->
00:02:10.599That Google group meets twice monthly on Zoom, and Chloe recently shared her Critique the Chatbot assignment, in which students evaluate ChatGPT's attempts to prove mathematical statements that students had previously disproved.
00:02:11.319 -->
00:02:24.039And I'm glad to welcome back onto the podcast Amy Langville, professor of mathematics at the College of Charleston, and Kathryn Pedings-Behling, online education and e-learning coordinator, also at the College of Charleston.
00:02:24.039 -->
00:02:35.240You may remember them from way back in episode 10 of the podcast in April of 2023 when they shared about their line of innovative mathematics textbooks called Deconstruct Calculus.
00:02:35.240 -->
00:02:45.400These books are packed with creative activities that invite students into embodied learning, often requiring students to do something in the physical world that helps explain the mathematical concept.
00:02:45.719 -->
00:02:47.960Yes, it's another four-guest episode.
00:02:47.960 -->
00:02:54.039And with all these smart people, the conversation runs a little long, so I'm going to wrap up this introduction right now.
00:02:54.039 -->
00:02:59.479Thank you all for being on Intentional Teaching Today.
00:02:59.479 -->
00:03:08.039I'm excited to have some fellow uh mathematicians on the podcast and to talk about how AI is changing the work of math education today.
00:03:08.039 -->
00:03:08.919Thanks for being here.
00:03:08.919 -->
00:03:10.359Very excited to have you all here.
00:03:10.359 -->
00:03:13.560Um I'm gonna start with my usual opening question.
00:03:13.560 -->
00:03:21.800Um, and it is this uh can you tell us about a time when you realized that you wanted to be an educator?
00:03:21.800 -->
00:03:24.759And uh Kathryn, maybe I'll start with you.
00:03:25.479 -->
00:03:25.879Sure.
00:03:25.879 -->
00:03:28.120Um, yeah, so I was the typical child.
00:03:28.120 -->
00:03:40.199I always knew I wanted to be an educator, so I'll answer this by saying when I knew I wanted to be a math educator, and that's when I had class um with a high school math teacher whom I had for my junior and senior years named Dr.
00:03:40.199 -->
00:03:43.960Nancy Long, who actually like completely changed my life.
00:03:43.960 -->
00:03:49.240Um, and I knew at that point that I wanted to be just like her and be a math educator just like her.
00:03:49.479 -->
00:03:50.120That's awesome.
00:03:50.120 -->
00:03:50.840That's awesome.
00:03:50.840 -->
00:03:51.400I love that.
00:03:51.400 -->
00:03:56.759I had I had a couple of key high school teachers too that kind of got me in that direction.
00:03:56.759 -->
00:03:58.199What about you, Amy?
00:03:59.080 -->
00:04:01.080Uh mine was later than Kathryn.
00:04:01.080 -->
00:04:16.360I was in graduate school trying to decide what to do with my uh degree in operations research, and I was a TA in the math department, and they forced me to teach my own class, and I didn't want to do it and protested, but they they made me and then I found out I loved it.
00:04:16.360 -->
00:04:22.519In hindsight, it made me made sense because I had been all throughout high school and um college.
00:04:22.519 -->
00:04:26.680I was an instructor of physical education type things like basketball and volleyball.
00:04:26.840 -->
00:04:29.960So uh kind of a natural teacher, perhaps.
00:04:29.960 -->
00:04:30.919Yeah.
00:04:30.919 -->
00:04:32.439What about you, Lew?
00:04:33.319 -->
00:04:34.840Um I would say so.
00:04:34.840 -->
00:04:40.360I kind of in college, I kind of landed in what was called our CLC, the computer learning center.
00:04:40.360 -->
00:04:52.039And uh I they had a very interesting thing set up where you would actually kind of coach or teach one-on-one different uh software packages, Lotus 123, Microsoft, or no, it wasn't even Microsoft back then.
00:04:52.039 -->
00:04:53.879It was MultiMate, all these crazy things.
00:04:53.879 -->
00:04:57.320But anyway, it was something that I kind of naturally took to and really enjoyed.
00:04:57.320 -->
00:05:03.000And the fact that I look at my pedigree and both parents were teachers, and I have a grandparent that was a teacher, and I married it.
00:05:03.000 -->
00:05:04.600It was just kind of like in the water there.
00:05:04.600 -->
00:05:06.519So yeah, always been interested.
00:05:06.759 -->
00:05:11.639Yes, I've had many folks on the podcast who point to a family pedigree of teaching.
00:05:11.639 -->
00:05:12.360Yeah.
00:05:12.360 -->
00:05:15.159Um, and Chloe, what about you?
00:05:15.799 -->
00:05:23.480Um, I was an undergraduate at Northern Michigan University, which is in the upper peninsula of Michigan, pretty remote rural area.
00:05:23.480 -->
00:05:28.519Um, and when I was there, actually, it was sort of undergoing a really big like economic shift.
00:05:28.519 -->
00:05:35.639Like we were transitioning out of being like a coal mining area into, I mean, it was just like Obama-era energy sort of efficiency kind of thing.
00:05:35.639 -->
00:05:37.320We're not going to be coal miners anymore, right?
00:05:37.320 -->
00:05:46.120So there was um a lot of money to like re-education grants for um people who were like returning to school after a really long time.
00:05:46.120 -->
00:05:51.080And I got the chance to work with the developmental mathematics course as an undergraduate while I was there.
00:05:51.080 -->
00:05:56.600And I just really liked the like diversity of a college classroom in that setting.
00:05:56.600 -->
00:06:00.840So that really made me want to be an educator and also to work at a public institution.
00:06:01.080 -->
00:06:01.799Oh, that's great.
00:06:01.799 -->
00:06:02.759That's great.
00:06:02.759 -->
00:06:07.399That's maybe a little closer to my own, is I knew I wanted to be a math major.
00:06:07.399 -->
00:06:13.639Um, but when I started actually kind of doing some tutoring as a college student, that's when I realized I really love this education bit.
00:06:13.639 -->
00:06:14.919Well, thank you all.
00:06:14.919 -->
00:06:19.320I think that gives us a little, a little sense, a little glimpse of the kind of who you are.
00:06:19.320 -->
00:06:23.159Um, but let's jump into this AI discussion.
00:06:23.159 -->
00:06:29.159Um and Lew, I'm gonna start with you mainly because I know that you're one of the organizers of this math and AI group.
00:06:29.159 -->
00:06:34.759Um, and I know that you've been very active in kind of writing and speaking about AI over the last few years.
00:06:34.759 -->
00:06:43.960What how would you describe AI's impact on college math teaching since Chat GPT was launched in 2022?
00:06:43.960 -->
00:06:46.519Have you seen any kind of trajectory there?
00:06:46.919 -->
00:06:47.240Yeah.
00:06:47.240 -->
00:06:50.200So okay, so the day that we'll live in infamy, right?
00:06:50.200 -->
00:06:57.000It was November 30th, 2022, as you pointed out, when ChatGPT 3.5, which is now retired when that landed.
00:06:57.000 -->
00:07:08.919Um, I think by this early spring of 2023, kind of the game for mathematicians, especially if you were teaching a proofs course, was hey, let's find the error that AI made in its proof.
00:07:08.919 -->
00:07:10.279And we'll laugh at the chat bot.
00:07:10.279 -->
00:07:13.399And wasn't that kind of a funny and cute little thing to do, right?
00:07:13.399 -->
00:07:16.120Um, that lasted about a semester.
00:07:16.120 -->
00:07:25.159Now you can only really reliably get errors if you specifically prompt the AI to produce mistakes that students typically make, right?
00:07:25.159 -->
00:07:26.759So the game has really shifted.
00:07:26.759 -->
00:07:34.120Um, I think also trajectory-wise, more significantly, AI has just decimated graded homework, uh, you know, with take-home stuff.
00:07:34.120 -->
00:07:38.679Uh as a result, myself, I've actually moved this semester to a standards-based grading.
00:07:38.679 -->
00:07:44.600You know, students demonstrate mastery of specific skills rather than just kind of accumulate points, that little game that we have.
00:07:44.600 -->
00:07:51.559Uh, the interesting thing is, this whole structure, if you've ever never tried this kind of standards-based grading, a lot of overhead.
00:07:51.559 -->
00:07:58.360I couldn't have done all this overhead without the very technology that broke my old system, right?
00:07:58.360 -->
00:08:01.720So the thing that broke my course actually has helped me to redesign it.
00:08:01.720 -->
00:08:09.799So, you know, the trajectory for me has been instead of trying to work around AI, is actually to kind of uh you know more lean into it and try to use it to my advantage.
00:08:09.799 -->
00:08:12.759And and for this semester, I really like where I've ended up with that.
00:08:13.159 -->
00:08:13.480Okay.
00:08:13.480 -->
00:08:14.200Okay.
00:08:14.200 -->
00:08:25.000Um, Amy, Kathryn, I want to turn to you to next, in part because I think you both teach online math courses and online education has been particularly impacted by AI.
00:08:25.000 -->
00:08:32.679Um, what's what's that trajectory been for you over the last few years uh as AI has has entered our world?
00:08:33.399 -->
00:08:40.679Similar to Lew, like there was the slow, what is this tool gonna do, and then realizing it just keeps getting better and better and better.
00:08:40.679 -->
00:08:48.919And so you can't uh ignore it, but on the same, on the same hand, it it's uh related to prior technology.
00:08:48.919 -->
00:08:54.360So like Kathryn and I've had many discussions with colleagues about tech tool use prior to AI.
00:08:54.360 -->
00:09:06.840And so we were already a little bit ahead of the curve leading towards the let's encourage tech tool use so that we can solve more interesting applied problems with students in an online space.
00:09:06.840 -->
00:09:16.680And I guess maybe one of the first things we did, we used that typical you see a lot of folks do red, yellow, green on particular assignments as whether it's prohibited or allowed.
00:09:16.680 -->
00:09:26.680And and Kathryn's got a little bit found a little bit more nuanced one where we have for every assignment it's clear to the student use is allowed or prohibited.
00:09:26.680 -->
00:09:42.440Um, and then we also with our uh textbooks that we use, we created our own thing for deconstruct calculus uh for our students, and it has become a really useful physical component of the class.
00:09:42.440 -->
00:09:47.480So much of an online class is on the screen, and to have this physical book has been very helpful.
00:09:47.480 -->
00:10:10.680And in having that, I think, also eased the transition into AI in some ways because the book itself, we we already focused on process over product, so every every page had answers on it, and you know, and maybe I'll let Kathryn jump in here and some more about the use of the book, maybe and how it helps with AI.
00:10:12.040 -->
00:10:17.000Yeah, so uh we Amy and I always love a new technology.
00:10:17.000 -->
00:10:27.240Um we we love to jump in, we love to learn about things, and we we realized pretty early on that this was gonna be a new norm for the world.
00:10:27.240 -->
00:10:36.360Um and so uh instead of putting our head in the sand, we really decided to figure out like, what does this mean for us?
00:10:36.360 -->
00:10:38.120What does this mean for our classes?
00:10:38.120 -->
00:10:40.759What does this mean for assessment of students?
00:10:40.759 -->
00:10:58.519Uh, what does this mean for the expected knowledge that students have in a class and what um skills they need to be learning, and and how that can then apply to their own lives when they leave college.
00:10:58.519 -->
00:11:04.200Um, so Amy and I in particular love to teach the business calculus class.
00:11:04.200 -->
00:11:07.720These are the non-STEM major students.
00:11:07.720 -->
00:11:16.360A lot of them are business majors, but we have quite a few psychology majors and um you know, we have some education majors, people like that in there.
00:11:16.360 -->
00:11:30.040And so we really started thinking about how we can teach them to use the tool in a way that it's going to be helpful when they leave the classroom, right?
00:11:30.040 -->
00:11:36.200And they may never do insert math topic here again, right?
00:11:36.200 -->
00:11:38.360This is what they want to tell you, right?
00:11:38.360 -->
00:11:47.800But we use math as an impetus for like how do we use these tools to help us gain knowledge when we need to learn new things.
00:11:48.040 -->
00:11:53.800Um, and so our textbook, again, our students really enjoy this physicality of the textbook.
00:11:53.800 -->
00:12:05.639Um, and something that we have always done that AI cannot replicate is we have quite a few three-dimensional things that students are required to do with their textbook.
00:12:05.639 -->
00:12:13.639They're writing in their textbook, they're tearing out pages of their textbook and making paper airplanes and throwing them and graphing functions based on that, right?
00:12:13.639 -->
00:12:18.759They're drawing on their hands to think about how three-dimensional maps work, right?
00:12:18.759 -->
00:12:24.920So there are these sort of physical features that AI cannot replicate at this point.
00:12:24.920 -->
00:12:29.320And so we're constantly thinking about that with our curriculum, right?
00:12:29.320 -->
00:12:30.440What can we do?
00:12:30.440 -->
00:12:38.279What can we do with our students that can't be replicated by a chatbot at this point in time?
00:12:38.279 -->
00:12:44.360Um, and so that's just a little, a little glimpse into sort of how we're using it and thinking about it in our classes.
00:12:44.759 -->
00:13:03.639Yeah, and one of the reasons I wanted to have the two of you back on the podcast, y'all were on a couple years ago to talk about um your very unique textbook, um, is that um I I hear from some faculty, uh mostly like English faculty who are who are moving the writing process back into the classroom.
00:13:03.639 -->
00:13:11.080I had a I had a colleague on the podcast recently, Jim Seitz, who's do who's has all of his students, they do all their writing in class.
00:13:11.080 -->
00:13:14.759Um and and there's lots of interesting things to say about that.
00:13:14.759 -->
00:13:18.040But I know for online courses, there's not an in-class option.
00:13:18.040 -->
00:13:23.960Um, but I love the kind of embodied elements of learning that you've built in um to that book.
00:13:23.960 -->
00:13:26.040Um Chloe, what about you?
00:13:26.040 -->
00:13:29.320What is the last three plus years uh in AI?
00:13:29.320 -->
00:13:31.399How does that look like in your teaching?
00:13:31.879 -->
00:13:36.279I don't usually consider myself like an early tech adopter.
00:13:36.279 -->
00:13:38.600I'm still hanging on to like my iPhone 7 or something, right?
00:13:38.600 -->
00:13:47.560But like I um sort of that fall when it first was released to the public, I was interested in it from a pedagogical standpoint.
00:13:47.560 -->
00:13:56.519Um, and so actually, I think the first assignment I gave, I was teaching uh uh advanced linear algebra class as a graduate student at Michigan State at this point.
00:13:56.519 -->
00:14:03.080And like one of the first kinds of exercises we had proved in that class was like, oh, prove that these two sets are equal.
00:14:03.080 -->
00:14:09.160And so they had like written down a proof of these two sets are equal on their homework, and then I gave them two other sets, and I was like, prove these two sets are not equal.
00:14:09.160 -->
00:14:12.759And they this was like homework one, they like submitted it.
00:14:12.759 -->
00:14:17.160I gave them feedback, they got to resubmit it if they wanted, like that was just sort of the normal process.
00:14:17.399 -->
00:14:31.639Um, in their final assignment, I I used to give a proof portfolio, which I don't assign anymore, um, because I think it's too easy to do with AI, but uh it was sort of a metacognitive reflection on like how do you think about proof?
00:14:31.639 -->
00:14:35.720And then it was also some concrete, like include some proofs from this class.
00:14:35.720 -->
00:14:48.680Um, and so as one of the sort of metacognitive reflective questions, I I asked ChatGPT to prove that these two sets are equal when they were not in fact equal, uh, and then asked the students to like assess the validity of that proof.
00:14:48.840 -->
00:14:53.560Um, that question has actually been sort of my like benchmark for the last three years.
00:14:53.560 -->
00:15:01.080Like uh still when falsely a lot of times now, if you command it to prove something falsely, it will say, This statement is not true.
00:15:01.080 -->
00:15:02.600Are you sure you want me to prove that?
00:15:02.600 -->
00:15:10.360Um this particular question of are these two sets equal, when I when I ask it to prove that the two sets are equal, it still returns a false proof every time.
00:15:10.360 -->
00:15:25.560It it has even on um one of my research students had um joined like the temporary pro plan um when it was like available to college students, and even that model still like wrote me a false proof of that statement.
00:15:25.560 -->
00:15:38.440So I guess that was the first thing I'd done with it, and then I've sort of continued that same um exercise in a lot of my classes, not just in proof-based mathematics classes.
00:15:38.440 -->
00:15:52.040So I teach a lot of our uh math for the little arts type classes here, uh, where these are students who not just are not math majors, but they are not STEM majors usually, they are art majors and music majors and things like this.
00:15:52.040 -->
00:15:58.279Um and so we cover, you know, voting theory and graph theory and topics like this.
00:15:58.279 -->
00:16:08.200Um I have in the past like asked Chat GPT to like produce me something that is false like related to voting theory.
00:16:08.200 -->
00:16:10.440Make me a voting theory table that does this.
00:16:10.440 -->
00:16:15.960Um, and so we'll do an exercise in class where we talk about can this happen, and then we go through and think about it.
00:16:15.960 -->
00:16:18.360And then I share a QR code with them.
00:16:18.360 -->
00:16:22.120And I'm like, all right, here's what ChatGPT said when I asked it to prove that what do you think?
00:16:22.120 -->
00:16:36.519Um, and one I think distinction that I've noticed when I give this type of exercise to my math for liberal art students, as opposed to when I give a similar type of exercise to my math students where I say, here's a proof that I made with AI, what do you think about it?
00:16:36.519 -->
00:16:42.840Um, is that the math real art students are much more likely to doubt their own answer, right?
00:16:42.840 -->
00:16:50.759Like we had just talked about it, we just come up to the conclusion before I even introduce anything about AI that was like, here's here's how this works, here's what we think.
00:16:50.759 -->
00:16:55.480But once they see the generated solution, they're like, Well, the computer can't be wrong.
00:16:55.480 -->
00:16:56.840I had a student say that to me one time.
00:16:56.840 -->
00:16:59.879Well, it's from the computer, and the computer can't be wrong, so we must be wrong.
00:16:59.879 -->
00:17:00.840What did we do wrong?
00:17:00.840 -->
00:17:07.000Whereas my math major students are so much more likely to like read through a proof and be like, this makes no sense.
00:17:07.000 -->
00:17:08.119Uh, this is stupid.
00:17:08.119 -->
00:17:09.240Like, why did it do this?
00:17:09.240 -->
00:17:09.480Right.
00:17:09.480 -->
00:17:18.759And so I I think that's a really interesting distinction that I've noticed in the way that students respond to AI generated output.
00:17:19.480 -->
00:17:22.200Well, do you have a conjecture as to why that would be?
00:17:22.200 -->
00:17:25.319I I have some, but I'm curious what you've what you've thought about.
00:17:25.879 -->
00:17:27.160I think it's twofold.
00:17:27.160 -->
00:17:35.400I think one, the culture of mathematics, or at least the culture of mathematics that I try to like build in my mathematics classrooms.
00:17:35.400 -->
00:17:37.160And again, these are more advanced math majors, right?
00:17:37.160 -->
00:17:39.480They've had math under their belt for a couple of years.
00:17:39.480 -->
00:17:43.880And um, is that like uh right?
00:17:43.880 -->
00:17:48.920We we are a we are a classroom that like asks a lot of questions, right?
00:17:48.920 -->
00:17:53.240So like I frequently like write something up on the board, I'm like, I don't know, do we think this is true?
00:17:53.240 -->
00:17:53.559Right.
00:17:53.559 -->
00:17:56.039And then we'll like have a discussion about whether or not we think that's true.
00:17:56.039 -->
00:17:59.400So that's sort of like questioning like, does this work or does this not work?
00:17:59.400 -->
00:18:04.039I think that is sort of inherent to the kinds of math proof classrooms that I built.
00:18:04.039 -->
00:18:07.160It's also just a broader part of the mathematics discipline, right?
00:18:07.160 -->
00:18:13.400As a professional mathematician, one of one function of your job is to, you know, you are refereeing a paper, right?
00:18:13.400 -->
00:18:16.279And so your whole job is to say, is this proof valid or not?
00:18:16.279 -->
00:18:18.360Like that's that's built into the job.
00:18:18.360 -->
00:18:33.960Um so I think that's one part of it is it's just this explicit training in argument validation that we get in mathematics, that is maybe not the same kind of training one would get in a different like kind of degree program.
00:18:34.279 -->
00:18:43.880Um, but I think the second theory I have is like the mathematical uh confidence in these students is very different.
00:18:43.880 -->
00:18:50.600Usually the kinds of students I see in a math for liberal arts course are the kind who write to me at the beginning of the semester and say, I hate math, I've never been good at math, right?
00:18:50.600 -->
00:18:52.759Like the the whole litany of everything.
00:18:52.759 -->
00:18:58.920But um I think one that makes them more doubtful of their own mathematical ability, right?
00:18:58.920 -->
00:19:07.319So even though we just came up with an argument for why this is true, that like just like is completely shot by the time they see something that a computer made.
00:19:07.559 -->
00:19:15.480I also wonder if like there's some, I think in the public, there's some kind of base knowledge about like how do large language models work, right?
00:19:15.480 -->
00:19:19.480Like there's some some sense that like, oh, these are statistical prediction models.
00:19:19.480 -->
00:19:24.200I think that's like a relatively well understood phenomenon by most of the population.
00:19:24.200 -->
00:19:29.000I think that what does that mean to be like a statistically predictive model?
00:19:29.000 -->
00:19:34.360I think that has a significantly different meaning to a student who is like an upper level math major, right?
00:19:34.360 -->
00:19:41.400They understand statistics and probability at a different kind of level than a student who is, you know, a self-proclaimed, oh, I don't like math kind of person.
00:19:41.400 -->
00:19:47.319Like I think that underlying understanding of how these models work is also a factor.
00:19:47.720 -->
00:19:48.839Yeah, yeah.
00:19:48.839 -->
00:19:53.079Um I want to throw this open to to the rest of you.
00:19:53.079 -->
00:20:03.559Do you see similar um similar situations where like some students are kind of over trusting the AI or or more skeptical?
00:20:03.559 -->
00:20:06.519What does that look like in your in your courses, Lew?
00:20:07.799 -->
00:20:10.519Um so I'm doing the standards-based grade thing, right?
00:20:10.519 -->
00:20:13.880And uh I do a lot of uh actually true-false with this.
00:20:13.880 -->
00:20:21.559It's it's kind of an interesting system, but there's a STEM question that has like maybe here's a linear system, and then I ask them a series of questions about that specific thing.
00:20:21.559 -->
00:20:24.839And anyway, we were talking about uh differential equations, right?
00:20:24.839 -->
00:20:29.559Linear differential equations and uh whether things would have the Wronskian or whatever.
00:20:29.559 -->
00:20:37.720I had a kid roll in, this is a sophomore level class, and I you know, my true-false question, he goes, but isn't it in this context, isn't this true due to Abel's theorem?
00:20:37.720 -->
00:20:48.200And I looked at him because my AI also pointed out in this very weird thing at the 600-level course, right, that you might invoke Abel's theorem, but this wouldn't fit in an undergraduate thing.
00:20:48.200 -->
00:20:50.200So I impressed him on him, like, where'd you get that?
00:20:50.200 -->
00:20:52.200He kind of like, you know, got it pulled back.
00:20:52.200 -->
00:20:57.799So, you know, it it's it's interesting how the students, I think, sometimes blindly just accept what's going on.
00:20:57.799 -->
00:21:05.160This was a computer science student taking a linear algebra differential equations course, but uh, you know, they were just uh, you know, bound to determine again.
00:21:05.160 -->
00:21:10.920There's kind of that mindset, I want to get this right, I need this problem to get my meet my expectations and the whole nine yards.
00:21:10.920 -->
00:21:17.880But it's just like, you know, he thought as long as I just throw out the word Abel's theorem, that that will be good enough, and that will kind of give me the passing grade that I need.
00:21:18.200 -->
00:21:28.759So Chloe, it sounds like part of your goal there is to help your students understand AI's strengths and limitations a little bit.
00:21:28.759 -->
00:21:30.360Is that is that fair to say?
00:21:31.079 -->
00:21:43.799I very much see it as like part of my job in in like working through these exercises in class is to uh sow doubt in the like complete and utter truth that is produced by a large language model, right?
00:21:43.799 -->
00:21:51.559Like that I think that's really important, especially for the students, like I said, who are taking this math for liberal arts class and are maybe not as comfortable with mathematics.
00:21:52.279 -->
00:21:52.600Okay.
00:21:52.600 -->
00:21:58.519Um, because I think you're also trying to give them a chance to practice their their proof skills, right?
00:21:58.519 -->
00:22:00.839It's it's kind of serving both goals, right?
00:22:00.839 -->
00:22:06.839A kind of math learning objective, but then also uh kind of an AI literacy, AI know-how piece.
00:22:06.839 -->
00:22:13.799Um, Amy and Kathryn, do you do you also see AI literacy as kind of part of what you're trying to teach now in your courses?
00:22:15.640 -->
00:22:24.200So I I will say something that I think um I think it's not 100% related just to AI literacy.
00:22:24.200 -->
00:22:33.640So what Chloe was talking about is a phenomenon I have seen with my non STEM major students, the whole time I've taught post-secondary.
00:22:33.640 -->
00:22:36.279So I used to teach secondary, I used to teach high school level.
00:22:36.279 -->
00:22:49.079The whole time I've taught post-secondary, it takes me almost until the second half of the semester for any non-STEM major student to correct me if I have made a mistake in class.
00:22:49.079 -->
00:22:55.400And most of the time it's just because they tr they blindly trust me, right?
00:22:55.400 -->
00:22:58.279They're like, well, I'm I'm not a math person.
00:22:58.279 -->
00:23:03.799So like if she's saying this is positive five instead of negative five, then that's gotta be what it is, right?
00:23:03.799 -->
00:23:07.160Even though inside they're like, wait a, wait a minute, is that right?
00:23:07.160 -->
00:23:08.360I don't know if that's right.
00:23:08.360 -->
00:23:18.920I spend almost half my class, the first half of the semester of my um, I also teach a contemporary math class, which is like math for the liberal arts, um, business calculus class.
00:23:18.920 -->
00:23:23.000I spend almost the first half saying, you have you belong here, right?
00:23:23.000 -->
00:23:24.519You have a place here.
00:23:24.519 -->
00:23:27.880Your voice matters, what you think matters.
00:23:27.880 -->
00:23:32.039You can think about mathematics, like you have that ability.
00:23:32.039 -->
00:23:51.079And so I think I don't know that I don't personally know that this is any more blind trusting of a large language model as it was a uh blind trusting of Kathryn Pedings-Behling standing in front of them, right, as their math teacher, right?
00:23:51.079 -->
00:23:54.600So I feel like I'm seeing the same stuff.
00:23:54.600 -->
00:24:08.519It's the to me, I I I don't really see a difference between um them questioning something that's coming out of our, you know, AI agent versus them questioning something that's coming out of my mouth, right?
00:24:08.519 -->
00:24:13.240It doesn't happen either way for those non-STEM major students.
00:24:13.240 -->
00:24:14.039I want to stress.
00:24:14.360 -->
00:24:15.400They doubt themselves.
00:24:15.400 -->
00:24:16.759They doubt themselves.
00:24:16.759 -->
00:24:22.039And so if if someone appears to have some expertise or authority, then they're just gonna go with that.
00:24:22.039 -->
00:24:22.440Yeah.
00:24:22.440 -->
00:24:26.120Um, and that could be their instructor or a confident sounding chat bot, either one.
00:24:26.279 -->
00:24:26.680Yes.
00:24:27.079 -->
00:24:27.559Yeah.
00:24:27.559 -->
00:24:28.440Yeah.
00:24:28.440 -->
00:24:31.240Um, Chloe, I'm gonna circle back to you again.
00:24:31.240 -->
00:24:33.400Uh, your your example is really interesting.
00:24:33.400 -->
00:24:40.200Um uh so you show your students an output that ChatGPT has generated.
00:24:40.200 -->
00:24:45.799You do not have the students go and kind of take that prompt and take it to Chat GPT themselves.
00:24:45.799 -->
00:24:56.120And so I'm wondering how you think about um your students using AI and and like your role in encouraging them or discouraging them from using AI.
00:24:56.120 -->
00:25:02.759I'm guessing that's an intentional choice on your part to kind of do it for them instead of have them use the tool.
00:25:03.319 -->
00:25:16.120Uh yeah, I don't want anybody to be required to register with any particular uh AI, like to get an account on any particular AI website in order to participate in the learning activities that we're doing in class.
00:25:16.120 -->
00:25:22.920So that has been the main motivation for generating uh all of the things myself and then sharing it with them.
00:25:22.920 -->
00:25:26.759I mean, ChatGPT makes it really easy to like share a link out to a chat and things like that.
00:25:26.759 -->
00:25:35.000We do, I mean, like all institutions or most institutions, I guess I should say now, we do have like uh institutional access to Microsoft Copilot.
00:25:35.000 -->
00:25:46.920Um but again, I I've not had this conversation with the student, but I've had this conversation with several other just people who have ethical and environmental objections to the use of AI.
00:25:46.920 -->
00:25:47.319Right.
00:25:47.319 -->
00:25:55.079So I I don't feel like I want to put anybody in a particular position where they would have to make such a choice.
00:25:55.079 -->
00:25:57.240Now, are most of them probably already using it?
00:25:57.240 -->
00:25:58.680Do they already have their own Chat GPT account?
00:25:58.680 -->
00:26:00.360Yeah, like I I I know that, right?
00:26:00.360 -->
00:26:10.440But I I do not want anybody to be forced to use something that they do not want to if uh like in order to like participate in the activities we are doing in class.
00:26:10.759 -->
00:26:11.000Sure.
00:26:11.000 -->
00:26:13.720And it sounds like for your goals, they don't need to either.
00:26:13.720 -->
00:26:14.600So Right.
00:26:14.600 -->
00:26:14.839Yeah.
00:26:15.000 -->
00:26:15.480Exactly.
00:26:15.480 -->
00:26:26.680Um I actually I wanted to like come back around to something that Kathryn said and bring in uh a particular uh reference to the literature um that I think is supporting what her point is.
00:26:26.680 -->
00:26:33.319Uh there is some work of Harel and Sowder in the 1990s about um student proof schemes.
00:26:33.319 -->
00:26:46.120So they sort of diagram like uh the things that like when someone is sitting there reading or watching someone prove something, like uh how do you how do you validate that proof?
00:26:46.120 -->
00:26:51.160Like, what is your sort of thinking process in order to determine do I think this is a valid proof or not?
00:26:51.160 -->
00:26:57.880Um and they studied the appearance of different types of proof schemes in different contexts and different classes of students.
00:26:57.880 -->
00:27:05.000Um, and one particular proof scheme that they found uh showed up in a lot of students was an authoritative proof scheme, right?
00:27:05.000 -->
00:27:11.559Which is exactly what Kathryn was just talking about, the idea that like this thing has to be true because my teacher said it, because it is printed in a textbook.
00:27:11.559 -->
00:27:12.759I think that is another, right?
00:27:12.759 -->
00:27:15.160It is like authority broadly construed.
00:27:15.160 -->
00:27:32.519And so uh Harel and Sowder sort of argue that these proof schemes kind of sit in a hierarchy of like, you know, uh what is I don't know, what is like the not good kind of mathematician do versus like what do you what is the robustly thinking mathematician do?
00:27:32.519 -->
00:27:36.759And sure authoritative proof schemes sit relatively low on that totem pole.
00:27:37.000 -->
00:27:40.600Um but I think more novice notion of proof, yeah.
00:27:40.600 -->
00:27:41.160Correct.
00:27:41.319 -->
00:27:49.400Yeah, but I just I wanted to add that uh to support her point that I I that this is one of the things that that we continue to observe from Harel and Sowder's work.
00:27:49.720 -->
00:27:50.440Yeah, yeah.
00:27:50.440 -->
00:27:51.880What about you, Lew?
00:27:51.880 -->
00:27:54.680Do you well how do you feel about your students using AI?
00:27:54.680 -->
00:27:55.720Do you encourage that?
00:27:55.720 -->
00:27:57.319Do you discourage that?
00:27:59.720 -->
00:28:04.680I assume that as soon as they walk out the door, they have access to AI, right?
00:28:04.680 -->
00:28:07.240I mean, there's just there's just really no way of getting around it.
00:28:07.240 -->
00:28:12.759I know uh Leon Furze is the other uh folks that we're referring to, the you know, the red, green light and all that kind of stuff.
00:28:12.759 -->
00:28:18.360Uh Leon, he he's first developed that and he's kind of modified, he's got some like bubblegum-colored things now.
00:28:18.360 -->
00:28:23.720But but he's uh thinking about that more kind of as an assessment way of how you might assess students.
00:28:23.720 -->
00:28:29.000But again, he kind of believes also that once you step out, it's just naive to think that they wouldn't have access to that.
00:28:29.079 -->
00:28:40.440So that's why I've really kind of directed things in a way to think with my students about how am I going to get to think about what they actually know and understand and can kind of demonstrate for me as opposed to just producing this product.
00:28:40.440 -->
00:28:44.360Because it used to be you produce this product, that was your homework, that was your thinking.
00:28:44.360 -->
00:28:46.759AI has kind of thrown all that out the window.
00:28:46.759 -->
00:28:52.279So that's why I really moved towards this uh this standards-based grading to try to kind of eliminate that.
00:28:52.279 -->
00:28:54.200Now, they know full on.
00:28:54.200 -->
00:28:56.440I mean, I'm very heavily involved with AI.
00:28:56.440 -->
00:28:57.240I talk about a lot.
00:28:57.240 -->
00:29:05.160A lot of my things at the bottom will have a little AI disclosure about how I use it to modify or you know, make formatting changes or whatever the case may be.
00:29:05.160 -->
00:29:09.400I talk to them about creating prompts for practice questions and stuff like that.
00:29:09.640 -->
00:29:14.200But we also talk about, you know, at the end of the day, they need to do the hard part of learning.
00:29:14.200 -->
00:29:17.079They need to do the you know, the friction that's involved.
00:29:17.079 -->
00:29:19.480Because otherwise, I mean, I have a lot of athletes, right?
00:29:19.480 -->
00:29:23.319You know, the old analogy of just, you know, let's just have the forklift lift the uh heavy weights for us.
00:29:23.319 -->
00:29:25.640They know that there's not going to be any benefit to that.
00:29:25.640 -->
00:29:32.759Uh the nice thing is, I think the uh the standard space grading, because they don't have to get it right the first time, gives them a little bit of a safety net.
00:29:32.759 -->
00:29:37.160And they're for once in my I mean, I've been teaching for a long time, man, 30 years plus.
00:29:37.559 -->
00:29:38.360You said Lotus Notes.
00:29:38.360 -->
00:29:40.759So that that aged you a little.
00:29:41.079 -->
00:29:41.240Right.
00:29:41.240 -->
00:29:41.559Yeah.
00:29:41.559 -->
00:29:42.440So yeah, exactly.
00:29:42.440 -->
00:29:46.200So they but they're actually learning from their mistakes, right?
00:29:46.200 -->
00:29:51.240So they they have to get like eight out of 10 or better, and they get a seven, and they don't just say, oh, it's a seven out of ten.
00:29:51.240 -->
00:29:55.240They look at that 30% and they focus on it for next time to do better, right?
00:29:55.240 -->
00:29:58.759Uh I never would have done that without AI messing everything up.
00:29:58.759 -->
00:30:00.120So it's kind of interesting.
00:30:00.120 -->
00:30:11.640I'm I'm using AI to redevelop my whole my class, but I'm still kind of, I guess, teaching in an old school, not really analog, but you know, kind of in an old school way as far as uh getting students to learn and think.
00:30:12.279 -->
00:30:21.319So I'm gonna go back to Amy and Kathryn, um, because I I I know you do have your students use a particular AI tool that you two have developed.
00:30:21.319 -->
00:30:23.160Um, can you tell us a little bit about that?
00:30:23.559 -->
00:30:34.279Kathryn took a class, I think it might have even been loose class on how to uh make your own custom GPT, basically.
00:30:34.279 -->
00:30:46.120And so we have um ChatGPT with uh custom instructions, and we fed it the knowledge base because we already had so many hundreds of pages of our deconstruct calculus book.
00:30:46.120 -->
00:30:53.319It was easy to just feed that as a PDF into the knowledge base, say only use this and give it some instructions.
00:30:53.319 -->
00:31:00.920And so we basically have a 24-hour tutor for our students that is customized to our book, to our language, to our notation.
00:31:00.920 -->
00:31:12.120And um, then Kathryn could talk about some of the student uh reactions to that, and we've started to collect data on custom versus general GPT, interacting with these as a tutor.
00:31:12.120 -->
00:31:15.240Uh, we've gotten some really great feedback.
00:31:16.279 -->
00:31:17.079Kathryn?
00:31:17.559 -->
00:31:24.360We at the College of Charleston, so we do not provide a license for open AI, ChatGPT.
00:31:24.360 -->
00:31:27.079They have a license for Microsoft Copilot.
00:31:27.079 -->
00:31:29.960Um, I'll leave my thoughts on that to myself.
00:31:29.960 -->
00:31:40.440And um but like Lew said, the assumption is they basically leave our classroom and they're using some sort of AI.
00:31:40.440 -->
00:31:59.240And um I don't want to get into this whole ethical thing, but this is what if I ever have students talk to me about it, my whole thing is that uh the college provides Microsoft Office 365, which sits in a data center that we require our students to use, right?
00:31:59.240 -->
00:32:01.559And we don't have a problem with that.
00:32:01.559 -->
00:32:08.279There's nobody talking about ethical implications of our students all having a Microsoft Office 365 account.
00:32:08.279 -->
00:32:14.360But then there's all this conversation about ethical implications of them using Chat GPT.
00:32:14.360 -->
00:32:24.200And so that anyway, my whole point is I can have a conversation all day long about um reducing our cloud usage as a whole.
00:32:24.200 -->
00:32:35.240I can have a conversation about that all day, but I have a lot of trouble pointing the finger at AI, which is why we decided to dive in on this hole.
00:32:35.480 -->
00:32:49.000So um I started doing a lot of thinking after this MAA workshop about AI agents that have a reduced um knowledge base, right?
00:32:49.000 -->
00:32:53.480So we don't let our AI agent access the web for answers.
00:32:53.480 -->
00:32:55.240Okay, so it has the knowledge base.
00:32:55.240 -->
00:32:59.799We have our book, we have our syllabus uploaded there, assessments, et cetera.
00:32:59.799 -->
00:33:17.400Um is that enough for our students to be able to ask it any questions that it has versus them just going to chatgbt.com, open, you know, not an agent, right?
00:33:17.400 -->
00:33:19.960Um, and asking it questions, right?
00:33:19.960 -->
00:33:22.440So how do they interact with those two things?
00:33:22.440 -->
00:33:29.720So I pulled a couple of tools together, um, and we have some survey and some open-ended questions.
00:33:29.720 -->
00:33:37.799Um, and so at the midterm in the final exam, we invite them uh to study with the AIs, right?
00:33:37.799 -->
00:33:43.079So the the AI agent that's specific to their class and then general chat GPT.
00:33:43.079 -->
00:33:44.600So general chat GPT.
00:33:44.600 -->
00:33:46.440And we give them a few, sorry, go ahead.
00:33:46.440 -->
00:33:46.839Please go ahead.
00:33:47.079 -->
00:33:48.440Yeah, let me jump in just to clarify.
00:33:48.440 -->
00:33:54.680So I think I'm hearing two attributes of your agent that are potentially relevant to what you're about to say.
00:33:54.680 -->
00:34:00.600One is that it is uh essentially trained, it has access to your course materials through your textbook.
00:34:00.600 -->
00:34:10.039And I think it's also been prompted to behave more like what Anna Mills calls an ethical tutor, someone who's not just giving away all the answers to students, right?
00:34:10.039 -->
00:34:13.719Which off-the-shelf chat GPT does not have that quality.
00:34:14.119 -->
00:34:14.599For sure.
00:34:14.599 -->
00:34:15.400A hundred percent.
00:34:15.400 -->
00:34:16.599Yes, absolutely.
00:34:16.599 -->
00:34:23.400Yes, we um we had played around, Amy and I had both played around with Khanmigo some before we did this, right?
00:34:23.400 -->
00:34:27.400And this idea of the Socratic type um model.
00:34:27.400 -->
00:34:34.920And I did do some playing around with bullying our agent AI and figuring out how long it would take it before it would give them the answer and all this kind of stuff.
00:34:34.920 -->
00:34:39.239And I I tried to help train it in ways to make that last longer, right?
00:34:39.239 -->
00:34:42.359To make it so you had to bully it longer before it gave you the answer.
00:34:42.359 -->
00:34:45.319So we give them some prompts to start off with, right?
00:34:45.319 -->
00:34:52.599Pick a topic that's gonna be on the midterm and ask the AI agent to make you flashcards for it, whatever, right?
00:34:52.599 -->
00:34:56.119Ask it to make you a multiple choice quiz, 10 questions.
00:34:56.119 -->
00:35:01.319Ask it about one of those questions on the quiz that you didn't understand and ask it to go through, right?
00:35:01.319 -->
00:35:03.559So we give them some prompts.
00:35:03.559 -->
00:35:16.119And the first time we did this, which was fall midterm, this past fall 2025, I had students being like, I literally had no idea that that AI could be used in this way.
00:35:16.119 -->
00:35:22.679Like I had no, they they said, I will never study the same way again, like for a test, period.
00:35:22.679 -->
00:35:26.279Like I had no clue that I could do this.
00:35:26.599 -->
00:35:30.119Um which I think anyway, yeah, go ahead, Amy.
00:35:30.119 -->
00:35:33.480I just think that's particularly important in our online class, right?
00:35:33.480 -->
00:35:36.439And face-to-face or students have access to the math lab.
00:35:36.439 -->
00:35:45.719These are distance ed students, and to be able to have this 24-hour access to a tutor that speaks the same language and and they understand it's a conversation.
00:35:45.719 -->
00:35:48.679It's not just a way to go grab an answer and cheat, right?
00:35:48.679 -->
00:35:54.439It's to have a conversation and to help me prepare for this midterm and final exam.
00:35:54.439 -->
00:35:58.039So they were using it because of the coaching that we've provided.
00:35:58.039 -->
00:36:02.519We, you know, there's it's hard to check on some of this, but they were using it in a constructive way, right?
00:36:02.519 -->
00:36:04.679So we coached them on how to use it.
00:36:05.079 -->
00:36:05.719Yeah.
00:36:05.719 -->
00:36:11.319Um, yeah, I don't know that I had uh any more to say about that, but we were collecting data again this semester.
00:36:11.319 -->
00:36:19.319We're excited this summer to analyze that um to figure out sort of uh what they liked about each one.
00:36:19.319 -->
00:36:23.959They have to answer things that they liked or didn't like about each one that they've studied with.
00:36:23.959 -->
00:36:31.959Um and I I mean, I personally think it's should it's not really a surprise, but I personally think agent tech AI is where we're going, right?
00:36:31.959 -->
00:36:35.559Like everybody's got an AI chatbot somewhere.
00:36:35.719 -->
00:36:47.239And I think I think the neat thing about that is for me to witness, you know, Kathryn went to Lew's class, but then to see um how easy it is to make one for every single class that you have.
00:36:47.239 -->
00:36:51.319Like it's it Lew, Lew had this set up, and it's like, oh, you make the custom instructions.
00:36:51.319 -->
00:36:56.359We're luckily had our own you know textbook material, but not every movie has that.
00:36:56.359 -->
00:36:59.399But there's it there's certain facets of your class that you can make.
00:36:59.399 -->
00:37:03.480And Kathryn did some really cool ones for an independent study student.
00:37:03.480 -->
00:37:20.919It was a student class of once, just one student in a class that you know it was able to sort of mimic and virtual classmates and have um conversations as if you were it was a um math ed student that was going to be teaching and mimic conversations that you might have with high school parents.
00:37:20.919 -->
00:37:27.079And so you can do these things rather easily with not much startup, not much know-how about coding.
00:37:27.079 -->
00:37:30.199Like that's I mean, we was talking about Lotus 123 .
00:37:30.199 -->
00:37:36.839Imagine if you had to do something like that 30 years ago now as an instructor, it's so easy to make these tools.
00:37:37.239 -->
00:37:37.879Yeah.
00:37:37.879 -->
00:37:48.919Chloe, I'm curious what you make of this idea of providing students with a kind of AI-powered tutoring resource.
00:37:48.919 -->
00:37:56.679What what what did you just hear that stood out in your mind as as interesting or useful or maybe concerning?
00:37:57.559 -->
00:38:06.199Um I haven't like uh set up any sort of particular tools for my students use.
00:38:06.199 -->
00:38:09.799Like I said, I mostly sort of bring them something already generated.
00:38:09.799 -->
00:38:18.519Um but I do have a lot of conversations with my students, especially I'm teaching abstract algebra this semester, um, with actually like a returning cohort of students that I'd already taught in geometry.
00:38:18.519 -->
00:38:20.279So it's like small class, nice size.
00:38:20.279 -->
00:38:21.879I know these students pretty well.
00:38:21.879 -->
00:38:29.799Um and uh all of my proofs classes are very like you try something once, I give you a lot of feedback on it, and then you try it again.
00:38:29.799 -->
00:38:35.319Uh not quite a standards-based grading, but like a specs grading kind of situation, I would say more.
00:38:35.319 -->
00:38:40.919Um and in a lot of those conversations, I guess my I'm just always sort of curious.
00:38:40.919 -->
00:38:42.599I'm like, Do we use AI?
00:38:42.599 -->
00:38:51.319And my students themselves are actually quite negative um on how well it does explaining mathematical ideas.
00:38:51.319 -->
00:38:57.239Again, these are abstract algebra students, so I might get a different response if I asked my calculus student something about this, right?
00:38:57.239 -->
00:39:04.919Um, but I I was talking to the student recently, uh, they're like working on some exam revisions right now.
00:39:04.919 -->
00:39:11.639And uh like before I'd even give them their test back, she was like, she's like, I couldn't figure this one out on the test, and I got here, here, and here.
00:39:11.639 -->
00:39:18.839And she's like, and I went and asked, you know, ChatGPT or Claude which everyone, she was like, the answer it gave me was like really bad.
00:39:18.839 -->
00:39:21.879I was like, she's like, it didn't, it didn't make sense.
00:39:21.879 -->
00:39:27.319And we sort of looked at it together and like flicked it into Claude, and it was like it wasn't all that great.
00:39:27.319 -->
00:39:32.119Um I I think there's like more promise in this sort of self-contained ecosystem.
00:39:32.119 -->
00:39:42.679Uh I I have a colleague at uh UW Stevens Point, uh, Grant Kopitzke, who uh also teaches a lot of distance learning, hybrid sort of things.
00:39:42.679 -->
00:39:55.879Um, and he like one of his assignments for his calculus class is like uh they like have to have a conversation with his particular uh calculus tutor, and like they have to like send the log of that conversation to him.
00:39:55.879 -->
00:39:58.599This is one of his assignments, and he really enjoys that.
00:39:58.599 -->
00:40:03.159Um so I I think those use cases are intriguing.
00:40:03.319 -->
00:40:20.679Um I also do like encounter quite often, I think, especially in my math for liberal arts class, I have a lot of English majors, not all of them, but I have a lot of English majors who take those sort of who feel very negatively about the future of generative AI as a whole, right?
00:40:20.679 -->
00:40:39.239And so uh when I like because I have them uh one one assignment in that class is we read uh Cathy O'Neil's Weapons of Math Destruction, uh, and then they do a journal assignment in response to that, uh, which is a really nice book, and it's got a great interchapter that is really accessible to a lot of people.
00:40:39.239 -->
00:40:44.919Um, but that is I sort of ask them to reflect on the role of, you know, did they see a weapon of math destruction in their own life?
00:40:44.919 -->
00:40:51.959And I get a lot of papers from English students that are like, what what is my future gonna be if if all of this exists?
00:40:51.959 -->
00:40:58.919And it feels uh insulting to or just incredibly negative, like sort of perspective on that.
00:40:58.919 -->
00:41:06.839So I like I do think there are students out there who who don't it like aren't using AI tools in this way.
00:41:06.839 -->
00:41:12.119Um whether that's like good for their employment prospects or whatnot later on, I'm not so sure.
00:41:12.119 -->
00:41:17.079But I but I do think this is an attitude that does exist among the population of our students.
00:41:19.319 -->
00:41:24.759Yeah, I just want to, as a uh innocent bystander, I just want to point out there's uh two different things going on.
00:41:24.759 -->
00:41:29.639So our group down in the Carolinas, they're using ChatGPT, but what's it called?
00:41:29.639 -->
00:41:31.319The corporate account, what's it called?
00:41:31.319 -->
00:41:32.599Um what is it?
00:41:32.919 -->
00:41:34.039Are you talking about teams?
00:41:34.279 -->
00:41:35.000Teams, yeah.
00:41:35.000 -->
00:41:37.639So what they're using, they're using it's a pro model.
00:41:37.639 -->
00:41:39.239So this is a deep thinking model.
00:41:39.239 -->
00:41:40.919This thing crushes stuff.
00:41:40.919 -->
00:41:46.279Gazam and I, when we ran the workshop, even last semester, she's like, yes, it will do all of undergraduate mathematics.
00:41:46.279 -->
00:41:49.239So you have to be careful when you say AI can't do things.
00:41:49.239 -->
00:41:55.399Free model, I always, when I give my talks, I usually put up like a three-speed Schwinn bicycle from the 1960s.
00:41:55.399 -->
00:41:56.599That's the free model.
00:41:56.599 -->
00:42:01.079The paid-for model is usually Doc Brown's DeLorean from Back to the Future.
00:42:01.079 -->
00:42:03.559Very fast, we'll take you, but dangerous, right?
00:42:03.559 -->
00:42:13.559But but again, the thing is these these thinking models, um Abel's theorem, again, there's hardly anything that the thinking models that the paid for $20 models cannot do.
00:42:13.559 -->
00:42:16.679So you're hearing slightly two different conversations going on.
00:42:16.679 -->
00:42:23.159Sometimes it's making stuff up, sometimes just keep in mind that we're talking about totally two different models of what's happening.
00:42:23.159 -->
00:42:24.759So just kind of point that out.
00:42:25.319 -->
00:42:30.039Hey, this is Future Derek jumping in with a recommended reading on Lew's Point.
00:42:30.039 -->
00:42:36.359Just last week, historian Mark Humphries wrote a post called The Agents Are Waking Up that made a similar point.
00:42:36.359 -->
00:42:44.119If you're basing your estimation of generative AI's capabilities on free chatbots, you're missing out on a lot of the power of these tools.
00:42:44.119 -->
00:42:49.639It's a good read that gives a sense of where AI is going in the near future, and it's a little scary.
00:42:49.639 -->
00:42:51.639See the show notes for a link.
00:42:52.599 -->
00:42:53.079Amy?
00:42:53.319 -->
00:42:54.519I'd like to jump in on that too.
00:42:54.519 -->
00:43:05.559Um, Lew, I agree that even further, uh besides just the free models, there are those explore custom GPTs that users contribute or companies contribute.
00:43:05.559 -->
00:43:07.719So like the Wolfram one, I think, is amazing.
00:43:07.719 -->
00:43:12.439The Wolfram GPT and the MATLAB one now, like, you know, the things that they can do.
00:43:12.439 -->
00:43:18.039And so I just want to come back to Lew's comment before how it just decimated he had to totally revamp his class.
00:43:18.039 -->
00:43:19.159I'm at that position.
00:43:19.159 -->
00:43:23.239I haven't taught a class, a graduate class that I haven't done in a couple years.
00:43:23.239 -->
00:43:32.679And now with these tools, especially with the MATLAB plus GPT, I feel like I need to totally redo my graduate class and start totally over.
00:43:32.679 -->
00:43:34.759And but Lew's right, there's opportunity, right?
00:43:34.759 -->
00:43:36.039That's that's something new.
00:43:36.039 -->
00:43:37.079I've been teaching a long time.
00:43:37.079 -->
00:43:38.359This is now fun and interesting.
00:43:38.359 -->
00:43:39.239What am I gonna do?
00:43:39.239 -->
00:43:41.399How and I come back to some of Chloe's things.
00:43:41.399 -->
00:43:43.399What is it we're trying to get students to do?
00:43:43.399 -->
00:43:47.719And I keep hearing over and over in hers is like verification and validation.
00:43:47.719 -->
00:43:49.319And that was prior to AI, right?
00:43:49.319 -->
00:43:57.799Because we talked about there are other tech tools, or there is the authoritative expert that, and so those skills become all the more heightened.
00:43:57.799 -->
00:44:02.359Like we really have to get students good at doing those, verifying and validating.
00:44:02.359 -->
00:44:03.959Because the tools aren't going anywhere.
00:44:03.959 -->
00:44:08.919I do think they're just going to become more and more ingrained, and it's hard to not use them.
00:44:08.919 -->
00:44:14.199Like you try you can't even do a regular search without a Google search anymore without getting AI response.
00:44:15.079 -->
00:44:19.559A couple of years ago on the podcast I had on Brett Becker, who's a computer science educator.
00:44:19.559 -->
00:44:24.679Um, and the the AI tools were pretty good at coding before they got pretty good at math.
00:44:24.679 -->
00:44:27.239Um, and so CS has had to deal with this for a while.
00:44:27.239 -->
00:44:29.480And and he argued that hey, it may be that.
00:44:29.480 -->
00:44:38.199Our first year computer science curriculum is more about um verifying that programs do what we want them to do than actually coding them ourselves.
00:44:38.199 -->
00:44:41.879And and he said, Look, this is all this is a useful skill already, right?
00:44:41.879 -->
00:44:47.000Like if you're a if you're a software engineer, you've got to read other people's code and make sure it does what you want it to do.
00:44:47.000 -->
00:44:54.119And so we just may get to that kind of uh earlier in the process um uh of CS education.
00:44:54.759 -->
00:44:58.679Um Yeah, and I want to say, if I could, can I just piggyback on that really quickly?
00:44:58.679 -->
00:45:07.000The College of Charleston has a newer major called Computing in the Arts, where they're focused on like the front-end design of things.
00:45:07.000 -->
00:45:16.839And these students used to stumble quite a bit over having to design the back end before they could do all the front-end art stuff that they were doing.
00:45:16.839 -->
00:45:27.000And I had a student in my business class who she and I were talking, and she said, AI has absolutely changed the way I'm able to do what I want to do with my major.
00:45:27.000 -->
00:45:33.079She's like, I'm no longer up all night trying to figure out how to code the back end of these things.
00:45:33.079 -->
00:45:36.119And I get to focus on the artistic piece, right?
00:45:36.119 -->
00:45:38.119On the front end of all of this.
00:45:38.119 -->
00:45:41.559That's what I want to be doing with my major, right?
00:45:41.559 -->
00:45:46.359And I just think that's a great example of um of what it can do, right?
00:45:46.359 -->
00:45:53.879What it can, what cognitive demand it can free up for our students who are in these artistic um type majors.
00:45:54.039 -->
00:45:56.199So Lew, I want to go back to you.
00:45:56.199 -->
00:46:08.039Um, you mentioned using, speaking of getting a little AI-powered assist in your professional work, you mentioned using AI to help you implement standards-based grading.
00:46:08.039 -->
00:46:15.559Um, I know you talk a lot about ways we've we focus mostly on student use of AI, and I think that's a great, a great place to be.
00:46:15.559 -->
00:46:20.199But but I know you talk a lot about how instructors can use AI in their work.
00:46:20.199 -->
00:46:30.279Are there are there a few kind of strategies or ideas that you find particularly useful to share with math educators who are maybe interested in using some AI in their their course prep?
00:46:31.079 -->
00:46:31.799Oh my God.
00:46:31.799 -->
00:46:33.319So the course prep.
00:46:33.319 -->
00:46:36.519So, you know, I'm looking at Amy when she's going to do that class.
00:46:36.519 -->
00:46:43.639She's going to have an AI graduate student, right, Amy, over your shoulder, like talking back and forth with this and pushing back and forth.
00:46:43.639 -->
00:46:50.679The thing that our math colleagues have to realize is that they've done a lot of hard work over the years and they become experts in their field, right?
00:46:50.679 -->
00:46:53.799They're experts about their content, their discipline.
00:46:53.799 -->
00:46:57.079They know their students very well and they know the context as well.
00:46:57.079 -->
00:47:01.480They can then take that and leverage that while using AI, right?
00:47:01.639 -->
00:47:03.639So I want to redesign my course.
00:47:03.639 -->
00:47:06.279So, okay, um, I'm teaching Calc 2, right?
00:47:06.279 -->
00:47:13.000And um okay, I know that students are gonna mess up some of the sequence tests and they're gonna, you know, what are they gonna do?
00:47:13.000 -->
00:47:15.159Uh they're gonna or sorry, the convergence test, right?
00:47:15.159 -->
00:47:19.719So they're gonna like um they're gonna confuse two different tests going on.
00:47:19.719 -->
00:47:29.399So I can ask AI, say, hey, give me a series where students will be tempted, maybe you like to use the ratio test, but the comparison test would actually be the better move, right?
00:47:29.399 -->
00:47:33.639So as an expert, you know your students will struggle with that and you can ask that question.
00:47:33.639 -->
00:47:39.959Then you can follow up with and also give me two follow-up uh questions that will actually build distinction for my students.
00:47:39.959 -->
00:47:42.279So now you're kind of channeling the AI.
00:47:42.279 -->
00:47:43.959I'm not having my students use it.
00:47:43.959 -->
00:47:45.159I mean that you could, right?
00:47:45.159 -->
00:47:54.199But I'm having them, you know, AI is giving me a better question that I can ask for my students because I'm tapping into that expertise that I've seen them struggle with for over the years, right?
00:47:54.359 -->
00:47:57.879Um my son is teaching AP stats as a 25-year-old.
00:47:57.879 -->
00:47:59.639It's like, holy cow, how's he doing that?
00:47:59.639 -->
00:48:03.239Because he leans into the AI and he says, hey, explain this to me.
00:48:03.239 -->
00:48:06.199You know, I know they're gonna, there's linear regression, they're not gonna understand this.
00:48:06.199 -->
00:48:08.599Give me three different ways to explain that, right?
00:48:08.599 -->
00:48:14.199And then he can go through and filter out which one works best for him and then help to explain that to the to his students.
00:48:14.199 -->
00:48:20.359So I think, you know, math colleagues, we have to understand that we are a bit in the driver's seat with AI.
00:48:20.359 -->
00:48:26.439If we ask it to do something, if it gives us the wrong answer, I mean, if a student gave you a wrong answer, you wouldn't say, Oh, stupid student.
00:48:26.439 -->
00:48:29.079You say, Oh, let me ask that in a slightly different way.
00:48:29.079 -->
00:48:34.679So use your power as a mathematician, use your power as an educator to kind of rechannel that energy.
00:48:34.679 -->
00:48:37.399And when you're working with the AI, treat it like a student.
00:48:37.399 -->
00:48:39.079If it didn't give you the right answer, oh, you know what?
00:48:39.079 -->
00:48:40.519Let me ask that in a slightly different way.
00:48:40.519 -->
00:48:42.599Let me kind of move that in a different direction.
00:48:42.599 -->
00:48:48.279Um, and man, you can just, as an expert, you can just make this AI stuff sing for you, right?
00:48:48.279 -->
00:48:49.799It does some really, really strong stuff.
00:48:49.959 -->
00:48:54.199It like I said, it's it's re it's revamped my whole course, or I should say, let me rephrase that.
00:48:54.199 -->
00:48:58.919I've revamped my whole course with it helping me, you know, kind of provide the structure of what's going on.
00:48:58.919 -->
00:49:01.719But it's just been a blast doing that.
00:49:01.719 -->
00:49:08.439Uh now again, I I, you know, I like I said, I don't know if it's chickened out as the right word, but I I have pulled things back in the classroom.
00:49:08.439 -->
00:49:12.599I'm not doing as much um deliberate homework, but I have these notes, right?
00:49:12.599 -->
00:49:14.759I create videos and I do a little stuff with things.
00:49:14.759 -->
00:49:19.239I have 61 students, my video notes, I'm getting like 80 hits.
00:49:19.239 -->
00:49:19.480Okay.
00:49:19.480 -->
00:49:28.439So either I have some psychotic student who's watching it 50 times, which I don't think is the case, but they're actually learning, they're looking at the notes because they know they will be tested on this material.
00:49:28.439 -->
00:49:33.719And again, getting back to that standards-based grading, they're trying to focus on the stuff that they missed.
00:49:33.719 -->
00:49:40.759So if they didn't get it the first time, that's why I'm getting more hits on a view than you know, just the minimal of 61 or something like that.
00:49:40.759 -->
00:49:42.199It's really amazing.
00:49:42.199 -->
00:49:43.959And I'm not even, I'm not giving any grades for this.
00:49:43.959 -->
00:49:44.919I'm not assigning it, right?
00:49:44.919 -->
00:49:46.439They're actually doing the work for once.
00:49:46.439 -->
00:49:47.559I'm I'm stunned.
00:49:47.719 -->
00:49:54.279So I um I keep referencing to old podcast episodes, but it's one reason I do this is to see these connections.
00:49:54.279 -->
00:50:01.480I had uh Sharona Krinsky and Robert Bosley on a couple of episodes ago who who run the grading conference and the Center for Grading Reform.
00:50:01.480 -->
00:50:08.679Um, and they said uh alternative grading practices just keep coming up, right?
00:50:08.679 -->
00:50:15.799We didn't come here today to talk about alternative grading practices, but like like it's one very sensible response to the advent of AI.
00:50:15.799 -->
00:50:20.439And so I hear it a lot in the guests that I have on my podcast, even if that's not the ostensible topic.
00:50:20.439 -->
00:50:23.239And so a little more proof of their their theory today.
00:50:23.399 -->
00:50:29.399Um, we are almost out of time, but I want to do um, let's do a round robin with one last question.
00:50:29.399 -->
00:50:36.839What do you wish more of your math colleagues knew about generative AI's impact on teaching and learning?
00:50:37.319 -->
00:50:38.119I will start.
00:50:38.119 -->
00:50:48.359Um I uh part of what I've been working on has been uh a research project with two of my undergraduate students, Ellah Olson and Madison Schwartz.
00:50:48.359 -->
00:50:50.679They're both uh mathematics education majors.
00:50:50.679 -->
00:51:03.480Um and we have been um testing how uh students respond differently to proofs when we tell them this has been generated by AAI versus when there's no mention of the author.
00:51:03.480 -->
00:51:05.639Like, are you more or less likely to believe that?
00:51:05.639 -->
00:51:14.439Um, so we've been working on this research project for the last couple of years, and um, as part of that, we just ended up prompting the AI to write and prove a lot of different mathematical theorems.
00:51:14.439 -->
00:51:20.359So um in doing so, we just were like, well, we kind of know a lot about what AI does with math.
00:51:20.359 -->
00:51:24.839And so we've we're talking at our um department meeting, the three of us.
00:51:24.839 -->
00:51:32.919Um, and I think the revelation of like you know, like it can do your entire take-home exam, like just full stop.
00:51:32.919 -->
00:51:35.399I I do think that was surprising to some people.
00:51:35.399 -->
00:51:44.759And so um for people who have like maybe really never tried those things, like I think they'd be shocked by like I said, it can it can do it all.
00:51:44.759 -->
00:51:47.319Well, it could give you an answer for everything, right?
00:51:47.719 -->
00:51:57.639So, and especially if you if you last checked in in 2023 or 2024, when when the tools were were often really bad at math, um, that has changed.
00:51:57.639 -->
00:51:58.039Yeah.
00:51:58.039 -->
00:51:59.079Kathryn?
00:52:00.439 -->
00:52:07.480Um, so I would like to stop having people ask me the question, how do I keep my students from cheating with AI?
00:52:07.480 -->
00:52:10.359That is the constant question people are asking.
00:52:10.359 -->
00:52:11.799How do I keep my students from cheating?
00:52:11.799 -->
00:52:13.239How do I keep them from cheating?
00:52:13.239 -->
00:52:26.119I wish we had so many more people in our department who thought like Lew thought, where it's like, this is an opportunity for me to completely revamp the way that I'm teaching mathematics, right?
00:52:26.119 -->
00:52:28.279And I have that opportunity, right?
00:52:28.279 -->
00:52:31.559Um so that's I don't I don't know.
00:52:31.559 -->
00:52:32.679That's that's what I'll say.
00:52:32.679 -->
00:52:45.000And I don't know if Amy's gonna bring up from Solve Khan's book, but we we like, we have really linked on to this idea of um now that AI is here, you can expect more of your students.
00:52:45.000 -->
00:52:51.000You can ask them to do more harder things because they have these tools.
00:52:51.000 -->
00:52:54.119And we're really taking advantage of that in our classes.
00:52:54.439 -->
00:52:55.959Amy, what would you add to that?
00:52:56.279 -->
00:53:00.599Um I've heard some people say, well, AI only knows what's already known.
00:53:00.599 -->
00:53:07.079And so what the human can, but I I don't know that that's necessarily I think it has elements of incremental creativity.
00:53:07.079 -->
00:53:12.679I don't think it'll ever do paradigm shifting creativity, but but so then that leaves like what is left for the human.
00:53:12.679 -->
00:53:22.359And uh if you have this encyclopedic resource, and Kathryn was saying, yeah, we can now ask grander assignments, they can expect more because you have leveraged this amazing assistant.
00:53:22.359 -->
00:53:26.519Um but I also come back to okay, well then we need to foster creativity.
00:53:26.519 -->
00:53:27.559And what exactly?
00:53:27.559 -->
00:53:29.639That's such a vague word, and what does that really mean?
00:53:29.639 -->
00:53:31.000And what does that mean in mathematics?
00:53:31.000 -->
00:53:37.000And so like these broader discussions about what other things could we focus on?
00:53:37.000 -->
00:53:46.759Um and I come back to Conrad Wolfram's four steps of computer-based math, and at one point he was like, oh, in step three, we're focusing on the computation step, and that's the thing that computers are good at.
00:53:46.759 -->
00:53:53.480Well, now it's like computers are actually getting better at even the first step, and sometimes the second, and certainly the fourth.
00:53:53.480 -->
00:53:56.759So so what is, you know, again, what is left then?
00:53:56.759 -->
00:53:59.480What what we need to rethink what we're doing.
00:53:59.799 -->
00:54:01.559I don't know if that's exciting or scary.
00:54:01.559 -->
00:54:04.039Um of both.
00:54:04.039 -->
00:54:08.599Lew, what do you wish more of our math colleagues knew about AI and teaching and learning?
00:54:08.839 -->
00:54:13.639Uh I'm gonna piggyback a little bit on what Kat or what Amy said about, you know, kind of getting involved with it.
00:54:13.639 -->
00:54:16.759Um, I I had a thing earlier today, and I was kind of thinking about this.
00:54:16.759 -->
00:54:23.719Uh for those of us who've had young children, right, and you try to get them to uh eat something new like broccoli, right?
00:54:23.719 -->
00:54:29.399You can't, you know, I'm not gonna have my kid decide whether they're gonna eat broccoli by listening to some other kid.
00:54:29.399 -->
00:54:31.000They have to try it themselves, right?
00:54:31.000 -->
00:54:34.279But moreover, it's one and done is not the deal, right?
00:54:34.279 -->
00:54:42.279In order to try something new and kind of get used to the flavors and all that kind of stuff, unless you have that weird gene for broccoli, uh, you know, it takes like five or six times.
00:54:42.279 -->
00:54:53.000So, you know, I think for for the math colleagues who have tried broccoli a long time ago, or maybe just listen to somebody else complain about broccoli, yeah, it's probably time to pony up and try the broccoli again.
00:54:53.000 -->
00:55:02.439Uh again, I would encourage, you know, if you do have the means to do it, uh, to poke around with one of the paid models or the thinking models because it will really surprise you.
00:55:02.599 -->
00:55:08.599But again, kind of spinning it from, oh, what are our students, you know, with Kathryn saying about our students cheating about how could I leverage this?
00:55:08.599 -->
00:55:13.079How could I make myself into, you know, there we've had a lot of issues with with education.
00:55:13.079 -->
00:55:17.319It wasn't like, you know, things were running along fine and all of a sudden AI hit and then things were bad, right?
00:55:17.319 -->
00:55:18.919There was a lot of stuff going on.
00:55:18.919 -->
00:55:21.480And I think AI just has kind of amplified that.
00:55:21.480 -->
00:55:39.399But I do think we can kind of use, we can kind of channel this negative thing that it does do on the on for ourselves and kind of actually turn it into something positive to help ourselves and help our students better with their learning if we you know use it appropriately and and kind of you know do all the stuff that we already knew was good.
00:55:39.399 -->
00:55:44.759We now actually have to pay attention to those, uh, you know, how students learn and cognitive science and all that stuff.
00:55:45.000 -->
00:55:45.480Yeah.
00:55:45.480 -->
00:55:48.119Well, um, that's a good place to leave it.
00:55:48.119 -->
00:55:52.039I know there's lots more in this topic we could explore together, but um our time is nigh.
00:55:52.039 -->
00:55:54.039Thank you so much for being here and being part of this panel.
00:55:54.039 -->
00:55:55.239I really appreciate it.
00:55:55.239 -->
00:55:57.480Um yeah, thanks so much.
00:55:57.799 -->
00:55:58.679Thank you.
00:55:59.159 -->
00:55:59.799Thank you all.
00:55:59.799 -->
00:56:00.519Nice to meet you guys.
00:56:00.839 -->
00:56:01.239Thank you.
00:56:03.959 -->
00:56:13.319Thank you to Amy Langville, Chloe Lewis, Lew Ludwig, and Kathryn Pedings Behling for coming on the show and sharing their current AI-aware teaching practices.
00:56:13.319 -->
00:56:30.439In the show notes, you'll find links to Lew's new book, The Science of Learning Meets AI, and to Amy and Kathryn's Deconstruct Calculus Project, and to Chloe's website, and to all the books, articles, and podcast episodes we mentioned during our conversation.
00:56:30.439 -->
00:56:36.599And if you're a math educator, I highly recommend joining the Teaching Mathematics with Generative AI Google Group.
00:56:36.599 -->
00:56:41.159That's where some of the best discussion of AI in math education is happening right now.
00:56:41.159 -->
00:56:43.879See the show notes for a link to that group too.
00:56:44.279 -->
00:56:49.719Intentional Teaching is sponsored by UPCEA, the Online and Professional Education Association.
00:56:49.719 -->
00:56:58.279In the show notes, you'll find a link to the UPCEA website where you can find out about their research, networking opportunities, and professional development offerings.
00:56:58.279 -->
00:57:04.119This episode of Intentional Teaching was produced and edited by me, Derek Bruff.
00:57:04.119 -->
00:57:12.279See the show notes for links to my website and socials and to the Intentional Teaching newsletter, which goes out most weeks on Thursday or Friday.
00:57:12.279 -->
00:57:18.359If you found this or any episode of Intentional Teaching useful, would you consider sharing it with a colleague?
00:57:18.359 -->
00:57:19.799That would mean a lot.
00:57:19.799 -->
00:57:22.279As always, thanks for listening.
