WEBVTT
1
00:00:00.000 --> 00:00:08.000
Foreign.
2
00:00:08.000 --> 00:00:16.000
Welcome to Examining, a technology focused podcast that dives deep. I'm Eric Christiansen.
3
00:00:16.000 --> 00:00:22.000
And I'm Chris Hans.
4
00:00:22.000 --> 00:00:36.000
And welcome to another episode of the Examining Podcast, the technology focused podcast that dives deep. We have a very special episode today. We are interviewing Saroosh Sabagan from the University of Calgary. Welcome, Siroosh.
5
00:00:36.000 --> 00:00:38.000
Thanks so much for having me on the show.
6
00:00:38.000 --> 00:01:01.000
Thank you for taking the time. I mean, especially on a weekend, we appreciate that people are taking their free time to, to chat with us. So just so our listeners know, I mean, we're kind of continuing the AI discussion here. So Saroosh, you have, you're from the. Well, we're hoping maybe you could tell us a bit about yourself first. And my understanding is that you're at the University of Calgary's Workland School of Education, is that correct?
7
00:01:01.000 --> 00:02:41.000
That is correct. I'm an associate professor at the Workland School of Education. I'm also the educational leader in residence for generative AI in the Taylor Institute for Teaching and Learning. If I would characterize my. If I had to characterize my work, I would say my work focuses on the intersection of generative AI and education, teacher training and responsible educational technology. I at the moment run several projects which are focused on supportive teaching, learning assessment, academic integrity, inclusive course design, human agency, professional judgment, and ethical responsibility. I have been coding for years now, so developed multiple different tools that use generative AI. And the goal here is again to support educators, co design, support early researchers with scholarly writing, lesson planning, program evaluation, that kind of thing. I do a lot of public scholarship. I work with public institutions, do a lot of advocacy around this technology, helping them to better understand how to use it responsibly.
8
00:02:41.000 --> 00:02:49.000
Interesting that you use the term public scholarship. I like that. Are you talking about more about knowledge mobilization?
9
00:02:49.000 --> 00:03:12.000
Yes. So work in schools, work with teachers, libraries, and I call these public institutions trying to get people to understand how the technology works, what kind of tools are out there, how do you approach the different tools? What's probably going to happen? What are the paths available to us? That kind of thing.
10
00:03:12.000 --> 00:03:19.000
Interesting. Very interesting. Well, that's a good segue. I mean, Chris can probably take the next one, but I think that's a good segue into our next question, actually.
11
00:03:19.000 --> 00:03:50.000
Yeah, no, I think. And so I had privilege of going and speaking with Swoosh on a panel that we had. And so the, the Taylor Institute hosts a conference on learning and teaching. And so there was four of us that were talking about digital transformation. But one of the things that I think we can maybe just. And you're already starting to. But, you know, how are you using AI in the classroom?
12
00:03:50.000 --> 00:04:10.000
Well, I can say that I had. I'll start like this. There were taboos for me initially when I started doing this work. Lines that I drew for myself that I would never cross. And I crossed every single one.
13
00:04:10.000 --> 00:04:11.000
I love it.
14
00:04:11.000 --> 00:09:22.000
So let's start from there. I promised myself that I would never use this technology to create material for my students to read for obvious reasons. It hallucinates, it's not accurate, it overstates, it doesn't understand that you can't take a strong position from a literature review paper or a position paper. It needs to be empirical, it needs to be large, that kind of thing. But last year I taught a course on the foundations of generative AI in education. And there was simply not enough literature out there for students to read. So I designed an application that would go and read content from Google Scholar and try to the best of my ability, produce content that's accurate with references and all of that. That's what I used. I gave students the option of reading relatively related work to the topic of discussion or directly related work to the topic of discussion, but generated by AI. And students were given that choice to choose which path they want to take. And it was 50 50. Some people read the content produced by Generative AI and others chose to use human written work. I promised myself that I would never use AI for assessment, ever. In that one particular course. Was a two week course, every day, an activity for students to do every day. There were 25 students. So that's 250 activities that I needed to grade over the span of two weeks. It would be impossible. So I created an application where an AI would. I would share my screen. The AI will see what's written on the screen. And then I would ask the AI to identify for me things that were important to me in that paper. So I would say, does the paper contain the following or so? And so then the AI would go and read the paper and highlight for me, yes, it does. Like, here it is. This conversation between myself and the AI was transcribed and I used that transcription to create feedback. And I actually modeled this in class. I said, this is how I'm going to grade your papers. And it would just. It's impossible to do all of this. And I did it live with a paper. And I said, we're going to do one, so you can see. And it was basically a conversation between me and an AI, both of us looking at the same thing. And then it generated feedback. And the feedback looked like exactly. I read the feedback and put it into the feedback box on into D2L, and I would send it that way. So spend about five minutes on a paper where normally I would spend maybe 20 minutes. And then this year I'm going to teach the same course, but I'm going to do it differently. So technology has come. A year in AI is like a decade everywhere else. Instead of assigning greetings, I am going to assign questions. I'm going to assign topics and questions. Three questions, three topics. I'm going to use the same technology I used before to create content. I'm going to give it to students and say, you create your own content now. You read it and come to class and we'll discuss that. And there's different paths that you can take. It's a lot more accurate. Now, the machine does understand the difference between empirical work and the literature review and a position papers, and understands not to overstate, understands not to put too much emphasis on content that comes from a position paper, for example, or actually say that this content comes from an empirical study, but the empirical study wasn't that big. Take it with a grain of salt, that kind of thing. These are some of the ways I've been thinking about, oh, I've been using and also thinking about how to improve generative AI in the classroom responsibly. I'm going to emphasize that. I'll stop there.
15
00:09:22.000 --> 00:11:28.000
It's interesting that you modeled it for your students and, and you're kind of using the AI I, I've got as kind of a. Not a Socratic opponent, but kind of a. As a teammate. One of the things that I've, I've talked to students about is the need to assign AI a very specific role because then it kind of keeps boundaries around it. And you can, of course, you can prompt it to do that. Chris and I have talked about this before. I probably brought it up on this podcast. But it reminds me of that old Apple concept video of the Knowledge Navigator where you're going back and forth between this person. I don't know if you've seen that. What you're describing also kind of reminds me of some work that's done in the technical writing space. So there's a technical writer at Google named Tom Johnson, and he talks about, you know, he does like technical writing, so release notes for updated APIs and stuff like that. So he's kind of a coder, but also a writer, which is an interesting space to live in, but he talked about how it's kind of a lot of professions, writing, teaching, have kind of shifted from starting with a blank page to maybe starting using judgment from this options generator, which is this AI. And then you're kind of having this back and forth all day rather than kind of struggling from scratch. And it's interesting to me, though, because one of the things he discusses in his profession, which is quite different, is that, you know, he does talks, he's written about it, but there's this really weird tension there, there's, there's, there's this palpable tension probably in a classroom, in a lecture hall, when people talk about AI and how you can use it in creative ways. And I'm wondering if you could speak a little bit to. Even though you've modeled it for your students, just like this person I'm discussing has kind of explained his rationale for using it in his profession. Is there, do you feel that uneasiness in the classroom? And why do you think that? If so, why do you think that is? What do you think people are nostalgic that they'll miss based on this new kind of computer interactive paradigm?
16
00:11:28.000 --> 00:14:18.000
It just might be public perception, but there seems to be the stigma that if you use this technology, you're lazy and that you have to do the work, so to speak, and that also the stigma that if you use this technology, you must be cognitively offloading. There is no other way to use this responsibility to use this technology in a way that enhances your cognitive thinking, that most people don't go into that space. Because when ChatGPT came out, people's first reaction to it was, I'm going to give ChatGPT to do my homework, which was cognitive offloading. And that kind of, I think, just stuck. And people are in this space of, oh, this is a tool for lazy people. This is a tool where you cognitively offload. If you use this tool, you really are not going to learn anything. And one of the areas that I work on is to show people that, no, you can actually use the tool to enhance your cognitive abilities, like the Socratic of back and forth. One of the other ways I use it in research is. I create a platform where I input in my own work and then input somebody else's work. And I say, how is this new work or new idea complement my work? And how does it not? It comes up with very, very interesting ideas that I would have missed. Like if I had read, I already know my own content, but reading this new paper, I wouldn't have seen it. So there are spaces to explore. But unfortunately the stigma is this is a tool where if you use it, you're definitely offloading cognitive labor onto it. And I think that another reason why that's the case is because we haven't taught people there's no literacy involved in how to use it properly. We just released it into the world and people just took it the way they wanted to. There was no building foundations of how to use this tool first. And I wish OpenAI had done that work before releasing ChatGPT, but that's not what happened.
17
00:14:18.000 --> 00:15:10.000
I don't want to hog because I'm sure Chris wants to ask a follow up, but because you mentioned cognitive offloading. So the paper that's on arxiv.org your brain on chatgpt has gone viral several times as I can tell. Now it seems like a very well done paper from a brain scan kind of perspective, but it's one study and so I'm always holding my tongue until I see a meta analysis or something like that. Perhaps that's the librarian me, I. I need to see multiple. But what, what is your opinion of the cognitive offloading? Do you think? Like what would. And if, if you. I guess my question is it, do you. Are there tenants of that paper that you agree with? And if so, where do you think that threshold is? Is it somewhere in between what you're doing and let it do everything for me, like where. Where would students start to cross the line?
18
00:15:10.000 --> 00:19:25.000
I. So if you go back to the fundamentals of learning and I'll frame it like this for years and years, maybe hundreds of years, we believe that if a student is able to produce an artifact that is well put together, which is fluent and correct, some sort of cognitive labor or intellectual labor must have taken place for that student to be able to produce that. So for us, the product and the process were the same. We evaluated it the same. If the product was good, there must have been some sort of intellectual labor. And of course you could cheat. You could have somebody else write it, you could have all that. But it was difficult. It was time consuming and expensive and out of reach for most people. When generative AI came around, you could produce that same perfect product without any learning at all. Completely bypasses this tradition that we've had for hundreds of years. That's where the panic comes in. This is the tool for cognitive offloading. Whereas I think that our approach to this is wrong. It has always been about the process. It has never been about the product. We've Equated the two in our education system because classes are large, because there's only one teacher, and it was just the most efficient way of doing it. But now we have to go back and rethink about what exactly is it that we're assessing. We've always assessed the process. So if you're using generative AI, you have to do two things. One, you have to focus, help people understand that a correct answer or a relatively correct answer is not the same as a complete answer. Those are two very different things. And in the learning process, we want a complete answer. We want you in it. Your voices, your senses, your experience. We want to see how you grew. That's a complete picture. We just don't want to see correctness all the time. And being complete is having an understanding of what a machine can do and what a machine can't do. So a machine can do literature review for you and give you this very neutral, relatively clean output, but there's no humanity. There's none of you in it. Right. If I ask a student to go and look at. Transit laws in the city of Calgary, for example, that's the assignment. You could just plug in a website and you could get. But if I ask the student, what does this paper miss, about your experience riding a bus, about your experience riding the C train, about the fears that you have, about the nervousness that you have, about how long you're going to use it, about all of that piece which makes the paper complete, the machine would have missed. The machine can't experience those things. And I think that we need to help people understand that. Where, yes, it's important to be correct, but it's more important to be complete. And it is the human that is able to provide a complete answer. The machine is just simply incapable.
19
00:19:25.000 --> 00:19:55.000
That's a terrific explanation. It reminds me of Daniel Pink. He gave a commencement address at, I think, Columbus School of Art and Design or something like that. And he said that the role of the futures, I mean, he. He used the same thing to develop your own voice, but he. He framed it in the context, in a design school of developing taste. And taste means to have taste or means you have to have a position. And to have a position, you have to have experiences and reflect on it. Yeah, very interesting.
20
00:19:55.000 --> 00:20:53.000
That is interesting. I mean, even, you know, that example that you brought up, I mean, right now there are some debates that are coming up because there's a conversation about removing the free fare, you know, downtown anyways, and some of the reasons that people are bringing it up is because of safety and other things. But I don't know. I mean, I, I think it's safe and it's kind of nice to be able to commute around and go to meetings. But, you know, if you, it's like you say, like the, the whole complete picture would be actually going in and experiencing it. What it's like, you know, one of the things that, you know, you mentioned in terms of, you know, there you touched on it with the like, AI literacy. Now, do you think that we should mandate AI literacy or AI competence or AI use, or are those different things?
21
00:20:53.000 --> 00:25:13.000
Well, I'll, I'll frame it this way. I know, I don't know if you're familiar with pisa, which is an international test that assesses reading numeracy literacy internationally. Right. So many countries participate. I know that in 2029, they're going to start bringing in a new form of literacy called AI literacy. And they're targeting, if I'm not mistaken, 2035 as their first instantiation of this exam. They're targeting basically people born during the pandemic. They want to know if people born in a pandemic have achieved some sort at some level of AI literacy. If you're born in the year 2020, you'll be 15 in 2035. Right. And Pisa targets that, that population group. So it's a global thing that AI literacy is important to have at a global level. That's why they're integrated into, into this exam. So they're saying numeracy, having ability to read and write is probably the same as having an understanding of how AI works and how to use them, so on and so forth. Globally, it's understood that this is a very important thing. That means that we have until. Let's assume they're going to actually make the happen in 2035. Let's just assume for a moment. Let's reverse engineer. So if, if we have there, if there is a test in 2035, we need to do, have a program ready in 2030, like to have teachers ready, have, have an AI literacy program so that we can teach our students so that in five years when they take that exam, they're ready to. It's the responsibility of our schooling system. That means that right about now, in 2025, 2026, we have to have a training plan ready for our teachers to go through governance so that it will be ready by 2030 to be implemented so that our students can pick it up and be ready in 2035. And we do not have that Right now, some countries in the world do have that. We don't have that. So going back to the question of should we mandate it, I think it's a very important part of the curriculum because there's so many ways that using this technology can go wrong. And people. It's not intuitive to pick it up and use it correctly. It's actually a skill. You have to teach people how to use it correctly, which means you have to show them the difference between wrong and right. The debate right now is if we are going to do this, it's going to be one more thing that we bring into the school system, which is already taxed. How are we going to do it? Who are we going to train? What kind of content should we bring? Given that the landscape is changing almost month to month, what are we actually going to say? There's so many questions that we get bogged down and we actually can't create the program. So it's not about a question of mandating, it's about questioning, of figuring out how we're going to do it first and then decide whether this should be mandated.
22
00:25:13.000 --> 00:25:41.000
Makes sense. But, you know, on the same note, like, you know, when we were at the conference, there was several people, and this came in afterwards that, you know, there were people who actually were refusing AI use for. Because of, you know, the. The ethical, privacy, environmental, labor reasons. So do you think that universities should also design for refusal without leaving people behind?
23
00:25:41.000 --> 00:27:50.000
So my, my position on that is that all technology, all technology carry that burden. There's an environmental impact. There's a labor impact, all of it. I think that singling out one technology without having a conversation about all technology, and then having a broader conversation about the impact that human beings and their laws have on the environment, I think we have to have a much larger conversation between before going in that direction. So this zoom call, for example, has an environmental impact. Me going to work on campus has an environmental impact. Netflix has an environmental impact. YouTube has an environmental impact. Encryption has environmental impact. Then we can go to other sectors, like the agriculture industry has major environmental impact. The rules we have for private jets travel have major environmental impact. We need to have a conversation there at that level first. Then we can single out a particular technology. If someone comes to me in my class, if I'm using AI and I'm saying, the university has provided a free version, I want you to use it, and if they will refuse, then I will ask them to drop the course because it would be as me assigning a particular reading and a student Coming to me and say, well, I'm not going to read that. I disagree with the author. I'm the expert here. This is what I think you should do in order to achieve the learning outcome. So if you're unhappy with the curriculum, then you should probably go with a different instructor.
24
00:27:50.000 --> 00:29:15.000
Yeah, I mean, those are good. Like, you know, aspects like you talk about the environmental. I mean, the one thing that I, I mean, I, I suppose if you go back in history, you could probably apply it to anything. Like, even, you know, you talk about, like, in this case, I don't know if in human history we've ever had this kind of ask for forgiveness. And instead of permission ever, or, you know, you're taking people's intellectual property to go and build the large language models, the labor that's going into it. You know, there's people getting paid $5 an hour to go and siphon through and make sure that there's only appropriate content that's going into the model and so on. Although now all of us that are using the free versions, I mean, we're basically getting paid nothing. And we're going and making the, the, the algorithms better. So. But yeah, I mean, it's, yeah, it's quite like that was one thing, I'll tell you personally, Surge. Like when. I never even thought about it when I was, Until I started reaching out to people, but I was looking at, okay, I'm gonna go. At first I was thinking of mandating it on every student. And then somebody asked me the question, you know, how about if somebody didn't want to use it for these reasons? And I'm like, okay, well, that's, that's a good point. And so I made it optional. You know, I still want them to go and experiment, but it's, you know, it's their choice if they want to use it or not.
25
00:29:15.000 --> 00:30:40.000
I used to do that about a year ago. I always had a plan B for students who didn't want to use it. And it was, it was because. Mainly because they had to pay for it. And I thought that would be, that was unfair. But once the university provided access to LLM to all students for free, then that changed the equation. Then it became a tool. I want you, you have to be prepared to go into society and use this tool responsibly. And I deeply believe that using it responsibly is a skill. I would explain all this not in my class, but outside in academia. And knowing that they are refusing to learn a particular skill to use this technology, responsibly knowing that it's going to be ubiquitous, it's going to be everywhere, and then once you're completely informed again, you decide not to use it. I respect your decision, but I would want you to know all the angles and the consequences of not knowing before you make that decision.
26
00:30:40.000 --> 00:30:43.000
Like not judging a book by its cover.
27
00:30:43.000 --> 00:30:47.000
Yes, that would be. Yep, good metaphor.
28
00:30:47.000 --> 00:31:23.000
I have a. I mean we'll probably do when we do rapid fire. I don't want to go too far off, but I know that you talked about institutionally provided tools, so that kind of speaks to something we were going to talk about regarding around equity and access. But just, you know, as from a personal standpoint, I guess the question is are you using, I guess primarily Copilot or the Microsoft side in the classroom, but also of the front of the frontier models available, do you have a preference or why or why not?
29
00:31:23.000 --> 00:33:53.000
I don't have a preference because each model is good at doing something better than others. So if I code, I use Claude. If I am trying to go through large pieces of text, I use Google. And if I want analytics, I use the OpenAI models. In the frontier models, if I'm not in a hurry, I would use Deep Seq API because it's as good. If I'm in a hurry and if I want to pay a little bit more, I'll use the OpenAI models. So it really depends on what it is that I want. Usually when I create AI programs, I use multiple models depending on what their task is, how much compute they use. I don't use the frontier models for every single API call. That would be insane. It depends on what is it that we want to do to try to keep speed, balance, cost, all of that. In terms of the equity and access pieces, I want to highlight one thing I think particularly for public institutions like the library, so they talk about access. They said we bring broad access to the Internet, broadband to the library so that people who need it could go to the library and access the Internet for free. And that's super important. I remember that I went to the library to get access to fast Internet because I didn't have it. But with this technology, it's not about access, it's about competency. Access is the first part. But we already know what happens when you give access to somebody. They don't know how to use it properly, so they cognitively offload. Access is the first step, the competency to use it. Like for example, find me. Go into Indeed.com and find me all the positions, all the Jobs that fit with the following description and then write my resume to fit that description. That will be command.
30
00:33:53.000 --> 00:33:54.000
Right?
31
00:33:54.000 --> 00:34:29.000
That's not cognitive offloading. That's using the tool for a very specific purpose so that you can apply for 40 jobs in five minutes. That's the competency that I'm hoping that people will get. It's about learning to command the machine. And I think that we're missing that. A lot of people are just focusing on access, but not enough on the competency, the actual skills that are required to use the tool properly.
32
00:34:29.000 --> 00:35:02.000
Kind of what Ethan Mollock says. He kind of says that these tools are. Some are better at others and some things. And I agree with you on cloud code though, I've been using Codex, the latest version, and it's a lot closer. So they've, it's more neck and neck now. But if somebody is going to commit to paying for these tools is their best option still, do you think OpenAI, if they want to develop competence, especially comparing outputs between models, because then that does have an extra cost associated.
33
00:35:02.000 --> 00:36:06.000
No, I still think that some of the open source models, models, even faster models or smaller models, they have enough capability to do something very specific. And that's what I mean. Having competency, knowing which model is good for which task so that you're using it responsibly. You're not putting maximum compute on something as simple as grammatical errors. That's a huge part of knowing how to use this technology responsibly. I think that people need to have this awareness that not all models are the same. They use different amounts of compute and water and resources. Having the understanding of what to use for what or which model to use for which purpose. That's a very important form of literacy.
34
00:36:06.000 --> 00:37:19.000
One thing that I was chatting with some students just this week about this, but you talk about knowing how to command the tool and working with these tools. But you know, if you Recall Back when 23, there was like this whole term of like prompt engineering and there was even people getting paid like $300,000 plus. But I would actually say at this point like prompt engineering is dead. And you know, I, I actually posed this to the students. I'm like, you know, how do you think is the best way to figure out a prompt? And so I'm like, what, what do you think about if. And I'm actually going to write this up, but asking the tool itself for the prompt and you know, you what it comes up with. I mean, who would know better than the tool itself to actually go and provide the instructions of how to get whatever you're looking at. And so I think it really comes down to, you know, I, I'm calling it more of like, you have to prepare, like a briefing, like tell, tell it the job, tell the, you know, the audience, give the source material, give the constraints, and, and that's what we're kind of looking for.
35
00:37:19.000 --> 00:37:23.000
I think one of the earliest tools that I built was a prompt engineer.
36
00:37:23.000 --> 00:37:25.000
Oh, okay.
37
00:37:25.000 --> 00:38:33.000
I gave it the tenants of what a good prompt should look like, and then I gave it some information and then it popped out the prompt that I would then use somewhere else. Yeah, I, I, I think, I'm not sure if prompt engineering is dead. The process, maybe, but the literacy of looking at different prompts and, and being able to say, okay, this one's a good prompt, but this one isn't. This one has too much context, this one has, is not very clear or leaves a lot of space for the machine to interpret. That kind of literacy, I think, is important. So it's a different form of engineering. It's not that you are actually doing it, but you're capable of looking at a prompt and say, this is probably going to work much better in cloud code versus that one, despite it being longer, for example. So, so yeah, I think, I think there are nuances. It's not dead yet.
38
00:38:33.000 --> 00:38:40.000
No. Yeah. I mean, what you're talking about, I would kind of categorize as more about the, the judgment around it.
39
00:38:40.000 --> 00:38:47.000
Absolutely.
40
00:38:47.000 --> 00:39:29.000
We kind of touched or alluded to this earlier when we talked about kind of the uneasiness around AI. But what do you think? I mean, we, you know, we make predictions all the time on the podcast of the caveat that we don't expect to be right. So we, we have that disclaimer up front, but what do you think the economic impact will be? Good, bad, or different? We're not trying to pin people down. Are we going to lose all our jobs? Are we going to create more? It was more just. What do you think will change? And I guess related to that, given what we know about the technology as it stands now, and that it'll surely get better, what are the skills that people are going to have to navigate that won't be automated?
41
00:39:29.000 --> 00:42:07.000
I think I talked a little bit about this in that panel. I think that using this technology creates a dividend, an AI dividend, if you will. And whether whether this is going to have a positive impact or a negative impact depends on what you do with that dividend. And the studies have shown that corporate Instinct is to do more of the same. The clear example is you were writing monthly reports, now you write weekly reports or bi weekly reports. And that doesn't actually contribute much to the institution. You just get more reports. It doesn't increase your efficiency in any meaningful way. And so then your workforce, I use the term AI generators become AI generators, trying to figure out whether the AI is hallucinated here or there or this piece is accurate or not. And that doesn't contribute to efficiency. But if you are mindful and purposeful about what are you going to do with that efficiency. So if it took you eight hours to write that monthly report and now it takes you two hours and now you have six hours of free time, that's the AI dividend. How are you going to use that to actually create value for your institution? Think strategically. Think about how the human component can help actually make better products, better customer experience, so on and so forth. If you're strategic about it and actually think about which directions you want to go and give opportunities to your workforce and say, tell me, give me ideas on what are you going to do with that extra six hours, then that's going to actually improve lives. But if you go the other direction of doing more of the same, that's actually going to go the other way. It's going to have a negative experience. People are going to get burned out because they're just AI generators. They're doing more, but they're not contributing in a meaningful, valuable way.
42
00:42:07.000 --> 00:42:12.000
I'm curious, as an educator, what do you do with your extra six hours?
43
00:42:12.000 --> 00:42:39.000
I have now more opportunity to read more, whereas I didn't have that opportunity before. So now I have much more breadth which is which down the line gives me more depth and that creates value for me.
44
00:42:39.000 --> 00:42:45.000
Well, Chris, should we, do you have another follow up or do are we ready for our rapid fire? What do you think?
45
00:42:45.000 --> 00:42:47.000
We could probably do the rapid fire.
46
00:42:47.000 --> 00:42:49.000
Rapid fire. Do you want to take the lead on it?
47
00:42:49.000 --> 00:43:03.000
Sure, sure. Like I mentioned to you, Sarosh, so we have this rapid fire there, you know, pretty simple questions like either or, you know, kind of thing. So are you ready for the rapid fire?
48
00:43:03.000 --> 00:43:04.000
Absolutely, go for it.
49
00:43:04.000 --> 00:43:12.000
Okay, so what's your favorite small language model? Online or offline?
50
00:43:12.000 --> 00:43:15.000
Oh, you have to define what small means.
51
00:43:15.000 --> 00:43:27.000
Well, I guess we're talking about. Yeah, I mean the definition. I think what we originally thinking is that things that either could be open source, that you could run locally.
52
00:43:27.000 --> 00:43:31.000
I don't know if you can. Quinn 3.6
53
00:43:31.000 --> 00:43:33.000
okay.
54
00:43:33.000 --> 00:43:35.000
I have my money on Gemma. But I was wrong.
55
00:43:35.000 --> 00:43:40.000
So, favorite tea?
56
00:43:40.000 --> 00:43:43.000
Green. Earl Grey? Green.
57
00:43:43.000 --> 00:43:46.000
All right. Mac or PC?
58
00:43:46.000 --> 00:43:48.000
Mac.
59
00:43:48.000 --> 00:43:50.000
Standing or sitting? Desk.
60
00:43:50.000 --> 00:43:53.000
Sitting.
61
00:43:53.000 --> 00:43:56.000
What's your favorite car?
62
00:43:56.000 --> 00:44:09.000
Oh, that is so hard to answer. Pass. I don't know.
63
00:44:09.000 --> 00:44:10.000
Ebook or paper?
64
00:44:10.000 --> 00:44:12.000
Ebook.
65
00:44:12.000 --> 00:44:16.000
Favorite Open source tech?
66
00:44:16.000 --> 00:44:19.000
Oh, my God.
67
00:44:19.000 --> 00:44:23.000
That could be hardware or software, by the way. So
68
00:44:23.000 --> 00:44:40.000
I like all the open source tech that help design robot movements. I can give you probably a name, but that's the area that I like so much. Very much.
69
00:44:40.000 --> 00:44:44.000
Synchronous. Asynchronous or hybrid learning?
70
00:44:44.000 --> 00:44:46.000
Synchronous.
71
00:44:46.000 --> 00:44:49.000
Web browser?
72
00:44:49.000 --> 00:44:52.000
Brave
73
00:44:52.000 --> 00:44:56.000
VR, AR or both?
74
00:44:56.000 --> 00:44:58.000
Both.
75
00:44:58.000 --> 00:45:01.000
And who inspires you?
76
00:45:01.000 --> 00:45:03.000
My father.
77
00:45:03.000 --> 00:45:04.000
That's very nice.
78
00:45:04.000 --> 00:45:11.000
That's very nice. That's a. That's a perfect conclusion to this interview. Well, thank you very much. It's been a pleasure chatting with you today.
79
00:45:11.000 --> 00:45:15.000
Yeah. And thank you so much for the wonderful questions and having me on the show.
80
00:45:15.000 --> 00:45:17.000
Absolutely.
81
00:45:17.000 --> 00:45:18.000
Thank you. Take care.
82
00:45:18.000 --> 00:45:19.000
Thank you.
83
00:45:19.000 --> 00:45:34.000
Take care.
1
00:00:00.000 --> 00:00:08.000
Foreign.
2
00:00:08.000 --> 00:00:16.000
Welcome to Examining, a technology focused podcast that dives deep. I'm Eric Christiansen.
3
00:00:16.000 --> 00:00:22.000
And I'm Chris Hans.
4
00:00:22.000 --> 00:00:36.000
And welcome to another episode of the Examining Podcast, the technology focused podcast that dives deep. We have a very special episode today. We are interviewing Saroosh Sabagan from the University of Calgary. Welcome, Siroosh.
5
00:00:36.000 --> 00:00:38.000
Thanks so much for having me on the show.
6
00:00:38.000 --> 00:01:01.000
Thank you for taking the time. I mean, especially on a weekend, we appreciate that people are taking their free time to, to chat with us. So just so our listeners know, I mean, we're kind of continuing the AI discussion here. So Saroosh, you have, you're from the. Well, we're hoping maybe you could tell us a bit about yourself first. And my understanding is that you're at the University of Calgary's Workland School of Education, is that correct?
7
00:01:01.000 --> 00:02:41.000
That is correct. I'm an associate professor at the Workland School of Education. I'm also the educational leader in residence for generative AI in the Taylor Institute for Teaching and Learning. If I would characterize my. If I had to characterize my work, I would say my work focuses on the intersection of generative AI and education, teacher training and responsible educational technology. I at the moment run several projects which are focused on supportive teaching, learning assessment, academic integrity, inclusive course design, human agency, professional judgment, and ethical responsibility. I have been coding for years now, so developed multiple different tools that use generative AI. And the goal here is again to support educators, co design, support early researchers with scholarly writing, lesson planning, program evaluation, that kind of thing. I do a lot of public scholarship. I work with public institutions, do a lot of advocacy around this technology, helping them to better understand how to use it responsibly.
8
00:02:41.000 --> 00:02:49.000
Interesting that you use the term public scholarship. I like that. Are you talking about more about knowledge mobilization?
9
00:02:49.000 --> 00:03:12.000
Yes. So work in schools, work with teachers, libraries, and I call these public institutions trying to get people to understand how the technology works, what kind of tools are out there, how do you approach the different tools? What's probably going to happen? What are the paths available to us? That kind of thing.
10
00:03:12.000 --> 00:03:19.000
Interesting. Very interesting. Well, that's a good segue. I mean, Chris can probably take the next one, but I think that's a good segue into our next question, actually.
11
00:03:19.000 --> 00:03:50.000
Yeah, no, I think. And so I had privilege of going and speaking with Swoosh on a panel that we had. And so the, the Taylor Institute hosts a conference on learning and teaching. And so there was four of us that were talking about digital transformation. But one of the things that I think we can maybe just. And you're already starting to. But, you know, how are you using AI in the classroom?
12
00:03:50.000 --> 00:04:10.000
Well, I can say that I had. I'll start like this. There were taboos for me initially when I started doing this work. Lines that I drew for myself that I would never cross. And I crossed every single one.
13
00:04:10.000 --> 00:04:11.000
I love it.
14
00:04:11.000 --> 00:09:22.000
So let's start from there. I promised myself that I would never use this technology to create material for my students to read for obvious reasons. It hallucinates, it's not accurate, it overstates, it doesn't understand that you can't take a strong position from a literature review paper or a position paper. It needs to be empirical, it needs to be large, that kind of thing. But last year I taught a course on the foundations of generative AI in education. And there was simply not enough literature out there for students to read. So I designed an application that would go and read content from Google Scholar and try to the best of my ability, produce content that's accurate with references and all of that. That's what I used. I gave students the option of reading relatively related work to the topic of discussion or directly related work to the topic of discussion, but generated by AI. And students were given that choice to choose which path they want to take. And it was 50 50. Some people read the content produced by Generative AI and others chose to use human written work. I promised myself that I would never use AI for assessment, ever. In that one particular course. Was a two week course, every day, an activity for students to do every day. There were 25 students. So that's 250 activities that I needed to grade over the span of two weeks. It would be impossible. So I created an application where an AI would. I would share my screen. The AI will see what's written on the screen. And then I would ask the AI to identify for me things that were important to me in that paper. So I would say, does the paper contain the following or so? And so then the AI would go and read the paper and highlight for me, yes, it does. Like, here it is. This conversation between myself and the AI was transcribed and I used that transcription to create feedback. And I actually modeled this in class. I said, this is how I'm going to grade your papers. And it would just. It's impossible to do all of this. And I did it live with a paper. And I said, we're going to do one, so you can see. And it was basically a conversation between me and an AI, both of us looking at the same thing. And then it generated feedback. And the feedback looked like exactly. I read the feedback and put it into the feedback box on into D2L, and I would send it that way. So spend about five minutes on a paper where normally I would spend maybe 20 minutes. And then this year I'm going to teach the same course, but I'm going to do it differently. So technology has come. A year in AI is like a decade everywhere else. Instead of assigning greetings, I am going to assign questions. I'm going to assign topics and questions. Three questions, three topics. I'm going to use the same technology I used before to create content. I'm going to give it to students and say, you create your own content now. You read it and come to class and we'll discuss that. And there's different paths that you can take. It's a lot more accurate. Now, the machine does understand the difference between empirical work and the literature review and a position papers, and understands not to overstate, understands not to put too much emphasis on content that comes from a position paper, for example, or actually say that this content comes from an empirical study, but the empirical study wasn't that big. Take it with a grain of salt, that kind of thing. These are some of the ways I've been thinking about, oh, I've been using and also thinking about how to improve generative AI in the classroom responsibly. I'm going to emphasize that. I'll stop there.
15
00:09:22.000 --> 00:11:28.000
It's interesting that you modeled it for your students and, and you're kind of using the AI I, I've got as kind of a. Not a Socratic opponent, but kind of a. As a teammate. One of the things that I've, I've talked to students about is the need to assign AI a very specific role because then it kind of keeps boundaries around it. And you can, of course, you can prompt it to do that. Chris and I have talked about this before. I probably brought it up on this podcast. But it reminds me of that old Apple concept video of the Knowledge Navigator where you're going back and forth between this person. I don't know if you've seen that. What you're describing also kind of reminds me of some work that's done in the technical writing space. So there's a technical writer at Google named Tom Johnson, and he talks about, you know, he does like technical writing, so release notes for updated APIs and stuff like that. So he's kind of a coder, but also a writer, which is an interesting space to live in, but he talked about how it's kind of a lot of professions, writing, teaching, have kind of shifted from starting with a blank page to maybe starting using judgment from this options generator, which is this AI. And then you're kind of having this back and forth all day rather than kind of struggling from scratch. And it's interesting to me, though, because one of the things he discusses in his profession, which is quite different, is that, you know, he does talks, he's written about it, but there's this really weird tension there, there's, there's, there's this palpable tension probably in a classroom, in a lecture hall, when people talk about AI and how you can use it in creative ways. And I'm wondering if you could speak a little bit to. Even though you've modeled it for your students, just like this person I'm discussing has kind of explained his rationale for using it in his profession. Is there, do you feel that uneasiness in the classroom? And why do you think that? If so, why do you think that is? What do you think people are nostalgic that they'll miss based on this new kind of computer interactive paradigm?
16
00:11:28.000 --> 00:14:18.000
It just might be public perception, but there seems to be the stigma that if you use this technology, you're lazy and that you have to do the work, so to speak, and that also the stigma that if you use this technology, you must be cognitively offloading. There is no other way to use this responsibility to use this technology in a way that enhances your cognitive thinking, that most people don't go into that space. Because when ChatGPT came out, people's first reaction to it was, I'm going to give ChatGPT to do my homework, which was cognitive offloading. And that kind of, I think, just stuck. And people are in this space of, oh, this is a tool for lazy people. This is a tool where you cognitively offload. If you use this tool, you really are not going to learn anything. And one of the areas that I work on is to show people that, no, you can actually use the tool to enhance your cognitive abilities, like the Socratic of back and forth. One of the other ways I use it in research is. I create a platform where I input in my own work and then input somebody else's work. And I say, how is this new work or new idea complement my work? And how does it not? It comes up with very, very interesting ideas that I would have missed. Like if I had read, I already know my own content, but reading this new paper, I wouldn't have seen it. So there are spaces to explore. But unfortunately the stigma is this is a tool where if you use it, you're definitely offloading cognitive labor onto it. And I think that another reason why that's the case is because we haven't taught people there's no literacy involved in how to use it properly. We just released it into the world and people just took it the way they wanted to. There was no building foundations of how to use this tool first. And I wish OpenAI had done that work before releasing ChatGPT, but that's not what happened.
17
00:14:18.000 --> 00:15:10.000
I don't want to hog because I'm sure Chris wants to ask a follow up, but because you mentioned cognitive offloading. So the paper that's on arxiv.org your brain on chatgpt has gone viral several times as I can tell. Now it seems like a very well done paper from a brain scan kind of perspective, but it's one study and so I'm always holding my tongue until I see a meta analysis or something like that. Perhaps that's the librarian me, I. I need to see multiple. But what, what is your opinion of the cognitive offloading? Do you think? Like what would. And if, if you. I guess my question is it, do you. Are there tenants of that paper that you agree with? And if so, where do you think that threshold is? Is it somewhere in between what you're doing and let it do everything for me, like where. Where would students start to cross the line?
18
00:15:10.000 --> 00:19:25.000
I. So if you go back to the fundamentals of learning and I'll frame it like this for years and years, maybe hundreds of years, we believe that if a student is able to produce an artifact that is well put together, which is fluent and correct, some sort of cognitive labor or intellectual labor must have taken place for that student to be able to produce that. So for us, the product and the process were the same. We evaluated it the same. If the product was good, there must have been some sort of intellectual labor. And of course you could cheat. You could have somebody else write it, you could have all that. But it was difficult. It was time consuming and expensive and out of reach for most people. When generative AI came around, you could produce that same perfect product without any learning at all. Completely bypasses this tradition that we've had for hundreds of years. That's where the panic comes in. This is the tool for cognitive offloading. Whereas I think that our approach to this is wrong. It has always been about the process. It has never been about the product. We've Equated the two in our education system because classes are large, because there's only one teacher, and it was just the most efficient way of doing it. But now we have to go back and rethink about what exactly is it that we're assessing. We've always assessed the process. So if you're using generative AI, you have to do two things. One, you have to focus, help people understand that a correct answer or a relatively correct answer is not the same as a complete answer. Those are two very different things. And in the learning process, we want a complete answer. We want you in it. Your voices, your senses, your experience. We want to see how you grew. That's a complete picture. We just don't want to see correctness all the time. And being complete is having an understanding of what a machine can do and what a machine can't do. So a machine can do literature review for you and give you this very neutral, relatively clean output, but there's no humanity. There's none of you in it. Right. If I ask a student to go and look at. Transit laws in the city of Calgary, for example, that's the assignment. You could just plug in a website and you could get. But if I ask the student, what does this paper miss, about your experience riding a bus, about your experience riding the C train, about the fears that you have, about the nervousness that you have, about how long you're going to use it, about all of that piece which makes the paper complete, the machine would have missed. The machine can't experience those things. And I think that we need to help people understand that. Where, yes, it's important to be correct, but it's more important to be complete. And it is the human that is able to provide a complete answer. The machine is just simply incapable.
19
00:19:25.000 --> 00:19:55.000
That's a terrific explanation. It reminds me of Daniel Pink. He gave a commencement address at, I think, Columbus School of Art and Design or something like that. And he said that the role of the futures, I mean, he. He used the same thing to develop your own voice, but he. He framed it in the context, in a design school of developing taste. And taste means to have taste or means you have to have a position. And to have a position, you have to have experiences and reflect on it. Yeah, very interesting.
20
00:19:55.000 --> 00:20:53.000
That is interesting. I mean, even, you know, that example that you brought up, I mean, right now there are some debates that are coming up because there's a conversation about removing the free fare, you know, downtown anyways, and some of the reasons that people are bringing it up is because of safety and other things. But I don't know. I mean, I, I think it's safe and it's kind of nice to be able to commute around and go to meetings. But, you know, if you, it's like you say, like the, the whole complete picture would be actually going in and experiencing it. What it's like, you know, one of the things that, you know, you mentioned in terms of, you know, there you touched on it with the like, AI literacy. Now, do you think that we should mandate AI literacy or AI competence or AI use, or are those different things?
21
00:20:53.000 --> 00:25:13.000
Well, I'll, I'll frame it this way. I know, I don't know if you're familiar with pisa, which is an international test that assesses reading numeracy literacy internationally. Right. So many countries participate. I know that in 2029, they're going to start bringing in a new form of literacy called AI literacy. And they're targeting, if I'm not mistaken, 2035 as their first instantiation of this exam. They're targeting basically people born during the pandemic. They want to know if people born in a pandemic have achieved some sort at some level of AI literacy. If you're born in the year 2020, you'll be 15 in 2035. Right. And Pisa targets that, that population group. So it's a global thing that AI literacy is important to have at a global level. That's why they're integrated into, into this exam. So they're saying numeracy, having ability to read and write is probably the same as having an understanding of how AI works and how to use them, so on and so forth. Globally, it's understood that this is a very important thing. That means that we have until. Let's assume they're going to actually make the happen in 2035. Let's just assume for a moment. Let's reverse engineer. So if, if we have there, if there is a test in 2035, we need to do, have a program ready in 2030, like to have teachers ready, have, have an AI literacy program so that we can teach our students so that in five years when they take that exam, they're ready to. It's the responsibility of our schooling system. That means that right about now, in 2025, 2026, we have to have a training plan ready for our teachers to go through governance so that it will be ready by 2030 to be implemented so that our students can pick it up and be ready in 2035. And we do not have that Right now, some countries in the world do have that. We don't have that. So going back to the question of should we mandate it, I think it's a very important part of the curriculum because there's so many ways that using this technology can go wrong. And people. It's not intuitive to pick it up and use it correctly. It's actually a skill. You have to teach people how to use it correctly, which means you have to show them the difference between wrong and right. The debate right now is if we are going to do this, it's going to be one more thing that we bring into the school system, which is already taxed. How are we going to do it? Who are we going to train? What kind of content should we bring? Given that the landscape is changing almost month to month, what are we actually going to say? There's so many questions that we get bogged down and we actually can't create the program. So it's not about a question of mandating, it's about questioning, of figuring out how we're going to do it first and then decide whether this should be mandated.
22
00:25:13.000 --> 00:25:41.000
Makes sense. But, you know, on the same note, like, you know, when we were at the conference, there was several people, and this came in afterwards that, you know, there were people who actually were refusing AI use for. Because of, you know, the. The ethical, privacy, environmental, labor reasons. So do you think that universities should also design for refusal without leaving people behind?
23
00:25:41.000 --> 00:27:50.000
So my, my position on that is that all technology, all technology carry that burden. There's an environmental impact. There's a labor impact, all of it. I think that singling out one technology without having a conversation about all technology, and then having a broader conversation about the impact that human beings and their laws have on the environment, I think we have to have a much larger conversation between before going in that direction. So this zoom call, for example, has an environmental impact. Me going to work on campus has an environmental impact. Netflix has an environmental impact. YouTube has an environmental impact. Encryption has environmental impact. Then we can go to other sectors, like the agriculture industry has major environmental impact. The rules we have for private jets travel have major environmental impact. We need to have a conversation there at that level first. Then we can single out a particular technology. If someone comes to me in my class, if I'm using AI and I'm saying, the university has provided a free version, I want you to use it, and if they will refuse, then I will ask them to drop the course because it would be as me assigning a particular reading and a student Coming to me and say, well, I'm not going to read that. I disagree with the author. I'm the expert here. This is what I think you should do in order to achieve the learning outcome. So if you're unhappy with the curriculum, then you should probably go with a different instructor.
24
00:27:50.000 --> 00:29:15.000
Yeah, I mean, those are good. Like, you know, aspects like you talk about the environmental. I mean, the one thing that I, I mean, I, I suppose if you go back in history, you could probably apply it to anything. Like, even, you know, you talk about, like, in this case, I don't know if in human history we've ever had this kind of ask for forgiveness. And instead of permission ever, or, you know, you're taking people's intellectual property to go and build the large language models, the labor that's going into it. You know, there's people getting paid $5 an hour to go and siphon through and make sure that there's only appropriate content that's going into the model and so on. Although now all of us that are using the free versions, I mean, we're basically getting paid nothing. And we're going and making the, the, the algorithms better. So. But yeah, I mean, it's, yeah, it's quite like that was one thing, I'll tell you personally, Surge. Like when. I never even thought about it when I was, Until I started reaching out to people, but I was looking at, okay, I'm gonna go. At first I was thinking of mandating it on every student. And then somebody asked me the question, you know, how about if somebody didn't want to use it for these reasons? And I'm like, okay, well, that's, that's a good point. And so I made it optional. You know, I still want them to go and experiment, but it's, you know, it's their choice if they want to use it or not.
25
00:29:15.000 --> 00:30:40.000
I used to do that about a year ago. I always had a plan B for students who didn't want to use it. And it was, it was because. Mainly because they had to pay for it. And I thought that would be, that was unfair. But once the university provided access to LLM to all students for free, then that changed the equation. Then it became a tool. I want you, you have to be prepared to go into society and use this tool responsibly. And I deeply believe that using it responsibly is a skill. I would explain all this not in my class, but outside in academia. And knowing that they are refusing to learn a particular skill to use this technology, responsibly knowing that it's going to be ubiquitous, it's going to be everywhere, and then once you're completely informed again, you decide not to use it. I respect your decision, but I would want you to know all the angles and the consequences of not knowing before you make that decision.
26
00:30:40.000 --> 00:30:43.000
Like not judging a book by its cover.
27
00:30:43.000 --> 00:30:47.000
Yes, that would be. Yep, good metaphor.
28
00:30:47.000 --> 00:31:23.000
I have a. I mean we'll probably do when we do rapid fire. I don't want to go too far off, but I know that you talked about institutionally provided tools, so that kind of speaks to something we were going to talk about regarding around equity and access. But just, you know, as from a personal standpoint, I guess the question is are you using, I guess primarily Copilot or the Microsoft side in the classroom, but also of the front of the frontier models available, do you have a preference or why or why not?
29
00:31:23.000 --> 00:33:53.000
I don't have a preference because each model is good at doing something better than others. So if I code, I use Claude. If I am trying to go through large pieces of text, I use Google. And if I want analytics, I use the OpenAI models. In the frontier models, if I'm not in a hurry, I would use Deep Seq API because it's as good. If I'm in a hurry and if I want to pay a little bit more, I'll use the OpenAI models. So it really depends on what it is that I want. Usually when I create AI programs, I use multiple models depending on what their task is, how much compute they use. I don't use the frontier models for every single API call. That would be insane. It depends on what is it that we want to do to try to keep speed, balance, cost, all of that. In terms of the equity and access pieces, I want to highlight one thing I think particularly for public institutions like the library, so they talk about access. They said we bring broad access to the Internet, broadband to the library so that people who need it could go to the library and access the Internet for free. And that's super important. I remember that I went to the library to get access to fast Internet because I didn't have it. But with this technology, it's not about access, it's about competency. Access is the first part. But we already know what happens when you give access to somebody. They don't know how to use it properly, so they cognitively offload. Access is the first step, the competency to use it. Like for example, find me. Go into Indeed.com and find me all the positions, all the Jobs that fit with the following description and then write my resume to fit that description. That will be command.
30
00:33:53.000 --> 00:33:54.000
Right?
31
00:33:54.000 --> 00:34:29.000
That's not cognitive offloading. That's using the tool for a very specific purpose so that you can apply for 40 jobs in five minutes. That's the competency that I'm hoping that people will get. It's about learning to command the machine. And I think that we're missing that. A lot of people are just focusing on access, but not enough on the competency, the actual skills that are required to use the tool properly.
32
00:34:29.000 --> 00:35:02.000
Kind of what Ethan Mollock says. He kind of says that these tools are. Some are better at others and some things. And I agree with you on cloud code though, I've been using Codex, the latest version, and it's a lot closer. So they've, it's more neck and neck now. But if somebody is going to commit to paying for these tools is their best option still, do you think OpenAI, if they want to develop competence, especially comparing outputs between models, because then that does have an extra cost associated.
33
00:35:02.000 --> 00:36:06.000
No, I still think that some of the open source models, models, even faster models or smaller models, they have enough capability to do something very specific. And that's what I mean. Having competency, knowing which model is good for which task so that you're using it responsibly. You're not putting maximum compute on something as simple as grammatical errors. That's a huge part of knowing how to use this technology responsibly. I think that people need to have this awareness that not all models are the same. They use different amounts of compute and water and resources. Having the understanding of what to use for what or which model to use for which purpose. That's a very important form of literacy.
34
00:36:06.000 --> 00:37:19.000
One thing that I was chatting with some students just this week about this, but you talk about knowing how to command the tool and working with these tools. But you know, if you Recall Back when 23, there was like this whole term of like prompt engineering and there was even people getting paid like $300,000 plus. But I would actually say at this point like prompt engineering is dead. And you know, I, I actually posed this to the students. I'm like, you know, how do you think is the best way to figure out a prompt? And so I'm like, what, what do you think about if. And I'm actually going to write this up, but asking the tool itself for the prompt and you know, you what it comes up with. I mean, who would know better than the tool itself to actually go and provide the instructions of how to get whatever you're looking at. And so I think it really comes down to, you know, I, I'm calling it more of like, you have to prepare, like a briefing, like tell, tell it the job, tell the, you know, the audience, give the source material, give the constraints, and, and that's what we're kind of looking for.
35
00:37:19.000 --> 00:37:23.000
I think one of the earliest tools that I built was a prompt engineer.
36
00:37:23.000 --> 00:37:25.000
Oh, okay.
37
00:37:25.000 --> 00:38:33.000
I gave it the tenants of what a good prompt should look like, and then I gave it some information and then it popped out the prompt that I would then use somewhere else. Yeah, I, I, I think, I'm not sure if prompt engineering is dead. The process, maybe, but the literacy of looking at different prompts and, and being able to say, okay, this one's a good prompt, but this one isn't. This one has too much context, this one has, is not very clear or leaves a lot of space for the machine to interpret. That kind of literacy, I think, is important. So it's a different form of engineering. It's not that you are actually doing it, but you're capable of looking at a prompt and say, this is probably going to work much better in cloud code versus that one, despite it being longer, for example. So, so yeah, I think, I think there are nuances. It's not dead yet.
38
00:38:33.000 --> 00:38:40.000
No. Yeah. I mean, what you're talking about, I would kind of categorize as more about the, the judgment around it.
39
00:38:40.000 --> 00:38:47.000
Absolutely.
40
00:38:47.000 --> 00:39:29.000
We kind of touched or alluded to this earlier when we talked about kind of the uneasiness around AI. But what do you think? I mean, we, you know, we make predictions all the time on the podcast of the caveat that we don't expect to be right. So we, we have that disclaimer up front, but what do you think the economic impact will be? Good, bad, or different? We're not trying to pin people down. Are we going to lose all our jobs? Are we going to create more? It was more just. What do you think will change? And I guess related to that, given what we know about the technology as it stands now, and that it'll surely get better, what are the skills that people are going to have to navigate that won't be automated?
41
00:39:29.000 --> 00:42:07.000
I think I talked a little bit about this in that panel. I think that using this technology creates a dividend, an AI dividend, if you will. And whether whether this is going to have a positive impact or a negative impact depends on what you do with that dividend. And the studies have shown that corporate Instinct is to do more of the same. The clear example is you were writing monthly reports, now you write weekly reports or bi weekly reports. And that doesn't actually contribute much to the institution. You just get more reports. It doesn't increase your efficiency in any meaningful way. And so then your workforce, I use the term AI generators become AI generators, trying to figure out whether the AI is hallucinated here or there or this piece is accurate or not. And that doesn't contribute to efficiency. But if you are mindful and purposeful about what are you going to do with that efficiency. So if it took you eight hours to write that monthly report and now it takes you two hours and now you have six hours of free time, that's the AI dividend. How are you going to use that to actually create value for your institution? Think strategically. Think about how the human component can help actually make better products, better customer experience, so on and so forth. If you're strategic about it and actually think about which directions you want to go and give opportunities to your workforce and say, tell me, give me ideas on what are you going to do with that extra six hours, then that's going to actually improve lives. But if you go the other direction of doing more of the same, that's actually going to go the other way. It's going to have a negative experience. People are going to get burned out because they're just AI generators. They're doing more, but they're not contributing in a meaningful, valuable way.
42
00:42:07.000 --> 00:42:12.000
I'm curious, as an educator, what do you do with your extra six hours?
43
00:42:12.000 --> 00:42:39.000
I have now more opportunity to read more, whereas I didn't have that opportunity before. So now I have much more breadth which is which down the line gives me more depth and that creates value for me.
44
00:42:39.000 --> 00:42:45.000
Well, Chris, should we, do you have another follow up or do are we ready for our rapid fire? What do you think?
45
00:42:45.000 --> 00:42:47.000
We could probably do the rapid fire.
46
00:42:47.000 --> 00:42:49.000
Rapid fire. Do you want to take the lead on it?
47
00:42:49.000 --> 00:43:03.000
Sure, sure. Like I mentioned to you, Sarosh, so we have this rapid fire there, you know, pretty simple questions like either or, you know, kind of thing. So are you ready for the rapid fire?
48
00:43:03.000 --> 00:43:04.000
Absolutely, go for it.
49
00:43:04.000 --> 00:43:12.000
Okay, so what's your favorite small language model? Online or offline?
50
00:43:12.000 --> 00:43:15.000
Oh, you have to define what small means.
51
00:43:15.000 --> 00:43:27.000
Well, I guess we're talking about. Yeah, I mean the definition. I think what we originally thinking is that things that either could be open source, that you could run locally.
52
00:43:27.000 --> 00:43:31.000
I don't know if you can. Quinn 3.6
53
00:43:31.000 --> 00:43:33.000
okay.
54
00:43:33.000 --> 00:43:35.000
I have my money on Gemma. But I was wrong.
55
00:43:35.000 --> 00:43:40.000
So, favorite tea?
56
00:43:40.000 --> 00:43:43.000
Green. Earl Grey? Green.
57
00:43:43.000 --> 00:43:46.000
All right. Mac or PC?
58
00:43:46.000 --> 00:43:48.000
Mac.
59
00:43:48.000 --> 00:43:50.000
Standing or sitting? Desk.
60
00:43:50.000 --> 00:43:53.000
Sitting.
61
00:43:53.000 --> 00:43:56.000
What's your favorite car?
62
00:43:56.000 --> 00:44:09.000
Oh, that is so hard to answer. Pass. I don't know.
63
00:44:09.000 --> 00:44:10.000
Ebook or paper?
64
00:44:10.000 --> 00:44:12.000
Ebook.
65
00:44:12.000 --> 00:44:16.000
Favorite Open source tech?
66
00:44:16.000 --> 00:44:19.000
Oh, my God.
67
00:44:19.000 --> 00:44:23.000
That could be hardware or software, by the way. So
68
00:44:23.000 --> 00:44:40.000
I like all the open source tech that help design robot movements. I can give you probably a name, but that's the area that I like so much. Very much.
69
00:44:40.000 --> 00:44:44.000
Synchronous. Asynchronous or hybrid learning?
70
00:44:44.000 --> 00:44:46.000
Synchronous.
71
00:44:46.000 --> 00:44:49.000
Web browser?
72
00:44:49.000 --> 00:44:52.000
Brave
73
00:44:52.000 --> 00:44:56.000
VR, AR or both?
74
00:44:56.000 --> 00:44:58.000
Both.
75
00:44:58.000 --> 00:45:01.000
And who inspires you?
76
00:45:01.000 --> 00:45:03.000
My father.
77
00:45:03.000 --> 00:45:04.000
That's very nice.
78
00:45:04.000 --> 00:45:11.000
That's very nice. That's a. That's a perfect conclusion to this interview. Well, thank you very much. It's been a pleasure chatting with you today.
79
00:45:11.000 --> 00:45:15.000
Yeah. And thank you so much for the wonderful questions and having me on the show.
80
00:45:15.000 --> 00:45:17.000
Absolutely.
81
00:45:17.000 --> 00:45:18.000
Thank you. Take care.
82
00:45:18.000 --> 00:45:19.000
Thank you.
83
00:45:19.000 --> 00:45:34.000
Take care.
