KIMBERLY NEVALA: Welcome to Pondering AI, brought to you by SAS. I'm your host, Kimberly Nevala. In this episode, we are pondering AI's impact on the practice of law with Colin Levy. Colin is a general counsel at Malbek, a professor at Albany Law School, and author of The Legal Tech Ecosystem. Colin, welcome to the show.
COLIN LEVY: Thanks so much for having me.
KIMBERLY NEVALA: Always our pleasure. So, to kick things off, I'd like to know how did your early studies in economics and political science impact or influence your approach to the practice of law?
COLIN LEVY: Yeah, I think it sort of just gave me an early taste of the fact that the world runs on data. And because of that, I realized fairly early on in my career as a lawyer that if I was going to bring the most value to my role, I needed to do so through understanding data. And it didn't all come to me at once, but it was just an ongoing journey to better understand data, how it's used, how it was understood in the business context. And from there, I started to learn more about it and integrate it into how I worked as a lawyer and communicated with my business clients.
KIMBERLY NEVALA: And I would think that folks may see an obvious connection between economics and the data question. How did data play into, or come out of, the study of poli sci? Or was that a totally separate adjunct interest?
COLIN LEVY: So it was sort of a combination of things. I had an interest in politics to some degree, at least from an academic perspective. But I also, as I started learning about political science, it occurred to me how much data was used in political science: in terms of polling data, in terms of voters, demographics, all sorts of different ways. And so the data theme kind of came out of that in a different way.
But if you put economics and political science together, they kind of complement one another and impact one another, obviously. So the data kind of theme was ever present, no matter what I was particularly studying at any given point.
KIMBERLY NEVALA: Oh, very good. Yeah, and I have a question for you coming up here about how economics may be implicated in how law is practiced today. Or at least how it's billed for, perhaps. But before we get there, one of the conversations we have a lot with folks is that before you want to apply any tech, AI included, you really have to understand the nature of the work that you are trying to apply or influence with that technology.
And law and the legal system is one area where, at least from my perspective - I do a little bit of advocacy work in the legal system - where I think the practice of law and what the law is is very different from, I think, what people's outside perspective of it is.
So what is the actual nature of the work that we're talking about here?
COLIN LEVY: Yeah, well, it's funny. Because when I graduated from law school and started working, I realized, what exactly am I doing here? Because what I realized law school was for was essentially learning how law came to be. But when it came to actually practicing, it was a whole different story.
And more specifically, it was much more of managing people, managing processes, type of work than it was kind of directly applying the law to a given set of facts. Now keep in mind I also was working as a corporate lawyer from the outset. So I wasn't writing persuasive briefs before a court or what have you. So in that case perhaps the applying law to a set of facts was a bit more sort of obvious. But in my work, it was much more a matter of, yeah, negotiating clauses and contracts.
But it was also just a matter of just managing people and getting the information you needed from those people to be able to do your job and properly negotiate various documents and agreements. And all of that came as a bit of a learning curve to me. Because law school didn't exactly do the best job at teaching a lot of those skills and those scenarios that I was faced with when I began practicing as a lawyer.
KIMBERLY NEVALA: And so as people maybe start to think about, from the outside in, looking at how law is practiced today. Where do you think people may assume that AI is a good fit where maybe it is not? Or where do you think they may overlook areas where AI may actually prove valuable?
COLIN LEVY: Well, I think there is a tendency to think, as has been the case with prior technologies, that AI is just the answer to everything. It's just the solution to whatever problem you have, which I think is definitely not the case.
I also think, though, that people are realizing that even in those instances where AI can help - whether it be with reviewing a brief, brainstorming ideas, researching cases, or what have you - it's not always going to-- it's not perfect. And so that means you need to approach it like you would approach any other tool. With an understanding that it's going to help you, but it's not going to get everything right. And so that's where your judgment and experience come into play.
And I think that this has shown up in a variety of different instances. For example, lawyers researching cases or citing cases that are made up because they just assume that those cases are real, because the AI came up with them, but there was no verification involved. So while AI is good at doing that type of research in certain cases it also has a tendency to provide you with what you want, as opposed to what you need. So you have to be careful there.
I would also say more to your question that AI can be very helpful with perhaps revising a workflow you have. Or perhaps finding a solution to a seemingly intractable problem. Or figuring out how to word something in a way that's perhaps more persuasive or more on point. That often gets overlooked in favor of more complex tasks that they think AI is automatically going to be good at because it's more complex. Which is not always the case.
And it's also dependent on which AI tool you're using as well. And I think people sometimes need to learn how to walk before they run with a lot of these tools, because they're not one size fits all.
KIMBERLY NEVALA: Now, one of the things that I have observed, and I think I've come to appreciate more and more - I say appreciate, but sometimes I find it intensely aggravating. That as much as we would like to think that the law and the rule of law - whether that's even in the context of criminal justice or dependency courts, where I do some advocacy, or just plain old contracts - is a matter of just strictly interpreting the rules. If A, then B. I've come to appreciate that the law in a lot of ways is a matter of interpretation.
The law is much more of a living contract - maybe not contract in the way that you might refer to it at Malbek - but it's a bit of a living ecosystem that has evolved. And there's so much to do - with how we interpret the law, how we apply the law - that has to do with context and judgment. Do you think that's a fair assessment? And what is it that ultimately makes a great lawyer great in your opinion?
COLIN LEVY: So to the judgment piece, as I've written a fair amount about this in the past, I certainly think that judgment is increasingly critical when using AI and other technologies. Because it doesn't have that judgment. These tools don't think in the way you and I think. And they're not intelligent in the way you and I are intelligent. Which I think a lot of people don't fully appreciate because of its being called artificially intelligent. So people think it's a computer version of human intelligence, which it really isn't at all.
As for what makes a great lawyer, in my experience, I think a great lawyer really is a balance of understanding people; being a really keen observer of people and their behavior, and what drives people to do what they do. Combined with being a great advocate in a way that comes off as natural and persuasive and human focused. And that can be exhibited in a variety of different ways, in a variety of different contexts, depending on what area of law you're working in and what type of lawyer you are.
But I do think that those two qualities tend to make for the most effective lawyer. And that really comes from, I think, applying judgment, thinking things through with intentionality, and fully appreciating the context of whatever situation you're working through.
KIMBERLY NEVALA: And one of the things that there's a lot of both debate and discussion around is what elements can a system such as AI take on? What elements shouldn't it? And you've written quite persuasively and beautifully about the three elements of judgment, experience, and knowledge.
So before we jump into the interplay of those things and how they show up - both in the practice of law and then as we think about applying AI and other technologies to law - how do you differentiate or think about judgment, experience, and knowledge? In this case, legal knowledge, and the relationship between them?
COLIN LEVY: Yeah, so knowledge really, I think, is just kind of the corpus of information you have that you know and you can apply. And that is a combination of laws, of knowing what laws are, how they apply, the situation you're in, the facts of that situation, and so on.
Judgment is the ability to discern within the context of whatever it is you're doing to know when something should be brought up, should be applied, should be discussed, what have you. And knowing when perhaps one solution may be better than another. Or perhaps something may be a bigger problem than people appreciate or what have you. That I think is where knowledge comes in.
And then experience really is just a matter, in my view, of what you have lived through. What you have personally felt. What you personally have had to deal with. And I'm not just talking about professional situations, but personal situations as well that you've lived through that kind of have made you who you are and act the way you act. And appreciating that in the context of judgment and knowledge.
Because I think the three kind of all impact one another and all play off one another. And the more you can understand how they do that, the better off you will be for navigating through what seem like intractable situations.
KIMBERLY NEVALA: Do you think understanding that interplay of these elements and where AI systems, today or in the future - again, I think we have to deal with the capabilities that are in front of front of us at any given moment in time - allows us to be more mindful and get more value from the systems? And if you're looking at those three, where do you think AI is most either showing up or having the most impact today?
COLIN LEVY: So I think that AI is having a great deal of impact upon one's ability to apply knowledge in new ways and perhaps learn new things as well. In other words, expand your knowledge. I think in terms of judgment, it is kind of testing the limits of your judgment. Because a lot of AI tools are purposely designed to give you what you're asking for. Which can often result in outputs that seem too good to be true, which sometimes they are. And you have to be able to understand that and figure out that that's the case and how to account for that.
I think in terms of your experience, for generations that have grown up with technology, where it's been just from day one a part of their lives, I think sometimes there's a tendency to take things for granted and just kind of expect technology to always be there. Whereas for other folks where technology has rapidly evolved; I remember a time when computers were very clunky and I wanted nothing to do with them to now where computers are just an inherent part of all of our lives. And we're likely carrying a pocket sized one in the form of a so-called phone all the time and don't fully appreciate its own capabilities.
And that, I think, speaks to experience. And knowing that you could still solve some of these problems we're solving now without technology, but it requires a different skill set and different way of thinking about things. And so I think not taking that for granted but understanding that is a useful skill set in and of itself is something to keep in mind going forward.
KIMBERLY NEVALA: And something that really jumped out at me, and I think I'm probably pulling things from two articles, is you had said AI changes information processing. But it doesn't without-- but it does it without replacing human judgment. And that judgment is the scarcest resource in the legal profession. Tell us more about that.
COLIN LEVY: Yeah, well, I think part of the problem, really, is the fact that now we have all these powerful technologies at our fingertips. There's a tendency to, as has been written in some articles, to offload our thinking to AI or to other technologies. And that's harmful to ourselves, because it kind of demands less of our brains. And we're kind of conceding or giving away our thinking to a technology that doesn't have the experience that we have and doesn't have all the feelings that we've felt over time.
And so I think because of that, judgment is becoming more and more important. But also becoming scarce in some ways because the allure of technology, and of AI specifically, is so strong. And sometimes we don't realize that we're conceding judgment when we're using AI. When really, we actually need to be applying it even more for the reasons I've mentioned before. But also because our brains are more powerful than we give them credit for. And we need to realize that we can actually solve problems with and without technology. And sometimes problems can't be solved with technology. And judgment helps you figure out which is which and also helps you figure out how to get from A to B when it seems like that path doesn't exist.
KIMBERLY NEVALA: How do we go about cultivating-- I was going to say cultivating judgment, which is kind of a weird statement. But how do we then, with the prevalence of tools and folks within practices that are seeing the influx of tools coming - I'm sure within legal is no different than any of the other industries - where does that then fall in how we train lawyers and how we help folks to understand how to operate within the practice? Especially as these tools come forward with, I don't know if it's an intention but at least the potential, to replace judgment.
COLIN LEVY: Yeah. Well, it starts with including and incorporating more practical experience into law school and into the educational process itself, whether it be through clinics or through internships or externships. I often think that the kind of apprenticeship model that the medical industry makes would be doctors go through - where there's a residency period where they work in a hospital under close supervision before they fully become doctors - would be helpful in this case. Because it gives you experience of situations that you likely would need to figure out how to navigate through but in a less intense way than if you were entirely on your own.
And certainly, in the health industry, look, you're literally often dealing with life or death. So those decisions are incredibly critical. And in legal, it's not quite that way, at least in the majority of cases. But nevertheless, you need to be able to figure out how to work through something under various constraints. And you get a better sense of how to do that through exposure. And that comes through practical exercises and experiences.
I mean, when I was in law school, kind of a funny, brief anecdote, but I learned quite a bit about negotiating not through my contracts class, but actually through this corporate tax class I was taking. Because it was an advanced corporate tax class and a couple of the classes were focused on actually negotiating a tax agreement between two parties. And I learned quite a bit about negotiation and about dealing with personalities through those experiences.
And that's precisely the type of experience that can help you better understand the role of judgment. And, also, what it means to actually practice law as opposed to just thinking about the law in some sort of theoretical sense.
KIMBERLY NEVALA: When it comes to the practice of law formally, how high is the bar for AI today? What are the set professional expectations, if you will, and are they keeping up with the rate of progress in the tech itself?
COLIN LEVY: Well, look, there is no regulation, no law, no human progress is never going to keep up with the rate of change in the tech industry. So I often say that instead of trying to keep up with it, you want to keep pace with it. Meaning that you always have it in your sights and you understand where you are with respect to AI and technology. Make sure that that distance remains relatively the same year in, year out, which means you have to adapt a certain amount.
In terms of working with AI in the legal industry, there are some rules currently in place with respect to double checking your work, understanding the risks and benefits of various technologies in the context of whatever your practice is, and understanding how to use AI so that you are not sacrificing your ethics and so on. But they're written in such a way that are fairly broad, and it's kind of up to you to figure out exactly how they apply in your given situation. And I think that's fairly unlikely to be markedly different anytime soon.
So it's really incumbent upon all of us to, again, do our own homework, if you will, and understand these technologies in the context of whatever work we're trying to do. And know that you're never going to keep up with technology. But at least you can understand where technology is and where you are and ensure that that always is kind of the framework through which you're looking at things through.
KIMBERLY NEVALA: Considering where the technology is today, what do you think are the most critical, I don't know if it's skills or capabilities, that legal professionals need to hone for this moment in time?
COLIN LEVY: I think it starts with really understanding that number one, technology - AI included - where it is now is not where it's going to be 10 years from now. We don't know where it's going to be 10 years from now. It would be very difficult to guess. Same with five years from now in that case.
And what that means for us is, I think, one of the most important skills is the ability to adapt. The ability to always be learning, always be adapting, always be understanding that we need to always be refining our skills. And we can't just operate in a static way because the world is very dynamic; more than ever I would say. And we need to be very dynamic as well, and that's hard.
Change is not easy for people, likely because just historically, once we as humans found a way that worked for us, we stuck with that because it allowed us to literally survive. And the risk was if we try to change, that could risk our survival. The stakes aren't quite as high now, but nevertheless, that resistance to change in that resistance to adapting remains strong inherent in us as part of us being human. But we have to fight against that, because technology doesn't care about that. It has no feelings. People creating these technologies, the technology itself is not trying to accommodate our humanity. We have to accommodate it.
KIMBERLY NEVALA: So again, through the lens from the legal profession that you sit in. You have said that organizations need to, or law practices, should really think about practicing innovation. Or thinking about practice innovation versus tech innovation. What did you mean by that? And why was it important to make that differentiation between practice innovation versus-- a focus on practice innovation versus a focus on the tech itself?
COLIN LEVY: Well, two things. One, technology is changing. And so technology inherently is going to be innovative, because there's always going to be a new version, a new something. Practice innovation refers to basically the way in which we work. And that can take many different forms. It doesn't have to be driven by technology.
It can be driven by an improvement in a process, an improvement in terms of what documents we use, what those documents are, how we deal with clients, how we manage clients, those sorts of things. All of which may or may not be helped by technology. But I don't need to innovate because of technology. You can innovate on your own through process improvement in other ways.
So I think it's important to distinguish that because I do think sometimes the two get conflated. Because people think - largely due to how a lot of the media and a lot of the news tends to refer to this - as innovation is just driven by technology and that's it. When that's not really the case. And I think we've seen technology help to actually prompt innovation in practice through alternative legal service providers, alternative business models, different ways of billing for work, and so on. And those are innovative in and of themselves, but they're not necessarily driven by technology.
KIMBERLY NEVALA: Interesting. Is there any element, or any way in which taking a sort of a technology first approach - or only looking through the lens of technology - is particularly risky when it comes to the legal arena? Or do you see that as really no different than any other industry?
COLIN LEVY: So I think that you don't want to view the world solely through the lens of the technology that exists, the industry that it is. I think you just want to see it as a major player in the world environment in which we operate, but not as the only player. Certainly it's an influential player, but there are other ones as well. And so I think it's important to just understand who the players are and how they all interact with one another.
But you don't want to necessarily view things through just one lens or another, because that can lead to tunnel vision and viewing things in a myopic way. Which give you blind spots. Which will not be helpful, because they'll leave you at a disadvantage because you will likely miss things as a result. So it's very easy for me to say that and very hard to actually do it in practice.
But really, what that means is just be very observant and just take things in and think with intention and act with intention as opposed to acting with impulse and just acting solely on impulse.
KIMBERLY NEVALA: And perhaps unfairly, I think legal practice, and maybe this comes from some of the-- it probably comes mostly from the billing side of the house or billing practices that we see as memorialized and sensationalized on many, many TV programs as well. Where folks from the outside are maybe looking at the practice of law and just seeing it as a fairly cutthroat business that is really leaning into things like hours and the number of documents, the number of hours you put in. And we've all heard the tropes and seen this sensationalized.
And there's no question that to date, I think, the economics, the business model, has rewarded or supported that kind of billing. When we get into a place-- so some of the discourse you see is, well, hey, now you don't have to pay. You don't need you don't need to pay someone to go do this because you can go and find that information.
And I suppose this comes back to that intersection of its judgment, experience, and knowledge. It's not just about understanding or being able to find that sort of piece of the law but understanding how to apply it. That being said, do you think AI is going to or is forcing a rethink for folks about where the value is and how to capture that value in law practice?
COLIN LEVY: It's certainly asking the question, for sure. In terms of whether-- I think it's definitely allowing for increased competition and putting more pressure on the billable hour model and how work is billed and how work is paid for. And certainly clients, I think, now are changing expectations in terms of how work is done, by whom, or by what, and how it's being paid for. It does remain to be seen whether the existing billable hour model remains the dominant model going forward.
As much as I love to say, it's just going to go by the wayside. Because others have predicted that many times before and been wrong. I don't think that's necessarily going to happen. But I certainly think that we're going to see increased competition in the form of other types of billing for work, project-based fees, flat fees, other types of ways of billing alongside or perhaps instead of billable hour models.
But I don't think that that model is just going to disappear simply because AI is here. At least certainly not in AI's current state. Now, AI will likely be vastly different in the future from what it exists as now. And so I can't say what will happen in the future. But thinking short term, I think that if nothing else, AI is certainly bringing increased pressure and competition, which I think is a good thing.
KIMBERLY NEVALA: Now, you wrote the - or maybe you didn't write it, but it was published, I believe, in 2023 - The Legal Tech Ecosystem. And I'm interested in since that time - there's been no shortage of excitement in the intervening years since - of some of the ideas that you were put forth in that book, what do you think are still the most, what do you find, just really most resonant? Would you really like folks to pay special attention today? And are there any sections that you think, well, maybe today I'd have to redact those because they're not so relevant?
COLIN LEVY: Yeah, I think, funnily enough, I think the parts that kind of project management on innovation are likely just as true now as they were when I wrote about them. I think that obviously a little bit of section I wrote that kind of talked about AI at the very end of the book I think is likely outdated at this point. But the intention wasn't to really write about it at that point, because things were moving very quickly. And also, the book really wasn't about AI in any real sense.
So I certainly think the people management, project management, and innovation sections remain as relevant today as they were when I wrote about them. Mainly because people remain complex, remain unpredictable, remain fickle. And so having the skills to be able to manage and deal with people, I think, is just as important now as it ever was.
KIMBERLY NEVALA: And in that book, you do break down the ecosystems and point out several different constituent components, I suppose, of the legal system. And too, I think what you just said: AI is a tech, it's the current tech. But whether it's this version of AI, a different version of AI, not AI at all, understanding all of those components of the ecosystem help us understand not only when to adopt it, when not to adopt it, but also how to operate with that.
So when you think about the legal environment or the ecosystem, the legal ecosystem, what are the most underappreciated or under-attended aspects of that practice?
COLIN LEVY: I've always been a very kind of fervent observer and student of people. And I think that when it comes to the legal ecosystem, I think that people, oddly enough, remain perhaps the most underappreciated part of that ecosystem. Because people are probably the least glamorous and interesting, at least ostensibly, part of that ecosystem. When in fact, they're the ones driving everything. They're the ones making decisions. They're the ones that are really pushing for things. And technology is a player that's kind of in the background, always doing things, and you never know quite what it's going to be doing.
So I do think that more than ever, it's really, really important to understand relationships, understand how people manage and build relationships, and understand the really big role and perhaps underappreciated role that emotions play in decision making. And as much as lawyers pride themselves on being logical and systematic, people are messy. Emotions are messy. And so you have to be able to figure out how to navigate through that mess. And understanding that most of the time, logic likely has nothing to do with it.
KIMBERLY NEVALA: Yeah. I imagine that means that even in the most logic driven, rational environment that we would like to believe legal and law is. And, as I said, having some intersection with the courts, I come to appreciate what you're saying ever so much more. Because I think it's often very little to do with what's on the page, per se, and very much about how we interpret it or how people relate to it and respond to it.
But one of the things that we see in organizations is that - and I'm wondering if in the practice of law and in organizations or the functions in which lawyers operate - whether there is the same divide or gap that we see very often between policy - maybe we think about that as the written law - and what happens on the ground in terms of how governance actually happens or doesn't happen day to day.
COLIN LEVY: Yeah, I mean, largely you can have all the policies in the world written down on various pieces of paper stored in a drawer somewhere. That's great. Good job on putting those policies in place. But it's often been said about other things. The proof really is in the pudding. And so you really have to look at people's actions more than look at whatever policies are.
And when it comes to AI, I've been speaking a little bit more about this. In terms of governance of AI, you really want to look at how people are using AI, what they're doing, what they're acting upon, and figure out how to govern people's behavior through the actions that they're likely to take. And that's hard, because it requires getting people to behave in a certain way.
And that's one of the most difficult things to do is, I think, trying to control how people behave. Because as much as we have policies and laws and what have you, as I mentioned before, emotions have a habit of overriding all of that. And so even the most logical person is likely going to operate in a very illogical way at times. Not because they want to necessarily, but because they're in a state where they just act due to how they're feeling. And that has to be accounted for in ways. And it's difficult, but it's just part of, I think, navigating a world in which the species that is operating are humans, and we're emotional creatures.
KIMBERLY NEVALA: So in that sense, where you may have popped the bubble of all of us who've just always seen lawyers as the most strictly rational folks at all times. Or at least maybe they've appeared to us that way when we've engaged in debate with them. How do you think, or how do you, at this moment of time, where do you think AI is or will have the most impact when it comes to legal practice? And where are areas that you think we need to really double down on humans?
COLIN LEVY: So I think when it comes to transactional type of work, drafting documents, reviewing documents, creating documents, that type of thing, AI is already having a big impact. But that's largely driven by the fact that AI right now is very much a content creation type of technology. And I'm not sure if that will be the only form of AI that will exist, but that's the dominant one right now.
But in those cases where there is a lot of emotion and a lot of context and complex contexts at play, say in the area of family law, for example, AI is going to help to some degree navigate through the noise a little bit. But it's not going to help with making a decision that has to accommodate the emotional baggage at play.
So I certainly think that AI can get you so far in a world in which emotion is not a big player. But in those areas of law where emotion and family dynamics and people and relationships are more at play, AI is going to have a limited impact currently. Is that likely to change at some point? Likely to change, yes. In terms of when? I have no idea.
KIMBERLY NEVALA: Yeah, interesting. Well, I suppose it begs the question as well. Because we've been talking throughout this, and you've said, that as much as we all like to think we're very rational creatures, and lawyers in particular, that we're all somewhat emotional.
And I would think even when we're thinking about this in a corporate context, maybe it's corporate contracts and things like that, that there's some human elements of what people want to lean into. What they don't want to lean into, What we're going to agree to. What we're not going to agree to that still take a little bit of that piece. Is that true, or do you think there's just some areas where it really is much more mechanistic in some ways?
COLIN LEVY: Yeah, I mean, at the end of the day, we can all say one thing and be more than capable of doing something quite different from that. So I do think that largely a lot of our behavior is, for better or worse, driven by how we're feeling at a certain point in our own lived experiences and our own passions and incentives. So I think you have to appreciate that and just understand that that's going to always be a factor, and it's not always going to be predictable as to how big a factor it's going to be.
KIMBERLY NEVALA: And what aspects do folks like me not question? What question should I be asking that I'm not asking about this intersection of AI and law today?
COLIN LEVY: I think you want to be asking, to the extent to which we define legal work, what it means to be a lawyer, what it means to practice law. Which for a long time meant, I think, fairly specific things. And now - and this has been undergoing this change, I think, for a little while thanks to technology - is what does it actually mean to practice law?
What is a lawyer to do in this world that is largely underpinned and driven by technology? And I think the answer to that question is driven in part by what clients are asking of their lawyers. And I would like to see actually more work, more academic study of that very question, because I don't think we have a very good answer right now or a very good understanding of what that answer is.
KIMBERLY NEVALA: Interesting. Fair. Is that something that you - types of research - that you are undertaking in conjunction with your professorial duties or with something that you--
COLIN LEVY: It's certainly something I'm thinking about, for sure. It's not something that I'm researching in any systematic way, but it's certainly something that's been top of mind for me. Particularly as I get ready to launch, actually, a new book coming out in October on AI and how lawyers should approach it.
KIMBERLY NEVALA: OK, excellent. Now, I have a fairly mechanistic, slightly a left turn here, but a somewhat mechanistic question, perhaps. And it was driven by something that I had read of yours when you were talking about contract considerations. And, in fact, I think with Malbek, you guys work on legal contract management, and you're going to correct me and put the right context out there.
But you were musing about how or what we should consider? How do we contract for capabilities not guaranteed or that you can't see yet? And it struck me as a really interesting question. How do you approach this perhaps nebulous feeling area?
COLIN LEVY: Yeah, I mean, I think you have to do it through a couple of ways. One is understanding what exists now and then think about how that will likely evolve in the short term. You can't go too far ahead. I mean, you can't think about long term, because you just don't know what that's going to look like.
But you have to account for the fact that as models get better, as things become more capable, that there has to be some accounting for that. And there has to be some transparency about what things are capable of doing and be given insight and notice of that type of thing. And that, I think, is how you can incorporate and contract for.
It is certainly tricky. But I do think as I've encountered more and more contracts that incorporate AI terms, that is certainly something that I think will become increasingly of concern to companies. Because I think they're realizing, as people are realizing, that this technology is moving very, very quickly.
KIMBERLY NEVALA: Yeah. And it strikes me that even as consumers, we see this a bit in my ever daily frustration with the purveyors of over at Google, where I get something new and then I have to go scrambling back down through the terms and conditions, through every setting known to man, because they tend to bury them 10 down, to then try to adjust how my data is shared, how it's used.
But it's always retroactive and it's always sort of an opt in permission that I have to go opt out of. And there's an expectation that I'm just automatically going to be pleased about it. I think there's probably more likely an expectation that most people are not going to either be aware or just not going to have the time and energy to go down and do that.
And so I'm wondering do we start to see any sort of a move where we just start to, as opposed to putting the onus on the consumer, or maybe it's even as a corporation or a business that has taken in and is using a piece of software, will we start to see more restrictive terms and conditions moving forward? Which is: I'm only sort of allowing you at this moment in time, with this. I don't know how this actually gets fixed, because you could envision the 27 page T&C of they're using your data in all these sorts of ways. But it does seem that we're a little bit backwards right now. And I'm wondering if you're seeing any shift in just how restrictive or how constrained these kinds of terms are.
COLIN LEVY: Yeah, I think that certainly companies that are negotiating with others that are offering AI services that companies are paying for are putting focus on restricting use of data, use of inputs, use of outputs, use of prompts and so on. And I think that's going to continue to be the case going forward.
Because I think on the flip side, the companies that are offering these technologies are going to want to ask for more and more flexibility and use of data, because that's a major constraint of large part of these models is the limited size of the data set. As much as we all think of these data sets that these models were trained on has been huge, they are actually limited when you think about the context of just how much data gets created every minute of every day.
So I think it's kind of going to be a balance and going to be a little bit of a battle between the two sides that is unlikely to get resolved anytime soon; but will likely continue to be in tension for quite some time.
KIMBERLY NEVALA: Are there any other shifts or changes in either attitude or contractual postures that you've noticed in relation to AI?
COLIN LEVY: I think there are some companies that are more sophisticated and have a better sense of how these models work, and that's reflected in some of the clauses that they insert and ask for. But largely, what I am seeing are clauses that tend to be one size fits all with respect to AI functionality or services. And so there is this sort of educational period that companies have to go through with others about who owns the model, how things work, who controls what.
That's not really a shift, more of just this ongoing tension, that I've also seen be present. There will likely be another shift in clauses as AI evolves and other forms of AI get created and start getting used. Because I do think it's important to keep in mind that in, with respect to all of this in general, we're very early in this AI journey. We're kind of at the very beginnings of it, as much as it may seem like we're not. And so we're going to, I think, be rather surprised and shocked at where we are 5, 10, 15 years from now.
KIMBERLY NEVALA: Yeah. Well, and it seems to me that the legal arena, of which there are many arenas, let's be clear. And as you said, the corporate versus criminal justice and everything in between are very, very different.
But it's an interesting area in which our ability to manage emerging risk, to account for those unknown unknowns, and to really think critically about the interplay of judgment, experience, and knowledge - which sometimes we tend to use those terms more interchangeably than we likely should. But it seems to me that the legal arena is an interesting place where a lot of these issues will come to roost. Or an interesting testing ground for a lot of those as well.
COLIN LEVY: It certainly is. It certainly is not a place for the faint hearted, I would say.
KIMBERLY NEVALA: So being someone who's clearly not a faint hearted, any final words or thoughts for the audience?
COLIN LEVY: I would just say thanks for listening. If you'd like to learn more, feel free to check out some of my work on LinkedIn, as well as on my website, colinslevy.com. That's C-O-L-I-N-S-L-E-V-Y dot com.
KIMBERLY NEVALA: Awesome. And we will provide links to all of that in the show notes as well. So thank you so much for your time and insights today.
COLIN LEVY: Thanks so much.
KIMBERLY NEVALA: Awesome. And to continue learning from doers, thinkers, and advocates such as Colin, you can find us wherever you listen to podcasts and also on YouTube.
ANNOUNCER: This has been a SAS podcast.
