Getting started with AI-enabled e-discovery: Do’s, don’ts, and lessons learned

WEBVTT
00:00:13.173 --> 00:00:16.813
Intro: Hello, and welcome to Tech Law Talks, a podcast brought to you by Reed Smith's Emerging Technologies Group. In each episode
00:00:16.813 --> 00:00:21.273
of this podcast, we will discuss cutting-edge issues on technology, data, and the law.
00:00:21.533 --> 00:00:26.333
We will provide practical observations on a wide variety of technology and data
00:00:26.333 --> 00:00:31.313
topics to give you quick and actionable tips to address the issues you are dealing with every day.
00:00:33.073 --> 00:00:36.293
Anthony: Hello, this is Anthony Diana, and welcome to Tech Law Talks.
00:00:36.533 --> 00:00:40.573
Today, we are starting a podcast series on AI-enabled e-discovery.
00:00:40.993 --> 00:00:45.793
The podcast series will focus on practical and legal issues when considering
00:00:45.793 --> 00:00:50.533
using AI-enabled e-discovery, with a focus on actual use cases,
00:00:50.713 --> 00:00:52.293
so not just on the theoretical.
00:00:52.953 --> 00:00:58.633
Joining me today are Therese Craparo and Marcin Krieger, colleagues of mine at Reed Smith.
00:00:58.933 --> 00:01:04.393
Welcome, guys, and let's get started. So let's start with just an idea,
00:01:04.573 --> 00:01:12.033
Therese, on and what are we seeing right now in terms of people actually using AI in e-discovery?
00:01:12.768 --> 00:01:16.688
Therese: Yeah, I think what's interesting is we're really starting to see an uptick in
00:01:16.688 --> 00:01:20.948
the use of AI in e-discovery in a few different areas.
00:01:21.168 --> 00:01:26.268
I think the first and where we're seeing really a lot of promising AI technology
00:01:26.268 --> 00:01:29.148
is in data knowledge and data farming.
00:01:29.368 --> 00:01:33.648
So speed to knowledge, we call early case assessment. It's taking that initial
00:01:33.648 --> 00:01:39.668
data set and using GenAI to quickly identify key concepts,
00:01:40.128 --> 00:01:45.108
key issues, anything that you want to do to get your handle on that data and
00:01:45.108 --> 00:01:49.348
the facts of the case, the potential facts of the case sooner than later.
00:01:49.568 --> 00:01:53.788
Really seeing a lot of that. And as a corollary to that, a use of GenAI for
00:01:53.788 --> 00:01:55.588
investigations, right?
00:01:55.688 --> 00:01:59.728
Because that's really where a lot of this, you know, early data assessment comes
00:01:59.728 --> 00:02:04.848
in handy. finding, zeroing in on the key issues and the key events really quickly
00:02:04.848 --> 00:02:09.348
to be able to make a quick assessment of what legal actions need to be taken.
00:02:10.208 --> 00:02:14.688
Also seeing in accelerated privilege models, so the ability to more effectively
00:02:14.688 --> 00:02:19.628
and quickly identify privileged documents and privileged information to sort
00:02:19.628 --> 00:02:23.328
of better facilitate privilege reviews, and along with that,
00:02:23.428 --> 00:02:24.428
of course, document review.
00:02:24.548 --> 00:02:29.168
We are absolutely seeing Gen AI tools popping up that are helping to facilitate,
00:02:29.428 --> 00:02:33.428
you We'll call them next-gen document review, automated review,
00:02:33.728 --> 00:02:38.028
again, ways of quickly, more efficiently and effectively identifying relevant
00:02:38.028 --> 00:02:40.948
documents to move them through the review process.
00:02:41.827 --> 00:02:46.347
Of course, we're also seeing use of AI for what we'll talk about as sort of
00:02:46.347 --> 00:02:48.167
data analysis and presentation, right?
00:02:48.267 --> 00:02:51.987
Now I've got the documents. I need to identify the data for my witness kits,
00:02:52.167 --> 00:02:56.787
for what I'm going to use at trial, to look for timelines of events and things
00:02:56.787 --> 00:03:01.987
like that to really help get handles around the actual relevant data and what's happening in the case.
00:03:02.427 --> 00:03:08.027
And then I think some really fun applications of Gen AI, like with translations.
00:03:08.027 --> 00:03:12.347
So on the fly translations, instead of hiring, you know, armies of folks who can,
00:03:12.507 --> 00:03:16.747
you know, in various languages, or even interpretation of the data without the
00:03:16.747 --> 00:03:20.807
translation, something that's in another language to get a sense of the content
00:03:20.807 --> 00:03:23.327
without actually having to translate the data.
00:03:23.547 --> 00:03:27.827
So I think there's a lot of really interesting and promising areas for AI-enabled
00:03:27.827 --> 00:03:31.507
e-discovery right now. And I'm sure Marsheen has thoughts on that as well.
00:03:32.186 --> 00:03:35.786
Marcin: Yeah, for sure. You know, it's interesting. There's been an absolute explosion
00:03:35.786 --> 00:03:41.566
of AI tools that have been coming out based upon generative AI because it's
00:03:41.566 --> 00:03:44.586
a very new way of analyzing documents.
00:03:44.746 --> 00:03:49.666
It's a really new way of getting at data. But the actual adoption appears to
00:03:49.666 --> 00:03:51.806
be a little slower than the interest.
00:03:52.026 --> 00:03:55.986
The interest is everywhere. Everybody wants it. There's a lot of hesitation
00:03:55.986 --> 00:03:59.966
in terms of how to adopt it. there are still many clients that are struggling
00:03:59.966 --> 00:04:02.826
with even the classic AI, the use of TAR.
00:04:03.026 --> 00:04:06.646
And I know that we're going to talk about that in future podcasts and how do
00:04:06.646 --> 00:04:11.246
we use all the different types of AI in the right way, in the right time, in the right place.
00:04:11.546 --> 00:04:14.646
But when you actually look at what's available in the market and the development,
00:04:14.886 --> 00:04:17.186
many of these tools are still in their infancy.
00:04:17.346 --> 00:04:22.106
And a lot of these tools are products that are looking for a problem to solve.
00:04:22.426 --> 00:04:26.046
And so what that means is that clients are seeing these tools and they're saying,
00:04:26.126 --> 00:04:28.366
you know, how do we adopt them? How do we best use them?
00:04:28.526 --> 00:04:31.526
And they're going to their outside council, their discovery council,
00:04:31.606 --> 00:04:33.726
and saying, how do you use this? How will you use this?
00:04:33.946 --> 00:04:38.526
And so there's just a forest fire of interest and demand.
00:04:38.906 --> 00:04:44.906
And yet there aren't as many advancements, surprisingly, in terms of just broad adoption.
00:04:45.426 --> 00:04:49.346
I think that everyone is tiptoeing to see who's going to be the first one to
00:04:49.346 --> 00:04:53.866
make the big splash with the use of GenAI in discovery, who is going to be
00:04:53.866 --> 00:04:58.566
brave enough to be the first one to gain judicial acceptance with the use of GenAI.
00:04:58.906 --> 00:05:03.166
And it's a very exciting time to see who's going to be, what's going to be the
00:05:03.166 --> 00:05:05.626
right case, what's going to be the right judge, what's going to be the right
00:05:05.626 --> 00:05:07.426
court that makes that possible.
00:05:07.646 --> 00:05:11.546
But what I will say, though, is the tools are fascinating. The tools are interesting.
00:05:11.906 --> 00:05:15.386
Like you said, Therese, speed to knowledge. We have tools that
00:05:15.820 --> 00:05:20.660
allow you to ask open-ended questions, your classic chatbot model,
00:05:20.700 --> 00:05:26.320
and you can take almost as much data as you want and glean key insights very, very quickly.
00:05:26.540 --> 00:05:31.620
Not only in ECA, but also in those midnight data dumps before the night of a
00:05:31.620 --> 00:05:37.740
deposition, opposing counsel drops 5,000 documents on you, hoping to bury the relevant information.
00:05:37.900 --> 00:05:42.340
Yet, if you have the right tool, you can ingest that and in moments sift through
00:05:42.340 --> 00:05:45.400
it to get to the most important information for tomorrow's witness.
00:05:45.880 --> 00:05:50.140
We're also seeing the advancements in the per document review,
00:05:50.460 --> 00:05:54.820
a totally different use of GenAI, which isn't just chatbot my whole corpus,
00:05:54.980 --> 00:05:58.440
but how do I code each document for relevance?
00:05:58.640 --> 00:06:03.400
This new version of TAR, we can't really call it TAR 3.0 because it's a completely
00:06:03.400 --> 00:06:08.520
different way of looking at the world of document review, but it is essentially the next evolution.
00:06:08.960 --> 00:06:13.400
So going to these conferences and watching these products evolve and watch them
00:06:13.400 --> 00:06:18.040
get better every week, every month is just absolutely fascinating and super exciting.
00:06:18.420 --> 00:06:22.740
Therese: Yeah, and I think just quick to pick up on that, when Machine's talking about
00:06:22.740 --> 00:06:27.380
the interest around it and the lag in adoption, which of course,
00:06:27.500 --> 00:06:33.020
as he said, we saw with TAR, I think what's really exciting about the GenAI tools is that
00:06:33.456 --> 00:06:39.756
There's such a variety of potential uses that in some ways, a lot of them have lower risk.
00:06:39.876 --> 00:06:43.096
Like a lot of the hesitation with TAR is, I don't want to have legal fights
00:06:43.096 --> 00:06:47.516
over it. I don't want to have discovery about discovery, which I think held a lot of people back.
00:06:47.676 --> 00:06:52.096
But the variety of these tools that are being developed gives the opportunity
00:06:52.096 --> 00:06:56.696
to start tipping your toe in the water without necessarily having to go to those
00:06:56.696 --> 00:07:02.076
areas where people are a little bit more nervous, which I am hopeful will really drive,
00:07:02.096 --> 00:07:04.696
you know, adoption. and, you know, development going forward.
00:07:04.996 --> 00:07:08.276
Anthony: Yeah, and I think it's telling, right? We've been, you know,
00:07:08.536 --> 00:07:11.716
we've talked about this a lot, TAR, the use of machine learning,
00:07:11.856 --> 00:07:15.476
artificial intelligence, has been around for more than 15 years, right?
00:07:15.616 --> 00:07:18.796
And it's been accepted by the courts and the like.
00:07:19.056 --> 00:07:21.836
Even with that, it's a relatively low adoption, right?
00:07:21.896 --> 00:07:25.296
I'm struck by the fact, I mean, I think that legal tech and stuff like that,
00:07:25.376 --> 00:07:27.256
I was talking to a lot of vendors and I was just asking the question,
00:07:27.436 --> 00:07:31.576
how often is this being used, right? How often do you actually use TAR? And
00:07:32.186 --> 00:07:37.986
generally, they said maybe 20% of the cases. Again, these are large cases,
00:07:38.086 --> 00:07:40.066
not just the small ones, which maybe doesn't make sense.
00:07:40.186 --> 00:07:44.006
But even in large cases, because of some of the things you talked about,
00:07:44.146 --> 00:07:48.626
even though there's just acceptance, people are like, well, I have to negotiate this or whatever.
00:07:48.866 --> 00:07:52.506
I'm going to put it off. But I think the legal industry as a whole has probably
00:07:52.506 --> 00:07:56.506
failed if TAR is only being used in 20% of large cases.
00:07:56.846 --> 00:08:01.706
And I don't think we have that option for GenAI. I think it's moving so quickly.
00:08:01.706 --> 00:08:03.266
I think we have to get past it.
00:08:03.466 --> 00:08:09.386
And I think importantly, as you mentioned, is in some ways, GenAI adoption,
00:08:09.426 --> 00:08:13.986
I think, may be faster, not in all use cases, but maybe faster because a lot
00:08:13.986 --> 00:08:16.926
of the discussions we're talking about in our podcast series is going to go
00:08:16.926 --> 00:08:17.986
into all these use cases.
00:08:18.286 --> 00:08:23.546
But a lot of them, you don't need the other side to agree, or you don't need the court to agree.
00:08:23.806 --> 00:08:26.626
And we're going to get into a little bit of that, of how do you make sure of
00:08:26.626 --> 00:08:28.106
that and protective workers alike.
00:08:28.326 --> 00:08:31.406
But generally, I think we're going to see much faster adoption.
00:08:31.546 --> 00:08:34.966
I think we're seeing it, I mean, you should know this, it's like every day,
00:08:35.186 --> 00:08:37.946
almost every one of the matters I'm working on, people are using.
00:08:38.587 --> 00:08:42.867
GenAI. We use Harvey here. But everyone is now used to it. They're trained
00:08:42.867 --> 00:08:44.447
on it. They're using it in their everyday.
00:08:44.707 --> 00:08:48.567
I think that'll actually help on the e-discovery side because people are using
00:08:48.567 --> 00:08:50.527
it. And they're like, but I do this all the time.
00:08:50.927 --> 00:08:55.247
So next is, okay, we talked about it. Generally, there's a lot of excitement.
00:08:55.647 --> 00:09:01.087
How do we get started? So, Marcin, they're sitting there, an in-house attorney.
00:09:01.287 --> 00:09:04.387
They want to start using it. But lots of issues.
00:09:04.647 --> 00:09:09.607
How do I just get started in starting to use AI-enabled. Marcin: Absolutely.
00:09:09.847 --> 00:09:14.047
So one thing that I might slightly disagree with you on is your statement that
00:09:14.047 --> 00:09:16.387
everyone's using AI and we're comfortable using it.
00:09:16.527 --> 00:09:19.547
I think it's a very interesting nuance to that.
00:09:19.847 --> 00:09:22.587
Everyone is using AI, but they don't actually know they're using it.
00:09:22.747 --> 00:09:28.007
We use AI every day, every time that you talk to your cell phone or now anytime you go to Google.
00:09:28.787 --> 00:09:34.807
And for attorneys that are hesitating with using AI in their practice,
00:09:34.807 --> 00:09:37.567
they just need to realize you've been using AI.
00:09:38.167 --> 00:09:42.467
And if you want to take yourself to the next step of using AI as part of your
00:09:42.467 --> 00:09:46.887
practice, one of the first ways to do it is you can just sign up for a free
00:09:46.887 --> 00:09:50.587
account with one of the existing models, not for client work.
00:09:50.727 --> 00:09:56.207
But for example, myself personally, I got an account with OpenAI on ChatGPT.
00:09:56.547 --> 00:09:59.847
And my daughter and I use it all the time Because, you know,
00:09:59.927 --> 00:10:02.207
with our hobbies, for me, it's vinyl records.
00:10:02.447 --> 00:10:07.127
I got very comfortable with learning the right way to write prompts,
00:10:07.287 --> 00:10:09.147
the right way to understand responses.
00:10:09.547 --> 00:10:12.867
That's step one. Get comfortable with the technology. And, you know,
00:10:12.927 --> 00:10:14.247
there's a lot of guidance out there.
00:10:14.347 --> 00:10:17.887
We've done CLEs on things like competence with the use of AI.
00:10:18.147 --> 00:10:21.987
But the first step is just get comfortable with how GenAI works generally.
00:10:22.590 --> 00:10:25.490
The next thing that you do is survey the market.
00:10:25.890 --> 00:10:30.310
Find the right tool. Don't find a tool to solve a problem. Have a problem and
00:10:30.310 --> 00:10:33.150
find the tool that solves it. You mentioned we use Harvey.
00:10:33.530 --> 00:10:37.950
Harvey is a fantastic tool, as are many others. It's a jack of all trades that
00:10:37.950 --> 00:10:41.330
lets you do a lot of things, but that may not necessarily be the right tool
00:10:41.330 --> 00:10:43.370
for a particular attorney's problem.
00:10:43.710 --> 00:10:46.650
Maybe you're a transactional attorney. Maybe you're an M&A lawyer.
00:10:46.950 --> 00:10:49.650
Maybe you're a discovery attorney. and if you're listening to
00:10:49.650 --> 00:10:52.450
this podcast and you want to learn how do i take that first
00:10:52.450 --> 00:10:55.330
step well there are a lot of attorneys out there that specialize in
00:10:55.330 --> 00:10:58.250
ai and we can advise you on it but the most important thing
00:10:58.250 --> 00:11:03.450
is don't just grab a tool and then try to force it into your toolkit have a
00:11:03.450 --> 00:11:08.410
problem go out there survey the market and that would be like the first big
00:11:08.410 --> 00:11:14.190
step we call it try before you buy right then there are a number of other steps
00:11:14.190 --> 00:11:16.970
you have to be aware of we can't really get into all of them here.
00:11:17.230 --> 00:11:20.590
But if you look at the ethical guidance that's out there, if you really want
00:11:20.590 --> 00:11:25.770
a great primer, the ABA guidance that was published a year ago actually has
00:11:25.770 --> 00:11:30.430
the framework for the things that you need to do to get started once you found the right tool.
00:11:30.730 --> 00:11:34.850
And in a future conversation, we'll go much deeper into these things.
00:11:34.970 --> 00:11:39.790
But there are steps that you have to take in terms of knowing how to protect your data.
00:11:40.290 --> 00:11:44.910
Knowing how to protect your client's confidentiality, understanding if there
00:11:44.910 --> 00:11:50.590
is a landscape, be it regulatory landscape or NDAs or maybe even restrictions
00:11:50.590 --> 00:11:52.930
from the court. But that's the next step.
00:11:53.310 --> 00:11:58.230
Find the tool, then make sure the tool is used in a way that complies with your
00:11:58.230 --> 00:11:59.690
obligations with data privacy.
00:12:00.210 --> 00:12:05.730
And then the last step, of course, is learning how to validate what the AI is doing.
00:12:05.850 --> 00:12:09.270
And that is a very product and use case specific conversation,
00:12:09.270 --> 00:12:10.970
which we will have down the road.
00:12:11.671 --> 00:12:18.091
Anthony: Yeah and I think one of the things that I hear from in-house folks is they're going to
00:12:18.091 --> 00:12:19.491
have to make the business case.
00:12:19.731 --> 00:12:23.691
It's not enough to just say, I'm going to use it because they may have policies
00:12:23.691 --> 00:12:28.151
and the like that are saying you have to be super careful when you're using AI.
00:12:28.811 --> 00:12:32.491
I think that helps. And that's why I think this podcast here is going to focus
00:12:32.491 --> 00:12:37.091
on use cases because that is have a use case that actually makes sense and then
00:12:37.091 --> 00:12:39.931
say, okay, here's the tool that fits that use case, you know,
00:12:39.991 --> 00:12:41.751
whether it's privilege or something like that.
00:12:41.751 --> 00:12:47.331
And then explain it to both whoever, all the stakeholders, as to why the present
00:12:47.331 --> 00:12:48.811
tools don't work, right?
00:12:49.091 --> 00:12:55.411
And this is a better solution. I do think it should not be sort of vendor-driven,
00:12:55.451 --> 00:12:58.711
which I think right now it's a little bit because every vendor is coming out
00:12:58.711 --> 00:13:01.951
and saying, here's another tool, here's another tool, here's another tool.
00:13:02.191 --> 00:13:07.291
It really should be client-driven, right? The client should be saying, this is what I need.
00:13:07.731 --> 00:13:11.491
Get me a tool that does this, because this is a pain point for me.
00:13:11.591 --> 00:13:13.891
And I think we're not quite there yet, but I think we'll get there.
00:13:13.891 --> 00:13:18.411
Therese: And I think for me, that's really the most important thing.
00:13:19.031 --> 00:13:26.271
There is this minute, I think, and Marcin touched on this, that the AI can do anything.
00:13:26.511 --> 00:13:30.071
Any AI tool can do anything because it's AI, right?
00:13:30.331 --> 00:13:35.011
And I think that's a dangerous thing for people to think because in reality,
00:13:35.271 --> 00:13:39.431
the most effective AI tools are the AI tools that are designed to do a specific thing.
00:13:40.010 --> 00:13:44.670
Right. General use AI tools can be fun and interesting, but they're not usually
00:13:44.670 --> 00:13:47.630
very good at solving the particular problem that you want to solve.
00:13:47.890 --> 00:13:51.970
And so I think that's for most people, it's really important to say you need
00:13:51.970 --> 00:13:56.090
to be focusing on tools that are developed to do the specific thing,
00:13:56.230 --> 00:13:58.870
whether that's a review, whether it's data analysis.
00:13:59.550 --> 00:14:05.010
Whatever it is that you're looking to do, look for the tools that do that and
00:14:05.010 --> 00:14:06.710
definitely test them out.
00:14:06.710 --> 00:14:09.090
Start with testing them, right?
00:14:09.310 --> 00:14:13.370
Because, you know, people are looking to sell, right? That their tools,
00:14:13.530 --> 00:14:16.070
their sell, AI is hot. Everybody wants people to buy.
00:14:16.270 --> 00:14:21.310
And it's really take your time to evaluate the tools to find the ones that are
00:14:21.310 --> 00:14:24.710
custom designed to do what you want to do. They can be incredibly effective.
00:14:24.930 --> 00:14:27.930
It will also be much more efficient and cost effective for you.
00:14:28.250 --> 00:14:31.790
And it avoids the, oh, that didn't work now.
00:14:31.910 --> 00:14:35.630
I never want to use AI again problem. And I think when you're looking at it
00:14:35.630 --> 00:14:41.390
that way, you're much more likely to both get comfortable and to truly find
00:14:41.390 --> 00:14:45.470
the tools that are going to help with the work that you're doing.
00:14:45.910 --> 00:14:49.190
Anthony: Yeah and I think the reality is is for particularly for
00:14:49.190 --> 00:14:52.110
this AI-enabled e-discovery and effective program and
00:14:52.110 --> 00:14:55.650
you probably should have a program and not you know just a you focus
00:14:55.650 --> 00:14:58.930
on a product, but you're probably gonna need lots of tools. And I think that's
00:14:58.930 --> 00:15:01.150
going to be a challenge right because you have to evaluate each of them but
00:15:01.150 --> 00:15:05.710
I think it is it's not going to be I can go to this one vendor for all my AI
00:15:05.710 --> 00:15:08.750
and e-discovery needs. I know the vendors aren't going to do that. But not
00:15:08.750 --> 00:15:12.230
all are going to be good at everything, right? It's impossible so they're
00:15:12.230 --> 00:15:15.430
going to have to focus and prioritized based on what they're hearing from their clients.
00:15:15.570 --> 00:15:17.910
So I think that's one of the things from an in-house person,
00:15:18.170 --> 00:15:22.110
even a law firm, like you have to figure out what tools make sense.
00:15:22.270 --> 00:15:26.850
And it's going to mean you're going to have to have lots of vendors, which is hard to manage.
00:15:27.070 --> 00:15:30.810
But okay, so now let's talk a little bit, Therese, about like,
00:15:30.890 --> 00:15:32.970
how do you avoid getting in trouble when you're getting started,
00:15:33.150 --> 00:15:35.130
right? Everybody wants to use it. There's a rush to use it.
00:15:35.752 --> 00:15:38.632
What are the things, like the high points, and we'll get into them more in the
00:15:38.632 --> 00:15:42.072
podcast series, but what are some key things to think about so you don't get
00:15:42.072 --> 00:15:43.172
in trouble when you're getting stuck?
00:15:43.752 --> 00:15:49.312
Therese: Look, I think the number one thing to always keep in mind to make sure that
00:15:49.312 --> 00:15:55.272
you don't get in trouble is simply to be mindful as your obligations as a lawyer,
00:15:55.712 --> 00:15:59.752
right? I mean, we have obligations as lawyers.
00:15:59.912 --> 00:16:03.952
And if we are following those the way that we should, you're going to be,
00:16:03.952 --> 00:16:08.672
you know, catching some of the potential gotchas that can get you in trouble. So number one, right?
00:16:09.672 --> 00:16:13.272
Confidentiality. We have an obligation to keep our clients' data confidential,
00:16:13.632 --> 00:16:16.672
right? Whether you are in an in-house counsel or an outside counsel.
00:16:16.912 --> 00:16:21.692
When you're looking at these tools, you need to make sure that they will properly
00:16:21.692 --> 00:16:25.992
protect that data, that they're not using that data for something else.
00:16:25.992 --> 00:16:28.152
They don't disclose that data to something else.
00:16:28.332 --> 00:16:30.752
And I think the most obvious one, of course, is.
00:16:31.332 --> 00:16:36.432
Don't use public tools. We're confidential, sensitive data.
00:16:36.752 --> 00:16:40.072
And I think that's number one, right? We are always mindful as lawyers.
00:16:40.312 --> 00:16:44.372
We need to protect our clients' information. You want to ask the same questions
00:16:44.372 --> 00:16:48.272
and do the same thing about any tool that you're using for AI is make sure that
00:16:48.272 --> 00:16:50.952
all the protections in the safe card to ensure that data is used,
00:16:51.152 --> 00:16:53.932
confidential and used properly. Number one.
00:16:54.312 --> 00:16:59.372
Number two, while we're talking about how you should use GenAI, don't trust GenAI.
00:16:59.612 --> 00:17:03.712
That doesn't mean, you know, it's not valuable. It means we have an obligation
00:17:03.712 --> 00:17:08.772
to check everything and to look at the responses we're getting,
00:17:08.892 --> 00:17:13.592
whether that's from a human being or a computer, to make sure that that information is accurate.
00:17:13.832 --> 00:17:18.332
We all know AI has the potential to hallucinate, to provide information that's
00:17:18.332 --> 00:17:22.432
inaccurate. It isn't actually thinking and giving you an analytical answer.
00:17:22.672 --> 00:17:27.132
And so I think always being aware of the fact that you can't trusted on itself.
00:17:27.312 --> 00:17:31.012
You have an obligation to check and to validate that data and to make sure that
00:17:31.012 --> 00:17:34.832
you have mechanisms in place whenever you're using it to make sure that data
00:17:34.832 --> 00:17:36.792
is checked and is validated.
00:17:37.232 --> 00:17:42.672
Being mindful of your obligations for what you can and cannot do in using GenAI.
00:17:42.972 --> 00:17:47.932
Does your client have requirements about whether or not you can or how you can
00:17:47.932 --> 00:17:50.212
use AI? Do they require you to disclose it?
00:17:50.432 --> 00:17:54.532
Does your firm have requirements of how you can use GenAI?
00:17:54.812 --> 00:17:59.252
Make sure you're aware of those obligations and those rules and that you are
00:17:59.252 --> 00:18:00.632
following them very clearly.
00:18:00.832 --> 00:18:05.332
Again, you always need to make sure that if you're using a particular AI tool,
00:18:05.472 --> 00:18:10.892
we see a lot of companies coming out, some saying, no, you can't use it without my permission.
00:18:11.072 --> 00:18:15.312
Some saying you absolutely better be using AI in my matters.
00:18:15.572 --> 00:18:20.172
So it's being mindful of what the parameters are for your organization and your
00:18:20.172 --> 00:18:24.772
client to make sure that whatever use you are doing is consistent with that.
00:18:25.463 --> 00:18:30.983
Disclosures, right? Being mindful of when and if you need to disclose the use.
00:18:31.163 --> 00:18:34.903
And again, whether that use is in a case or with other people involved,
00:18:34.903 --> 00:18:39.743
if the AI is being used on their voices or their information,
00:18:40.003 --> 00:18:45.043
be aware of what the requirements are and whether or not you have an obligation to disclose.
00:18:45.163 --> 00:18:47.963
And if so, what those disclosures may be.
00:18:48.163 --> 00:18:53.283
And I think when you're going down the road is protecting the data.
00:18:53.463 --> 00:18:58.043
Is someone else going to use AI on your client's data?
00:18:58.403 --> 00:19:01.383
And if so, right, think of the other side or the other party.
00:19:01.583 --> 00:19:04.843
Are there parameters you want to set? Do you want information in protective
00:19:04.843 --> 00:19:09.283
orders and the like that are going to protect the use of that data?
00:19:09.703 --> 00:19:13.303
And then I think finally, and this is what, you know, tripped everyone else
00:19:13.303 --> 00:19:15.643
up quite a bit with TAR is validation.
00:19:16.223 --> 00:19:23.143
If you are using AI, what are your validation mechanisms for defensibility purposes?
00:19:23.143 --> 00:19:27.343
If you are going to use it in a case and you are going to have to defend it,
00:19:27.483 --> 00:19:31.363
you have to be able to say not just, I was using AI and it's fine and I have
00:19:31.363 --> 00:19:33.123
a really good vendor and it's a really good tool.
00:19:33.363 --> 00:19:38.263
You have to say, and here's the mechanism I use to validate the results so I
00:19:38.263 --> 00:19:43.223
know that it's working and that my response, my production, whatever it may
00:19:43.223 --> 00:19:45.643
be met my legal obligations.
00:19:46.263 --> 00:19:50.183
Marcin: So I'm going to go backwards then because I'm going to start where Therese ended.
00:19:50.643 --> 00:19:55.743
You don't want to become a headline. Check your sites, check your sites, check your sites.
00:19:55.983 --> 00:20:00.643
It is the easiest thing that you can do when you're using AI,
00:20:00.883 --> 00:20:04.883
and yet it's the number one thing that will land you in the news.
00:20:05.323 --> 00:20:09.983
From there, things get a little more subtle because we only hear in the news
00:20:09.983 --> 00:20:12.983
about hallucinations when it comes to things like sites.
00:20:13.283 --> 00:20:18.063
But AI has the tendency to come up with incorrect conclusions,
00:20:18.543 --> 00:20:20.823
the summaries may be slightly off.
00:20:21.502 --> 00:20:26.242
You have to know how your tool works in order to be able to get an inherent
00:20:26.242 --> 00:20:31.442
sense of when it's not doing what it needs to be doing. That comes with practice and familiarity.
00:20:31.802 --> 00:20:37.562
But ultimately, attorneys are skeptical. And when you use AI at any level,
00:20:37.782 --> 00:20:40.622
the level of validation that you need, of course, depends on the task and the
00:20:40.622 --> 00:20:43.242
kind of AI you're using. But remain skeptical.
00:20:43.902 --> 00:20:49.562
Automation bias, our inherent desire to trust computers, is the number one way
00:20:49.562 --> 00:20:51.722
that you fail to validate.
00:20:52.402 --> 00:20:57.642
So we can talk a lot about validation in future podcasts, but that's how I'm
00:20:57.642 --> 00:20:59.862
going to end my little rant here.
00:21:00.082 --> 00:21:04.542
The easy thing is check your sites and everything else from there becomes a
00:21:04.542 --> 00:21:07.082
lot more nuanced, but remain skeptical.
00:21:07.622 --> 00:21:11.902
Learn the way your tool works so that you have a sense of when things aren't
00:21:11.902 --> 00:21:13.062
going the way they're supposed to be.
00:21:13.402 --> 00:21:16.362
Anthony: Yeah. And I'm going to go back to, and this is, I think one of the lessons learned
00:21:16.362 --> 00:21:20.602
from Tar and the use of Tar is a lot of people were scarred.
00:21:20.802 --> 00:21:24.642
It's people process technology, right? You have to have people who understand it.
00:21:24.862 --> 00:21:28.662
That's number one. But process is the one thing that people forget all the time.
00:21:28.942 --> 00:21:32.622
Validation is part of it. But we saw with Tar where you were like,
00:21:32.682 --> 00:21:36.542
oh, we're going to use TAR. It's going to save you all this money. Again, whether you're
00:21:36.869 --> 00:21:41.769
AI-enabled e-discovery saves money. It remains to be seen, but it should be
00:21:41.769 --> 00:21:44.509
more effective, right? And it's the effectiveness that matters.
00:21:44.949 --> 00:21:47.809
Maybe there's cost savings associated with it, but in the end,
00:21:47.969 --> 00:21:50.069
you want a process that makes sense.
00:21:50.329 --> 00:21:54.969
One of the things we saw with TAR was there wasn't developed processes around
00:21:54.969 --> 00:21:59.329
it, not just validation, but things like we load it in, we do all this training,
00:21:59.369 --> 00:22:01.789
and then there's another set of custodians you can load in.
00:22:01.929 --> 00:22:04.709
And suddenly it's like, oh, now we have to do the training again. And people are like,
00:22:04.749 --> 00:22:07.989
well, that's not what you sold me it's not costing no money so
00:22:07.989 --> 00:22:10.809
I think really going through as you're
00:22:10.809 --> 00:22:14.129
doing and figuring out how I'm going to use AI is developing
00:22:14.129 --> 00:22:16.989
what the process is going to be to make sure you're dealing
00:22:16.989 --> 00:22:21.409
with all these anomalies so that everyone is aware of what's going to happen
00:22:21.409 --> 00:22:25.669
when something happens right and I think that's that's key too so I think as
00:22:25.669 --> 00:22:28.829
we talk and we're going to have a podcast series on on the podcast series we're
00:22:28.829 --> 00:22:31.309
going to talk about Tar because I think there's a lot of good lessons learned
00:22:31.309 --> 00:22:34.829
about what we did in Tar and how we can apply it to these.
00:22:35.029 --> 00:22:39.809
So thank you, Therese and Marcin. I think this was a good episode that sort
00:22:39.809 --> 00:22:43.569
of started us off and laid the groundwork for our series.
00:22:43.709 --> 00:22:49.469
So I hope everyone joins us as we go through this journey and talk about AI-enabled
00:22:49.469 --> 00:22:53.489
e-discovery over the next few months and years. Thank you all. Marcin: Thank you.
00:22:55.789 --> 00:23:00.189
Outro: Tech Law Talks is a Reed Smith production. Our producers are Ali McCardell and
00:23:00.189 --> 00:23:04.309
Shannon Ryan. For more information about Reed Smith's Emerging Technologies
00:23:04.309 --> 00:23:07.489
Practice, please email techlawtalks@reedsmith.com.
00:23:08.602 --> 00:23:12.502
You can find our podcast on Spotify, Apple Podcasts, Google Podcasts,
00:23:12.682 --> 00:23:15.062
Readsmith.com, and our social media accounts.
00:23:15.762 --> 00:23:19.962
Disclaimer: This podcast is provided for educational purposes. It does not constitute legal
00:23:19.962 --> 00:23:23.242
advice and is not intended to establish an attorney-client relationship,
00:23:23.242 --> 00:23:27.402
nor is it intended to suggest or establish standards of care applicable to particular
00:23:27.402 --> 00:23:29.102
lawyers in any given situation.
00:23:29.562 --> 00:23:33.282
Prior results do not guarantee a similar outcome. Any views,
00:23:33.402 --> 00:23:37.222
opinions, or comments made by any external guest speaker are not to be attributed
00:23:37.222 --> 00:23:39.902
to Reed Smith LLP or its individual lawyers.
00:23:40.202 --> 00:23:42.402
All rights reserved.