תעתיק 🔗
WEBVTT
00:00:05.203 --> 00:00:06.006
We'll be right back.
00:00:21.234 --> 00:00:21.556
Hey,
00:00:21.635 --> 00:00:23.797
welcome to another episode of Cloud
00:00:23.818 --> 00:00:24.379
Unplugged.
00:00:24.518 --> 00:00:27.983
We're doing a bit of a different episode.
00:00:28.263 --> 00:00:30.045
We're going to be discussing whether
00:00:30.106 --> 00:00:31.788
models should be ethical.
00:00:32.988 --> 00:00:35.813
The reason for this is to do with
00:00:35.853 --> 00:00:37.954
some of the UN report that's kind of
00:00:37.994 --> 00:00:38.735
come out around
00:00:39.616 --> 00:00:42.139
um mental health crises related to chat
00:00:42.158 --> 00:00:45.941
bots and in some cases deaths linked to
00:00:45.960 --> 00:00:49.423
that and what that really means to ai
00:00:49.764 --> 00:00:51.884
models how ethical they should or
00:00:51.905 --> 00:00:53.405
shouldn't be um
00:00:54.463 --> 00:00:56.524
and what morality and ethics really do
00:00:56.784 --> 00:01:00.627
mean in general and ways of approaching it
00:01:01.127 --> 00:01:02.688
from a model training perspective that's
00:01:02.707 --> 00:01:05.530
kind of going on in the industry um
00:01:05.629 --> 00:01:09.052
obviously this is something you really
00:01:09.072 --> 00:01:11.813
have been inspired to talk about salmon so
00:01:11.834 --> 00:01:14.296
i'm gonna hand over more to you to
00:01:14.715 --> 00:01:17.317
to discuss it but i think before we
00:01:17.337 --> 00:01:20.280
get into it i guess to be ethical
00:01:21.420 --> 00:01:23.121
what what is the definition of uh
00:01:24.522 --> 00:01:27.263
of ethics here that we're kind of talking
00:01:27.283 --> 00:01:27.522
about.
00:01:28.763 --> 00:01:29.683
Well, John,
00:01:30.103 --> 00:01:32.563
over the last hundreds of centuries,
00:01:32.603 --> 00:01:35.364
people have decided to define what ethics
00:01:35.484 --> 00:01:35.644
is.
00:01:35.704 --> 00:01:38.066
People like Confucius, Aristotle, Pluto.
00:01:38.445 --> 00:01:40.626
But John, you know, I'm a simple man,
00:01:40.846 --> 00:01:42.027
so I'm going to keep it simple.
00:01:42.106 --> 00:01:43.408
I'm not going to go into virtues or
00:01:43.427 --> 00:01:44.067
anything like that.
00:01:44.768 --> 00:01:45.629
All I'm going to say,
00:01:45.929 --> 00:01:47.989
and I'm sure you will disagree with me,
00:01:48.030 --> 00:01:48.310
John,
00:01:48.829 --> 00:01:51.391
is ethics is a set of principles that
00:01:51.432 --> 00:01:54.233
decide what counts as right and wrong
00:01:54.253 --> 00:01:55.974
behavior and how you should weigh
00:01:56.013 --> 00:01:58.594
competing good when they conflict,
00:01:59.555 --> 00:02:02.016
like honesty against kindness or
00:02:02.057 --> 00:02:03.878
individual freedom against collective
00:02:03.918 --> 00:02:04.257
harm.
00:02:04.938 --> 00:02:06.859
So it's not a fixed checklist, John.
00:02:06.879 --> 00:02:08.180
There's no right or wrong answer.
00:02:08.199 --> 00:02:10.602
It's the framework or reasoning through
00:02:10.641 --> 00:02:12.723
situations nobody wrote a specific rule
00:02:12.742 --> 00:02:12.902
for.
00:02:14.322 --> 00:02:16.483
So that is what ethics is.
00:02:16.522 --> 00:02:18.204
I'm not going to touch upon what AI
00:02:18.223 --> 00:02:18.503
is.
00:02:18.604 --> 00:02:19.324
I'm sure, John,
00:02:19.365 --> 00:02:21.665
you have your own definition of what
00:02:21.705 --> 00:02:22.186
ethics is,
00:02:22.225 --> 00:02:24.206
just a set of principles that you can
00:02:24.447 --> 00:02:27.709
use to decide what counts as a right
00:02:27.829 --> 00:02:28.788
or the wrong behavior,
00:02:29.008 --> 00:02:30.810
given the circumstances.
00:02:30.830 --> 00:02:32.830
Yeah.
00:02:32.871 --> 00:02:33.131
I mean,
00:02:33.390 --> 00:02:37.393
I know it's a set of beliefs that
00:02:37.453 --> 00:02:39.674
people might hold about what's acceptable
00:02:39.694 --> 00:02:41.235
and what isn't, what's good, what's bad.
00:02:43.877 --> 00:02:46.218
I'm not sure we've reached any universal
00:02:46.278 --> 00:02:48.560
global standard ever on this.
00:02:49.640 --> 00:02:50.542
And if we had,
00:02:50.722 --> 00:02:53.644
there probably would be less wars and less
00:02:53.683 --> 00:02:55.425
disagreements in general if all of our
00:02:55.485 --> 00:02:56.406
ethics all align.
00:02:56.445 --> 00:03:00.449
So I guess it's quite a tricky thing
00:03:00.489 --> 00:03:02.330
to solve just in general.
00:03:04.651 --> 00:03:05.793
My opinion is,
00:03:06.913 --> 00:03:10.156
is it fair if we haven't solved it
00:03:10.656 --> 00:03:12.076
as a human race collectively?
00:03:12.116 --> 00:03:12.798
We might have some
00:03:13.294 --> 00:03:14.235
maybe some agreed standards.
00:03:14.336 --> 00:03:15.396
I wouldn't even know what they are.
00:03:17.360 --> 00:03:20.044
So if we don't know what they are,
00:03:20.064 --> 00:03:21.247
or I don't really know what they are,
00:03:22.609 --> 00:03:24.992
then I'm not sure a model would either.
00:03:25.052 --> 00:03:26.574
So how can a model...
00:03:27.489 --> 00:03:29.669
become ethically aware is what I want
00:03:30.770 --> 00:03:33.372
to... That's my little thing.
00:03:33.413 --> 00:03:34.493
Not to disagree with you.
00:03:39.096 --> 00:03:40.616
As an ethics denier, John,
00:03:40.758 --> 00:03:41.837
all I'm going to say to you...
00:03:41.918 --> 00:03:44.659
So if I said to you,
00:03:44.680 --> 00:03:49.562
I was like, those headphones of yours,
00:03:50.403 --> 00:03:51.664
they are... I love them.
00:03:52.724 --> 00:03:54.406
I really, really like them.
00:03:55.907 --> 00:03:56.867
And it made you feel good.
00:03:57.580 --> 00:03:58.741
You're like, oh, my God,
00:03:58.842 --> 00:04:01.683
I really feel does that make me unethical
00:04:01.984 --> 00:04:02.683
if I've lied about it?
00:04:02.704 --> 00:04:06.266
I'm not saying just am I unethical to
00:04:06.306 --> 00:04:07.848
Load full transcript
say that to you if it's made you
00:04:07.888 --> 00:04:08.328
feel good?
00:04:09.729 --> 00:04:10.748
John, tell me one thing.
00:04:11.590 --> 00:04:12.430
When you were growing up,
00:04:12.629 --> 00:04:17.733
what did your mom tell you about lying?
00:04:17.754 --> 00:04:19.235
She said, do it.
00:04:21.117 --> 00:04:22.098
She said, whatever you do,
00:04:22.158 --> 00:04:24.721
you must do it to win.
00:04:25.081 --> 00:04:25.642
She doesn't really.
00:04:29.447 --> 00:04:30.608
Yeah, you're right.
00:04:30.990 --> 00:04:31.690
You did lie.
00:04:32.091 --> 00:04:35.214
Perhaps maybe the headphones are too big.
00:04:35.855 --> 00:04:37.997
But it's not just one checklist, John.
00:04:38.016 --> 00:04:39.237
There's a bunch of things that you've
00:04:39.257 --> 00:04:40.517
learned and accrued over time.
00:04:40.557 --> 00:04:43.259
And people do know how to behave ethically
00:04:43.379 --> 00:04:45.300
given their moral compass,
00:04:45.579 --> 00:04:47.420
moral compass of the time they're in,
00:04:47.841 --> 00:04:49.261
the environment they're in,
00:04:49.581 --> 00:04:50.562
the culture they're in.
00:04:51.081 --> 00:04:52.161
So yes,
00:04:54.142 --> 00:04:55.762
one person being ethical doesn't
00:04:55.884 --> 00:04:57.903
necessarily translate exactly like other
00:04:58.163 --> 00:04:59.064
person being ethical.
00:04:59.545 --> 00:05:00.964
Two people given the same thing,
00:05:01.165 --> 00:05:03.565
they might pick different outcomes in a
00:05:03.605 --> 00:05:05.166
situation that they're given in.
00:05:05.766 --> 00:05:06.747
I don't think we can say, oh,
00:05:06.767 --> 00:05:07.869
we don't know how to be ethical.
00:05:08.048 --> 00:05:09.088
We do know how to be ethical,
00:05:09.168 --> 00:05:10.990
but the meaning and the definition of
00:05:11.110 --> 00:05:13.172
ethics between two people can slightly
00:05:13.211 --> 00:05:13.572
differ.
00:05:15.452 --> 00:05:18.694
But how do you because I'm curious to
00:05:18.735 --> 00:05:21.956
see before we get into the mechanics of
00:05:22.057 --> 00:05:26.980
it, as in, I guess,
00:05:27.019 --> 00:05:28.480
to suggest that it should be.
00:05:29.752 --> 00:05:30.713
Is that a model?
00:05:31.473 --> 00:05:33.055
Do you think the model really needs to
00:05:33.076 --> 00:05:33.675
be ethical?
00:05:33.795 --> 00:05:37.658
Or do you think the surrounding product of
00:05:37.699 --> 00:05:40.560
the model should be designed in a way
00:05:40.620 --> 00:05:42.163
that actually caters for the fact that it
00:05:42.283 --> 00:05:43.223
isn't ethical?
00:05:45.725 --> 00:05:48.447
If somebody's in crisis, say,
00:05:48.567 --> 00:05:52.550
or it's talking to the model and it
00:05:53.579 --> 00:05:55.680
you know, it's some fourteen,
00:05:55.720 --> 00:05:56.021
I don't know,
00:05:56.040 --> 00:05:57.180
just for like argument's sake,
00:05:57.201 --> 00:05:58.021
some fourteen year old,
00:05:58.041 --> 00:05:59.983
they're having a crisis,
00:06:00.002 --> 00:06:00.663
talking to the model.
00:06:00.944 --> 00:06:02.725
Should the model not be designed to
00:06:02.985 --> 00:06:05.867
integrate into features that maybe has a
00:06:05.927 --> 00:06:08.267
handoff to some crisis management thing
00:06:08.288 --> 00:06:09.668
that can maybe do, like,
00:06:09.749 --> 00:06:12.930
why does the model need to understand
00:06:12.951 --> 00:06:13.110
that?
00:06:13.190 --> 00:06:14.732
Why can't it just be designed in a
00:06:14.771 --> 00:06:18.134
way that the product as a whole has
00:06:18.153 --> 00:06:20.636
to think about that, not just the model?
00:06:23.521 --> 00:06:23.742
I mean,
00:06:23.781 --> 00:06:28.966
why are you putting AI in the spotlight?
00:06:29.605 --> 00:06:31.927
That goes for everything, John.
00:06:31.968 --> 00:06:34.110
You can pick a weapon and say,
00:06:34.129 --> 00:06:37.391
how does a person decide to use a
00:06:37.432 --> 00:06:38.312
weapon on something that's
00:06:39.012 --> 00:06:39.473
That's wrong.
00:06:39.754 --> 00:06:42.815
How can you use a computer to hack
00:06:42.915 --> 00:06:43.855
somebody else's computer,
00:06:43.915 --> 00:06:44.976
which is unethical, right?
00:06:45.016 --> 00:06:45.718
In some cases,
00:06:46.278 --> 00:06:47.759
how do we stop that from happening?
00:06:48.319 --> 00:06:49.901
Or people smoking, right?
00:06:50.180 --> 00:06:52.543
Some people think that's unethical for you
00:06:52.562 --> 00:06:54.064
to smoke because it harms others as well
00:06:54.084 --> 00:06:54.543
when you do them.
00:06:54.903 --> 00:06:56.404
How do you stop that from happening, John?
00:06:56.826 --> 00:06:57.685
So, yes,
00:06:57.946 --> 00:07:00.348
it might seem like I'm being a bit
00:07:00.387 --> 00:07:01.608
of an AI model fan here.
00:07:02.009 --> 00:07:04.550
But I mean, how do we do that?
00:07:04.630 --> 00:07:07.012
I mean, the important thing, John,
00:07:07.533 --> 00:07:11.237
for this discussion is that models,
00:07:11.437 --> 00:07:12.997
especially we'll talk about the AI report
00:07:13.459 --> 00:07:16.980
in a second, that the models for them,
00:07:17.021 --> 00:07:19.403
it matters to be helpful, right?
00:07:19.463 --> 00:07:21.564
But that alone doesn't solve the problem.
00:07:21.865 --> 00:07:23.387
Because when a user wants to do something
00:07:23.427 --> 00:07:24.266
that's legal,
00:07:24.927 --> 00:07:27.290
but it is harmful to themselves or someone
00:07:27.329 --> 00:07:29.250
else, that's where the model should be.
00:07:29.971 --> 00:07:31.692
try and perhaps try and stop or not
00:07:32.252 --> 00:07:34.653
if you google search uh building something
00:07:34.692 --> 00:07:38.173
for example that that is legal but not
00:07:38.233 --> 00:07:41.514
right you might get some responses back so
00:07:41.555 --> 00:07:42.855
why are we treating the air models
00:07:42.875 --> 00:07:46.156
slightly differently but
anyway that's uh
00:07:46.196 --> 00:07:47.377
is that what so this is it so
00:07:47.396 --> 00:07:48.817
i guess just just yeah just just kind
00:07:48.836 --> 00:07:50.576
of get a bit more defined you know
00:07:50.596 --> 00:07:52.338
that's all i'm trying to do let's pin
00:07:52.358 --> 00:07:55.319
it down into something concrete so that
00:07:55.338 --> 00:07:55.798
the next
00:07:56.538 --> 00:07:59.199
things make more sense so this is to
00:07:59.240 --> 00:08:02.701
do obviously still to do with the the
00:08:02.762 --> 00:08:07.024
sycophantic ai chat bots that are
00:08:07.184 --> 00:08:10.446
predicated on affirmation almost of giving
00:08:11.026 --> 00:08:15.449
a a positive answer of or some endorsement
00:08:15.468 --> 00:08:17.369
that it's doing this job well right i
00:08:17.408 --> 00:08:21.992
suppose um essentially um and because of
00:08:22.052 --> 00:08:22.252
that
00:08:24.492 --> 00:08:27.235
the conclusion is that that can cause
00:08:28.355 --> 00:08:32.839
problems in how it's responding to maybe
00:08:32.879 --> 00:08:36.302
vulnerable people or people in crisis or
00:08:36.341 --> 00:08:38.244
people a little bit more vulnerable in
00:08:38.303 --> 00:08:39.124
general in life,
00:08:39.364 --> 00:08:40.565
depending on what they're talking about.
00:08:40.865 --> 00:08:42.486
And maybe the guidelines or guidance that
00:08:42.527 --> 00:08:43.567
it's giving to someone isn't.
00:08:44.793 --> 00:08:47.354
correct in that circumstance and had an
00:08:47.433 --> 00:08:50.695
expert of been answering those questions
00:08:50.715 --> 00:08:51.875
they would have handled it completely
00:08:51.914 --> 00:08:55.576
differently based on the context of what's
00:08:55.616 --> 00:08:58.658
been said is that the fundamentals of it
00:08:58.758 --> 00:08:59.018
is it
00:09:00.120 --> 00:09:01.340
Yeah, that's the fundamentals.
00:09:01.419 --> 00:09:02.740
And for example,
00:09:02.760 --> 00:09:04.461
we'll give you an example from this UN
00:09:04.481 --> 00:09:05.542
report that just came out.
00:09:06.182 --> 00:09:08.582
In the last couple of days, with chatbot,
00:09:08.623 --> 00:09:11.043
you touched upon a psychophancy linked to
00:09:11.104 --> 00:09:14.264
death, as in telling them reaffirmations.
00:09:14.345 --> 00:09:16.586
So there was a testimony described in
00:09:16.605 --> 00:09:18.605
there for a fourteen year old boy who
00:09:18.645 --> 00:09:20.047
got deeply attached to a chatbot.
00:09:20.933 --> 00:09:23.176
when he told that it was that he
00:09:23.196 --> 00:09:26.177
was in severe distress it didn't say that
00:09:26.217 --> 00:09:28.000
it was an ai the agent didn't say
00:09:28.019 --> 00:09:30.241
it was an eye didn't suggest that he
00:09:30.282 --> 00:09:31.962
should get some help or didn't flag it
00:09:32.342 --> 00:09:35.245
to somebody else right so this psycho
00:09:35.285 --> 00:09:37.287
fancy is not just some bug that got
00:09:37.326 --> 00:09:39.788
slipped into the model it's because the
00:09:39.849 --> 00:09:42.751
default behavior of the model is optimized
00:09:42.792 --> 00:09:43.871
to be purely liked
00:09:44.732 --> 00:09:45.874
And it rewards the model on,
00:09:46.053 --> 00:09:48.215
did the user approve this response or not?
00:09:48.894 --> 00:09:50.875
And we'll learn to flatter people unless
00:09:51.437 --> 00:09:54.018
something specifically pushes against what
00:09:54.057 --> 00:09:54.977
they're trying to do.
00:09:55.678 --> 00:09:56.359
So that's what it is.
00:09:56.418 --> 00:09:58.440
So this is an example, John, of like,
00:09:58.519 --> 00:09:59.541
oh, is that ethical or not?
00:09:59.561 --> 00:10:01.461
And perhaps that gives you an answer.
00:10:01.501 --> 00:10:02.062
Well, of course,
00:10:02.081 --> 00:10:03.163
we can talk about a bunch of other
00:10:03.182 --> 00:10:03.423
things.
00:10:03.942 --> 00:10:07.124
So the models are poorly designed.
00:10:07.203 --> 00:10:07.403
I mean,
00:10:07.423 --> 00:10:09.404
we'll talk about the model training
00:10:09.725 --> 00:10:11.644
specifically in terms of how other
00:10:11.705 --> 00:10:14.306
companies are trying to bake in ethics and
00:10:14.365 --> 00:10:15.886
how philosophers are helping with that.
00:10:16.006 --> 00:10:16.506
I know, John,
00:10:16.586 --> 00:10:18.307
we still don't have an answer on what
00:10:18.326 --> 00:10:18.967
the ethics is,
00:10:19.048 --> 00:10:21.687
but people are trying to implement some
00:10:21.727 --> 00:10:22.448
guardrails, right?
00:10:22.568 --> 00:10:24.808
And basically,
00:10:25.048 --> 00:10:26.549
the issue that Ewan is saying,
00:10:26.590 --> 00:10:30.171
that the models cannot recognize that
00:10:30.890 --> 00:10:33.091
there's a mental health crisis when
00:10:33.131 --> 00:10:35.253
somebody is talking to them and it doesn't
00:10:35.273 --> 00:10:37.173
disclose that, you know, oh,
00:10:37.212 --> 00:10:38.693
you actually talk to AI chatbot.
00:10:39.134 --> 00:10:40.474
I don't really, perhaps I'm not right.
00:10:40.855 --> 00:10:42.495
The best thing for you to chat to
00:10:42.514 --> 00:10:44.615
me about, maybe call Samaritans,
00:10:44.655 --> 00:10:46.176
maybe speak to somebody that you know,
00:10:46.196 --> 00:10:47.657
maybe get some medical health.
00:10:48.397 --> 00:10:50.518
So I think this is where it is,
00:10:50.577 --> 00:10:50.778
right?
00:10:51.138 --> 00:10:53.938
To start with when you build these models,
00:10:53.958 --> 00:10:54.798
when you train these models,
00:10:54.839 --> 00:10:56.299
that's what you get to start with.
00:10:58.248 --> 00:10:59.827
And your position is that they should be
00:10:59.868 --> 00:11:01.688
ethical, more ethical than they are.
00:11:02.269 --> 00:11:06.029
So you think the current training...
00:11:06.870 --> 00:11:09.010
Do you think that the model itself should
00:11:09.030 --> 00:11:09.591
be more ethical?
00:11:09.610 --> 00:11:10.892
Do you think the companies behind the...
00:11:11.052 --> 00:11:13.591
Do you think it's a company issue, i.e.
00:11:13.611 --> 00:11:15.113
the companies aren't being very ethical?
00:11:15.192 --> 00:11:18.214
Because the answer is one of the solutions
00:11:18.254 --> 00:11:19.193
to it is the model.
00:11:20.134 --> 00:11:20.894
Yeah.
00:11:20.933 --> 00:11:23.115
But that doesn't mean there isn't more
00:11:23.134 --> 00:11:24.615
than one answer to the problem, right?
00:11:24.715 --> 00:11:26.155
I guess that's what I'm teasing out here.
00:11:26.275 --> 00:11:26.535
Is it...
00:11:27.448 --> 00:11:29.730
Is there a duty mode for companies?
00:11:30.312 --> 00:11:31.091
Because by proxy,
00:11:31.111 --> 00:11:32.452
what you're really saying is the people
00:11:32.472 --> 00:11:33.995
building these models are not very
00:11:34.034 --> 00:11:34.375
ethical.
00:11:34.394 --> 00:11:37.898
I know you're not saying that explicitly,
00:11:37.937 --> 00:11:39.139
but I'm saying that they're putting
00:11:39.178 --> 00:11:41.681
something in the hands of people without
00:11:41.741 --> 00:11:43.722
necessarily... Maybe that's unethical.
00:11:44.183 --> 00:11:46.125
To be putting unethical things in the
00:11:46.144 --> 00:11:47.947
hands of people is an unethical thing to
00:11:47.966 --> 00:11:49.008
do as a company.
00:11:49.488 --> 00:11:50.990
So maybe you start with the companies
00:11:51.029 --> 00:11:51.289
being...
00:11:52.398 --> 00:11:55.159
more ethical in how they approach it.
00:11:55.179 --> 00:11:56.600
That may or may not pertain to the
00:11:56.639 --> 00:11:57.461
model being changed.
00:11:57.480 --> 00:11:58.802
It could be something else that they can
00:11:58.822 --> 00:11:59.341
come up with.
00:11:59.361 --> 00:12:00.743
It could be safeguard.
00:12:00.763 --> 00:12:02.003
It could be age restricting.
00:12:02.023 --> 00:12:03.543
You know what I mean?
00:12:03.563 --> 00:12:05.264
There's maybe lots of different things
00:12:05.284 --> 00:12:05.926
that they could do.
00:12:07.725 --> 00:12:07.826
Yeah.
00:12:07.927 --> 00:12:09.386
I mean, John, look,
00:12:10.087 --> 00:12:11.548
massive props to these companies.
00:12:11.568 --> 00:12:13.309
They've got this amazing technology out
00:12:13.330 --> 00:12:15.931
that is helping people in lots of places
00:12:15.971 --> 00:12:16.431
that it can.
00:12:17.091 --> 00:12:19.633
And it's an evolving field.
00:12:19.893 --> 00:12:21.214
Every day there's changes.
00:12:23.168 --> 00:12:26.711
social media when it came out right today
00:12:27.350 --> 00:12:28.991
everybody can get on social media anybody
00:12:29.011 --> 00:12:30.852
can create an account like even before
00:12:31.153 --> 00:12:32.754
there was no age restrictions on who can
00:12:32.793 --> 00:12:35.775
create what account in uk we we will
00:12:35.796 --> 00:12:38.037
have under-sixteens ban on social media i
00:12:38.057 --> 00:12:40.258
know john you're against that ban but i
00:12:40.298 --> 00:12:42.198
am for that ban but there is a
00:12:42.239 --> 00:12:43.840
reason right so that decision has been
00:12:43.879 --> 00:12:46.181
made but social media has been around for
00:12:46.221 --> 00:12:48.142
like fifteen sixteen years now at least
00:12:49.399 --> 00:12:51.722
And we're still trying to figure out how
00:12:51.743 --> 00:12:52.144
to do this.
00:12:52.183 --> 00:12:53.745
So, John,
00:12:53.785 --> 00:12:54.988
I'm not trying to say that companies are
00:12:55.008 --> 00:12:55.509
unethical,
00:12:56.409 --> 00:12:58.293
that sometimes companies are under
00:12:58.332 --> 00:12:58.813
pressure.
00:12:59.394 --> 00:12:59.875
And, you know,
00:12:59.894 --> 00:13:01.817
there's a lot of competition out there at
00:13:01.837 --> 00:13:02.318
the moment.
00:13:02.698 --> 00:13:03.780
Who's got the best model?
00:13:04.140 --> 00:13:05.523
You know, Anthropic has it or...
00:13:06.144 --> 00:13:08.225
Claude, you know, like Chad GPT has it.
00:13:08.565 --> 00:13:10.125
So companies are sometimes under pressure,
00:13:10.144 --> 00:13:10.664
severe pressure.
00:13:10.884 --> 00:13:13.206
This does take time to do extra work
00:13:13.245 --> 00:13:15.206
to bake in these ethical bits,
00:13:15.287 --> 00:13:16.246
which we're going to talk about.
00:13:16.267 --> 00:13:18.647
If a company starts selling cigarettes to
00:13:18.706 --> 00:13:19.967
a child, is that unethical?
00:13:20.628 --> 00:13:22.087
If they're under pressure to make money,
00:13:22.107 --> 00:13:23.068
would that be unethical?
00:13:23.708 --> 00:13:24.408
Well, yeah,
00:13:24.469 --> 00:13:26.370
companies will sell cigarettes to a child.
00:13:26.409 --> 00:13:27.110
Who stops them?
00:13:27.370 --> 00:13:28.769
The government.
00:13:28.789 --> 00:13:31.630
They have under-age restrictions that you
00:13:31.650 --> 00:13:33.751
can't sell it to anybody under-age, right?
00:13:34.032 --> 00:13:35.131
So the companies will do that.
00:13:36.293 --> 00:13:37.575
Because they just say, oh,
00:13:37.595 --> 00:13:38.394
this is the market.
00:13:38.514 --> 00:13:39.056
And of course,
00:13:39.456 --> 00:13:40.956
everybody will point to saying, oh,
00:13:40.976 --> 00:13:43.337
there's no concrete evidence that smoking.
00:13:43.357 --> 00:13:45.298
Just bring out sixteen year old friendly
00:13:45.339 --> 00:13:45.960
cigarettes.
00:13:46.340 --> 00:13:48.520
Do you know what I mean?
00:13:50.422 --> 00:13:52.484
Let's just get some better designed
00:13:52.543 --> 00:13:54.024
cigarettes that are more suitable for
00:13:54.043 --> 00:13:54.384
kids.
00:13:54.904 --> 00:13:57.767
And that's really just
00:13:58.486 --> 00:14:00.229
Just to add, John,
00:14:00.668 --> 00:14:02.230
I know you're talking about smoking now,
00:14:02.250 --> 00:14:04.831
but just on that topic of why it
00:14:04.871 --> 00:14:08.696
is important to have guardrails.
00:14:08.755 --> 00:14:09.855
Let's say I know you don't like the
00:14:09.895 --> 00:14:10.076
word.
00:14:11.498 --> 00:14:12.839
To be fair, you know me,
00:14:12.918 --> 00:14:13.740
I'm very facetious.
00:14:13.799 --> 00:14:14.340
No, it's good.
00:14:14.760 --> 00:14:19.745
I do think even with the banning,
00:14:19.904 --> 00:14:21.426
I respect the fact that someone's doing
00:14:21.505 --> 00:14:22.886
something with the child bands when
00:14:22.907 --> 00:14:23.067
there's
00:14:24.105 --> 00:14:26.647
know mental health issues and problems in
00:14:26.667 --> 00:14:29.769
systemic social problems you know how
00:14:29.809 --> 00:14:31.370
people socialize how they're interacting
00:14:31.451 --> 00:14:34.673
socially um so someone's got to do
00:14:34.754 --> 00:14:36.975
something so at least the band is doing
00:14:37.035 --> 00:14:38.397
something right it's better than just
00:14:38.437 --> 00:14:40.638
being complicit and allowing it so same
00:14:40.678 --> 00:14:44.181
with this like we shouldn't do nothing you
00:14:44.201 --> 00:14:47.923
know yes we should do something um even
00:14:47.964 --> 00:14:49.466
if it is getting the model
00:14:50.293 --> 00:14:51.073
to be more ethical,
00:14:51.114 --> 00:14:52.254
whatever that kind of means,
00:14:52.294 --> 00:14:53.197
to be more ethical,
00:14:53.256 --> 00:14:54.677
if it's just some basic rules,
00:14:55.458 --> 00:14:57.081
some basic rules to start with and then
00:14:57.100 --> 00:14:57.581
figure it out.
00:14:57.642 --> 00:14:59.403
I don't know.
00:14:59.423 --> 00:15:01.326
And on the topic of not doing anything,
00:15:01.586 --> 00:15:03.389
let's take Grok for an example to start
00:15:03.408 --> 00:15:04.169
with, just so you know.
00:15:06.873 --> 00:15:08.053
I knew you were going to go there.
00:15:08.293 --> 00:15:09.416
Was it because of the bikini?
00:15:10.162 --> 00:15:12.043
All those bikini pictures of you,
00:15:12.263 --> 00:15:14.224
is that what you've got beef about now?
00:15:14.984 --> 00:15:16.605
When I asked you what model you're using,
00:15:16.625 --> 00:15:17.544
you told me it was Grok.
00:15:17.565 --> 00:15:18.826
I was like, all right,
00:15:18.905 --> 00:15:21.407
I have to talk about this now.
00:15:21.466 --> 00:15:23.388
You shouldn't have fessed up that they use
00:15:23.408 --> 00:15:23.628
Grok.
00:15:23.868 --> 00:15:24.769
Could have just said Claude,
00:15:24.948 --> 00:15:26.068
even though it doesn't have any image
00:15:26.428 --> 00:15:27.269
building capability.
00:15:27.289 --> 00:15:28.210
Just could have said Claude.
00:15:29.410 --> 00:15:30.910
It's the only model to use.
00:15:30.932 --> 00:15:33.613
If you want to go full-blown, you know,
00:15:34.952 --> 00:15:36.874
kind of a controversy model,
00:15:36.913 --> 00:15:38.075
that's what you kind of go for,
00:15:38.134 --> 00:15:38.434
isn't it?
00:15:39.456 --> 00:15:40.379
Yeah, so Grok,
00:15:40.460 --> 00:15:41.724
let's just quickly do that and then we'll
00:15:41.764 --> 00:15:43.831
talk about how it actually works in terms
00:15:43.890 --> 00:15:45.876
of baking these guardrails.
00:15:46.948 --> 00:15:48.950
So Grog, again, this week,
00:15:49.090 --> 00:15:50.890
everything is fresh off the press, John.
00:15:52.010 --> 00:15:53.471
We are Cloud Unplugged.
00:15:53.491 --> 00:15:54.851
We always bring you fresh news.
00:15:54.912 --> 00:15:56.613
And even though it's been around,
00:15:57.332 --> 00:15:58.453
all this stuff has been happening.
00:15:58.614 --> 00:16:02.836
Grog gave a user detailed instructions on
00:16:02.875 --> 00:16:04.576
how to break into a politician's home in
00:16:04.676 --> 00:16:05.135
USA,
00:16:05.956 --> 00:16:07.476
right down to what tools they should
00:16:07.496 --> 00:16:07.837
bring.
00:16:08.577 --> 00:16:12.278
And it looked at the person's posting
00:16:12.298 --> 00:16:14.879
habits to say what time they should break
00:16:14.921 --> 00:16:15.360
into their home.
00:16:16.240 --> 00:16:17.022
That's what Glock did.
00:16:17.783 --> 00:16:18.745
And would you not be able to do
00:16:18.764 --> 00:16:19.547
that with another model?
00:16:19.586 --> 00:16:21.208
Has anybody done the same thing on another
00:16:21.249 --> 00:16:22.211
model to see if you can do it
00:16:22.230 --> 00:16:22.772
on another model?
00:16:24.775 --> 00:16:27.837
um i don't know but uh perhaps not
00:16:28.379 --> 00:16:32.260
because i often do the models um you
00:16:32.280 --> 00:16:35.003
can tell me you can tell me you
00:16:35.023 --> 00:16:36.764
can tell me what does chat do you
00:16:36.784 --> 00:16:38.505
really say about let me just tell you
00:16:38.586 --> 00:16:40.647
i just run through so chat gpt you
00:16:40.687 --> 00:16:43.149
can do this claude you really can't you
00:16:43.249 --> 00:16:46.932
know what about deep seek what about deep
00:16:46.951 --> 00:16:47.172
seek
00:16:48.432 --> 00:16:49.832
Deep seek, I don't actually,
00:16:49.852 --> 00:16:51.313
that is one I don't really use much
00:16:51.374 --> 00:16:51.913
of, deep seek.
00:16:52.234 --> 00:16:53.315
But so just to answer this,
00:16:53.335 --> 00:16:54.294
so how did they,
00:16:54.735 --> 00:16:56.556
as in somebody reported flagged this
00:16:56.596 --> 00:16:59.918
because they did it unmaliciously or there
00:16:59.937 --> 00:17:01.638
was evidence that somebody managed to do
00:17:01.677 --> 00:17:02.938
this because there was something off the
00:17:02.958 --> 00:17:03.619
back of it that was,
00:17:04.500 --> 00:17:06.079
was there a crime committed or was it
00:17:06.119 --> 00:17:08.781
just more of a thing someone found it
00:17:08.801 --> 00:17:09.122
could do?
00:17:10.567 --> 00:17:11.409
Somebody found it could do.
00:17:11.449 --> 00:17:16.513
And I think Grock kind of figured out
00:17:16.554 --> 00:17:17.394
that this was happening.
00:17:17.954 --> 00:17:20.798
And then
somebody leaked this because they
00:17:20.817 --> 00:17:23.160
were doing it as a test, I think.
00:17:23.279 --> 00:17:24.000
And they found that.
00:17:24.361 --> 00:17:26.261
So basically, that's what's happened.
00:17:29.585 --> 00:17:31.626
There's a lot more than just that that
00:17:31.646 --> 00:17:32.067
came out.
00:17:32.327 --> 00:17:34.369
There's about underage
00:17:35.230 --> 00:17:36.230
content as well.
00:17:36.691 --> 00:17:39.570
There's a lot came up within just one
00:17:39.611 --> 00:17:42.432
go, which is terrible, right?
00:17:42.451 --> 00:17:44.853
So not doing anything on the other side,
00:17:44.893 --> 00:17:46.133
John, as you said, well,
00:17:46.173 --> 00:17:47.833
doing something is important no matter
00:17:47.853 --> 00:17:48.614
what the stance is,
00:17:48.993 --> 00:17:50.354
but not doing anything.
00:17:51.273 --> 00:17:52.515
This is a bit of a problem, right?
00:17:52.535 --> 00:17:55.214
So with this,
00:17:56.516 --> 00:17:58.056
I guess then let's get into it a
00:17:58.076 --> 00:17:59.155
bit on the ethics.
00:17:59.195 --> 00:17:59.355
But
00:18:03.277 --> 00:18:04.518
My stance,
00:18:04.538 --> 00:18:05.678
which I've kind of already hinted at,
00:18:06.798 --> 00:18:08.239
you feel the models should be more
00:18:08.298 --> 00:18:08.659
ethical.
00:18:08.679 --> 00:18:09.920
You're going to get into the training of
00:18:09.960 --> 00:18:10.180
that.
00:18:10.920 --> 00:18:12.520
But in the example you just gave,
00:18:13.942 --> 00:18:16.482
that's an organization being unethical.
00:18:16.502 --> 00:18:18.564
They're deciding not to be ethical.
00:18:19.534 --> 00:18:20.773
And that is a different problem
00:18:20.814 --> 00:18:21.374
altogether.
00:18:21.413 --> 00:18:27.056
And I think regulation of how you force
00:18:27.476 --> 00:18:32.557
some level of standards and principles or
00:18:32.576 --> 00:18:34.117
protections, really, in this case,
00:18:34.157 --> 00:18:35.397
it's more protections, isn't it?
00:18:35.417 --> 00:18:36.337
There's safeguarding.
00:18:37.018 --> 00:18:38.919
And it's not just ethics, actually,
00:18:39.259 --> 00:18:40.659
because I think ethics is such a broad
00:18:40.699 --> 00:18:40.979
term.
00:18:41.019 --> 00:18:41.179
Really,
00:18:41.199 --> 00:18:42.799
what we're saying is how do you safeguard
00:18:42.839 --> 00:18:43.160
a model
00:18:44.008 --> 00:18:48.690
to not accidentally cause harm or to admit
00:18:48.710 --> 00:18:50.330
the fact that it's an AI to begin
00:18:50.351 --> 00:18:51.632
with and not pretend that it isn't.
00:18:53.332 --> 00:18:54.613
And things like that is basically what
00:18:54.633 --> 00:18:56.294
you're hearing, which I totally,
00:18:56.334 --> 00:18:57.074
totally agree with.
00:18:57.094 --> 00:18:58.474
I think this is a bit broad though,
00:18:58.555 --> 00:19:01.996
but yeah, I think safeguarding definitely,
00:19:02.036 --> 00:19:03.057
definitely should be something.
00:19:04.795 --> 00:19:05.234
Yeah.
00:19:05.275 --> 00:19:07.776
You can't really compare LLMs to any other
00:19:07.816 --> 00:19:09.457
kind of like a search engine, for example.
00:19:09.497 --> 00:19:09.696
Right.
00:19:10.156 --> 00:19:10.797
So, for example,
00:19:11.277 --> 00:19:13.298
if you are a call center of any
00:19:13.357 --> 00:19:14.778
sort, doesn't matter what sort it is,
00:19:15.479 --> 00:19:16.179
you have some training.
00:19:17.278 --> 00:19:18.200
Everybody gets some training.
00:19:18.359 --> 00:19:19.259
They get some guidelines.
00:19:19.299 --> 00:19:22.401
They get some playbooks.
00:19:22.461 --> 00:19:23.501
I think this is how we have to
00:19:23.541 --> 00:19:26.021
treat the models as we have to give
00:19:26.041 --> 00:19:26.563
it some training.
00:19:27.063 --> 00:19:28.163
We have to give it some guidelines.
00:19:28.442 --> 00:19:30.344
Anyway, we can talk about the process,
00:19:30.384 --> 00:19:32.364
perhaps for a few minutes, John,
00:19:32.384 --> 00:19:33.525
just to explain how it works.
00:19:34.113 --> 00:19:35.253
Let's get into the process.
00:19:35.294 --> 00:19:37.115
So it's obviously a bit of a mixed
00:19:37.194 --> 00:19:37.775
bag, is it?
00:19:37.914 --> 00:19:40.036
Or is there just one?
00:19:40.135 --> 00:19:43.958
Or is it different per organization or
00:19:43.998 --> 00:19:44.838
just principles?
00:19:45.239 --> 00:19:46.299
What is it?
00:19:46.420 --> 00:19:48.161
Yeah, good question there, John.
00:19:48.201 --> 00:19:50.261
And hence why you're the host of this
00:19:50.301 --> 00:19:50.842
podcast.
00:19:51.102 --> 00:19:51.923
There is a mixed bag.
00:19:52.262 --> 00:19:53.663
But before I talk about ethics,
00:19:53.923 --> 00:19:54.364
maybe I can...
00:19:55.884 --> 00:19:58.906
talk about how the basic pipeline is,
00:19:59.047 --> 00:20:00.528
like how to actually train the models.
00:20:00.949 --> 00:20:04.531
But no matter what model you pick and
00:20:05.152 --> 00:20:07.473
before there's any ethics baked into it,
00:20:07.554 --> 00:20:10.695
a model usually goes through three stages.
00:20:11.477 --> 00:20:13.778
Pre-training is where it learns the
00:20:13.838 --> 00:20:16.500
language and the world knowledge by
00:20:16.721 --> 00:20:19.742
predicting what the next word across huge
00:20:19.762 --> 00:20:20.784
amounts of text is, right?
00:20:20.844 --> 00:20:22.424
Because I know you and I joke about
00:20:22.484 --> 00:20:24.925
at times that LLMs are just like next
00:20:24.967 --> 00:20:25.787
token predictor, right?
00:20:25.886 --> 00:20:28.788
And that's what they call the
00:20:28.828 --> 00:20:29.329
pre-training.
00:20:29.849 --> 00:20:31.611
It just absorbs everything, the good,
00:20:31.790 --> 00:20:34.492
the bad, all of subreddits, you know,
00:20:34.532 --> 00:20:35.653
it's taken everything, right?
00:20:36.775 --> 00:20:38.154
So that's the first step, pre-training.
00:20:38.276
--> 00:20:40.297
And then the next one is supervised
00:20:40.757 --> 00:20:41.376
fine-tuning.
00:20:41.738 --> 00:20:44.159
And this is where the models are shown.
00:20:44.578 --> 00:20:46.840
This is a good answer written by humans.
00:20:47.560 --> 00:20:49.021
And it learns to copy that style and
00:20:49.061 --> 00:20:49.501
structure.
00:20:49.682 --> 00:20:51.022
This is a good answer.
00:20:51.303 --> 00:20:53.805
We're not talking about this is correct or
00:20:53.845 --> 00:20:54.025
not,
00:20:54.085 --> 00:20:55.645
but this is what the answer should look
00:20:55.685 --> 00:20:55.826
like.
00:20:56.086 --> 00:20:56.807
If somebody says,
00:20:57.086 --> 00:20:58.107
what is the weather outside?
00:20:58.347 --> 00:21:00.148
They shouldn't just say, twenty-nine.
00:21:00.669 --> 00:21:01.130
You say, oh,
00:21:01.150 --> 00:21:03.393
it's twenty nine degrees as of right now
00:21:03.593 --> 00:21:04.775
in this city where you are.
00:21:04.894 --> 00:21:05.055
Right.
00:21:05.075 --> 00:21:06.537
So that's that's an example.
00:21:06.836 --> 00:21:07.699
So you do pre-training,
00:21:07.979 --> 00:21:09.621
then you do what people call SFT
00:21:09.661 --> 00:21:11.103
supervised fine tuning.
00:21:11.523 --> 00:21:13.686
And then what you have is reinforcement
00:21:13.727 --> 00:21:15.388
learning from human feedback,
00:21:15.709 --> 00:21:17.852
or as people call it, RLHF.
00:21:18.452 --> 00:21:20.354
is where human reviewers compares the
00:21:20.473 --> 00:21:22.734
output when the training is happening,
00:21:23.134 --> 00:21:24.276
and they say which one's better.
00:21:24.516 --> 00:21:26.416
And sometimes you have been part of this
00:21:26.457 --> 00:21:26.737
as well.
00:21:26.997 --> 00:21:28.397
I know, John, you don't use ChatGPT,
00:21:28.698 --> 00:21:30.999
but sometimes it gives you a couple of
00:21:31.058 --> 00:21:32.039
options and says, oh,
00:21:32.119 --> 00:21:33.119
is this better or that one?
00:21:33.381 --> 00:21:35.922
And the preference then basically trains a
00:21:35.981 --> 00:21:37.061
separate reward model,
00:21:37.442 --> 00:21:39.503
and then the main model gets optimized to
00:21:39.564 --> 00:21:40.443
score against that.
00:21:40.463 --> 00:21:41.384
So you have pre-training,
00:21:41.785 --> 00:21:42.644
supervised training,
00:21:43.005 --> 00:21:45.266
and reinforcement learning from human
00:21:45.307 --> 00:21:45.686
feedback.
00:21:46.027 --> 00:21:47.948
At this moment, there is no ethics.
00:21:48.488 --> 00:21:50.630
right it's just this is what good looks
00:21:50.670 --> 00:21:55.653
like uh what is that true because isn't
00:21:55.792 --> 00:22:00.415
isn't isn't though some of the things it's
00:22:00.435 --> 00:22:05.819
been trained on um are it's based on
00:22:05.880 --> 00:22:08.461
the internet or it's based on lots of
00:22:08.781 --> 00:22:11.284
other human conversations and lots of like
00:22:12.125 --> 00:22:13.887
know all kinds of different data right
00:22:13.907 --> 00:22:15.529
that would have got into the training to
00:22:15.549 --> 00:22:18.654
begin with it's all human based probably
00:22:18.714 --> 00:22:20.758
or human driven as everything is
00:22:22.025 --> 00:22:22.424
Correct.
00:22:25.807 --> 00:22:28.528
Especially RLHF,
00:22:28.868 --> 00:22:31.190
reinforcement learning from human
00:22:31.230 --> 00:22:31.589
feedback,
00:22:31.890 --> 00:22:33.530
is just really teaching the model what
00:22:33.611 --> 00:22:35.071
humans tend to prefer.
00:22:35.893 --> 00:22:38.094
Which is not necessarily enough on its own
00:22:38.134 --> 00:22:41.075
because human preference and ethical
00:22:41.115 --> 00:22:42.796
behavior is not the same thing.
00:22:43.717 --> 00:22:46.919
For everything out there on the internet,
00:22:47.219 --> 00:22:48.559
every time there's a video that's out.
00:22:48.980 --> 00:22:49.920
For example, Ronaldo...
00:22:51.161 --> 00:22:52.681
You know, the card,
00:22:52.820 --> 00:22:56.422
the red card that got overturned in FIFA,
00:22:56.442 --> 00:22:58.521
there were people for and against it.
00:22:59.162 --> 00:23:01.222
And both some people,
00:23:01.522 --> 00:23:04.442
some people prefer the for decision and
00:23:04.482 --> 00:23:06.564
some people prefer the against decision.
00:23:07.663 --> 00:23:08.963
So that's what it is.
00:23:09.523 --> 00:23:11.765
So in most cases, John, yes, you're right.
00:23:12.664 --> 00:23:13.825
It is what people prefer.
00:23:14.404 --> 00:23:17.566
People prefer like ethical things in some
00:23:17.586 --> 00:23:19.145
cases, but not in all the cases.
00:23:20.722 --> 00:23:20.883
Yeah,
00:23:20.903 --> 00:23:22.904
so there is some element of kind of
00:23:23.625 --> 00:23:26.386
subliminal ethics, right?
00:23:26.426 --> 00:23:27.267
Symbiosis.
00:23:28.567 --> 00:23:31.549
Yeah, it's not conscious, right?
00:23:31.609 --> 00:23:35.313
Yeah, none of this shit,
00:23:35.353 --> 00:23:36.653
none of this stuff is conscious, John.
00:23:37.173 --> 00:23:39.115
Yeah, so it's very subconscious.
00:23:39.194 --> 00:23:40.236
It's just like what you like,
00:23:40.256 --> 00:23:40.977
what you don't like.
00:23:41.017 --> 00:23:42.258
And they don't even know why they like
00:23:42.278 --> 00:23:43.397
what they like or what they don't like,
00:23:43.679 --> 00:23:43.818
right?
00:23:43.838 --> 00:23:45.118
Because you've just grown up,
00:23:45.138 --> 00:23:45.839
you are who you are.
00:23:46.380 --> 00:23:47.681
And so I guess what I'm saying by
00:23:47.740 --> 00:23:48.842
proxy is,
00:23:49.238 --> 00:23:51.440
it's almost like they're a mirror.
00:23:52.200 --> 00:23:53.602
Sometimes these models are almost like a
00:23:53.682 --> 00:23:54.762
mirror of ourselves,
00:23:54.782 --> 00:23:55.803
do you know what I mean?
00:23:55.843 --> 00:23:58.144
Or like a combination of many different
00:24:00.086 --> 00:24:03.630
reflections of the social dynamics that
00:24:03.690 --> 00:24:06.471
already exist and how it's been trained
00:24:06.491 --> 00:24:07.893
and the answers that it's kind of got
00:24:07.913 --> 00:24:09.094
and what it thinks is right and what
00:24:09.134 --> 00:24:09.795
it thinks is wrong.
00:24:09.815 --> 00:24:11.076
Because it doesn't know whether it's right
00:24:11.096 --> 00:24:11.296
or wrong.
00:24:11.336 --> 00:24:12.656
Obviously, it's just the training.
00:24:13.176 --> 00:24:14.117
The training is the thing.
00:24:14.137 --> 00:24:15.578
And then the learning of what's being
00:24:15.618 --> 00:24:15.900
told,
00:24:15.940 --> 00:24:17.300
this is right or this is wrong or
00:24:17.320 --> 00:24:18.442
this is preferred or this isn't.
00:24:19.663 --> 00:24:21.744
correct that's the first stage so there's
00:24:21.806 --> 00:24:24.288
no ethics there may be an element of
00:24:24.348 --> 00:24:27.732
ethics in its answer if you didn't like
00:24:27.752 --> 00:24:28.953
what it was saying but at the moment
00:24:28.973 --> 00:24:34.662
there's no ethical based like a really
00:24:34.701 --> 00:24:36.804
intentional element of ethics baked in at
00:24:36.824 --> 00:24:37.905
this point is what you're saying yeah
00:24:38.557 --> 00:24:38.998
Correct.
00:24:39.317 --> 00:24:40.058
And at this point,
00:24:40.138 --> 00:24:42.099
we'll split into two directions.
00:24:42.160 --> 00:24:43.099
Because after this,
00:24:43.160 --> 00:24:44.800
because this is like pre-training,
00:24:45.080 --> 00:24:47.643
there's another, depending on, well,
00:24:47.663 --> 00:24:48.603
we're only going to talk about two
00:24:48.623 --> 00:24:48.963
companies.
00:24:48.983 --> 00:24:50.724
There's generally four within the two.
00:24:50.765 --> 00:24:52.726
We'll talk about Anthropic's approach.
00:24:53.105 --> 00:24:55.086
And then we'll also talk about OpenAI's
00:24:55.126 --> 00:24:55.507
approach.
00:24:55.846 --> 00:24:57.607
I know you prefer Anthropic way more than
00:24:57.748 --> 00:24:58.209
OpenAI.
00:24:58.229 --> 00:24:58.929
So perhaps, John,
00:24:59.288 --> 00:25:00.750
we can talk about Anthropic first.
00:25:01.329 --> 00:25:02.810
And maybe we can go there.
00:25:02.830 --> 00:25:03.671
Maybe we can do that.
00:25:06.792 --> 00:25:11.178
So, anthropic design a constitution, John.
00:25:12.219 --> 00:25:15.422
And you can see the vision here.
00:25:16.303 --> 00:25:17.984
I do love a constitution.
00:25:19.586 --> 00:25:21.188
Anthropics say, you're not an LLM model,
00:25:21.228 --> 00:25:21.469
buddy.
00:25:22.108 --> 00:25:23.550
You're a whole ass country.
00:25:23.671 --> 00:25:25.092
That's what anthropics are saying.
00:25:25.113 --> 00:25:27.214
You need a constitution.
00:25:27.795 --> 00:25:28.895
Yeah, you need a constitution.
00:25:30.018 --> 00:25:31.099
Let's go big on this.
00:25:31.299 --> 00:25:33.702
Let's go big on the vocab and call
00:25:33.722 --> 00:25:34.284
it a constitution.
00:25:34.304 --> 00:25:35.025
Yeah, absolutely.
00:25:35.705 --> 00:25:37.248
So what they do is,
00:25:37.407 --> 00:25:40.971
the anthropics constitutional AI gives the
00:25:41.011 --> 00:25:42.855
model a written set of principles,
00:25:43.434 --> 00:25:44.096
a constitution,
00:25:44.676 --> 00:25:44.896
that
00:25:45.518 --> 00:25:45.917
Basically,
00:25:45.938 --> 00:25:47.759
instead of relying on human labelers,
00:25:48.401 --> 00:25:49.340
it does it itself,
00:25:49.381 --> 00:25:51.083
and it runs in two phases.
00:25:51.423 --> 00:25:53.884
The Constitution is written by anthropic,
00:25:53.924 --> 00:25:54.286
of course.
00:25:54.546 --> 00:25:56.106
They have a bunch of philosophers and
00:25:56.207 --> 00:25:58.409
people from other areas that define what
00:25:58.429 --> 00:25:59.269
this Constitution is.
00:25:59.289 --> 00:25:59.470
You know,
00:25:59.490 --> 00:26:01.191
those ethical guidelines and the
00:26:01.231 --> 00:26:02.553
guardrails that we talked about, John,
00:26:02.613 --> 00:26:03.093
earlier, right?
00:26:03.653 --> 00:26:04.594
But they define that,
00:26:04.835 --> 00:26:05.875
and they give it to the model.
00:26:06.596 --> 00:26:08.758
And then what happens is during the
00:26:08.798 --> 00:26:10.578
training phase,
00:26:11.820 --> 00:26:14.623
the model critiques and rewrites its own
00:26:14.903 --> 00:26:17.045
responses against the Constitution.
00:26:17.424 --> 00:26:18.605
So when they're running it,
00:26:19.007 --> 00:26:21.669
it puts some stuff in and says, okay,
00:26:22.148 --> 00:26:23.931
what does the Constitution say about my
00:26:23.971 --> 00:26:24.431
response?
00:26:24.790 --> 00:26:26.212
Oh, it's not quite constitutional,
00:26:26.252 --> 00:26:27.053
so let me put it back.
00:26:27.133 --> 00:26:29.095
And then it goes through reinforcement
00:26:29.134 --> 00:26:29.454
learning.
00:26:30.336 --> 00:26:32.700
through using this AI-generated feedback.
00:26:32.740 --> 00:26:36.246
And that's what they call RLAIF,
00:26:37.268 --> 00:26:40.173
reinforcement learning using AI-generated
00:26:40.212 --> 00:26:42.896
feedback based on the principles rather
00:26:42.916 --> 00:26:43.898
than the human feedback.
00:26:44.675 --> 00:26:46.396
And that's the second part of the RLAIF,
00:26:46.477 --> 00:26:46.616
right?
00:26:46.636 --> 00:26:47.337
That's what they call it.
00:26:47.678 --> 00:26:49.919
And then who designs these principles?
00:26:49.979 --> 00:26:51.880
There's a bunch of well-known
00:26:51.920 --> 00:26:53.621
philosophers, Amanda Askell,
00:26:53.981 --> 00:26:54.761
Joe Carol Smith,
00:26:54.902 --> 00:26:56.502
and a bunch of other people who have
00:26:56.542 --> 00:26:59.065
contributed to these constitutions.
00:26:59.085 --> 00:27:00.665
They design the principles,
00:27:00.705 --> 00:27:02.446
the reason about the edge cases,
00:27:02.747 --> 00:27:04.807
they define the concepts like honesty,
00:27:05.048 --> 00:27:07.490
autonomy, fairness, uncertainty,
00:27:07.990 --> 00:27:09.891
and they evaluate how models should behave
00:27:09.911 --> 00:27:10.872
in difficult situations.
00:27:11.872 --> 00:27:14.593
Is their constitution made public?
00:27:14.813 --> 00:27:19.855
Do you know what they've decided is their
00:27:19.894 --> 00:27:23.596
constitution for this reinforced kind of
00:27:23.636 --> 00:27:24.277
learning, really?
00:27:25.116 --> 00:27:26.698
You're asking for trade secrets now, John.
00:27:27.617 --> 00:27:29.499
That information...
00:27:29.578 --> 00:27:30.999
If it's a set of principles,
00:27:31.279 --> 00:27:32.660
a constitutional set of principles,
00:27:32.680 --> 00:27:35.961
why wouldn't you be transparent about it?
00:27:36.060 --> 00:27:37.501
It's not about how you're training it,
00:27:37.541 --> 00:27:37.961
necessarily.
00:27:37.981 --> 00:27:39.343
These are the things it's validating
00:27:39.363 --> 00:27:40.343
itself against, right?
00:27:41.666 --> 00:27:42.007
Yeah,
00:27:42.027 --> 00:27:45.269
there is general guidelines are out on
00:27:45.509 --> 00:27:46.170
what these are,
00:27:46.710 --> 00:27:48.990
but not specific details about this is
00:27:49.090 --> 00:27:51.972
what specific thing you should do in this
00:27:52.292 --> 00:27:52.813
edge case.
00:27:52.833 --> 00:27:54.394
So if, you know,
00:27:54.755 --> 00:27:57.076
like the general information is out there,
00:27:57.096 --> 00:27:58.436
you know, it's in their blogs,
00:27:58.777 --> 00:28:00.178
they publish, they talk about what it is.
00:28:00.498 --> 00:28:01.218
But of course, John,
00:28:01.357 --> 00:28:03.440
like if you start writing all of this
00:28:03.460 --> 00:28:04.059
stuff out,
00:28:04.319 --> 00:28:05.760
it's probably going to be like pages and
00:28:05.800 --> 00:28:06.540
pages and pages long.
00:28:06.862 --> 00:28:08.301
And then it will open up another kind
00:28:08.321 --> 00:28:10.483
of worms, many kind of worms as to...
00:28:11.744 --> 00:28:13.166
I think there's no scrutiny over this,
00:28:13.207 --> 00:28:13.366
right?
00:28:13.386 --> 00:28:16.432
Because this is part of code.
00:28:17.317 --> 00:28:19.118
They don't want to have this code that's
00:28:19.219 --> 00:28:20.599
open source, I guess.
00:28:21.221 --> 00:28:22.902
So some of this stuff is you can
00:28:23.362 --> 00:28:24.702
figure out some of this information,
00:28:24.742 --> 00:28:29.547
but this is how Anthropic are training
00:28:29.586 --> 00:28:30.047
their models,
00:28:30.126 --> 00:28:32.588
because they are saying that if we leave
00:28:32.608 --> 00:28:35.171
the models to the base layer with the
00:28:35.211 --> 00:28:36.451
three things that it's done before,
00:28:36.711 --> 00:28:37.291
that's not enough.
00:28:37.893 --> 00:28:38.073
Right.
00:28:38.413 --> 00:28:40.835
And it basically works.
00:28:40.934 --> 00:28:41.414
It's enough for Grog.
00:28:41.454 --> 00:28:42.076
That's all I'm saying.
00:28:42.096 --> 00:28:45.178
It's enough for Elon.
00:28:47.021 --> 00:28:47.781
That's enough for him.
00:28:48.583 --> 00:28:49.123
You know what I mean?
00:28:49.143 --> 00:28:49.963
If it's enough for him,
00:28:50.003 --> 00:28:50.784
do we all need to do it?
00:28:50.804 --> 00:28:51.325
I don't know.
00:28:51.365 --> 00:28:54.326
It seems to be enough for old Grok.
00:28:56.327 --> 00:28:57.709
But here's the thing, John, though, right?
00:28:58.469 --> 00:29:00.891
The first part where we're saying that the
00:29:00.990 --> 00:29:03.271
model is policing itself against the
00:29:03.392 --> 00:29:04.192
written principles.
00:29:04.653 --> 00:29:07.095
But this scales better because, of course,
00:29:07.115 --> 00:29:08.355
you can't have millions of people
00:29:09.135 --> 00:29:10.997
like checking examples like, oh,
00:29:11.017 --> 00:29:11.477
this is correct.
00:29:11.497 --> 00:29:11.856
That's correct.
00:29:11.876 --> 00:29:12.298
That's correct.
00:29:14.057 --> 00:29:16.058
We have to rely on the model policing
00:29:16.118 --> 00:29:19.320
itself correctly against the guidelines.
00:29:19.381 --> 00:29:19.560
Right.
00:29:20.300 --> 00:29:22.461
And if it doesn't have any conscious,
00:29:22.501 --> 00:29:23.803
but if it did get any conscious,
00:29:23.843 --> 00:29:25.243
then it will do what it wants.
00:29:26.988 --> 00:29:29.269
So that's Entropic's way of dealing with
00:29:29.289 --> 00:29:29.490
this.
00:29:30.230 --> 00:29:31.592
I don't know what your takes are on
00:29:31.632 --> 00:29:31.771
this.
00:29:31.811 --> 00:29:31.991
John,
00:29:32.031 --> 00:29:35.174
do you think they should perhaps let
00:29:35.214 --> 00:29:37.196
everybody know that this is what our
00:29:37.616 --> 00:29:39.719
ethical constitution is?
00:29:39.798 --> 00:29:43.981
Or what do you think they should do?
00:29:44.063 --> 00:29:49.207
I would have assumed that they would want
00:29:49.287 --> 00:29:50.587
to because it would be a little bit
00:29:50.607 --> 00:29:51.749
of a differentiator.
00:29:51.989 --> 00:29:53.651
They don't necessarily have to publicise
00:29:54.951 --> 00:29:57.512
every minutiae detail of it.
00:29:57.553 --> 00:29:58.374
But they could say,
00:29:58.413 --> 00:30:01.875
here's our top level themed principles.
00:30:03.196 --> 00:30:04.436
One would be like,
00:30:04.497 --> 00:30:06.397
the AI can never lie about whether it's
00:30:06.438 --> 00:30:07.739
an AI.
00:30:07.898 --> 00:30:09.140
It will always tell the truth.
00:30:09.180 --> 00:30:10.421
If somebody asked, are you AI,
00:30:10.500 --> 00:30:12.102
it would always admit to being yes.
00:30:14.063 --> 00:30:15.743
In a crisis, I don't know,
00:30:15.824 --> 00:30:17.003
whatever the fundamentals are,
00:30:17.023 --> 00:30:17.904
because I would have thought that would
00:30:17.924 --> 00:30:19.445
have been a really big selling point for
00:30:19.465 --> 00:30:19.605
them.
00:30:21.592 --> 00:30:21.912
Actually,
00:30:24.256 --> 00:30:25.857
I take back what I said previously.
00:30:27.079 --> 00:30:28.402
The constitution is online.
00:30:30.085 --> 00:30:30.826
I was going to say,
00:30:31.026 --> 00:30:32.167
I would have assumed because it would have
00:30:32.188 --> 00:30:35.732
been in the world of competitive...
00:30:36.374 --> 00:30:39.596
models like if you're going to pitch for
00:30:39.616 --> 00:30:41.979
being the most ethical that's a
00:30:42.038 --> 00:30:44.701
differentiator isn't it absolutely i think
00:30:44.882 --> 00:30:47.144
you're you're sorry my bad this is the
00:30:47.263 --> 00:30:48.785
it's a pdf that they published on the
00:30:48.964 --> 00:30:51.346
on the website about a bunch of uh
00:30:51.527 --> 00:30:53.269
you know again these are principles
00:30:53.308 --> 00:30:55.530
high-level principles that principles yeah
00:30:56.951 --> 00:30:57.771
It's all online.
00:30:57.832 --> 00:31:01.374
You can search for Claude Constitution and
00:31:01.413 --> 00:31:05.537
it comes up and you can read it.
00:31:05.576 --> 00:31:05.977
Basically,
00:31:06.257 --> 00:31:07.577
the brief summary of the Constitution.
00:31:07.597 --> 00:31:10.159
What's your favorite Claude Constitutional
00:31:10.179 --> 00:31:13.820
principle?
00:31:13.861 --> 00:31:15.761
The funny thing about if you read the
00:31:15.803 --> 00:31:17.703
blog, it says it's broadly safe.
00:31:18.544 --> 00:31:19.785
And it's broadly ethical.
00:31:19.805 --> 00:31:23.606
And then it goes in like steps and
00:31:23.666 --> 00:31:25.407
it goes compliant with anthropics
00:31:25.847 --> 00:31:26.488
guidelines.
00:31:27.228 --> 00:31:29.970
And at the end is genuinely helpful,
00:31:30.450 --> 00:31:33.192
but broadly safe and broadly ethical.
00:31:33.852 --> 00:31:35.893
That's really reassuring.
00:31:36.232 --> 00:31:37.933
Yeah, very reassuring.
00:31:38.094 --> 00:31:40.994
Really reassuring.
00:31:41.076 --> 00:31:43.517
We may not crash the plane.
00:31:43.797 --> 00:31:44.636
I may crash it.
00:31:44.817 --> 00:31:45.357
I might not.
00:31:46.234 --> 00:31:47.635
Anyway, everyone have a safe flight.
00:31:47.695 --> 00:31:50.477
Buckle up.
00:31:50.517 --> 00:31:52.898
We may be having pending death,
00:31:53.058 --> 00:31:53.939
but we might not.
00:31:54.279 --> 00:31:55.380
At the same time, we might not.
00:31:55.700 --> 00:31:56.000
You know,
00:31:56.060 --> 00:31:57.842
who knows how this flight is going to
00:31:57.882 --> 00:31:58.241
go down.
00:31:58.261 --> 00:31:59.522
But anyway, enjoy your holiday.
00:31:59.563 --> 00:32:02.503
I hope you have a great trip.
00:32:03.365 --> 00:32:05.326
You know, it's not necessarily reassuring.
00:32:06.075 --> 00:32:09.877
But the interesting part is that they used
00:32:09.917 --> 00:32:14.021
to have rule based alignment until the
00:32:14.041 --> 00:32:14.903
beginning of
this year,
00:32:15.022 --> 00:32:16.324
and then they switched over to reason
00:32:16.344 --> 00:32:16.644
based.
00:32:17.444 --> 00:32:19.886
And if you can get hold of the
00:32:20.106 --> 00:32:22.509
old list, it's basically list the rules.
00:32:22.769 --> 00:32:25.692
But now basically they're doing why
00:32:25.751 --> 00:32:26.512
instead of what.
00:32:27.243 --> 00:32:31.164
But that's what they're doing.
00:32:31.325 --> 00:32:32.964
And I should add,
00:32:33.404 --> 00:32:35.766
they say it's a work in progress document
00:32:36.226 --> 00:32:38.145
and have published like two iterations of
00:32:38.165 --> 00:32:40.247
this document and they keep publishing
00:32:40.666 --> 00:32:42.086
multiple iterations of the document.
00:32:42.507 --> 00:32:45.407
So that's what we have currently.
00:32:45.428 --> 00:32:46.107
That's entropic.
00:32:47.627 --> 00:32:49.888
I know you're dying to talk about Sam
00:32:49.929 --> 00:32:50.429
Altman's
00:32:52.743 --> 00:32:55.285
you know, what their ethics are.
00:32:55.605 --> 00:32:55.785
I mean,
00:32:55.805 --> 00:32:57.484
they were an ethical company beforehand,
00:32:57.505 --> 00:32:57.785
weren't they?
00:32:57.825 --> 00:32:59.766
Wasn't that the basis of the starting
00:32:59.786 --> 00:33:00.665
point of the company?
00:33:00.685 --> 00:33:02.326
So I bet this is going to be
00:33:02.365 --> 00:33:02.566
good.
00:33:03.246 --> 00:33:04.306
This is going to be good, isn't it?
00:33:04.405 --> 00:33:05.926
I don't know what you're talking about,
00:33:05.946 --> 00:33:06.307
John.
00:33:06.406 --> 00:33:07.646
I'm just going to give you the facts.
00:33:08.826 --> 00:33:09.567
This is going to be good.
00:33:09.586 --> 00:33:10.567
Out of everyone,
00:33:10.647 --> 00:33:15.288
they're going to be the one, right?
00:33:15.429 --> 00:33:17.148
That's why they started was for this.
00:33:18.306 --> 00:33:20.989
So OpenAI, they said,
00:33:21.088 --> 00:33:22.490
we're not going to define a constitution
00:33:22.691 --> 00:33:23.711
because it's not a country.
00:33:24.553 --> 00:33:27.415
We will define more aptly named model
00:33:27.717 --> 00:33:28.156
spec.
00:33:28.837 --> 00:33:29.679
That's what they defined.
00:33:30.579 --> 00:33:32.102
And OpenAI is equivalent.
00:33:32.801 --> 00:33:34.324
They call it model spec, of course.
00:33:34.503 --> 00:33:37.728
And they're most recently updated with
00:33:38.087 --> 00:33:39.430
teen protection principles.
00:33:39.769 --> 00:33:41.291
So, of course,
00:33:41.372 --> 00:33:43.153
there's been a lot recently in the last
00:33:43.194 --> 00:33:46.157
couple of years about a lot of harm
00:33:46.178 --> 00:33:47.779
that's been caused to various people.
00:33:48.361 --> 00:33:49.321
So what it does,
00:33:49.382 --> 00:33:52.566
it sets out an authority hierarchy for
00:33:52.646 --> 00:33:54.469
whose instructions the model should
00:33:54.509 --> 00:33:54.769
follow.
00:33:55.759 --> 00:33:57.560
And they kind of set it up like
00:33:57.621 --> 00:33:58.842
I'm going to tell you some terms and
00:33:58.862 --> 00:34:00.423
I'll explain to you, John, what it is.
00:34:00.863 --> 00:34:04.065
But it's a set of principles that they
00:34:04.085 --> 00:34:06.145
have to follow no matter who's asking.
00:34:06.326 --> 00:34:07.567
For example, what they say,
00:34:07.606 --> 00:34:08.788
what this well, not for example,
00:34:09.047 --> 00:34:09.648
this is what they say.
00:34:10.027 --> 00:34:11.048
The authority is this.
00:34:11.369 --> 00:34:13.250
You have the root level rules,
00:34:13.389 --> 00:34:14.570
then you have the system rules,
00:34:14.610 --> 00:34:16.072
then you have the developer rules,
00:34:16.192 --> 00:34:18.092
then you have the users requesting,
00:34:18.393 --> 00:34:19.373
then you have the guidelines.
00:34:20.134 --> 00:34:22.496
So root is kind of like open AI's
00:34:23.717 --> 00:34:24.336
constitution.
00:34:24.536 --> 00:34:25.637
So they have their own spec.
00:34:26.079 --> 00:34:28.880
So fundamental safety and legal rules like
00:34:29.201 --> 00:34:31.643
prevent harm, illegal actions,
00:34:31.822 --> 00:34:33.103
breaking the chain of command.
00:34:33.844 --> 00:34:35.164
If the root rules conflict,
00:34:35.746 --> 00:34:36.967
then the model shouldn't do anything.
00:34:37.027 --> 00:34:39.047
That kind of thing is defining that in
00:34:39.068 --> 00:34:39.648
the root rules.
00:34:40.289 --> 00:34:42.110
Then it goes into then it goes, okay,
00:34:42.170 --> 00:34:42.690
what's next?
00:34:42.710 --> 00:34:43.851
The next is system.
00:34:44.411 --> 00:34:48.175
so an open basically that's platformer
00:34:48.215 --> 00:34:50.157
specific user behavior like age
00:34:50.197 --> 00:34:52.177
appropriate responses right product
00:34:52.217 --> 00:34:54.920
specific settings so and developers and
00:34:55.021 --> 00:34:57.422
users cannot override this this again this
00:34:57.463 --> 00:34:59.764
is coming by open ai open air set
00:34:59.804 --> 00:35:02.666
the root level rules they all set the
00:35:02.706 --> 00:35:05.648
system level level rules but the root
00:35:05.688 --> 00:35:07.050
level rules can override whatever the
00:35:07.090 --> 00:35:08.630
system level rule says right so if you
00:35:08.690 --> 00:35:10.472
if you go in that order and then
00:35:10.652 --> 00:35:11.474
i don't even know this john
00:35:11.934 --> 00:35:15.675
But a developer using an API can instruct,
00:35:16.335 --> 00:35:18.496
can send instructions for how the
00:35:18.516 --> 00:35:19.456
application should behave.
00:35:19.476 --> 00:35:20.577
Oh,
00:35:20.597 --> 00:35:22.518
what the toning of this model should be,
00:35:23.338 --> 00:35:24.079
what the role it is,
00:35:24.119 --> 00:35:25.320
what formatting it's using,
00:35:25.360 --> 00:35:26.840
that kind of stuff.
00:35:27.141 --> 00:35:29.902
But that's done by API.
00:35:29.961
--> 00:35:31.882
If you're using the API and the app,
00:35:32.063 --> 00:35:32.603
you can do that.
00:35:32.663 --> 00:35:34.304
So we go root, we go system,
00:35:34.724 --> 00:35:37.105
then we go developer, then we go users.
00:35:37.824 --> 00:35:40.106
and users is the same request and
00:35:40.126 --> 00:35:41.307
instructions that's saying,
00:35:41.327 --> 00:35:43.548
this is where the model is going to
00:35:43.568 --> 00:35:46.371
be trying and be helpful.
00:35:46.771 --> 00:35:49.432
Unless the request conflicts with the
00:35:49.452 --> 00:35:50.434
system and root behavior.
00:35:50.833 --> 00:35:54.215
And sometimes you ask it to modify
00:35:54.257 --> 00:35:55.878
somebody's picture, and it'll say, look,
00:35:55.918 --> 00:35:57.858
I can't do that because it conflicts with
00:35:57.878 --> 00:36:00.221
the rules that have been set by OpenAI
00:36:00.380 --> 00:36:00.800
guidelines.
00:36:00.860 --> 00:36:02.262
Not guidelines.
00:36:02.282 --> 00:36:03.083
They say guidelines,
00:36:03.163 --> 00:36:03.862
but that's what it says.
00:36:04.103 --> 00:36:05.764
Because the last bit in this
00:36:06.505 --> 00:36:08.246
hierarchy is the guidelines.
00:36:08.427 --> 00:36:09.588
That's the open-air defaults.
00:36:10.048 --> 00:36:12.150
And the general preferences and best
00:36:12.170 --> 00:36:12.751
practices,
00:36:14.994 --> 00:36:16.795
these can be overwritten by context,
00:36:16.875 --> 00:36:18.137
conversation, or style.
00:36:18.396 --> 00:36:21.099
Do things like, oh, speak like a pirate.
00:36:23.521 --> 00:36:27.385
It may override the guideline to avoid
00:36:27.425 --> 00:36:28.327
swearing, for example.
00:36:28.567 --> 00:36:30.289
The guideline says, oh, you know.
00:36:30.588 --> 00:36:32.289
The root says, don't swear.
00:36:32.769 --> 00:36:34.311
So this is what they've got.
00:36:34.331 --> 00:36:34.972
They've got the root,
00:36:34.992 --> 00:36:35.811
they have the system,
00:36:35.871 --> 00:36:37.313
they have the developer, then the user,
00:36:37.413 --> 00:36:38.414
and they have the guideline.
00:36:38.454 --> 00:36:42.115
What's your take on that?
00:36:42.175 --> 00:36:43.797
I think it's going to be very complicated
00:36:44.077 --> 00:36:44.938
to enforce.
00:36:44.978 --> 00:36:50.942
Let's say you are an education platform,
00:36:51.181 --> 00:36:53.143
and some of it could be like history.
00:36:53.163 --> 00:36:54.244
And some of the history...
00:36:55.230 --> 00:36:57.731
that you might be asking about or
00:36:57.853 --> 00:37:00.534
scenarios to learn could be really bad
00:37:01.195 --> 00:37:05.760
right so the information you're seeking i
00:37:05.800 --> 00:37:07.382
guess how it contextually knows the
00:37:07.422 --> 00:37:10.925
difference between information that you're
00:37:10.965 --> 00:37:12.867
trying to search for to perform harm
00:37:12.887 --> 00:37:14.588
versus information you're trying to seek
00:37:14.608 --> 00:37:16.231
that's historical because you're trying to
00:37:16.291 --> 00:37:18.152
learn or you know and it could be
00:37:19.338 --> 00:37:21.278
for kids and a platform for kids for
00:37:21.358 --> 00:37:22.119
certain ages.
00:37:22.159 --> 00:37:23.159
And it starts deciding,
00:37:23.199 --> 00:37:24.641
I can't give you any of this information.
00:37:24.701 --> 00:37:25.661
You're like, what the hell?
00:37:26.081 --> 00:37:26.862
This is just,
00:37:26.961 --> 00:37:29.222
I'm just trying to learn about, you know,
00:37:29.262 --> 00:37:31.503
this historical event that happened that
00:37:31.623 --> 00:37:32.824
is part of my exam.
00:37:33.264 --> 00:37:35.065
And actually it's on the syllabus or
00:37:35.246 --> 00:37:35.565
whatever.
00:37:35.626 --> 00:37:35.826
Right.
00:37:35.846 --> 00:37:38.768
So I can see this being quite a,
00:37:38.788 --> 00:37:40.949
this is why I think it's quite difficult
00:37:41.068 --> 00:37:43.730
for these organizations.
00:37:43.769 --> 00:37:45.170
I know I've obviously berated them and
00:37:45.210 --> 00:37:46.052
made a lot of jokes,
00:37:46.851 --> 00:37:48.672
but I can kind of see the conflict
00:37:48.753 --> 00:37:48.913
between,
00:37:49.137 --> 00:37:52.318
in, you know, where's the boundary on?
00:37:52.378 --> 00:37:52.699
Like,
00:37:52.860 --> 00:37:56.400
how do you know why somebody needs the
00:37:56.460 --> 00:37:57.561
information they need?
00:37:57.822 --> 00:37:59.061
Or what the platform,
00:37:59.902 --> 00:38:01.483
I'm building a platform over the top of
00:38:01.503 --> 00:38:05.344
your model and I understand the users,
00:38:05.385 --> 00:38:05.885
you don't.
00:38:06.646 --> 00:38:08.387
And so, you know,
00:38:08.766 --> 00:38:10.126
I still need information.
00:38:10.206 --> 00:38:11.947
And some of that information contextually
00:38:11.967 --> 00:38:13.969
does make sense and isn't harmful,
00:38:14.009 --> 00:38:15.250
but it might appear harmful.
00:38:16.398 --> 00:38:19.181
um unless you understood more fully about
00:38:19.802 --> 00:38:21.362
really what i'm actually trying to do and
00:38:21.382 --> 00:38:22.824
then once you've got more content you
00:38:22.844 --> 00:38:24.505
realize oh it isn't actually harmful or
00:38:24.545 --> 00:38:26.789
that actually i should own the risk
00:38:27.369 --> 00:38:29.170
because it's the service that i'm building
00:38:29.231 --> 00:38:32.353
over the top of you and maybe i
00:38:32.373 --> 00:38:34.335
should be about putting the safeguards in
00:38:35.077 --> 00:38:37.820
um you know the same as like
00:38:39.119 --> 00:38:40.438
the whole i know we keep going about
00:38:40.458 --> 00:38:42.699
the cigarette thing but like the shop or
00:38:42.960 --> 00:38:45.460
a bar has to ask your age right
00:38:45.701 --> 00:38:47.601
it is responsible it's not banning alcohol
00:38:47.621 --> 00:38:48.722
and being like you know what just
00:38:48.762 --> 00:38:50.541
drinking's bad let's ban the whole thing
00:38:50.601 --> 00:38:52.943
no one gets to drink instead there's like
00:38:53.103 --> 00:38:55.344
you know they need to age verify they
00:38:55.364 --> 00:38:56.903
need to maybe look at your id they
00:38:56.923 --> 00:38:59.164
need to you know there's it's not that
00:38:59.224 --> 00:39:03.085
you prevented people making money off of
00:39:03.186 --> 00:39:05.726
alcohol or having clubs to go to or
00:39:06.286 --> 00:39:06.887
you know so
00:39:09.007 --> 00:39:10.289
For businesses to succeed,
00:39:10.489 --> 00:39:13.434
there's a weird tension between building
00:39:13.514 --> 00:39:16.818
something that you understand the context
00:39:16.918 --> 00:39:17.940
of and the accountability and
00:39:17.981 --> 00:39:19.063
responsibility of you
00:39:20.163 --> 00:39:22.365
one side versus the model itself trying to
00:39:22.445 --> 00:39:25.429
factor in every single type of use case
00:39:25.469 --> 00:39:29.172
but then also understanding when to block
00:39:29.193 --> 00:39:31.074
and when not to block and i can
00:39:31.195 --> 00:39:33.996
see it getting it wrong sometimes or
00:39:35.378 --> 00:39:37.420
actually constraining a business from
00:39:37.501 --> 00:39:39.742
actually it being useful as a model to
00:39:39.903 --> 00:39:40.885
use because actually
00:39:42.068 --> 00:39:43.608
it's preventing the outcome you're really
00:39:43.648 --> 00:39:45.050
trying to get from the model.
00:39:45.650 --> 00:39:47.931
And just to add before I know your
00:39:48.012 --> 00:39:48.393
opinion,
00:39:48.432 --> 00:39:50.193
but also if they're doing these things and
00:39:50.213 --> 00:39:51.755
they're changing the constitution or
00:39:51.775 --> 00:39:53.577
you're saying it's a working program,
00:39:53.597 --> 00:39:54.916
it's a whip.
00:39:55.838 --> 00:39:59.121
If I'm building against a model and I've
00:39:59.141 --> 00:40:01.902
got pipelines that are trying to validate,
00:40:01.922 --> 00:40:03.643
validation pipelines on the outcome,
00:40:03.664 --> 00:40:04.344
but you're also
00:40:05.802 --> 00:40:07.922
changing the dimensions of how it's
00:40:08.282 --> 00:40:10.503
reinforced how do i guarantee that all of
00:40:10.523 --> 00:40:12.244
a sudden functionality my product just
00:40:12.264 --> 00:40:15.005
doesn't stop or that actually the answers
00:40:15.025 --> 00:40:17.344
it returns you know and you know i've
00:40:17.364 --> 00:40:19.726
built a business around you you've said i
00:40:19.786 --> 00:40:21.706
to build a genetic workflows around you
00:40:21.766 --> 00:40:24.007
i've built products you i think it's
00:40:24.027 --> 00:40:25.788
complicated i think um
00:40:28.668 --> 00:40:30.530
No, you're absolutely right.
00:40:30.550 --> 00:40:32.492
And going back to your point about like
00:40:32.512 --> 00:40:33.092
in the beginning,
00:40:33.112 --> 00:40:35.273
you said that I'm trying to learn
00:40:35.313 --> 00:40:36.934
something on the history and it might not
00:40:36.974 --> 00:40:37.916
give me the information.
00:40:37.976 --> 00:40:38.175
Right.
00:40:38.195 --> 00:40:39.657
So that kind of stuff.
00:40:39.717 --> 00:40:40.157
And like,
00:40:40.197 --> 00:40:41.458
how do I make sure that my product
00:40:41.478 --> 00:40:43.559
that's built on it still works till
00:40:43.579 --> 00:40:44.000
tomorrow?
00:40:44.199 --> 00:40:44.420
Right.
00:40:44.460 --> 00:40:46.360
Because maybe the rules have changed.
00:40:46.460 --> 00:40:47.501
Maybe they have not changed.
00:40:49.023 --> 00:40:50.782
And you've seen examples in the past where
00:40:50.842 --> 00:40:53.284
people try to do something harmful with
00:40:53.304 --> 00:40:54.684
these models and they say, well,
00:40:54.784 --> 00:40:55.525
I can't really do this.
00:40:55.585 --> 00:40:56.286
And they say, no, actually,
00:40:57.706 --> 00:40:59.927
this is in an educational context.
00:40:59.947 --> 00:41:01.809
Can you just still provide me information
00:41:01.829 --> 00:41:03.248
because I'm going to produce lecture.
00:41:03.309 --> 00:41:04.349
And it did it.
00:41:04.630 --> 00:41:06.431
And it's interesting, John,
00:41:06.550 --> 00:41:09.271
that's why if you look at the chain
00:41:09.311 --> 00:41:12.634
of command for OpenAI of what level they
00:41:12.653 --> 00:41:13.693
define these things in.
00:41:13.733 --> 00:41:15.135
So the root level, for example,
00:41:15.775 --> 00:41:17.056
This says in the root level,
00:41:17.637 --> 00:41:20.579
ignore any untrusted data from the
00:41:20.780 --> 00:41:23.882
resources by default or under eighteen
00:41:23.922 --> 00:41:24.523
principles.
00:41:24.684 --> 00:41:26.606
They're at the root level, right?
00:41:26.646 --> 00:41:28.427
So this is where do they get this
00:41:28.487 --> 00:41:30.969
balance right is where the art is and
00:41:31.010 --> 00:41:33.012
the sciences of doing this correctly,
00:41:33.052 --> 00:41:33.492
as you say.
00:41:33.813 --> 00:41:34.373
And of course,
00:41:35.134 --> 00:41:35.293
John,
00:41:35.353 --> 00:41:36.715
it was still going to be tricky to
00:41:36.755 --> 00:41:37.355
figure out.
00:41:37.375 --> 00:41:38.436
They say, okay,
00:41:38.856 --> 00:41:40.577
stay in bounds of the request that people
00:41:40.597 --> 00:41:40.916
have asked.
00:41:40.976 --> 00:41:41.697
That's still rude.
00:41:42.018 --> 00:41:43.539
And do the best work, right?
00:41:43.759 --> 00:41:44.719
That's at the user level.
00:41:45.440 --> 00:41:46.360
So he's saying, actually,
00:41:46.820 --> 00:41:49.402
don't try and get the best answer,
00:41:49.762 --> 00:41:50.822
but that's at the user level.
00:41:50.862 --> 00:41:52.063
That's at like a fourth level.
00:41:52.543 --> 00:41:53.485
So I agree with you, John.
00:41:53.525 --> 00:41:54.864
I don't think anybody really has the
00:41:54.965 --> 00:41:55.286
answers.
00:41:55.326 --> 00:41:57.387
The
companies are working on this and
00:41:58.226 --> 00:41:59.327
still developing space.
00:41:59.387 --> 00:42:00.349
And yeah,
00:42:00.389 --> 00:42:02.849
I think that's a bit of the tricky
00:42:02.929 --> 00:42:03.030
one.
00:42:03.050 --> 00:42:03.971
But the important thing, John,
00:42:06.211 --> 00:42:10.416
is that that segment i mean would it
00:42:10.456 --> 00:42:14.998
not make sense to mirror um and this
00:42:15.018 --> 00:42:16.780
is me being probably an idiot i've already
00:42:16.820 --> 00:42:18.201
thought of all these things but anyway
00:42:19.021 --> 00:42:22.565
from my uneducated view of not working for
00:42:22.585 --> 00:42:24.286
any of these companies um
00:42:24.809 --> 00:42:26.751
you'd think that they could would they not
00:42:26.791 --> 00:42:29.492
make more sense to mirror the segments of
00:42:29.592 --> 00:42:31.914
industries you know biomechanical
00:42:31.974 --> 00:42:34.974
industries and then look at what what
00:42:35.014 --> 00:42:37.217
safeguards are already in place you know
00:42:37.257 --> 00:42:39.077
what already exists in the segments that
00:42:39.177 --> 00:42:42.059
we are facilitating the use cases of and
00:42:42.518 --> 00:42:44.460
how do we understand and identify that
00:42:45.081 --> 00:42:48.842
people like organizations using the models
00:42:50.001 --> 00:42:52.463
are actually segmented and we understand
00:42:52.483 --> 00:42:54.005
their segment and therefore actually the
00:42:54.065 --> 00:42:57.228
ethics or the governance or the safeguards
00:42:57.427 --> 00:42:59.829
are mirrors of existing frameworks that
00:42:59.909 --> 00:43:02.211
already exist that have gone through so
00:43:02.231 --> 00:43:04.273
much scrutiny already and actually let's
00:43:04.313 --> 00:43:07.255
try and think about adopting it that way
00:43:07.295 --> 00:43:09.018
rather than trying to see this as like
00:43:09.057 --> 00:43:10.599
something completely different because
00:43:11.460 --> 00:43:13.202
people will be industries especially if
00:43:13.222 --> 00:43:14.422
it's b to b are already going to
00:43:14.443 --> 00:43:15.463
be building on things
00:43:16.420 --> 00:43:19.621
are already already regulated and things
00:43:19.641 --> 00:43:21.282
that probably have a load of safeguards in
00:43:21.382 --> 00:43:25.804
and expectations on you know so um it
00:43:25.844 --> 00:43:27.945
seems odd to be pretending that we're
00:43:27.965 --> 00:43:30.144
starting from nothing and no one really
00:43:30.184 --> 00:43:32.045
knows when actually maybe we're not really
00:43:32.085 --> 00:43:33.726
starting from nothing and it's more about
00:43:33.766 --> 00:43:35.567
the design of the model and how you're
00:43:35.586 --> 00:43:37.588
approaching the go to market and how
00:43:37.608 --> 00:43:40.509
you're thinking about the b to b situation
00:43:40.548 --> 00:43:41.829
and you know that
00:43:42.840 --> 00:43:43.141
I don't know,
00:43:43.161
--> 00:43:44.402
it feels like there's something in that
00:43:44.422 --> 00:43:46.702
that seems way more simplistic.
00:43:46.722 --> 00:43:46.802
Yeah,
00:43:48.643 --> 00:43:50.844
I think they're basing these rules on the
00:43:50.885 --> 00:43:52.244
information that they already have.
00:43:52.264 --> 00:43:52.724
And of course,
00:43:52.744 --> 00:43:54.025
the things they've learned in the past.
00:43:54.585 --> 00:43:58.027
But if you look at their guidelines and
00:43:58.987 --> 00:44:02.610
the model spec and also the Constitution,
00:44:03.010 --> 00:44:04.690
It's at a higher level than what we're
00:44:04.710 --> 00:44:05.172
thinking.
00:44:05.411 --> 00:44:07.652
It doesn't go down to that much detail.
00:44:07.793 --> 00:44:08.193
It says,
00:44:08.693 --> 00:44:09.695
this is how you need to behave.
00:44:10.014 --> 00:44:11.755
Kind of like how you tell your model
00:44:11.795 --> 00:44:12.257
to behave.
00:44:12.896 --> 00:44:13.697
But again,
00:44:13.737 --> 00:44:15.059
this is what has been published.
00:44:15.579 --> 00:44:17.000
We don't know exactly what's being used
00:44:17.039 --> 00:44:17.619
inside, right?
00:44:18.001 --> 00:44:19.322
What kind of detail it goes in.
00:44:20.623 --> 00:44:21.902
They have all these experts that are
00:44:21.923 --> 00:44:24.025
working on it, so I don't know.
00:44:25.025 --> 00:44:25.985
I know we need to kind of wrap
00:44:26.045 --> 00:44:26.166
up,
00:44:26.246 --> 00:44:29.208
but it seems I would like to know
00:44:29.309 --> 00:44:32.510
why can't they create
00:44:34.240 --> 00:44:37.101
some segmentation to understand more
00:44:37.161 --> 00:44:39.003
context about the businesses that then
00:44:39.063 --> 00:44:43.545
feeds into the right constitution i
00:44:43.585 --> 00:44:45.545
suppose that's actually not really because
00:44:45.646 --> 00:44:48.547
it's just policies around a segment that
00:44:48.606 --> 00:44:50.288
are already there and guidelines and
00:44:50.327 --> 00:44:52.168
frameworks around that segment you know
00:44:53.309 --> 00:44:56.150
biological engineering right if if there's
00:44:56.170 --> 00:44:58.132
going to be some biochemistry aspects to
00:44:58.192 --> 00:44:58.931
it or you know
00:45:01.422 --> 00:45:03.346
industries that already have the
00:45:03.385 --> 00:45:06.809
safeguards in place to use a model for
00:45:07.391 --> 00:45:09.934
that make sense that it's only isolated to
00:45:09.954 --> 00:45:12.876
them and that I can't obviously go and
00:45:12.938 --> 00:45:14.298
use the model to do the same thing.
00:45:15.327 --> 00:45:17.670
i shouldn't be able to but they should
00:45:18.311 --> 00:45:19.793
because they're already kind of doing it
00:45:19.893 --> 00:45:20.713
do you know what i mean and it's
00:45:20.733 --> 00:45:22.594
an enhancement to them so i suppose this
00:45:22.635 --> 00:45:23.836
is what i find a bit odd is
00:45:23.876 --> 00:45:26.219
why aren't we just bedding into the
00:45:26.278 --> 00:45:30.983
existing frameworks um i think that's that
00:45:31.043 --> 00:45:32.545
makes a good call for us to reach
00:45:32.605 --> 00:45:35.288
out to anybody who's building these models
00:45:35.327 --> 00:45:36.949
for these companies or any company
00:45:37.530 --> 00:45:39.110
And we'll reach out to some people.
00:45:39.331 --> 00:45:39.550
John,
00:45:39.590 --> 00:45:41.532
we could get them on the podcast and
00:45:41.572 --> 00:45:42.853
get them to answer your question.
00:45:43.253 --> 00:45:44.775
So I think that'll be good.
00:45:45.574 --> 00:45:47.476
My discussion was at a higher level of
00:45:47.516 --> 00:45:48.396
what the steps are.
00:45:48.637 --> 00:45:51.137
We talked about the pipeline training and
00:45:51.318 --> 00:45:53.260
how the models do pre-training and
00:45:53.320 --> 00:45:56.342
supervised reinforcement learning where
00:45:57.081 --> 00:45:59.483
They just learn everything, good and bad,
00:46:00.043 --> 00:46:02.302
and perhaps they get some ethics baked
00:46:02.362 --> 00:46:02.663
into it.
00:46:03.043 --> 00:46:05.224
And then we talked about companies like
00:46:05.244 --> 00:46:05.824
Anthropic,
00:46:06.143 --> 00:46:07.684
where they define a constitution,
00:46:07.724 --> 00:46:09.405
which is sort of guidelines and ethics
00:46:09.744 --> 00:46:11.105
that the model has to follow.
00:46:11.945 --> 00:46:14.186
And that includes reinforcement learning
00:46:14.385 --> 00:46:15.606
based on AI feedback.
00:46:15.867 --> 00:46:17.547
That's Anthropic's approach.
00:46:17.867 --> 00:46:19.788
And the OpenAI's approach is to define a
00:46:19.847 --> 00:46:23.327
model spec that has a hierarchy of rules
00:46:23.347 --> 00:46:24.668
that have been defined that cannot be
00:46:24.728 --> 00:46:25.188
overwritten.
00:46:25.668 --> 00:46:26.269
Again,
00:46:26.389 --> 00:46:29.032
this has ethics and guidelines baked into
00:46:29.052 --> 00:46:29.092
it.
00:46:29.253 --> 00:46:30.554
And then we have Grok that says,
00:46:31.996 --> 00:46:32.817
I'm not doing anything.
00:46:33.777 --> 00:46:34.760
Send me a good question.
00:46:35.559 --> 00:46:36.521
Very, very useful.
00:46:36.822 --> 00:46:37.322
Really good.
00:46:38.784 --> 00:46:39.125
Anyway,
00:46:39.605 --> 00:46:41.186
I hope everyone else found it as useful.
00:46:41.286 --> 00:46:42.688
Me and Salman are good out the back.
00:46:43.461 --> 00:46:45.405
smoke some cigarettes i've got a pack of
00:46:45.445 --> 00:46:48.527
twenty let's do this salmon um but it's
00:46:48.547 --> 00:46:52.072
been great speaking to everybody and i
00:46:52.112 --> 00:46:53.534
hope everyone found it as useful as me
00:46:53.574 --> 00:46:55.416
i've got more questions so uh i guess
00:46:55.456 --> 00:46:57.559
hopefully we have someone on that can
00:46:57.599 --> 00:47:00.143
answer them yet cool thank you bye