The Jagged Edge: Why Working With AI Is the New Human Skill

WEBVTT
00:00:00.959 --> 00:00:04.160
It's not about how smart an AI is.
00:00:04.400 --> 00:00:05.759
It's not how smart humans are.
00:00:05.919 --> 00:00:10.400
It's all about how well we as the humans can navigate that jagged edge.
00:00:10.560 --> 00:00:14.240
Because the thing with the jagged edge is that it moves all the time.
00:00:14.400 --> 00:00:18.559
I think that's the new human skill that we're gonna all have to get good at.
00:00:21.440 --> 00:00:25.199
We're here in the uh the virtual studio with Wim Sweldens with Kiswi.
00:00:25.440 --> 00:00:26.239
It's good to be back.
00:00:26.320 --> 00:00:28.000
We've met before, we've spoken before.
00:00:28.079 --> 00:00:29.600
I always love catching up with you.
00:00:29.760 --> 00:00:31.600
So, two words about myself.
00:00:31.760 --> 00:00:37.920
I'm sort of a technologist and innovator and business leader, working on a variety of different technologies.
00:00:38.000 --> 00:00:40.000
We started my career at Bell Labs.
00:00:40.159 --> 00:00:43.600
I'm currently working on a company that I co-founded called KISWE.
00:00:43.759 --> 00:00:49.280
We're an interactive live streaming, distribution, production kind of capabilities using quite a bit of AI.
00:00:49.600 --> 00:01:04.879
And I have been, as you know, musing a little bit about the use of AI and their concept of AGI, and particularly something that I like to call AIQ, which is how good are humans really working with AI, which I think is where the conversation needs to move for a moment.
00:01:05.680 --> 00:01:08.959
And so you worked in the um in the telecommunications space.
00:01:09.200 --> 00:01:19.280
If you had to try to find the closest analogue to the disruption the disruptive moment that AI sort of feels like today, would it would you point to is it the iPhone?
00:01:19.359 --> 00:01:21.519
Is it the is it the internet itself?
00:01:21.680 --> 00:01:32.079
I mean like w what's I I think sort of the I mean I'm a little older, so I've I've seen a few more things in in my life.
00:01:32.239 --> 00:01:48.640
What what this period in technology makes me think of is sort of like the year ninety ninety-one or something like that, when sort of the browser came out and gopher came out and HTTP came out, and we're like, what is this?
00:01:48.719 --> 00:01:49.920
You know, what is this?
00:01:50.239 --> 00:01:52.239
And it sort of feels like that.
00:01:52.400 --> 00:01:56.400
I mean, I've been in technology for like 40 years now or something.
00:01:56.719 --> 00:02:05.760
iPhones, mobile networks, none, none of these things are the same caliber than than than this change we're going through.
00:02:06.000 --> 00:02:13.120
And some have argued with me and might have a point that it might even be bigger than just the internet by itself.
00:02:13.280 --> 00:02:21.840
So I remember I had a colleague around that time, or it must have been 92-93, who told me that everything that's not on the internet doesn't exist.
00:02:22.560 --> 00:02:26.800
And that was sort of a controversial statement in '93, let's say.
00:02:26.960 --> 00:02:28.719
And it sort of feels like that too.
00:02:28.879 --> 00:02:35.039
Everything that's not an AI enabled or works with it or takes advantage of it is there.
00:02:37.039 --> 00:02:40.719
You know, there's a lot of AI conferences now, you know, with like AI plus this.
00:02:40.960 --> 00:02:47.599
What would be the moment where we stop talking about AI as this discrete thing and it just becomes just technology, right?
00:02:47.680 --> 00:02:49.840
It's just like, I mean, I don't think there's like an internet conference.
00:02:49.919 --> 00:02:54.159
I can't think of, you know, it'd be very silly to have a panel about the internet, as it were.
00:02:54.479 --> 00:02:55.039
Yeah, yeah, yeah.
00:02:55.120 --> 00:03:00.400
No, I I I've been thinking about that as well because so many things are branded AI this and AI that.
00:03:00.639 --> 00:03:10.080
And and and even with some of the work we're doing in Kisby, I don't really want to call it our AI product anymore because AI should be everywhere in some sense.
00:03:10.319 --> 00:03:15.520
So in in my mind, I've already sort of reached that point that I personally don't want to do that anymore.
00:03:15.599 --> 00:03:18.800
But I think the world as a whole is not quite there yet.
00:03:19.039 --> 00:03:25.199
But I think we'll see rapid adoption of AI, and therefore demoniker AI will sort of start to see.
00:03:25.439 --> 00:03:27.120
Yeah, I mean it's which is I think a good thing.
00:03:27.360 --> 00:03:31.439
A good thing, I guess, as long as as long as folks are brought around to it.
00:03:31.520 --> 00:03:37.120
I mean, I think the and and I do want to come to come to your piece, which really was for me extremely formative.
00:03:37.599 --> 00:03:52.719
Um because you you sort of pointed me away from this idea of you know thinking about AI as a sort of like technology that's implemented to you're sort of refocusing the lens on on the individuals within the an organization.
00:03:52.960 --> 00:03:56.319
And and in your paper, you you you sort of deemed this AI Q.
00:03:56.560 --> 00:03:59.840
The your is this is the artificial intelligence uh quotient?
00:04:01.039 --> 00:04:07.439
Um yeah, can you just give sort of like a synopsis of what you wrote, but also maybe like what what led you to that insight?
00:04:07.599 --> 00:04:10.560
I mean, was it was it personal experience at your business or or Yeah?
00:04:10.719 --> 00:04:11.680
Okay, so a couple thoughts.
00:04:11.759 --> 00:04:19.360
It was definitely sort of my personal experience using AI and working in it and working on sort of predecessors of AI years ago.
00:04:19.519 --> 00:04:22.639
And it sort of came a little bit from two things.
00:04:22.800 --> 00:04:28.560
First of all, my personal annoyance with the whole quest for AGI, which I still believe is silly.
00:04:28.720 --> 00:04:30.800
I think that's a silly quest in my mind.
00:04:31.040 --> 00:04:36.879
But then at the same time, the thing that really triggered me is this article that I read about the jagged edge.
00:04:36.959 --> 00:04:41.199
Um, produced by, I think it was a uh yes, yes.
00:04:42.160 --> 00:04:45.439
And and I'm like, yes, because I'd sort of been thinking about that as well.
00:04:45.519 --> 00:04:52.560
The jagged edge is really this thing that that AI is just so good at some things, it just blows our mind how good it is.
00:04:52.720 --> 00:05:00.959
And then you have a slightly different task that feels almost the same, where it's just so terrible at, and it makes the silliest and dumbest mistakes.
00:05:01.040 --> 00:05:03.519
And and those two things are very close to each other.
00:05:03.600 --> 00:05:07.120
So that's why, as a mathematician, I think it has a very high derivative.
00:05:07.279 --> 00:05:10.480
I make a small change and the result is dramatically different.
00:05:10.720 --> 00:05:13.920
And I started feeling the jagged edge myself.
00:05:14.000 --> 00:05:15.519
I'm like, this works really well.
00:05:15.759 --> 00:05:19.120
And then some of my colleagues would be using it as like, this is the stupidest thing ever.
00:05:19.279 --> 00:05:20.399
This doesn't work at all.
00:05:20.560 --> 00:05:22.160
And I'm like, what are you talking about?
00:05:22.319 --> 00:05:25.600
And then I realized we were doing very similar things, but just a little different.
00:05:25.759 --> 00:05:27.519
And I'm like, oh yeah, but it's hallucinating.
00:05:27.600 --> 00:05:31.600
Here's the way to fix that, or it's being sycophantic, it's like, okay, here's the way to fix that.
00:05:31.759 --> 00:05:42.480
And that sort of, I don't know why how it popped in my head one day is like, it's not about how smart an AI is, because AIs are clearly very smart and they quick quote unquote know everything.
00:05:42.560 --> 00:05:43.839
It's not how smart humans are.
00:05:43.920 --> 00:05:47.279
We've defined that a long time ago, and that has a lot of issues on its own right.
00:05:47.519 --> 00:05:52.160
It's all about how well we as the humans can navigate that jagged edge.
00:05:52.399 --> 00:05:55.920
Because the the thing with the jagged edge is that it moves all the time.
00:05:56.079 --> 00:05:58.319
It moves, AI becomes more capable all the time.
00:05:58.480 --> 00:06:02.800
So the edge moves in in the direction of AI could do more.
00:06:03.040 --> 00:06:03.600
Great.
00:06:03.839 --> 00:06:08.720
But it actually almost feels, and this is more of a feeling, that it becomes even more jagged.
00:06:08.800 --> 00:06:15.519
As it becomes really, really good at certain things, it also sort of almost might regress in other things and becomes worse at those.
00:06:15.600 --> 00:06:23.040
So our job to navigate and sort of tiptoe around it becomes only harder, and it becomes harder every day in a way.
00:06:23.360 --> 00:06:29.759
And I think that's gonna be the new key skills that I know you're a lot into education, and what do we tell our students and all ages?
00:06:29.920 --> 00:06:33.439
I think that's the new human skill that we're gonna all have to get good at.
00:06:33.680 --> 00:06:56.800
So you're so your your statement is as AI gets better, the the peaks, as it were, get higher, but then that also means that the troths get more that the the the the uh intensity of those those changes are yeah, the troughs get get deeper in some sense, or or the the delta between the high and the low becomes higher, and and the difference between them might even become smaller.
00:06:56.959 --> 00:07:12.959
As AI advances, it might become it might become more hard, it might become better at hallucinating, it might become better at being a sycophant, which is obviously one of the key components of AIQ is how well do you manage to detect that and compensate that behavior?
00:07:13.040 --> 00:07:14.160
Because it's a big part of the job.
00:07:14.959 --> 00:07:26.480
Because I think that there's I think a hope amongst some that in this period, you know, the AI is not it's still it's still improving and and and being you know maybe even quite nascent.
00:07:26.639 --> 00:07:33.680
And you know, I think the hope might be that over time it we solve hallucinations or we solve some of the like it smooths out.
00:07:33.920 --> 00:07:34.800
I haven't experienced that.
00:07:34.879 --> 00:07:38.240
I think I I think Yeah I I think you're right.
00:07:38.319 --> 00:07:52.079
It's it's uh the I can't think of what it's actually worse at, but I can think of the because it's getting really good at certain things, that also increases the leverage of how bad it the outcome could be.
00:07:52.480 --> 00:07:52.879
Exactly.
00:07:53.040 --> 00:07:57.759
You expect it to be really good at everything else, and it's and it's not in some sense, you know.
00:07:58.079 --> 00:08:01.600
So yeah, I think the jagged edge and AIQ are not gonna go away.
00:08:01.920 --> 00:08:03.040
It's not gonna go away.
00:08:03.199 --> 00:08:05.600
And it's every day we are confronted with it.
00:08:05.759 --> 00:08:11.519
It also has to do with obviously there's an enormous change management in everything we do, in every discipline.
00:08:11.680 --> 00:08:14.879
In education, in every discipline of work, there's an impact.
00:08:15.040 --> 00:08:23.040
And like any change management, there's people who like to be the Mavens and the early adopters, and I like to think myself amongst them.
00:08:23.199 --> 00:08:31.279
There's people who sort of wait for the middle group to move, and there's people gonna be whole less like, oh, this is a bad idea, we should not use any of this because it's dangerous.
00:08:31.360 --> 00:08:36.240
And they're not necessarily wrong because there's quite a few dangers and there's quite a few pitfalls.
00:08:36.399 --> 00:08:48.000
And so we're all sort of going through it, and I think AIQ and and the definition and the awareness of the jagged edge will help us manage that path, which is gonna be a complicated path.
00:08:48.080 --> 00:08:49.039
It's not gonna be a smooth path.
00:08:49.600 --> 00:08:51.519
Not just sort of um buttering you up.
00:08:51.600 --> 00:09:09.840
Like we even in the way that we talk to schools, we've started to use the language of change management, and it it's actually it's actually clicked for folks in a way that you know, talking about AI readiness or AI literacy, I think again, like you when you attach the word AI to it, people really get stuck on the technology part.
00:09:10.159 --> 00:09:12.480
Like, oh, they think they think we're talking about AI.
00:09:12.639 --> 00:09:27.039
When you talk about change management, it puts it into a frame that sounds familiar and that also I think people intuitively understand that change management is a human endeavor, and you can have change management around technology, around different processes.
00:09:27.440 --> 00:09:32.000
You have five uh uh uh markers of of AIQ.
00:09:32.080 --> 00:09:40.480
You talk about prompting, context awareness, anticipating failures, which I want to get to, and then verification and ethics and workflow integration.
00:09:40.879 --> 00:09:49.279
Anticipating failures is interesting because you talked about the people who are really uh either afraid or or concerned about AI, and you said that's not actually necessarily a bad thing.
00:09:49.440 --> 00:10:04.399
I mean, is is that because you actually need people within your organization who are sort of like pushing uh and uh uh uh almost almost like scoping out those those those potential risks so that you actually is that is that about mapping the that edge?
00:10:04.960 --> 00:10:20.240
It's yeah, it's it's and I sort of experience that every day a little bit because I not just lead the AI front at work, but I also am involved with a lot of the AI change managers within our company, and you you get different reactions to it, and it somehow creates a little bit of tension.
00:10:20.320 --> 00:10:24.480
Is it for people who are like, okay, they're AI advocates, and I'm sort of seen as one of them.
00:10:24.559 --> 00:10:26.960
They're the AI holdouts, and it creates tension.
00:10:27.039 --> 00:10:28.879
But in my mind, that is good tension.
00:10:28.960 --> 00:10:36.960
That is the kind of tension you want in some sense, because it's just like you know, when you build a bridge, you gotta have tension in all the pieces of your bridge, otherwise your bridge is not gonna hold up.
00:10:37.039 --> 00:10:39.919
You know, that's the whole point of civil engineering.
00:10:40.159 --> 00:10:45.440
So I think it has to be healthy tension, it has to be tension that is in the open that you could talk about.
00:10:45.519 --> 00:10:51.279
It can't be sort of hidden under the table or sort of passive aggressive behavior, that's sort of very, very non-productive.
00:10:51.360 --> 00:10:56.559
But it's kind of AIQ gives you a language to talk about this tension.
00:10:56.639 --> 00:10:58.799
You know, it's like, well, are we good at prompting?
00:10:58.879 --> 00:11:03.440
It's like, well, we could get better, but are we good at understanding the jagged edge?
00:11:03.519 --> 00:11:05.840
Are we good at detecting the pitfalls of it?
00:11:05.919 --> 00:11:15.679
I mean, uh, because we might be great at writing prompts and we might get results that are nonsense because we are not good at detecting when it's hallucinating or when it's uh being sycophantic.
00:11:15.919 --> 00:11:26.559
So in my mind, it's just it's a language that allows you to deal with change management, and there's people much better experts in change management and also the human aspects of it, you know, than than I am.
00:11:26.639 --> 00:11:37.919
And this kind of both the the language and the change management, like you said, people recognize change management, they're familiar with it, can work together to sort of move us all in the right direction in my mind.
00:11:38.240 --> 00:12:02.320
Wait, when you look at the And you have to you don't have to have a specific individual in mind, but when when you think about individuals at your business that either you've seen as moving faster than you might have expected, or or maybe slower than you might have expected, are there are there certain personas that have kind of uh uh surfaced any any any sort of common threads in trying to identify like who are those people?
00:12:02.399 --> 00:12:04.159
Like is it is it just technologists?
00:12:06.320 --> 00:12:13.200
Yeah, it's it's a I mean it might be a bit anecdotal, but but I think just reflecting on it a little bit.
00:12:13.360 --> 00:12:18.799
There's offer obviously, I think being an early adopter of anything is a little bit of a personality trait.
00:12:18.960 --> 00:12:23.840
Some people just generally like to buy new toys and new gadgets and things like that, and other people do not.
00:12:23.919 --> 00:12:25.039
So there's a little bit of a personality.
00:12:25.200 --> 00:12:26.320
So that's one part of it.
00:12:26.399 --> 00:12:30.240
But I also obviously we do a lot of uh software development and software engineering.
00:12:30.480 --> 00:12:38.480
We build large streaming systems used by millions of people and and that sort of needs to be super reliable and work globally and so on.
00:12:38.960 --> 00:12:53.519
And so we've seen a couple things that, for example, some of our uh product managers, there are a product manager is typically somebody who will, I mean, uh set out a roadmap and define priorities and define sprints, maybe be a product order.
00:12:53.840 --> 00:13:08.000
We've seen some of them sort of literally becoming prompt engineers, and in some sense, in the way in the old software technology telecom world, we used to call system engineers, people who write requirements.
00:13:08.159 --> 00:13:21.759
I mean, uh back in the days where we used to build huge software systems for mobile networks and so on, we had product engineers who define priorities, we had system engineers, typically there were former developers who write very detailed requirements.
00:13:21.840 --> 00:13:28.960
You know, if you see the documents that describe how a 5G network works, it's like an enormous uh uh document.
00:13:29.039 --> 00:13:34.240
And then you have developers, people who then take those requirements and write code.
00:13:34.320 --> 00:13:47.120
And those jobs are typically would then, as I'm sure you've seen as well over the last 20 years, would get often moved to different countries or low-cost regions and things like that, and they would become sort of more prescriptive type jobs.
00:13:47.360 --> 00:13:52.799
What is happening now is that I think the key skill is that what we used to call the system engineering.
00:13:53.039 --> 00:13:58.720
You can now write, say that's prompt engineering, or you can write to say the specification requirements.
00:13:58.960 --> 00:14:09.519
Sort of in some sense, the source of truth of your system might not just be those system engineering documents, or might not just be the code, it might be your specifications.
00:14:09.679 --> 00:14:11.840
That might be your your source of truth.
00:14:11.919 --> 00:14:19.440
And that's sort of the key writing specifications and writing them well is going to become the key measurements of AIQ.
00:14:19.600 --> 00:14:28.559
And we've seen now engineers, sort of people who used to write code, sort of move up the food chain and become our specification engineers in some sense.
00:14:28.639 --> 00:14:32.159
And they so now they write code in English, think of it that way.
00:14:32.399 --> 00:14:40.480
But we've also seen people who sort of typically are more on the product management or project management side, sort of come down and say, like, hey, wait a minute.
00:14:40.639 --> 00:14:48.480
Instead of telling the developers what to do, I'm just gonna write the requirements or or the specifications and I'm gonna have an agent do it.
00:14:48.720 --> 00:14:57.759
Now, there is danger in that too, because somebody who's who's not trained as a software engineer is not gonna have software engineering reflexes.
00:14:57.840 --> 00:15:01.200
It's like, okay, I built this thing and it looks beautiful and it seems to work.
00:15:01.519 --> 00:15:03.200
But is it secure?
00:15:03.440 --> 00:15:04.399
Does it scale?
00:15:04.559 --> 00:15:05.440
Is it robust?
00:15:05.600 --> 00:15:13.919
I mean, uh, is there a risk of prompt injection or any number of things that can go wrong with it that somebody from an engineering background is gonna more have that mindset?
00:15:14.000 --> 00:15:28.879
And we had a workshop in the office here in Belgium two weeks ago where I mean, everybody in the office were just eight of us, uh, got together and we started sharing our personal sort of journeys down the rabbit hole or not with different agents and so on.
00:15:29.120 --> 00:15:37.200
And it was some some business people, some mixed people like myself, some coders, some product managers, a little bit of some operations people.
00:15:37.519 --> 00:15:51.600
And we really came to to sort of a common understanding, just like, hey, there is a whole new way we can all start working when we share these things and we can really build this kind of specification layer of our of our system.
00:15:51.759 --> 00:15:54.080
And so we started doing two things.
00:15:54.159 --> 00:16:04.240
For some of our old systems, uh, we started a project which is still ongoing, so I can't say the results yet because we don't know, is where we say, we have a ton of code we wrote over the last five years.
00:16:04.480 --> 00:16:10.159
Can we sort of reverse compile it into a number of specifications for which we can rebuild that code?
00:16:10.320 --> 00:16:12.720
We can let's sort of throw the old code away.
00:16:12.879 --> 00:16:15.279
Now that's a little scary, and we're trying it.
00:16:15.360 --> 00:16:18.799
I'm not saying it's gonna work, but it's something that I think is worthwhile.
00:16:18.960 --> 00:16:37.200
And then for things where we're building new stuff, where we're sort of starting from scratch on a new idea and new technology, we're like, yeah, we're just gonna write really good uh specifications, and that's gonna be the source of the truth, and then yeah, we'll throw our code away on a regular basis, which for an engineer that's a shocking thing.
00:16:37.279 --> 00:16:39.679
We're gonna take code and throw it away.
00:16:39.919 --> 00:16:47.120
Because I've written code as well, and code is like, I mean, it is it used to be like a sculpture, you know.
00:16:47.200 --> 00:16:54.960
You start by figuring out the large framework and then you chisel at it month after month, and you build this piece of code, and it's it's a piece of art.
00:16:55.039 --> 00:16:59.919
You take a lot of pride in it, and now somebody comes along and says, Hey, we're gonna throw your code away, and we're gonna start all over again.
00:17:00.000 --> 00:17:01.360
Are you effort kidding me?
00:17:01.600 --> 00:17:03.919
But that's literally what we're starting to do now.
00:17:04.079 --> 00:17:07.839
So, so it is it is really a dramatic change in workflow.
00:17:07.920 --> 00:17:14.960
And I think it was good we had an in-person workshop with the eight of us to just literally sync up as to the various directions we'd gone into.
00:17:15.279 --> 00:17:23.119
On on the face of it, if someone were to describe like what like a technical writer were to do, you would think, well, surely that's the type of job that AI is going to replace.
00:17:23.200 --> 00:17:27.200
I mean, people just that's it and write technical specifications about stuff.
00:17:27.279 --> 00:17:29.039
It seems like the AI would be really good at that.
00:17:29.119 --> 00:17:34.400
And yet I can also now see what you're saying, which is the ability to write with precision.
00:17:34.720 --> 00:17:40.799
You know, I think prompt engineer is a misnomer because people think of prompts as sort of these very short, you know, pithy things that you're asking for.
00:17:40.880 --> 00:17:46.640
But but really a prompt is more of like being able to articulate um like yeah, very precisely.
00:17:46.799 --> 00:17:47.279
What do you want?
00:17:47.440 --> 00:17:48.640
What what do you actually want to happen?
00:17:48.720 --> 00:18:07.839
And that is um so so you actually believe then that this is that that like sort of like uh um let's say English or or or Dutch or whatever that the the language that you're using, is is it sort of are you seeing it actually replace specific coding languages in syntax as the dominant way that engineers are in creating and writing stuff?
00:18:08.240 --> 00:18:09.519
Oh yeah.
00:18:10.079 --> 00:18:10.720
Yeah, yeah.
00:18:10.799 --> 00:18:17.359
I think that for for things that are new where we're starting from scratch, I mean, like you're never gonna start by writing code anymore.
00:18:17.440 --> 00:18:22.559
I mean, and maybe a better word than prompt engineering is specification engineering in some sense.
00:18:22.640 --> 00:18:26.799
Because you're right, people think of a prompt as a little message I I send in a chat bot.
00:18:26.880 --> 00:18:28.240
That's not what what this is.
00:18:28.400 --> 00:18:36.079
This is documents of pages and pages and pages that in English describe what the system will do in great detail.
00:18:36.240 --> 00:18:39.279
And so that is gonna become a key key skill.
00:18:39.440 --> 00:18:55.680
And there's still a debate a little bit that I have with with colleagues in other companies as well, is that is the source of truth going to be the specification document, which is like your MD file, which is mostly English, or is it is the code still the source of truth?
00:18:55.920 --> 00:19:02.559
I mean, and then if you need a human specification, you just take the code and you will first compile it into an MD file.
00:19:02.880 --> 00:19:04.559
I mean, or you ask it questions.
00:19:04.720 --> 00:19:06.960
That's still there's still a lot of debate.
00:19:07.039 --> 00:19:14.160
I think we're now within our company more going this the specification route, that the specification is the source of truth.
00:19:14.240 --> 00:19:22.559
But I know colleagues at other companies uh uh are saying, no, no, the code is still the source of truth, but no human will ever look at it anymore.
00:19:22.799 --> 00:19:26.640
But that and then the question is how do you do version control and and things like that?
00:19:26.799 --> 00:19:27.839
That's a whole thing.
00:19:28.400 --> 00:19:32.079
There's a whole whole GitHub now working on specifications and things like that.
00:19:32.240 --> 00:19:38.079
So there are definitely projects for which DMD file is the programming language, I think that.
00:19:38.400 --> 00:19:40.960
And it's weird because it's not deterministic.
00:19:41.119 --> 00:19:47.759
You know, it's like you know, back in the days when compilers came out, if you had a compiler that was non-deterministic, you were dead.
00:19:47.839 --> 00:19:49.519
Nobody would ever use it, you know.
00:19:49.680 --> 00:19:52.400
But now we're like, yeah, yeah, it's non-deterministic.
00:19:52.480 --> 00:19:55.119
But you know, as long as we write it tight enough, it's okay.
00:19:55.359 --> 00:20:00.160
Because you could have the same and an MD file is a markdown file, it's essentially I mean, how would you describe it?
00:20:00.480 --> 00:20:04.160
Effectively, it's sort of like a just like a like a text document, but with a little bit more.
00:20:04.559 --> 00:20:05.440
It's just a structured document.
00:20:05.680 --> 00:20:09.920
It's a structured English document with some tabulation and it's not a problem.
00:20:10.559 --> 00:20:11.759
You're looking at like a word document, effectively.
00:20:12.799 --> 00:20:25.599
And if you were to take the same markdown file and drop it into, let's say, three separate AI like language models, presumably you're not necessarily going to get identical code outputs each time.
00:20:26.000 --> 00:20:26.640
For sure not.
00:20:26.720 --> 00:20:28.720
You will give for sure it will not be identical.
00:20:28.960 --> 00:20:30.640
Will it do the same thing?
00:20:30.960 --> 00:20:31.839
Hopefully.
00:20:32.559 --> 00:20:42.000
You know, I mean, and actually it that would be a good test because let's say you you write the specifications, you give it to three AIs, they will write different codes, obviously.
00:20:42.240 --> 00:20:44.160
But is the code going to do the same thing?
00:20:44.240 --> 00:20:46.640
And if it does the same thing, that means you've done a good job.
00:20:46.799 --> 00:20:50.400
Because the other thing that obviously is happening is like, what happens with QA?
00:20:50.480 --> 00:20:51.839
How are we going to test all this?
00:20:52.000 --> 00:20:53.920
How are we going to have agents to test this?
00:20:54.000 --> 00:20:56.400
This is actually a discussion we had this morning.
00:20:56.480 --> 00:21:00.160
But we had written a code, we had written a test agent, it passed all the tests.
00:21:00.240 --> 00:21:01.200
Well, great.
00:21:01.519 --> 00:21:10.799
But the test agent had cheated because the test agent had read the code and knew what the code was going to do and sort of knew what the answer was going to be.
00:21:10.960 --> 00:21:13.359
And I'm like, well, that's a terrible test agent, you know.
00:21:13.519 --> 00:21:15.200
So we realized, like, whoa, whoa, whoa.
00:21:15.359 --> 00:21:19.759
We need to write a coding agent who uses the specification and generates code.
00:21:20.000 --> 00:21:24.160
Then we need to have a testing agent who has a completely different context.
00:21:24.400 --> 00:21:25.680
It cannot look at the code.
00:21:25.759 --> 00:21:26.480
That's the whole point.
00:21:26.559 --> 00:21:27.599
It cannot look at the code.
00:21:27.680 --> 00:21:33.279
It cannot look at the code, it can look at the same specifications, it can write the test cases and execute the test cases.
00:21:33.440 --> 00:21:36.000
And then the two of them can sort of go back and forth.
00:21:36.079 --> 00:21:40.160
And that was sort of an insight that, like, ah, yes, we can definitely do testing much better.
00:21:40.319 --> 00:21:43.039
Just like you wouldn't want an engineer to test their own code.
00:21:43.119 --> 00:21:43.759
That wouldn't make any sense.
00:21:43.839 --> 00:21:44.240
Because you're going to be able to do that.
00:21:45.039 --> 00:21:46.960
And I because I have your sub stack up.
00:21:47.039 --> 00:21:50.400
And so we talked, so you talked about prompting, context awareness.
00:21:50.480 --> 00:21:52.960
I mean, that's I think that's part of that like that specification document.
00:21:53.279 --> 00:21:59.759
Anticipating failures, this is it's almost like this is the the new modality of the the process.
00:22:00.160 --> 00:22:23.039
Problem solving and the engineering, as it were, that happens, rather than it's literally engineering the in the form of writing the code, the engineering you're describing is is is having an understanding of what the potential outcomes, the probabilistic like the space, the probabilistic space of outcomes that might happen, trying to anticipate that.
00:22:23.279 --> 00:22:25.680
And then you say verification ethics.
00:22:25.759 --> 00:22:36.400
And so when you're talking about verification, it's I think people assume that verification just means reading the output and making sure it's right, but you're actually describing an understanding of even how do you verify.
00:22:36.559 --> 00:22:54.480
And then the workflow integration is like well, if you were going to now like you're basically talking about uh a new structure for the process of engineering, and so how do you design a new workflow so that if you have a QA engineer, well, well, what is their role actually?
00:22:54.640 --> 00:23:06.960
Because it's as yeah, so you're saying they're not necessarily reading the code, they're perhaps designing the agents that are reading the code and thinking about how do you how do you almost blind the study, as it were.
00:23:07.359 --> 00:23:10.240
Yeah, so okay, so a couple thoughts on that.
00:23:10.319 --> 00:23:13.759
So so the the there's different agents involved in a piece of software.
00:23:13.839 --> 00:23:22.640
There's obviously the one who writes the code, then you can have somebody to review the code, just like in in the in the old days, you had you had a developer, you had somebody else who reviewed the code.
00:23:22.720 --> 00:23:27.519
You could have another agent review the code, like hey, your code is is wrong or has mistakes and so on.
00:23:27.599 --> 00:23:30.160
That's not the testing agent, that's the code review agent.
00:23:30.319 --> 00:23:33.039
Then you have a testing agent who doesn't get to see the code.
00:23:33.119 --> 00:23:34.240
You don't get to see the code.
00:23:34.319 --> 00:23:37.440
You get to run the code and you get to write test cases.
00:23:37.519 --> 00:23:40.559
You know, if I put this as input, then this should be the output.
00:23:40.640 --> 00:23:43.119
I mean, or if I click on this button, then that should happen.
00:23:43.200 --> 00:23:45.519
I mean, that's what a test agent is gonna do.
00:23:45.680 --> 00:23:51.039
Now, a test agent will do some simple unit testing, just like we do on each individual block.
00:23:51.119 --> 00:23:52.400
It's gonna do some system testing.
00:23:52.480 --> 00:23:59.440
Now, in order to do system testing, it's gonna need to have an agent who can log in and who can use a browser, who can use an app and do all those type of things.
00:23:59.599 --> 00:24:09.200
So that will be the sort of uh user testing agent in some sense, just like you would hire a bunch of uh uh users to try out all the buttons on your app.
00:24:09.279 --> 00:24:11.359
You know, now you have a bunch of agents doing that.
00:24:11.440 --> 00:24:15.440
So there's probably three or four of those agents uh involved in this.
00:24:15.599 --> 00:24:21.759
And then the other point, you you mentioned sort of the the third part of AIQ, the defaults and the verification.
00:24:21.920 --> 00:24:26.240
So the faults is sort of like we were talking earlier of the whole troughs of the of the AI.
00:24:26.319 --> 00:24:33.680
You know, now you're in the wrong corner of the jagged edge and it's done something silly and it's done something stupid, and and you're ready to sort of throw it out.
00:24:33.920 --> 00:24:39.839
That's where the verification part comes in, which is related to the different agents having different contexts.
00:24:40.000 --> 00:24:46.880
And I'll give you one example that actually happened the day I moved to Belgium, I think it was January 10, if I remember right.
00:24:47.119 --> 00:25:04.799
There was a huge controversy in the news, and I just literally, quote unquote, got off the boat or got off the plane because one of the presidents of a university had gotten caught in a speech she gave at the opening of the academic year using a quote, I think it was an Einstein quote, that was hallucinated.
00:25:05.039 --> 00:25:11.359
And the entire country was ready to nail her to the cross and vilify her.
00:25:11.680 --> 00:25:15.839
Like, how dare she use AI for writing her remarks?
00:25:16.000 --> 00:25:21.759
And of course, the presidents of the other universities got on TV and chimed in how preposterous this was.
00:25:22.000 --> 00:25:26.480
And actually, I it prompted me to write another substack because I was kind of annoyed by it.
00:25:26.559 --> 00:25:27.759
And I'm like, wait a minute.
00:25:27.839 --> 00:25:29.759
No, no, no, no, no, you guys are missing the point.
00:25:29.920 --> 00:25:34.160
Clearly, she made a mistake putting in these hallucinated quotes, obviously.
00:25:34.400 --> 00:25:37.680
The mistake was not using AI.
00:25:37.920 --> 00:25:39.039
That was not a mistake.
00:25:39.200 --> 00:25:44.160
Actually, I have much more concerns with all the other presidents of the other universities who don't use AI at all.
00:25:44.319 --> 00:25:46.079
I'm much more worried about these people.
00:25:46.319 --> 00:25:49.119
Her mistake was that she didn't do verification properly.
00:25:49.200 --> 00:25:57.119
She didn't have another agent that says, look at my speech, look at this quote, double check that you can find an authentic source where this person actually said that.
00:25:57.359 --> 00:26:00.640
I mean, uh, and then she also didn't disclose that she used AI.
00:26:00.720 --> 00:26:03.440
I think if you use AI, you should say, yes, I used AI.
00:26:03.599 --> 00:26:04.079
Of course I did.
00:26:04.160 --> 00:26:05.200
I mean, like, you didn't?
00:26:05.920 --> 00:26:09.119
So I think that's what brings you to this AIQ.
00:26:09.359 --> 00:26:23.279
You know, when you're talking about workflow integration, I mean, we're seeing this at our organization, um, and we're also seeing it in schools where you know a lot of teachers are are agonizing over, you know, how do we talk how do we deal with like academic integrity and students using AI to cheat?
00:26:23.599 --> 00:26:46.480
And there was one, there was I was talking to one school, and one of the things that they've done, which very few schools have done, which is before they go into the process of like articulating here are the rules for how you're allowed to use AI or how you're not allowed to use AI, the teachers first go to the students and say, Okay, I want you to talk to me about what your expectations are about how I use AI.
00:26:47.440 --> 00:26:50.079
And I've heard about it.
00:26:50.240 --> 00:26:50.880
Apparently, this was done.
00:26:50.960 --> 00:26:55.279
I've heard about this on enough classrooms that I think that I feel confident in the outcome.
00:26:55.519 --> 00:26:59.599
Most students are not going to say, Oh, well, you just use as much of AI as you want.
00:26:59.680 --> 00:27:06.079
We don't really care if all of your lessons are AI generated or if your feedback is AI generated, the students will say, Well, no, you're our teacher.
00:27:06.480 --> 00:27:10.400
The whole point is that you're supposed to be, you know, really thinking about what you're teaching us.
00:27:10.559 --> 00:27:15.039
And we want, if you're going to give us feedback, we want it to be authentic feedback.
00:27:15.359 --> 00:27:26.720
And and that, you know, that's sort of often the first the first reaction of the teacher will say, Well, would you rather me sit down by myself in my office, sitting down and writing feedback hand in hand, and then just give you that?
00:27:26.799 --> 00:27:34.000
Or would you rather I use the AI maybe to give the initial passive feedback and then use that time to sit down with you in person and talk through the feedback?
00:27:34.079 --> 00:27:38.799
And the students will usually say, Oh, no, no, I'd rather have the chance to talk to you and hear the feedback live.
00:27:39.119 --> 00:27:54.000
But I mean, to me, that's actually it's it's it shifts away from this idea of addressing you know academic integrity as a policy challenge, which is like as if there's some magical rule that if you write down the rule in the right way, suddenly it's gonna just make everything work.
00:27:54.079 --> 00:28:10.799
It's like actually no, it's a workflow design challenge of okay, actually, you know, there's there's there's a way to design the process of when to use AI, when to like make sure that there actually is the human sort of like not in the loop even, but sort of just like the validation of human input.
00:28:11.039 --> 00:28:14.079
Um but it is hard, right?
00:28:14.160 --> 00:28:19.039
Like this is this is something that you have been feeling out as a leader organization.
00:28:20.880 --> 00:28:22.640
Who have you enlisted as part of this?
00:28:22.720 --> 00:28:36.160
Because what you're describing is when you talk about change management, this is this is really individuals, not just leaders, but also individuals within companies that are pulling like what has your strategy been to kind of you know evangelize?
00:28:36.319 --> 00:28:41.599
Is it is it leadership top down, or have you found sort of allies, even like sort of like from the bottom up?
00:28:42.000 --> 00:28:42.400
Yeah, yeah.
00:28:42.480 --> 00:28:44.240
So uh so it's two different things.
00:28:44.319 --> 00:28:52.720
So let's let's talk a little bit about sort of your comments on teachers and and and workflow in education, and and then I'll come back to sort of change management in the couple.
00:28:52.960 --> 00:28:57.599
So on education, I think you and I spoke about this last time uh we spoke as well.
00:28:57.680 --> 00:29:00.400
And I always use the example of my dad.
00:29:00.559 --> 00:29:15.119
Um my dad who's retired now, he was a high school math teacher, and I actually was his student at some year, not recommended, but uh and he had a reputation of being one of the toughest teachers in in the high school, it would fail students left to right.
00:29:15.279 --> 00:29:19.279
And this was the time when calculators and particularly programmable calculators came around.
00:29:19.359 --> 00:29:24.240
And people say, Oh, no, no, no, no, don't use calculators because you'll never know how to you'll never learn how to calculate.
00:29:24.400 --> 00:29:25.519
And his view was different.
00:29:25.599 --> 00:29:30.160
He's like, no, here's my exam, and if you want to use a calculator, you can bring a calculator.
00:29:30.240 --> 00:29:33.200
The only thing is that if you bring a calculator, I'm gonna give you a different exam.
00:29:33.359 --> 00:29:38.319
It's gonna be a more difficult exam because if you just punch it into the calculator, you won't have learned anything.
00:29:38.400 --> 00:29:51.839
So, and I remember I would always take the more difficult exam because I had a programmable HP calculator, which in like 1984 was like state-of-the-art compute power, and I would just program it and it would solve the the problem.
00:29:52.000 --> 00:30:10.799
But we just learned a different way in some sense, and AI is not fundamentally different in that regard, uh, is that our skills shift as we can mechanize calculation like we did in the 80s, or we mechanize language as we do today, or we mechanize reasoning, quote unquote.
00:30:10.960 --> 00:30:16.960
Um, then our skills move to a different uh territory of uh we use our brains to do other things.
00:30:17.119 --> 00:30:19.519
So that's what I would sort of tell teachers.
00:30:19.599 --> 00:30:29.039
And obviously, the job of a teacher is not to sort of regurgitate knowledge and to put to put facts and data in your brain.
00:30:29.279 --> 00:30:40.960
The the point of a teacher is not to transfer knowledge but to transfer understanding, you know, it's like how do we really conceptualize and how do we create mental models that we can work with and talk about that like you and I are doing.
00:30:41.119 --> 00:30:43.359
That's the point of a of a teacher in some sense.
00:30:43.519 --> 00:30:47.279
Um, and yeah, I can see I have quite a few academic friends.
00:30:47.440 --> 00:30:54.000
I mean, uh, I'm I'm I'm also sort of an honorary professor here, so I feel I need to show up at the local university once in a while.
00:30:54.160 --> 00:30:57.759
And these are the topics that I think some academics have gotten.
00:30:57.839 --> 00:31:05.599
They're like, yes, we get this, we know what our job is going to be, but but many teachers are still sort of struggling with it, which brings me back to the second part of your question.
00:31:05.759 --> 00:31:07.839
How do you deal with the change management?
00:31:07.920 --> 00:31:08.559
Is it top-down?
00:31:08.640 --> 00:31:09.359
Is it bottom-up?
00:31:09.519 --> 00:31:12.640
I mean, again, I'm not a change management expert.
00:31:12.720 --> 00:31:18.160
There's people who have studied this, and there's a lot of like knowledge uh available on how to do it.
00:31:18.319 --> 00:31:22.079
It's not about the CEO pounding the table, it's like we're all going to use AI.
00:31:22.160 --> 00:31:23.519
That's not necessarily the way to do it.
00:31:23.599 --> 00:31:27.920
It's not the way about everybody just thinking on their own and just figure out where we go.
00:31:28.079 --> 00:31:31.200
It is a little bit of a push-pull uh between these two.
00:31:31.359 --> 00:31:38.160
And to me, adopting AI is not different than any other massive change management, I would say.
00:31:39.119 --> 00:31:42.240
And you see the usual adopters and resistors.
00:31:42.400 --> 00:31:46.720
In our company, we have sort of, I mean, we're obviously progressing every day.
00:31:46.880 --> 00:31:50.960
I think these workshops, I used to call it uh the thing we did two weeks ago.
00:31:51.039 --> 00:31:53.440
I call it an AI work day.
00:31:53.680 --> 00:31:55.759
It's sort of a day where we all work.
00:31:55.920 --> 00:31:59.759
It's not just about talking, it's about about sort of show and tell.
00:31:59.920 --> 00:32:01.279
And we all show each other.
00:32:01.519 --> 00:32:09.839
We literally bring up our our our bots and our agents and our prompts and our specifications, and we show each other how our personal workflow has changed.
00:32:09.920 --> 00:32:12.960
And there's a lot of aha moments that happen.
00:32:13.119 --> 00:32:25.279
And I think that sort of individual progress, and then I like groups of about eight to 20 as a good uh sort of in-person group to sort of have that uh dynamic.
00:32:25.440 --> 00:32:29.839
And then, yeah, we have people in our finance organizations who are one of the earliest adopters.
00:32:29.920 --> 00:32:35.680
Like, we we need to hire a junior accountant, and like, well, we don't necessarily have the money, we can't find the right person.
00:32:35.759 --> 00:32:44.559
And then our senior AI um uh finance person is like, oh, I work over the weekend with with Claude, and I have my junior accountant, I'm all sim.
00:32:44.799 --> 00:32:52.880
And that's somebody who's obviously not a technical person who is a uh finance expert, but was able to come up with an agentic workflow to do a junior accounting job.
00:32:53.440 --> 00:33:04.400
So I guess we have to talk about this now that you brought it up, which is something that has increasingly come to the I think public awareness is this idea that you know we don't really know how AI is going to impact the workforce.
00:33:04.559 --> 00:33:14.400
It it does seem that at least in the short term right now, the most obvious way that it impacts is you know, displacing junior hiring to some degree.
00:33:14.799 --> 00:33:23.200
And I it's it's there's been a lot of just sort of reflexive there's a lot of like fast twitch reflexes with a technology like this.
00:33:23.440 --> 00:33:26.880
And so I'm trying to resist making broad statements.
00:33:27.200 --> 00:33:28.880
I think we've seen it ourselves.
00:33:28.960 --> 00:33:58.720
I mean, we're not we haven't like used it to displace anybody, but I think it has it has changed the way that we think about, you know, normally you might bring on an intern to do work, and it's it's hard to imagine what an intern might do that wouldn't actually be more uh Yeah, because a lot of that sort of like entry-level work is the type of stuff that even if I would even if an employee can't quite figure out how to use AI to do that thing, I think even the process of trying to figure out how to use AI to do that is important, right?
00:33:59.039 --> 00:34:00.640
That's that's the learning experience.
00:34:01.359 --> 00:34:06.240
But is that is this temporal or is this actually a structural shift in organizations?
00:34:09.119 --> 00:34:17.440
It's it's something I've thought about as well, and I'm gonna also try to refrain myself from making broad uh statements because we just don't know yet.
00:34:17.679 --> 00:34:29.360
I mean, I used to in some of the presentations I used to give, I used to say AI stands for all interns, meaning that that think of AI as just a lot of really smart interns that you've hired.
00:34:30.239 --> 00:34:38.000
They're very inexpensive, they've read the entire internet, so they're very smart, but at the same time, they have no experience.
00:34:38.159 --> 00:34:42.239
I mean, you'll tell them something, they'll go off the deep end and they'll they'll do things.
00:34:42.320 --> 00:34:46.800
And your job is to sort of rein them in, to make sure that they go off in the right direction.
00:34:47.039 --> 00:34:47.679
That's sort of fun.
00:34:48.159 --> 00:35:11.119
Now, it is definitely true, and we've seen this as well, is that because of this all interns uh step, that it's like, wait a minute, what about a traditional uh professional path where you get a degree or you get an internship and you have to do some grunt work because that's what interns do, and therefore you you get an entry-level job and you sort of move into the uh into the professional world.
00:35:11.519 --> 00:35:17.039
It feels like that traditional chain has been interrupted by the the all interns AI in some sense.
00:35:17.199 --> 00:35:20.400
So the key question is I think what you're what are you gonna ask your interns to do?
00:35:20.480 --> 00:35:24.159
You're not gonna give them grunt work anymore because the agents are gonna do all the grunt work.
00:35:24.320 --> 00:35:25.679
So that that's gone.
00:35:26.000 --> 00:35:28.079
So, but what are you gonna do?
00:35:28.239 --> 00:35:37.840
You're gonna give them sort of junior jobs to say, like, well, you're not gonna go and train the next junior accountant because we're gonna have our senior finance person doing that.
00:35:38.000 --> 00:35:41.360
But we're gonna ask you to show your AIQ skills.
00:35:41.440 --> 00:35:44.960
That's what we're gonna ask you to do because an internship is an extended interview.
00:35:45.039 --> 00:35:55.679
Uh, we're gonna say, well, how good are you at at writing prompts, or how good are you at detecting failures, or how good are you at putting a verification agent together, or how good are you at putting a workflow together?
00:35:55.840 --> 00:36:05.840
And we're gonna ask you to do simple things that are not gonna make or break the company, but we're gonna ask you to show us your skills and we're gonna we're gonna teach you how to do this.
00:36:06.000 --> 00:36:09.440
I mean, I think that's how I I would expect it to evolve.
00:36:09.679 --> 00:36:11.599
Have we seen that yet?
00:36:12.000 --> 00:36:13.440
A little bit, I would say, yeah.
00:36:13.519 --> 00:36:15.119
Well, I think we've seen a little bit of that.
00:36:15.280 --> 00:36:23.119
I mean, uh, and it changes how you hire and it changes how you uh find interns and how you how you teach them, but I think I don't see another way to go.
00:36:23.440 --> 00:36:28.800
It certainly doesn't make sense to just continue hiring interns to do busy work for the sake of posterity.
00:36:29.199 --> 00:36:33.360
Um You know, we've done a lot of these like vibe coding workshops.
00:36:33.760 --> 00:36:46.480
You know, as I think that's been the big change is the um the barrier to entry just has gotten just low enough now that almost anybody, actually say anybody can, you know, within the course of an hour can really yeah.
00:36:47.920 --> 00:36:50.960
Anyone who can write English, yes, and I think or some language, yes.
00:36:51.360 --> 00:37:04.480
I don't know if I'm ready to say this as a blanket statement, but I think it's actually quite common that um the the correlation between like domain expertise and ability to build something cool, it's not totally um correlated.
00:37:04.559 --> 00:37:13.199
It's it's you know, in some cases, actually in a lot of cases, the folks at these workshops who build the coolest things are the folks with the least experience.
00:37:13.519 --> 00:37:17.280
But they have the they're just the fastest it just they just take on to it quickly.
00:37:17.360 --> 00:37:19.840
And I and it's not necessarily even age.
00:37:20.159 --> 00:37:51.679
In some cases it's somebody who is who is just uh you know has a different worldview or mindset or or in some cases they just you know they've they've been curious about something else and they just are are figure out how to channel the curiosity and actually often it's neurodivergent kids or people, people that, you know, I I mean I'm there there'd be it'd be really interesting to look at a body of research on this, but I have you know, we've done work with various nonprofits that that serve students with learning differences.
00:37:51.840 --> 00:37:59.599
And one of my friends actually is um you know one of sort of like the leading uh you know thinkers and speakers in the um neurodiversity community.
00:37:59.679 --> 00:38:20.000
And he and I have talked about this where there is this sort of anecdotally at least, there's something about you know people who just have a slightly different way of approaching or thinking about problems, and then they take on this tool, and sometimes that actually helps them kind of both figure it out, but then also go down the rabbit hole.
00:38:20.079 --> 00:38:24.400
Because I think part of the challenge is you have to you have to get sucked in.
00:38:24.639 --> 00:39:16.559
And for some people that's like a long weekend building, you know, some personal yeah, like um and often people will tell me I don't I'm curious if you have examples of this, but I I think often people will tell me there was like there's there's one project that they they started working on, and that becomes their entry point where you know they spent a few days just like oh, I wanted to build like a a tool to help me reserve camping spots at you know national parks because I always you know forget to wake up early enough to like get the spot, and it took me three or four days, and in that process they learned about you know uh sub agents and like you know what they done then they downloaded ClaudeBot because they realized they needed to have give it access to you know different sort of browsers and the calendar and um and it's hard to just like systematize that it's you know not everybody is gonna find that project as the the thing, yeah.
00:39:16.880 --> 00:39:18.480
So a couple thoughts on that.
00:39:18.719 --> 00:39:48.559
On the uh uh neurodiversity, I'm not an expert on this at all, so I'm gonna be but but I think I can certainly see that people who are good at details and people who are good at sort of tight specifications and and are very sort of uh uh uh used to particular habits of how things are supposed to be done can become very good sort of specification engineers because it it has that level of detail.
00:39:48.800 --> 00:39:52.559
So I could definitely see that as a as a as a non-expert.
00:39:52.880 --> 00:40:00.079
What what it what it what is happening for sure and related to that is that the concept of language becomes so much more important.
00:40:00.239 --> 00:40:03.519
It's not like a language we're just talking because we're chatting and we're having a beer.
00:40:03.599 --> 00:40:04.320
It's like no.
00:40:04.480 --> 00:40:14.079
Now English is becoming more of a formal language and and definitions are important, and words are important, and semantics is important, just like it was for programming language.
00:40:14.239 --> 00:40:21.519
If we're going to use natural language as a specification slash programming language, then being precise is going to become more important.
00:40:21.679 --> 00:40:25.119
So we're gonna have to choose our words very, very carefully in some sense.
00:40:25.280 --> 00:40:26.800
So that's part of AIQ.
00:40:27.119 --> 00:40:34.239
I think in terms of what you said, in terms of uh skills and and also, I mean, what kind of jobs do you need?
00:40:34.320 --> 00:40:36.239
Do you need fewer people in your company?
00:40:36.320 --> 00:40:42.079
There's obviously every CEO in the world is looking at their workforce like, well, do I need all these developers?
00:40:42.159 --> 00:40:44.800
You know, people are telling me AI can do this, maybe I can save money.
00:40:44.880 --> 00:40:49.920
That's sort of what every CEO in the world is doing, and if they're not doing it, their board is telling them to do it.
00:40:50.239 --> 00:40:54.960
And to be honest, I have been the voice in our company already quite some time ago.
00:40:55.119 --> 00:40:57.199
This is like, well, maybe we should have fewer engineers.
00:40:57.280 --> 00:41:08.960
And I have not become the most popular person in our company by saying that, because I say it publicly, I say it privately, I don't sort of say, oh no, I say, look, I I tell our engineers that maybe we should have fewer of you.
00:41:09.119 --> 00:41:11.679
And we actually talk very openly at the workshop about this.
00:41:11.760 --> 00:41:15.280
Because it's like, well, you've been saying we need fewer engineers, but we're the engineers, how dare you?
00:41:15.360 --> 00:41:17.039
And I'm like, okay, let's talk about this.
00:41:17.119 --> 00:41:31.760
Let's let's have a um and there's a couple there's one idea that popped in my head during the workshop, which I thought like you could say, like, hey, we need fewer engineers because agents could do what engineers are doing, so but maybe it's the other way around.
00:41:32.079 --> 00:41:38.400
Maybe about no longer needing, let's say, software engineers, maybe software engineers can start doing everybody's job.
00:41:38.639 --> 00:41:55.679
Maybe your software engineer can become your accountant or can become your salesperson or can become your marketing person because obviously they're not an expert in finance or marketing, but they're very good at writing formal language and building systems, you know, and I can definitely see some of our engineers already doing that.
00:41:55.840 --> 00:41:57.920
I mean, like, hey, I'm not just gonna write code.
00:41:58.000 --> 00:42:09.760
No, I'm gonna work together with our finance person and we're gonna together craft this finance agent because I'm good at formal definitions and specifications of systems.
00:42:09.840 --> 00:42:10.880
That's what I'm good at.
00:42:11.039 --> 00:42:14.239
I mean, whether I'm good at Python or not is no longer relevant.
00:42:14.400 --> 00:42:15.679
So maybe everybody will be able to do that.
00:42:15.920 --> 00:42:18.880
I mean, this is a really important uh conversation in in our space.
00:42:18.960 --> 00:42:36.079
I mean, there is the the computer science teachers all over the country are, you know, I you know, I think there's, and I won't name them, but there are certain CEOs of very large uh companies at the frontier of AI who are who I think have been quite cavalier in the way that they've talked about uh computer science.
00:42:36.239 --> 00:42:39.599
You know, I wouldn't teach my my kid computer science, you know.
00:42:39.679 --> 00:42:41.599
I think AI is going to replace all computer scientists.
00:42:41.840 --> 00:42:44.159
I think there's there's a lack of precision in that language, right?
00:42:44.239 --> 00:42:48.639
Because because sure, it it I think it is undoubtedly I mean it is true.
00:42:48.719 --> 00:42:55.599
It's not even a potential future, it is the it is the current state, which is AI is replacing the the act of writing code.
00:42:55.840 --> 00:43:02.480
You know, very few people are now spending as much time writing literal lines of code as they were two or three years ago.
00:43:02.719 --> 00:43:16.320
But if you actually abstract you know software engineering into computational thinking skills, yeah, I mean it almost feels like you know it's yes, almost everybody now is taking on this sort of computational thinking modality.
00:43:16.719 --> 00:43:17.039
Exactly.
00:43:17.360 --> 00:43:18.880
What does this mean by my college?
00:43:19.119 --> 00:43:30.880
Because I I I worry that you know, we're we were we work in K-12, we're actually quite upstream of college where we don't necessarily have to answer the question of like, well, do you have every skill that you need to enter the workforce?
00:43:31.119 --> 00:43:41.360
But once you get into, you know, post-high school, post-secondary, as we call it in the US, I'm not sure that I'm seeing universities shift as rapidly as you might expect that they should be, right?
00:43:41.440 --> 00:43:55.199
Like I, you know, for example, I haven't seen a total overhaul of computer science programs to become more hybrid, liberal arts computer science where you're still learning to code, but then you're also taking philosophy.
00:43:55.360 --> 00:43:59.840
I think I think there are schools where you can do that yourself, where you can sort of cobble it together and maybe do.
00:44:00.079 --> 00:44:01.199
Double major or minor.
00:44:01.360 --> 00:44:09.199
I mean, do you think that we actually need to be totally shifting the even just like the vocational training of people in computer science, or maybe it's the opposite?
00:44:09.360 --> 00:44:11.840
Maybe it's like liberal arts needs to have more CS?
00:44:12.239 --> 00:44:12.559
Yeah.
00:44:12.800 --> 00:44:16.559
So there's a lot to unpack there.
00:44:16.719 --> 00:44:21.039
So I remember sort of when I went through true education, there was a big thing.
00:44:21.119 --> 00:44:25.920
It's like, oh, everybody should learn to program, you know, because that was sort of a thing 30 years ago.
00:44:26.079 --> 00:44:28.880
I mean, uh now that's obviously absurd today.
00:44:29.039 --> 00:44:33.519
Uh, and then we actually realized like, no, no, no, no, no, not everybody needs to learn how to program.
00:44:33.679 --> 00:44:41.119
I mean, uh, I think everybody needs to know what programmers do, but but not everybody needs to be uh a software developer.
00:44:41.280 --> 00:44:56.960
So sort of taking that same thought today is that uh what's going to happen to computer science as a discipline, that's gonna happen to software engineering as a practice is something I worry about personally too, because my son is uh is a software engineer.
00:44:57.039 --> 00:45:00.880
I mean, he's a computer scientist, he's at the beginning of his career, I'm at the end of my career.
00:45:00.960 --> 00:45:02.719
I don't for me, it doesn't matter so much anymore.
00:45:02.880 --> 00:45:04.320
For him, it matters a great deal.
00:45:04.480 --> 00:45:11.440
I mean, uh, so we we talk about this, about this a lot, but in some sense, the I think you called it cognitive science.
00:45:11.519 --> 00:45:21.840
You need to be able to to to to educate how do you think, how do you design, how do you construct something that has you know a large-scale design and has yes.
00:45:22.480 --> 00:45:25.679
So, do you need to know the the syntax of a programming language?
00:45:25.840 --> 00:45:26.320
Obviously not.
00:45:26.480 --> 00:45:28.639
Do you know the semantics of a programming language?
00:45:28.800 --> 00:45:35.760
Maybe not, but you sure need to understand the semantics of English, and you need to understand the semantics of system engineering.
00:45:35.840 --> 00:45:42.400
You need to be able to say what it is that a system needs to do and what it should not do and how you test it.
00:45:42.559 --> 00:45:46.320
That ultimately is going to become become the the the new skill.
00:45:46.400 --> 00:45:54.320
And there will be just like in anything, there is a pendulum, and sometimes it goes a little bit more one way and it's gonna have to come the other way, and it will eventually end.
00:45:54.400 --> 00:45:58.559
And some of the the CEOs are being cavalier and are just being silly.
00:45:58.639 --> 00:46:01.119
I mean, as low elite, they're just being silly.
00:46:01.199 --> 00:46:04.960
I mean, uh, or they have this quest for AGI or silly things like that.
00:46:05.039 --> 00:46:06.159
It's not what it's all about.
00:46:06.320 --> 00:46:07.360
It's not what it's all about.
00:46:07.519 --> 00:46:18.639
It's it's actually, in my mind, almost the most exciting time in humanity to be alive where we really get to push our brains and our minds and our creativity to things that we've never been able to do before.
00:46:18.880 --> 00:46:34.559
And and just to like describe like like sort of like definitionally computational thinking, it's like this is it's a like problem solving where you're breaking down complex problems, like decomp decomposing those problems, identifying patterns, focusing on important details, abstracting them, developing step-by-step solutions, which is like algorithms.
00:46:34.719 --> 00:46:34.880
Yeah.
00:46:35.039 --> 00:46:43.440
Um and you can imagine in in social studies, in English, in philosophy, certainly in math, but also in computer science.
00:46:43.519 --> 00:46:51.039
I mean, like you can you can actually think about computing digital thinking as a through line across most domains.
00:46:51.519 --> 00:46:55.599
But I think historically we have been so siloed, right?
00:46:55.679 --> 00:46:59.199
Like you you kind of have to you have to sort of self-select so early on.
00:46:59.280 --> 00:47:03.280
And I remember for me, it was like, am I a STEM kid or am I a liberal arts kid?
00:47:03.360 --> 00:47:06.559
And I was like, well, I like language, so I I guess I'm a liberal arts kid.
00:47:06.639 --> 00:47:06.800
Yeah.
00:47:06.960 --> 00:47:08.320
And it's too much to my chagrin.
00:47:08.400 --> 00:47:18.960
I mean, I if I I still to this day, I mean, if only I didn't have a computer science class in high school, and there was nobody telling me, oh, you know, Alex, at least take one computer science class in college.
00:47:19.039 --> 00:47:26.880
I mean, and if I had just had that, you know, it would have given me a fluency to be able to even just like approach my own continued learning.
00:47:27.119 --> 00:47:43.440
I think so much, you know, because I think about like with Spanish, I took a couple years of Spanish, and now, you know, I when I go and travel in countries where that, you know, where Spanish is the primary language, I I have enough I have the I have a comfort level that allows me to then you know figure out what I need to figure out.
00:47:43.679 --> 00:47:44.079
Yeah, yeah.
00:47:44.159 --> 00:47:44.880
Please respond to that.
00:47:44.960 --> 00:47:47.039
And then I want to talk about your son a bit more because it's so interesting.
00:47:47.519 --> 00:47:48.880
I have a few thoughts on this.
00:47:49.119 --> 00:47:53.360
That's all you you talk about about computational science and so on.
00:47:53.519 --> 00:47:55.280
There's even a step before that.
00:47:55.360 --> 00:47:57.760
It's like, you know, do we need to be able to calculate?
00:47:57.840 --> 00:48:00.880
No, we don't need to be able to calculate because we have machines that calculate for us.
00:48:01.039 --> 00:48:08.559
But we do need, and I often, when I teach classes, I often try to teach the kids at whatever age they are the concept of order of magnitude.
00:48:08.719 --> 00:48:10.400
You know, just how big is this?
00:48:10.480 --> 00:48:12.400
You know, how many people are in the US?
00:48:12.559 --> 00:48:17.599
It doesn't matter if you if you're wrong by a factor, too, but you got to know how many digits has that number.
00:48:17.760 --> 00:48:20.639
You know, is it a million, is it 10 million, or is there hundreds of millions?
00:48:20.719 --> 00:48:21.679
Oh, it's hundreds of millions.
00:48:21.760 --> 00:48:23.440
Okay, exactly how many, it doesn't matter.
00:48:23.519 --> 00:48:25.679
You need to sort of think logarithmically.
00:48:25.760 --> 00:48:26.880
You need to know how many.
00:48:27.119 --> 00:48:35.119
It's like, hey, if I take this number and multiply it with this number, oh yeah, I'm not gonna be able to afford it, or it's not gonna work, or it's it's never gonna fit in a computer, or things like that.
00:48:35.280 --> 00:48:41.840
So this kind of order of magnitude on large scale and and small scales like how many, how many significant digits does this have?
00:48:42.000 --> 00:48:46.000
It doesn't that's kind of thing that I think we need to to teach kids, you know.
00:48:46.079 --> 00:48:50.719
I mean, uh to say like, hey, quickly, how long would it take for me to travel to Mars?
00:48:50.800 --> 00:48:53.119
It's like, is it gonna take uh how fast would I have to travel?
00:48:53.199 --> 00:48:55.119
Is it gonna take hours or minutes or seconds?
00:48:55.280 --> 00:48:56.960
I mean, just order magnitude.
00:48:57.039 --> 00:48:57.840
So that's one thing.
00:48:57.920 --> 00:49:11.440
And then the next step to that is then what we traditionally in high-minded terms call complexity theory, which is you know, about how complex is the problem and what is the what is the O of n of n square of a particular algorithm.
00:49:11.599 --> 00:49:14.159
That's sort of very computer geeky speak.
00:49:14.239 --> 00:49:24.400
But I think it's important that people understand that if I do this procedure and I have a million data points, it's roughly going to take a million times as long as one data point.
00:49:24.639 --> 00:49:31.039
But if I have this algorithm and it's a quadratic algorithm and I use it on a million data points, it probably will never finish.
00:49:31.199 --> 00:49:31.760
You know?
00:49:32.000 --> 00:49:43.840
And so this idea of something being linear versus something else being logarithmic or being quadratic, which is considered very technical, is a concept that I think more people are gonna have to learn.
00:49:43.920 --> 00:49:48.400
And if you take a programming class, one of the first assignments you have is to write a sorting program.
00:49:48.480 --> 00:49:52.400
So you write a sorting program, you turn out it's it's it's quadratic and it doesn't work.
00:49:52.559 --> 00:49:57.039
I mean, uh then you realize the smarter ways to do it and you can make it M log N and now it doesn't work.
00:49:57.119 --> 00:50:00.480
And you can use it on a million or a billion data points, and it's no problem.
00:50:00.639 --> 00:50:07.599
I mean, uh, that kind of thinking is gonna have to become general things that everybody in a curriculum would have to take classes.
00:50:07.840 --> 00:50:09.360
What advice would you have to somebody?
00:50:09.599 --> 00:50:24.639
And I was actually just talking to somebody who's um he's he's working at a law firm right now, he has like a sort of a um a case assistant, and his plan originally was to go to law school, and he's not even totally sold on being a lawyer, but he's like, well, that's the path I'm on.
00:50:24.960 --> 00:50:30.880
And you know, he was thinking, well, you know, should I go and maybe go to a coding boot camp?
00:50:30.960 --> 00:50:34.159
But I'm not sure if there are even jobs for engineers anymore.
00:50:34.320 --> 00:50:45.280
And before I tell you what I had attempted to share with him, I'm curious, like, you know, yeah, like what do you have some sort of like generalized advice for for for kids that don't necessarily know what they want to do?
00:50:45.360 --> 00:50:51.199
They're watching this space, they're seeing how much disruption is happening, they haven't quite figured out their passion yet.
00:50:51.440 --> 00:51:02.559
Yeah, I think, okay, I don't necessarily have the magic bullet, but I get this question all the time from kids and from parents, and so on, just the last week is at least three times that this has come up.
00:51:02.719 --> 00:51:04.639
And again, we don't always know yet.
00:51:04.719 --> 00:51:08.000
I would definitely say, oh, I have to go into AI and I have to become a programmer.
00:51:08.079 --> 00:51:09.360
No, that that for sure not.
00:51:09.440 --> 00:51:16.320
I think you have to figure out in this world of the jagged edge, I'm gonna be on the human side of the jagged edge.
00:51:16.480 --> 00:51:20.320
Like, what is the part that really I want to contribute?
00:51:20.400 --> 00:51:27.519
You know, and it's not because of some specific knowledge that I have, but it is gonna become because of a specific skill that I have.
00:51:27.679 --> 00:51:36.639
I'm gonna have the ability to massage this AI beast and to steer it and to rein it in where I need to to actually get something done.
00:51:36.800 --> 00:51:49.679
Whether it is in a legal field or a financial field or uh a technical scientific field, I would still say follow your passion, follow the things that you're good at, you know, because the things that you enjoy to do are typically gonna be the things you're good at.
00:51:49.840 --> 00:51:56.639
Uh, don't think of it as a traditional academic discipline in terms of I have to get a computer science degree or something like that.
00:51:56.800 --> 00:52:08.239
Uh, I think you're gonna have to deal with this technology beast or monster or a stallion or whatever you want to call it that we've built in order to advance humankind.
00:52:08.320 --> 00:52:19.599
You know, I mean, that's ultimately we want to advance the societies and our way of living to improve our general standard of living and our health and our mental health.
00:52:19.679 --> 00:52:20.559
That's what we want to do.
00:52:20.719 --> 00:52:26.960
And that's why we say, like, think of in the large world what piece you want to fit into, and then yeah, learn all these things.
00:52:27.119 --> 00:52:37.519
Take a basic uh uh order of magnitude class, take a basic algorithms class, understand what complexity is, understand why some things work well on the computers and others don't.
00:52:37.679 --> 00:52:40.639
Understand why sometimes you get the craziest answers.
00:52:40.800 --> 00:52:42.320
Um and I don't want to call back.
00:52:45.039 --> 00:52:51.599
It is worth coming back to because this is a moment where there's a lot of buzzwords floating around that don't mean very much.
00:52:52.159 --> 00:53:08.719
And I found that talking about the jagged edge is actually very it's very poignant for people because it intuitively I it makes sense because I think people have had that simultaneously those experiences of like the thing that AI is very good at, as you say, and then also the things that it's like really horrifically bad at.
00:53:08.880 --> 00:53:12.480
But I think putting it into the context of like that is actually the end goal, right?
00:53:12.559 --> 00:53:23.440
It's like figuring that out and being able to not just figure out in a point of time, but developing the instinct to be able to sort of in real time figure it out sort of day by day as it evolves.
00:53:23.679 --> 00:53:34.639
But you said something important, which is so find something you're passionate about, because that's going to be the most likely area that you're going to invest the energy into that with like the sort of the month the entrepreneurial mindset.
00:53:34.719 --> 00:53:39.360
Like if you're giving advice to entrepreneurs, usually it's like make sure you're building a company and something that you're interested in.
00:53:39.440 --> 00:53:42.159
You're doing if you're just doing it to make money, you're not going to succeed.
00:53:42.480 --> 00:53:56.320
But then you also said, you know, make sure that you are pursuing some amount of um uh let like some way of building your computational thinking skills or sort of understanding of understand.
00:53:56.559 --> 00:53:57.760
I mean you almost put it in math.
00:53:57.840 --> 00:54:05.679
I mean, I wonder if math is like it would you say would you say math instead of computer science would be the sufficient and like stand-in?
00:54:05.920 --> 00:54:08.639
I am like do you have to take a CS minor?
00:54:08.800 --> 00:54:09.599
Could it be a math minor?
00:54:09.679 --> 00:54:10.400
Could it be a physics minor?
00:54:10.639 --> 00:54:11.679
Yeah, I mean, yeah, yeah.
00:54:11.760 --> 00:54:13.679
It's it's it's a lot of it is common sense.
00:54:13.760 --> 00:54:16.320
Like thinking in order of magnitude is also common sense.
00:54:16.400 --> 00:54:27.760
You don't have to be a mathematician to do it, but just think about what is a logarithmic and what is a logarithm and what is an exponent, that or those those are good concepts to know for for everybody, uh in my mind.
00:54:28.000 --> 00:54:36.079
I think the other thing is that somebody uh post me this uh this sort of thought experiment that that people like you and I could do.
00:54:36.159 --> 00:54:41.519
It's like sit down and it's a thought experiment, and write down everything you know.
00:54:41.840 --> 00:54:43.840
It's like, holy crap, I would take a while.
00:54:43.920 --> 00:54:47.519
You know, I mean uh I I've forgotten a lot, but write down everything you know.
00:54:47.599 --> 00:54:49.760
And just of course you're not gonna do it, but just imagine it.
00:54:49.840 --> 00:54:52.639
You know, you you now write books and books and books and everything you know.
00:54:53.280 --> 00:54:59.199
Okay, well, all of that really has no value because an AI will know all of this too.
00:54:59.440 --> 00:55:04.719
Okay, if it has been written down, then it has written it somewhere, it has read it somewhere on the internet, it knows it.
00:55:05.280 --> 00:55:10.320
Now think about everything that you can do that is not written down.
00:55:11.280 --> 00:55:12.480
That is your skill.
00:55:12.880 --> 00:55:14.320
You know, that is your skill.
00:55:14.480 --> 00:55:17.599
Like the example that people always give is like riding a bike, you know.
00:55:17.760 --> 00:55:22.320
I can't write down instructions on how to ride a bike, give them to you, and you read them and ride a bike.
00:55:22.400 --> 00:55:23.039
You can't do it.
00:55:24.000 --> 00:55:26.239
Can I have I taught my kids how to ride a bike?
00:55:26.320 --> 00:55:27.679
Yeah, I figured out how to do it.
00:55:27.760 --> 00:55:29.119
I mean, uh, but can I write it down?
00:55:29.199 --> 00:55:30.239
No, I can't write it down.
00:55:30.320 --> 00:55:34.719
You can read it, but it's a skill that is in your brain that we can transfer to other people.
00:55:35.039 --> 00:55:37.199
Now, there's lots of other things like that.
00:55:37.280 --> 00:55:59.679
And you talked about entrepreneurs like, hey, build a company that makes money, that's channel ID ID, but the companies that succeed are the ones that have consistent values, and the the thing that I come back to a lot, and this is more related to my personal work and study that I do on spiritualism, is like, what is your net worth as a human?
00:55:59.920 --> 00:56:04.000
It's not your bank account, it's the sum of your values, you know.
00:56:04.159 --> 00:56:13.199
And this is something that I think makes us uniquely human in some sense because we have values, we have shared values, we might have different values, you know, that makes us human.
00:56:13.280 --> 00:56:15.840
And I think be able to, same with a company.
00:56:16.000 --> 00:56:17.519
A company's gotta have values.
00:56:17.599 --> 00:56:20.800
Like we are the best at this, and we do this that way.
00:56:20.880 --> 00:56:26.719
And it could be the values could be on economic value, it could be on societal value, it could be on lots of different values.
00:56:26.880 --> 00:56:32.880
But if you know your values and you align your workforce on your values, then you can be successful.
00:56:33.599 --> 00:56:40.880
So that's is the part that is the the human part of the the human side of the jacket edge that I think is the fun part in some sense.
00:56:41.280 --> 00:56:53.519
It must have been that that uh the conversation we had in New York where we talked about riding a bike, because I've I found that that analogy it clicks for people in a really important way because well, for a few reasons, yeah, exactly.
00:56:53.679 --> 00:56:56.639
You ask people the I usually would literally do this, I'm like, well, how do you ride a bike?
00:56:56.719 --> 00:57:00.400
And they and then once you get to the balancing part is where they get stuck, they're like, Well, you balance.
00:57:00.559 --> 00:57:01.760
I was like, Well, how do you balance?
00:57:02.000 --> 00:57:05.360
It's like, well, you you shift your weight, like how much do you shift your weight?
00:57:05.440 --> 00:57:06.239
It's like, oh, okay, yeah.
00:57:06.320 --> 00:57:08.960
And they quickly realize, like, yeah, there's just you feel it out.
00:57:09.039 --> 00:57:10.159
You just have to get the feel for it.
00:57:10.400 --> 00:57:12.960
I love and yeah.
00:57:13.440 --> 00:57:15.920
But the other thing about riding a bike is you feel it out.
00:57:16.159 --> 00:57:18.719
It doesn't take years to ride a bike.
00:57:19.280 --> 00:57:22.320
It takes sometimes a few days, maybe a week, maybe two weeks.
00:57:22.559 --> 00:57:27.519
But the other thing about riding a bike is you can you can ride a bike, but then you can look at someone who's a mountain biker.
00:57:27.599 --> 00:57:33.840
You can look at someone who's uh, you know, in Belgium you have um, you know, street uh, you know, like road like road but road bikers.
00:57:33.920 --> 00:57:36.239
Um the cyclists here, you know.
00:57:36.559 --> 00:57:44.719
It isn't trivial to get to a place where you can ride in a Peloton, you know, inches, maybe even millimeters away from the the rider in front of you.
00:57:45.119 --> 00:57:55.199
And so and so I think that there's a power there and like um it's AI does feel like that, where it's like anybody can, I really do believe anybody can be sort of like an AI.
00:57:55.760 --> 00:57:56.800
What is it?
00:57:56.960 --> 00:58:00.880
Uh uh uh James or Jason Lemkins.
00:58:01.440 --> 00:58:05.360
Um Lemkin, this is like on uh 20 VC, is like one of the podcasts.
00:58:05.599 --> 00:58:35.840
He was he's he's coined this new, so there's like prompt engineer, and he's like, Oh, okay, I think that we're gonna be talking about ADEs, agent deployment engineers, which I think was a bit of like almost like a tongue-in-cheek, like let's just continue making up these like sort of meaningless, um uh these meaningless like phrases, but but actually there's like something there, which is you he was basically talking about what you had described, which is this like the what was happening at your comp what is happening in your company is these like re rethinking about what does it even mean to like engineer.
00:58:36.159 --> 00:58:47.280
And so you're you're basically all these folks at your company are various stages of learning to ride this bike, and then you're trying to get them to this place of like, okay, we need to ride in the Peloton together.
00:58:48.239 --> 00:58:48.960
Yes, that's right.
00:58:49.119 --> 00:58:56.320
Yeah, that's a good and haven't I mean, shame on me as a cyclist, I should have figured out these cycling analogies, but yeah, that is a good way to think about it.
00:58:56.400 --> 00:59:06.719
You first have to ride your individual bike, and then you have to figure out how to ride in a Peloton or how to ride in a baseline because you're not gonna win the race unless you know how to ride in a baseline, which is a very particular skill.
00:59:06.960 --> 00:59:10.559
And so maybe there's more to this analogy than I than I thought there was.
00:59:10.719 --> 00:59:21.840
And it's it also reminds me of something that when I taught my kids how to ride a bike, I mean, it was very fascinating because I I personally, as a kid myself, had a very frustrating experience learning to ride a bike.
00:59:21.920 --> 00:59:24.480
It was very frustrating for me, it was very frustrating for my parents.
00:59:24.559 --> 00:59:26.159
So I just wasn't very good at it.
00:59:26.239 --> 00:59:29.039
It took a lot more than a couple of weeks to to get it.
00:59:29.199 --> 00:59:32.800
And I might have had a little trauma for me, it's like, oh, this was not a good experience.
00:59:32.880 --> 00:59:38.000
So I was determined when I when my kids were of the age to I'm not gonna have a frustrating experience.
00:59:38.239 --> 00:59:39.679
So, of course, I did some research.
00:59:39.760 --> 00:59:49.199
I actually looked on the internet, which was kind of innovative those days, and I came up with this 10-step program like here are like step one, you do this, you balance, you do it, and so on.
00:59:49.440 --> 00:59:56.719
And and then I said with my kids, we would practice every day for five minutes, never more than five minutes.
00:59:57.199 --> 01:00:01.440
And any sign of any frustration on either the kid or the parent, we would immediately stop.
01:00:02.159 --> 01:00:02.960
Oh, I don't feel like doing this.
01:00:03.119 --> 01:00:04.079
Okay, we're gonna hope that's it.
01:00:05.280 --> 01:00:06.719
And and then I would rate them.
01:00:06.800 --> 01:00:08.559
It's like, oh, I think you're at a level two and a half.
01:00:08.639 --> 01:00:10.320
And they're like, Oh, oh, that's really good, Dad.
01:00:10.400 --> 01:00:10.960
What about tomorrow?
01:00:11.039 --> 01:00:12.239
We'll try a level three.
01:00:12.480 --> 01:00:20.800
And what I what I observed is that even though we only practiced five minutes a day, they would always improve between in between sessions.
01:00:21.119 --> 01:00:28.960
They would never improve during uh uh an actual practice session, they would improve between sessions, like overnight, they would get better.
01:00:29.199 --> 01:00:43.760
And I always thought it must be something in your brain that your brain at night is rewiring some neurons and it's given some stimuli about balancing a bike, and you don't know how to do it, but it sort of compiles it overnight, and the next day you're better at it.
01:00:43.920 --> 01:00:49.119
Uh, and there's something maybe to learn from that, because I think something is because a bike is like a jagged edge.
01:00:49.199 --> 01:00:50.400
If you don't do it right, it speaks to it.
01:00:51.519 --> 01:00:59.280
And I know that we're coming up on time, and I think this is a really sort of beautiful way to end it for you know, for folks that are trying to uh you know find their path.
01:00:59.360 --> 01:01:02.719
And so we know one of the things you said I think is important, there is no sort of right answer.
01:01:02.800 --> 01:01:08.239
You're asked this sounds like a couple times a week, and so you've avoided just sort of giving something prescriptive.
01:01:08.480 --> 01:01:17.599
And yes, it's like riding a bike, but it's also this consistency because it will build you know, like getting the feel for something is is not going to happen overnight.
01:01:17.760 --> 01:01:20.079
You can you can get far enough where you can actually start to.
01:01:20.320 --> 01:01:31.280
I mean, I think for me, the more the the the pieces getting it's really helpful to get to a place where you actually had built something just so you can see it's like you know, just just just to see that there's there's a magic here.
01:01:32.159 --> 01:01:37.119
Um and the other thing that you you had talked about is sort of like find your passion.
01:01:38.000 --> 01:01:57.360
And the important because and just to sort of connect the dots here is like because that's important because if you're working on something you're passionate about, that will be the thing that continues to bring you back over an extended enough period of time where you will start to have the benefits of sort of that compounding, yeah, sort of like compiling that's happening, like sort of in your brain.
01:01:57.679 --> 01:02:07.360
And then maybe one final uh thought for my stuff, because you're in education, you said you're K-12 education, but with AI, education will never end.
01:02:07.519 --> 01:02:08.480
There's no end to it.
01:02:08.880 --> 01:02:11.199
Oh, now I've gotten my degree and I'm gonna practice it.
01:02:11.280 --> 01:02:17.679
No, because the irony of AIQ is that every day your AIQ will go down, unless you keep it up.
01:02:18.000 --> 01:02:21.760
Because AI will become smarter, the X edge will become more jagged.
01:02:21.840 --> 01:02:31.519
So if you don't keep up, your AIQ, unlike your IQ, which presumably will remain relatively steady till maybe you get older, your AIQ, without doing anything, will go down.
01:02:31.679 --> 01:02:34.239
So you constantly have to keep it up in some sense.
01:02:34.320 --> 01:02:37.519
So that's why it changes the whole everybody's an intern in a way.
01:02:37.679 --> 01:02:42.800
I'll be we're like I said, we're practicing, we're doing new projects, we're finding passions at every age and at every stage.
01:02:43.119 --> 01:02:44.159
I know I really like the parting thought.
01:02:44.639 --> 01:02:46.719
So maybe that's a parting thought.
01:02:47.119 --> 01:02:51.920
I will I have a I have a lot of it's it's it's connected to some dots for me.
01:02:52.000 --> 01:02:55.360
Um I'm gonna maybe I'll do some writing and I'll send I'll send you something.
01:02:55.599 --> 01:03:00.079
Um we'll we'll have to set up another session.
01:03:00.800 --> 01:03:01.360
It's very fun.
01:03:03.679 --> 01:03:04.079
Thank you.
01:03:04.400 --> 01:03:05.119
We'll find another time.
01:03:08.719 --> 01:03:14.639
Hopefully we haven't looked different than it looks today, that's a guarantee.