The Problem With AI Observability Nobody Wants To Admit

WEBVTT
00:00:00.160 --> 00:00:05.120
It's shocking to me that observability of our systems is not table stakes.
00:00:05.280 --> 00:00:20.719
You need to have someone on staff who can understand and look at that observability at a system level and be able to discern that impact, be able to communicate that to the relevant business.
00:00:20.879 --> 00:00:26.239
You need to make sure that you've got your internal observability with anything you build or anything that you buy.
00:00:26.399 --> 00:00:33.280
You need to have someone internally who can leverage those at an intelligence level, not just a log level.
00:00:33.439 --> 00:00:36.799
If AI has ever made you stop and think, wait, what is happening?
00:00:36.960 --> 00:00:38.079
You're not alone.
00:00:38.320 --> 00:00:41.679
I'm Mo, and I'm a security researcher asking the same questions.
00:00:41.920 --> 00:00:55.920
On Curiouser and Curiouser, we're having open conversations with experts, researchers, and leaders working at the edge of this space, talking through how AI is taking shape, what's shifting, and how people inside the work are thinking about it as it happens.
00:00:56.159 --> 00:00:59.439
So join us and listen in as the conversation takes shape.
00:01:00.799 --> 00:01:01.840
Welcome back.
00:01:02.079 --> 00:01:05.599
Another week of me and my friends.
00:01:05.760 --> 00:01:07.760
We are back on Curiouser and Curiouser.
00:01:07.920 --> 00:01:08.799
Uh I'm Mo.
00:01:08.879 --> 00:01:10.560
Uh, you've probably had enough of me already.
00:01:10.879 --> 00:01:15.920
But someone you haven't had enough of is my friend Alison Cassette.
00:01:16.079 --> 00:01:23.040
Um, not cassette, cossette, uh founder of Clara Trace, uh and an overall AI nerd.
00:01:23.120 --> 00:01:27.280
She's super cool, met her at a conference, and was like, we got to talk some more.
00:01:27.519 --> 00:01:30.239
And it's it's been an awesome, it's been awesome.
00:01:30.400 --> 00:01:35.120
So I'm gonna let you talk about yourself because I've found that I'm horrible at talking about other people.
00:01:35.359 --> 00:01:37.599
I don't know if I'm so good at talking about myself either.
00:01:37.680 --> 00:01:40.159
So between the two of us, we're we're in a spot.
00:01:40.400 --> 00:01:55.359
Um, as you mentioned, I'm Alison Cossette, started out as a healthcare data scientist, been in the AI world since before it was cool, when it was just a bunch of math nerds somewhere in a corner building algorithms and waiting for somebody else to put them in production.
00:01:55.599 --> 00:01:58.640
Since then, obviously the world has evolved, as have I.
00:01:58.799 --> 00:02:16.240
And most of what I'm doing these days is deep AI research, seeing where we're going, trying to build systems that will allow us to have an AI architecture and an AI ecosystem that makes what we do and what we do for humans better.
00:02:16.560 --> 00:02:16.960
Yeah.
00:02:17.120 --> 00:02:23.039
Um, and speaking of that, actually, like uh just like the whole AI thing.
00:02:23.280 --> 00:02:38.719
Um, I know for I know sometimes you've kind of um just as like this awesome AI person, you've said that it's easy to call AI like a black box and that it's just like an excuse for bad design.
00:02:40.400 --> 00:02:43.599
So those are really those are honestly fighting words.
00:02:43.759 --> 00:02:46.240
Um, especially when I I don't know.
00:02:46.400 --> 00:02:49.520
I'd say that the the industry kind of loves the complexity, right?
00:02:49.599 --> 00:02:54.159
Because it just makes it uh it makes the end solution seem so much more interesting.
00:02:54.400 --> 00:02:59.039
So maybe you can tell me a little bit more about that kind of like thought process.
00:02:59.199 --> 00:03:05.199
Like what makes it so that like maybe this black box is smoke screening something else that's a lot bigger?
00:03:05.360 --> 00:03:06.560
Like what are we missing in there?
00:03:06.800 --> 00:03:07.840
It's a really great question.
00:03:07.919 --> 00:03:09.039
And I'm glad you brought it up.
00:03:09.199 --> 00:03:12.639
So am I saying that there is no black box?
00:03:13.280 --> 00:03:14.080
Not really.
00:03:14.479 --> 00:03:21.120
But what I want everyone to understand is that at the end of the day, everything that we've built is technically math.
00:03:21.280 --> 00:03:27.039
It's very complex math, it's very high computational math, but ultimately it really is math.
00:03:27.280 --> 00:03:42.960
So whereas in a traditional ML linear regression, this is the formula for how much your house should cost, so many dollars per bedroom, so many dollars per square foot, so many dollars per lot size was very explainable.
00:03:43.199 --> 00:03:51.759
Now we're in a generative moment where we use these foundation models and we don't necessarily know exactly where this piece came from or that piece came from.
00:03:52.080 --> 00:04:05.759
The good news is while those weights are not open in most cases, there are people doing really phenomenal work around explainability, even within the foundation models.
00:04:05.919 --> 00:04:17.759
Think what Anthropic's doing with mechanics, mechanistic interpretability, where um you can actually did you ever hear of the Golden Gate boost that they had done at one point?
00:04:18.079 --> 00:04:18.800
No, I haven't.
00:04:18.879 --> 00:04:19.600
What's that?
00:04:19.920 --> 00:04:26.399
So, what they had done is they found within the model where the Golden Gate Bridge was and they cranked it up, right?
00:04:26.639 --> 00:04:33.600
Won't get into the deep technical linear algebra of um neural networks, but they cranked up the golden gate bridge.
00:04:33.759 --> 00:04:35.759
And then they asked it, what's two plus two?
00:04:36.079 --> 00:04:44.560
And it said, two plus two is two golden gate bridges, and another two beautiful golden gate bridges gives you four glorious golden gate bridges.
00:04:44.720 --> 00:04:45.600
I paraphrase.
00:04:45.839 --> 00:04:51.120
But what they found is that it wasn't just the golden gate bridge, but it was the awe of the golden gate bridge.
00:04:51.439 --> 00:04:54.399
And so that's what really got me thinking.
00:04:54.800 --> 00:05:07.920
We know that it's math, we recognize that there are activation functions in different places, we can clearly see when we have insight into these models themselves that we can manipulate.
00:05:08.240 --> 00:05:13.759
So when we follow that forward, what does that make possible?
00:05:14.160 --> 00:05:28.480
Is it that yes, it's complex, but it means it's very plausible that the ability to steer the AI, the ability to steer the foundation model even, is not as far off as a concept as we think.
00:05:28.959 --> 00:05:33.920
So that's how I ended up at the concept of control native AI, which is where you and I met.
00:05:34.240 --> 00:05:37.839
Yeah, and control native AI was actually really interesting.
00:05:38.000 --> 00:05:40.319
When you were explaining it, it was really cool.
00:05:40.480 --> 00:05:47.439
So maybe for folks that haven't heard about it, um, why don't you give a really short roundup on what control native AI is in your own words?
00:05:47.759 --> 00:05:48.240
Sure.
00:05:48.480 --> 00:05:52.079
So that's sort of the technical insides of it.
00:05:52.160 --> 00:05:57.279
But the way I like to, the analogy I like to use is AI is somewhat like a bowling ball.
00:05:57.439 --> 00:06:06.720
It has a lot of power behind it, it has a lot of force, it can do really great things, and we throw it down the bowling alley, and boom, things happen and it's glorious and magical.
00:06:06.959 --> 00:06:14.079
And then we put on guardrails because we realize that there are certain things coming out of generative AI that maybe we don't want coming out.
00:06:14.240 --> 00:06:21.279
It's built on what humans have said and written, not all things that we've said and written we necessarily want repeated.
00:06:21.439 --> 00:06:23.040
And so we put in these guardrails.
00:06:23.360 --> 00:06:30.720
So, okay, you're in the bowling alley, you're bowling the ball, it doesn't go into the gutters, you've got the guardrails are great.
00:06:31.600 --> 00:06:38.720
But we also recognize as we move into agentic systems, and again, we're in generative systems, so they're not bounded.
00:06:39.120 --> 00:06:44.000
What happens if, say, you know, agents start to swarm?
00:06:44.160 --> 00:06:45.839
We've seen them do really interesting things.
00:06:45.920 --> 00:06:48.160
I'm sure we're gonna hit on open claw today.
00:06:48.399 --> 00:06:55.759
Or we've seen agents um in labs working together to manipulate markets, they were given trading capabilities.
00:06:56.000 --> 00:07:01.439
And so as we get, it's it's almost as if the bowling balls can now fly.
00:07:01.600 --> 00:07:04.240
They're in three dimensions, they're moving in all different places.
00:07:04.399 --> 00:07:11.839
And now, if the bowling ball's mission is to knock over the pins, what if all the balls start working together?
00:07:12.000 --> 00:07:16.720
And then they just destroy the pin resetter because they're focused on this one mission.
00:07:16.959 --> 00:07:23.360
This is the the optimization that they're going for, this is the reward that they're optimizing for.
00:07:23.600 --> 00:07:39.839
And so I'm thinking about this and I'm thinking, well, if we take it back to this small concept of mechanistic interpretability, what would it mean if instead of having the guardrails around where the balls were going, what if we had sensors inside?
00:07:40.000 --> 00:07:42.959
What if we could see the directions that they were heading?
00:07:43.199 --> 00:07:50.079
Now, what this means mathematically, we can certainly get into, but that's what I mean when I'm talking about control native systems.
00:07:50.240 --> 00:08:00.720
How is it that we can build not just the models, but the systems around them to have an understanding of where they're going, not just where they've gotten.
00:08:01.040 --> 00:08:08.240
So control native AI, um, just the high level from what it sounds like is more about all the things around it, right?
00:08:08.399 --> 00:08:11.360
So that you can understand the directions in which it goes.
00:08:11.600 --> 00:08:25.040
So having the right observability, the traceability, and being able to understand maybe um all of the different factors that cause it to go in a certain direction, have certain responses, et cetera.
00:08:25.279 --> 00:08:25.519
Yeah.
00:08:25.600 --> 00:08:29.360
And I think the important thing is to see where it's going.
00:08:29.439 --> 00:08:33.600
So the way I like to think of it is the first thing that we need to have is the observability.
00:08:33.679 --> 00:08:35.919
And I think as an industry, we've done that really well.
00:08:36.080 --> 00:08:37.919
There are companies that are phenomenal at it.
00:08:38.080 --> 00:08:39.840
We have Arise, they're really great.
00:08:40.000 --> 00:08:42.159
There's a lot of folks that are doing that well.
00:08:42.960 --> 00:08:47.360
From there, we need to have the understanding and the interpretability.
00:08:47.519 --> 00:08:53.200
And when it comes to some of the agentic systems, in the last couple of months, it's kind of starting to catch up a little.
00:08:53.440 --> 00:09:01.600
Um, but only at that point when we have the clarity and we have the understanding, can we start diving into what it means to steer?
00:09:01.840 --> 00:09:04.879
And that's where I think we need to start looking forward to.
00:09:05.039 --> 00:09:06.879
How do we steer the system?
00:09:07.039 --> 00:09:10.240
Or how do we understand the direction that it's going?
00:09:10.639 --> 00:09:14.559
My view on that is I'm a big fan of the digital twin.
00:09:14.720 --> 00:09:29.039
I'm a big fan of understanding, creating the digital twin of the system, really looking at the physics of which tools are we using, which agents are collaborating more or less, what are the live physics of the actual system?
00:09:29.279 --> 00:09:35.600
And that's going to give us the understanding of where it's going rather than just a log level output.
00:09:35.840 --> 00:09:44.320
You know, when we were talking about it, I remember we were talking about how like uh you have this right now, guardrails are like the really like hot topic, right?
00:09:44.480 --> 00:09:46.559
For being able to stop that output.
00:09:46.799 --> 00:09:52.320
And um, I remember talking about guardrails, and you were just like, guardrails are useless.
00:09:52.399 --> 00:09:53.039
They're the worst.
00:09:53.360 --> 00:09:55.679
Guardrails are are fences in space.
00:09:55.840 --> 00:09:56.080
Yes.
00:09:56.320 --> 00:09:58.960
And I was like, that makes a lot of sense, right?
00:09:59.120 --> 00:10:08.000
Um, because again, they're stopping one thing, but I mean, there are so many directions in which AI is pulled in, and you can't always predict where it's gonna go.
00:10:08.080 --> 00:10:16.720
And guardrails are almost uh exclusively made because you know exactly where the response is going, right?
00:10:16.960 --> 00:10:23.120
Um, but I think you miss out on all these other um these tertiary effects and outcomes, right?
00:10:23.279 --> 00:10:38.320
Um, while the response may be blocked, um, again, in a deterministic system, it's possible that just the prompt that was getting you to that response is causing a shift or a drift in the system and how it responds in the future.
00:10:38.639 --> 00:10:49.600
So you're kind of opening up to uh out, let's call it like um the like this AI native control is almost like looking for like this AI systems intelligence piece, right?
00:10:49.759 --> 00:10:53.200
Where you're really trying to understand the intelligent piece of that.
00:10:53.360 --> 00:11:00.799
I think like when you when you look at it, right, that's not really something that companies have ever had the budget for or they've ever had to really think about, right?
00:11:00.960 --> 00:11:12.320
So like you there's I don't think uh we've ever walked into a room with a CTO or like the chief AI officer, new, new new uh new position that's been uh kind of coming out, right?
00:11:12.559 --> 00:11:18.159
Um like how do you even open up to these people and say, like, hey, you need more AI systems intelligence?
00:11:18.320 --> 00:11:33.279
Like, how do you demonstrate the value of traceability when you've got a room full of people that literally just want the best models, they want them to work better, and they want to create business value or like the meme says, shareholder value, right?
00:11:33.600 --> 00:11:39.039
We've been working on ClaraTrace for almost two years, and it's not an easy conversation to have.
00:11:39.279 --> 00:11:45.840
We, you know, we were told early on you're ahead of the market, people aren't there yet, they're not ready to take that on.
00:11:46.000 --> 00:11:50.799
Uh, so it is a tough conversation and it's not something that folks are thinking about.
00:11:50.960 --> 00:11:54.240
It's starting to come into the conversation a little bit.
00:11:54.559 --> 00:12:09.840
Um, I had had a conversation with one of the heads of AI for one of the big consulting companies, and they had said, now we've built this great platform and we have hundreds and hundreds of agents, but we don't know what they're doing or how they're collaborating.
00:12:10.000 --> 00:12:13.600
And all of a sudden I was like, oh, the moment is coming over getting there.
00:12:13.840 --> 00:12:16.159
And it's because people aren't thinking about it yet.
00:12:16.240 --> 00:12:20.399
They don't know that it matters, they don't know that it's an issue.
00:12:20.720 --> 00:12:30.399
And I mean, when we think about even where agents were a year ago, last February was the first AI engineer agent summit in New York.
00:12:30.639 --> 00:12:35.919
Now we've got agents as the topic of many Super Bowl commercials a year later.
00:12:36.159 --> 00:12:41.440
So agents as a regular everyday uh commonality were there.
00:12:41.679 --> 00:12:50.559
What I think we have are behind on is the risk debt that we have built up by this rapid adoption.
00:12:50.799 --> 00:12:53.279
And we're starting to see it come through.
00:12:53.440 --> 00:12:59.840
We're starting to see some of the challenges that people have run into when they let things run wild.
00:13:00.000 --> 00:13:01.120
So we're getting there.
00:13:01.200 --> 00:13:09.679
But honestly, I the sad thing is it's until it goes wrong for somebody on a really high level, people aren't going to be talking about it.
00:13:09.919 --> 00:13:10.240
Right.
00:13:10.399 --> 00:13:15.200
People don't, people don't buy people don't think about their life insurance when they're 18.
00:13:15.360 --> 00:13:18.000
They think about it once they have children and there's something at stake.
00:13:18.159 --> 00:13:20.080
They think about it after they've lost someone.
00:13:20.240 --> 00:13:28.399
And it's that same kind of viewpoint is people aren't there yet because it's still new and they don't recognize what's coming.
00:13:28.559 --> 00:13:37.120
And so part of my directive and what's really important to me is to get out and talk about it as much as possible to get that conversation moving.
00:13:37.440 --> 00:13:48.000
So all of the chief AI officers and the CTOs can really start thinking through how do we address this in this macro way?
00:13:48.559 --> 00:13:53.840
I think addressing it is it's just not an even conversation, especially where we're at right now.
00:13:54.000 --> 00:13:59.039
Um things are moving way faster than I think any of us have anticipated.
00:13:59.279 --> 00:14:10.240
Um but I mean, if we had to, if we had to take a step back and really think about how we got here, um, let's go back in the days, like you said, last year that there's so many changes.
00:14:10.399 --> 00:14:15.360
Um, you know, we had the first AI AI conference for engineering or AI engineering conference.
00:14:15.519 --> 00:14:24.799
Um, you go even further back, you go to 2022, you've got the first you know, instance of like ChatGPT getting big, right?
00:14:25.679 --> 00:14:41.840
And before then, right, you actually just had like people installing Anaconda on their systems and taking a really long time to build very rigorous pipelines uh around machine learning, right?
00:14:42.159 --> 00:14:47.519
And it felt like um during yeah, like we we've been there.
00:14:47.840 --> 00:15:02.240
And you know, the amount of of I I was at a shop that was doing this, and there were so many, there was so much concern around compliance and just data sovereignty, like where's the data residing?
00:15:02.399 --> 00:15:08.480
Um, making sure that there was a lot of data integrity, a ton of validation, a ton of testing, right?
00:15:08.639 --> 00:15:14.799
Because no one wanted anything in machine learning to like go us astray, even 1%.
00:15:15.519 --> 00:15:34.159
Like you had to be so sure that everything you were getting, whether it was uh forecasting or predictive model, um, all the way to just um to understanding how um a model was getting to a conclusion, um, everything needed to be like super like just sealed.
00:15:34.320 --> 00:15:40.879
It feels like we've kind of lost a little bit of that in just trying to keep up with the velocity.
00:15:41.279 --> 00:15:46.320
So, what is it going to take to make the pendulum swing back to like this extreme rigor?
00:15:46.480 --> 00:15:51.360
Or do we even need to go back to a place where um we need that same rigor again?
00:15:51.679 --> 00:15:54.000
Yeah, all really great observations.
00:15:54.159 --> 00:16:00.960
One, yes, the the level of rigor and responsibility we had when we were building them by hand was immense.
00:16:01.200 --> 00:16:12.480
And it's why when AI first became a commonly used tool by people who didn't understand it, I kind of felt like we gave the middle schoolers cars to drive.
00:16:12.639 --> 00:16:14.399
Like you don't know how to drive it yet.
00:16:14.480 --> 00:16:15.679
Just slow down a little bit.
00:16:15.840 --> 00:16:17.360
Can we start with a go-kart?
00:16:17.600 --> 00:16:23.200
Um and so acknowledged A.
00:16:23.519 --> 00:16:28.320
Um, I think the point that you bring up that's the most important part is the velocity.
00:16:28.639 --> 00:16:58.399
That because the technology and the development of technology and deployment and our ability to deploy is moving at such a breakneck speed that it is very hard as a business is and as a C-suite and as leaders to know how do we we don't even necessarily understand the entirety of the risk, but we do know what our competitors are doing.
00:16:58.480 --> 00:17:00.399
We're very familiar with that risk.
00:17:00.559 --> 00:17:07.920
We clearly see that if we don't embrace this, we don't move quickly, we're going to be left behind, especially for small media businesses.
00:17:08.000 --> 00:17:09.440
This can be existential.
00:17:10.319 --> 00:17:25.680
And so unfortunately, it's just the nature of how fast things are moving and how much people have a comprehension of the risk involved, that they have a good, clear idea on their business risk, they know how to manage that.
00:17:26.000 --> 00:17:38.559
And while it's not necessarily a black box, the depth of technical understanding and technical prowess isn't necessarily there for people making decisions.
00:17:38.720 --> 00:18:04.559
So I think to your point, part of the reason why the chief AI officer is predominant, and I would encourage everybody to have someone in your organization that is the AI expert because you need someone to bring that understanding and to really become an important part of your risk identification profile to know where things can be.
00:18:04.720 --> 00:18:14.400
You know, I think at the conference that we met, there was this one moment where I said, when you're in generative circumstances, the risk plane is infinite.
00:18:15.200 --> 00:18:21.279
And to have an understanding of that, you know, it used to be, like you said, getting to the data.
00:18:21.440 --> 00:18:22.400
Is it valid?
00:18:22.640 --> 00:18:24.480
Is somebody trying to hack into our system?
00:18:24.640 --> 00:18:29.599
There are these very clear risk moments and you could identify them.
00:18:29.759 --> 00:18:35.599
But now that risk is more ethereal in some ways.
00:18:35.839 --> 00:18:47.519
And so I think unfortunately, as the case with most things, it's not going to become a common conversation until something goes tragically wrong.
00:18:47.759 --> 00:18:55.599
Um, you know, in the military, they say until somebody dies, or in business, it's until somebody loses a lot of money.
00:18:55.759 --> 00:19:01.759
And one of those things is going to occur and everybody's going to say, oh no, we should probably look at this.
00:19:02.000 --> 00:19:04.960
The good news is there's lots of people doing phenomenal research.
00:19:05.200 --> 00:19:06.559
Companies are working on it.
00:19:06.799 --> 00:19:11.200
So the information and the tooling and the answers are out there.
00:19:11.519 --> 00:19:12.960
People just have to be ready for it.
00:19:13.279 --> 00:19:19.759
I wish I was half as persistent as AI was because it's so driven to a goal, right?
00:19:19.839 --> 00:19:21.359
It's driven to an objective.
00:19:21.440 --> 00:19:25.200
And it will find any path to that objective as it can.
00:19:25.359 --> 00:19:41.759
Um, you know, I in I've been in a couple of startups, and there's always that one engineer that finds a way to complete a task in a way that like totally would flag security if we figured out how to how to catch it fast enough, right?
00:19:42.000 --> 00:19:46.240
Uh again, in this case, those flags aren't being flipped at all.
00:19:46.400 --> 00:19:49.039
You know, and these agents are going to find ways.
00:19:49.119 --> 00:19:52.480
It's not when, it's not if they're going to find a way, it's when.
00:19:52.640 --> 00:19:59.519
Because again, 24-7, always trying to complete a task, it's all about when are we going to complete the task.
00:19:59.759 --> 00:20:00.319
Yeah.
00:20:00.640 --> 00:20:05.200
So it's just the the area of risk is is so hard to imagine.
00:20:05.279 --> 00:20:10.640
And just um catching it and kind of figuring out a good control for it is really difficult.
00:20:10.880 --> 00:20:15.279
But I think that's kind of what you're what you're kind of thinking about as well.
00:20:15.359 --> 00:20:16.640
It's kind of top of mind for you.
00:20:16.880 --> 00:20:23.359
You know, I always go back to the series finale of Silicon Valley, which is probably 10 years ago now.
00:20:23.519 --> 00:20:30.720
I don't even know how long it's been, where the AI got so good they had to, they had to kill it because they knew they couldn't unleash it.
00:20:31.039 --> 00:20:34.000
Well, we clearly learned nothing.
00:20:34.240 --> 00:20:38.079
Um, but to your point, it is, it really is relentless.
00:20:38.319 --> 00:20:43.200
And so that's why I guardrails don't work, right?
00:20:43.359 --> 00:20:50.319
Guardrails don't work because the AI will come up with so many other places.
00:20:50.480 --> 00:20:54.960
There's no way we can even imagine all the edge cases or all the possible ways.
00:20:55.200 --> 00:20:56.400
And then what do we do?
00:20:56.559 --> 00:21:02.240
We have one AI trying to figure out all the ways to stop it, and the other one is trying to find all the ways to break it.
00:21:02.319 --> 00:21:05.200
And like what are we even doing?
00:21:05.359 --> 00:21:21.200
So that's why I feel like if we move towards this control native place where we watch where it's going, we watch the way it's steering, we are sort of partners of observability in the actions themselves.
00:21:21.519 --> 00:21:27.440
We can start to really look at it from the inside rather than from the after effect, right?
00:21:27.680 --> 00:21:33.839
So we've got you've got an open claw that's running all night to try and execute a task.
00:21:34.559 --> 00:21:36.240
Have we looked at what that looks like?
00:21:36.400 --> 00:21:40.559
Do we know all the places that it went and what it was trying and what it was understanding?
00:21:40.720 --> 00:21:45.279
Do we see what those patterns of thinking are?
00:21:45.599 --> 00:21:47.200
Where when does it turn back?
00:21:47.359 --> 00:21:49.119
When does it follow down a branch?
00:21:49.200 --> 00:21:57.759
When does it prune the branch and try another mechanism to really begin to map those mechanisms of understanding?
00:21:58.000 --> 00:21:59.759
Because we're never going to.
00:22:01.279 --> 00:22:06.799
Cognitively imagine, or even with computation, imagine all the infinite outputs.
00:22:07.279 --> 00:22:12.400
But the question becomes can we see going in certain directions?
00:22:12.559 --> 00:22:18.799
So for example, let's think about mechanistic interpretability.
00:22:18.880 --> 00:22:28.880
It's like the most basic element, mechanistic interpretability in some output that is clearly part of a guardrail that it shouldn't be going.
00:22:29.200 --> 00:22:42.799
It's not just the words that came out, it was what was the direction, which different pieces were activated, what inside the model was being leveraged so that we understand when someone's going in that direction.
00:22:43.200 --> 00:22:43.519
Right.
00:22:43.599 --> 00:22:47.359
One of the things that I haven't done any research on this, but I'm really curious about.
00:22:47.599 --> 00:22:56.160
So we have these algorithms that can take photos of humans and turn them into things that are not okay.
00:22:56.720 --> 00:22:57.039
Right.
00:22:57.920 --> 00:23:05.279
So you know, we don't need to get into the details, don't want to trigger anybody, but we take humans and we put them in situations that are not okay.
00:23:06.720 --> 00:23:17.839
The distance between the original photo and the output, if we think just linear algebra, the difference between the original and the output is a given distance, is a given vector.
00:23:19.039 --> 00:23:23.200
We know what that vector is, we know where that is in vector space, right?
00:23:23.359 --> 00:23:30.640
The I think we completely under-leverage vector distances as a mechanism, right?
00:23:30.799 --> 00:23:46.240
Because we don't necessarily need to look at the output and reclassify it and you know if you know that's the case, you can make it so that that area is either a blank space, it can't be done, or like ways that it gets understood, right?
00:23:46.319 --> 00:24:01.039
So it doesn't have to be the exact guardrail, but how can we use the math to understand where something is going and possibly redirect or block, but take a take a really deep mathematical approach to what's happening.
00:24:01.279 --> 00:24:06.079
I haven't done the research on it, but I definitely think there's there's something there that we can leverage.
00:24:06.400 --> 00:24:08.799
I mean, there's a lot of credibility to it as well, right?
00:24:08.880 --> 00:24:16.559
Like for instance, I remember back in the day when computer vision was like still a huge hot topic, right?
00:24:16.640 --> 00:24:17.839
Um, there was some study.
00:24:18.000 --> 00:24:24.799
I wish I had it just in front of me, but essentially you could um paint over a stop sign, right?
00:24:24.960 --> 00:24:29.599
And a computer could read it as right, go 45, whatever, right?
00:24:29.759 --> 00:24:31.039
It can just go through.
00:24:31.359 --> 00:24:43.200
And that was because it was reading pixels, and it would basically read a couple of pixels, and it would uh it's see that this part was covered or it was a different color than it expected for stop sign.
00:24:43.279 --> 00:24:49.279
I was like, oh, this must not be stop, it must be a different sign that has a pixel here, and the pixel is this color.
00:24:49.599 --> 00:24:51.920
So it makes sense, right?
00:24:52.079 --> 00:25:06.880
Because, or at least in some in this case, the machine was very easily fooled by this literal, like uh, I don't know if this particularly would be considered map, but by its own expectations, right?
00:25:07.039 --> 00:25:11.279
It expected this type of thing and it got something entirely different.
00:25:11.519 --> 00:25:21.279
I think in this sense, and maybe this actually will lead us into a conversation about the what an impl a good implementation of control native AI looks like.
00:25:21.599 --> 00:25:29.279
But when we think, when we go back to your scenario with the um the you know images being generated, right?
00:25:29.440 --> 00:25:33.680
And you have image A, um, let's just say it's an apple, right?
00:25:34.000 --> 00:25:40.000
And image Z is an apple that is still on the tree, right?
00:25:40.400 --> 00:25:50.079
And like how do you know this person is actually generating an apple that is on a tree versus an apple that is somewhere we don't want to talk about, right?
00:25:50.319 --> 00:25:54.240
Like what are the steps that are actually determined between these things?
00:25:54.480 --> 00:26:05.519
I'm wondering, is there a combination of like prompt analysis, maybe understanding like uh like the tokens that are being used for the request and what these typically take?
00:26:05.680 --> 00:26:10.400
Like, I don't know if there, and again, you're coming, I'm a I'm just a security guy, right?
00:26:10.480 --> 00:26:11.759
I'm just making assumptions.
00:26:11.920 --> 00:26:20.240
I'd assume that like a tree in like a certain dimension or uh of an image has a certain expected like value for creation.
00:26:20.319 --> 00:26:23.759
So like it'll take X amount of tokens to create this tree, right?
00:26:24.079 --> 00:26:34.240
So I'm wondering if there's any way to do like some predictive analysis and kind of make an estimate that like, hey, there's way more tokens than we need for this type of request.
00:26:34.480 --> 00:26:38.240
Maybe we like look a little bit deeper into the thought process, right?
00:26:38.400 --> 00:26:55.279
Because there are like um concealed in some models, they have these like uh thinking tags or they use um idea tags, or they, you know, they have these different types of things that going back to what you were speaking about are black boxes and are not really exposed to observability.
00:26:55.440 --> 00:27:00.400
But this is where the model kind of thinks, but again, this is also where the money's made.
00:27:00.640 --> 00:27:12.880
So let's go back to control native AI because this is technically all of the observability that you have in your control and purview that you can use to get an understanding of where your AI is going.
00:27:13.119 --> 00:27:18.480
So, what would a control native approach to this type of solution to generative content look like?
00:27:18.720 --> 00:27:26.880
Yeah, I mean, this is where I wish I was on the research team at Anthropic because I'd be very happy spending my days exactly trying to solve this problem.
00:27:27.440 --> 00:27:32.480
Um I I the the fact is that the math is very deep.
00:27:32.640 --> 00:27:35.440
The number of people that are working on this is very small.
00:27:35.680 --> 00:27:39.359
Um, as I said, they're my favorite people working on it.
00:27:39.519 --> 00:27:47.440
So I and there's a lot of work to be done to get it to the place where it surface enough to be consumable.
00:27:48.160 --> 00:27:57.039
So the question for me is how can we, on the enterprise side, on the non-foundation side, what can we do to do that?
00:27:57.440 --> 00:28:03.680
I mean, I know what my old school, you know, mathematical approach would be.
00:28:03.839 --> 00:28:07.279
I don't know that it's scalable, right?
00:28:07.440 --> 00:28:14.720
So, like I said, I would I would look at anything that I could extract, because most of the time these aren't open models, right?
00:28:14.960 --> 00:28:20.400
I would probably do all my research on an open model where I could actually look at the weights and I could track things.
00:28:20.640 --> 00:28:25.519
Um, but that's kind of silly because that's not what everybody's really using.
00:28:25.599 --> 00:28:27.599
So it's not really that helpful, right?
00:28:27.680 --> 00:28:40.559
So that's part of the challenge we have is we get back to the velocity moment, the velocity of I'm gonna use the hyperscaler model to get what I need.
00:28:40.720 --> 00:28:51.519
And I'm going to hope that their approach to it makes sense and kind of cross your fingers that things don't go wrong.
00:28:51.759 --> 00:28:53.359
I mean, it's not a good answer.
00:28:53.519 --> 00:28:55.359
Like, I want to have the answer for you, Mo.
00:28:55.599 --> 00:28:56.559
I just I don't have it.
00:28:56.880 --> 00:28:58.079
No, I mean, that's fine.
00:28:58.160 --> 00:29:00.799
And I think that's where we also are, right?
00:29:01.119 --> 00:29:08.799
We are in an area where we're super curious about figuring out what is the thing that we should be doing, right?
00:29:09.039 --> 00:29:11.279
We are really trying to understand.
00:29:11.839 --> 00:29:22.240
I don't think any of the conversations that we have necessarily have to end in solutions, um, but they should at least take us into steps towards finding those solutions.
00:29:22.480 --> 00:29:27.200
And on that, I actually think you're working on some pretty interesting stuff.
00:29:27.440 --> 00:29:32.400
Um, now I know that you were kind of working with NIST on a couple of guidelines, right?
00:29:32.640 --> 00:29:35.200
So I would love to hear a little bit more about that.
00:29:35.680 --> 00:29:49.200
So most of my work um in the in that area was around understanding what are some of the mathematical pieces and really speaking to how do we frame risk?
00:29:49.359 --> 00:29:53.839
How do we classify the different um the different elements of it?
00:29:53.920 --> 00:30:00.240
So you take the the classic um you know AI risk framework and how do we put Gen AI onto it?
00:30:00.559 --> 00:30:12.880
And what we came out with was like a pretty decent matrix of understanding, but the again, it's it's all about what people are gonna do with it, right?
00:30:13.039 --> 00:30:19.920
So we have frameworks that we put out and you know guide people in ethical choices.
00:30:20.160 --> 00:30:22.160
Most of it is about application, though.
00:30:22.240 --> 00:30:31.759
It's not about the actual build, it's about um, you know, understanding what what is the decision being made on this output, what's at risk.
00:30:32.000 --> 00:30:44.079
Where I think things are more interesting, I would love to see a foundation model that's just built on, say, children's books or is built just on something that is very bounded, right?
00:30:44.480 --> 00:30:50.160
Because what we know to be true is we can't have something come out of the model that it hasn't seen.
00:30:50.480 --> 00:30:56.880
Well, that's actually not entirely true, but you know, it because the models have gotten so sophisticated.
00:30:56.960 --> 00:30:58.880
I have to I have to alter that a little bit.
00:30:59.119 --> 00:31:12.880
But I would love to see some smaller models built on very closed cases that can hopefully avoid some of that negative noise that can come out of them.
00:31:13.279 --> 00:31:21.440
You know, the smaller models are really good to look at in terms of you know, how do you scale certain solutions and certain controls, right?
00:31:21.599 --> 00:31:25.359
And you can see how like you could see the impacts a little bit easier.
00:31:25.519 --> 00:31:32.880
Um at the same time, the small, I will say the smaller models are getting way bigger um just by the day.
00:31:32.960 --> 00:31:36.480
Like the things that are considered small are actually quite large.
00:31:36.720 --> 00:31:40.960
Um and it it's getting it's getting it's getting interesting.
00:31:41.119 --> 00:31:43.119
It's getting interesting all across it.
00:31:43.440 --> 00:31:48.160
So guess going back all the way to the beginning, right?
00:31:48.319 --> 00:31:55.279
Where we were trying, where we were in the room with the CTO and the chief AI officer, and they want to get their things done, right?
00:31:55.839 --> 00:32:10.400
What are kind of the ways we enable these folks to feel comfortable about the risk that they're taking while also maybe providing their teams with some level of observability and transparency and traceability in their AI systems?
00:32:10.640 --> 00:32:27.920
What are like the kind of the high-level, if you had to say like maybe three things that you would say, these are the three decisions you should make and the three things that you should have implemented in your product or your um your AI pipeline that you're using in your organization?
00:32:28.160 --> 00:32:29.759
What would those kind of look like?
00:32:30.000 --> 00:32:34.000
And how would you apply like some of those control native AI solutions here?
00:32:34.319 --> 00:32:36.400
The first thing is observability.
00:32:36.720 --> 00:32:43.039
You know, it's shocking to me that observability of our systems is not table stakes.
00:32:43.279 --> 00:32:44.160
It's just not.
00:32:44.400 --> 00:32:48.000
Partially because So what does that look like?
00:32:48.960 --> 00:32:54.079
I I mean, honestly, if you're if for if it's me and I'm buying a product, right?
00:32:54.240 --> 00:32:56.640
Most people aren't necessarily building your own agents.
00:32:56.880 --> 00:33:04.079
If I'm getting something, all of the OTE has to be built into the system and it has to be easily accessible.
00:33:04.720 --> 00:33:04.960
Right?
00:33:05.279 --> 00:33:06.480
It's a non-negotiable.
00:33:06.559 --> 00:33:09.119
You have to have the you have to have the OTEL traces, right?
00:33:10.400 --> 00:33:10.720
A.
00:33:11.200 --> 00:33:24.880
B, you need to have someone on staff who can understand and look at that observability at a system level and be able to discern that impact.
00:33:25.599 --> 00:33:30.480
And that person needs to be able to communicate that to the relevant business.
00:33:31.359 --> 00:33:31.680
Right.
00:33:31.839 --> 00:33:38.319
So you need to make sure that you've got your internal observability with anything you build or anything that you buy.
00:33:38.559 --> 00:33:46.480
You need to have someone internally who can leverage those at an intelligence level, not just a log level.
00:33:46.880 --> 00:33:55.839
And then you need to make sure that they have an understanding of what's at stake for the business for each of the areas that this is hitting.
00:33:56.000 --> 00:34:03.279
And that communication, that person needs to be able to educate, just like we did when we had our basic linear regression models.
00:34:03.440 --> 00:34:13.199
Our job was always to provide a model to explain the risk so that the business unit could make a decision that they feel confident about.
00:34:13.599 --> 00:34:20.480
We need to have that now in the same way that we did before, but we need to have it at this agenc level.
00:34:20.800 --> 00:34:25.440
So we talked a little bit about traceability, but like let's dig a little bit deeper into that.
00:34:25.519 --> 00:34:27.440
So, what does traceability mean here, right?
00:34:27.519 --> 00:34:33.039
Is it actually figuring out how a decision is made, or is it understanding everything?
00:34:33.280 --> 00:34:43.199
You know, it can get conflated with observability because I'm even talking about it out loud, and I'm like, well, observability should be about being able to see how you're getting to something.
00:34:43.360 --> 00:34:45.280
Traceability, what does that really look like?
00:34:45.440 --> 00:34:50.960
Is it the actions that the AI or an AI agent is actually taking across the environment?
00:34:51.199 --> 00:35:03.039
Is it more about the things that are happening around the agent that can be now you can look back at and attribute them as a root cause for an action that an agent is taking?
00:35:03.199 --> 00:35:06.719
So, like what is traceability uh here and what does that look like?
00:35:06.880 --> 00:35:11.920
Uh the way I view it, I view I view traceability as the unit, right?
00:35:12.079 --> 00:35:26.719
The actual trace, the ability to follow a decision or an outcome from an AI back through all of the agents, trace to all of the data sources, to everything that went into it, all the way back to the source of that data, right?
00:35:27.039 --> 00:35:31.679
That's the traceability of the actual decision and the branching that led to it.
00:35:32.000 --> 00:35:35.599
Observability for me is the one layer up.
00:35:35.760 --> 00:35:38.159
We see all the traces, we see all the movement.
00:35:38.400 --> 00:35:45.039
Observability to me is saying, okay, these are all the actions that are being taken, but what does it mean?
00:35:45.280 --> 00:35:48.159
What does that understanding?
00:35:48.320 --> 00:35:53.599
Because even if we look back at traditional ML ops, drift isn't one output, right?
00:35:53.760 --> 00:35:55.920
All of a sudden we have one thing that's a little bit different.
00:35:56.079 --> 00:35:59.599
That doesn't mean the entire model has drifted or the environment has drifted.
00:35:59.760 --> 00:36:07.840
It's when we see a certain tipping point, a threshold, you say, oh, now we can see this is considered drift.
00:36:08.079 --> 00:36:11.039
And so for me, that's that's where that is.
00:36:11.280 --> 00:36:21.760
I I think that it's really important for us because of the nature of agents and these interactions, we cannot look at it at log level.
00:36:22.000 --> 00:36:25.280
We cannot even look at it in two dimensions.
00:36:25.920 --> 00:36:40.079
We have to take time elements into account when we're looking at observability, because velocity shift of importance is going to be one of those big indicators that we want to start looking at.
00:36:40.559 --> 00:36:51.840
Because when we're looking at the system, what's anomalous is not really the gold standard for me when I'm looking at it.
00:36:52.079 --> 00:36:54.480
What I want to look at is what's influential.
00:36:55.679 --> 00:36:56.000
Right.
00:36:56.239 --> 00:37:04.719
I want to look at what is the I don't know, the tool call or the data source that is really influencing many things.
00:37:05.039 --> 00:37:13.199
What is like a big shift, like example, you looked at how quickly someone was responding in your WhatsApp chat, right?
00:37:13.679 --> 00:37:17.519
It was the it was the velocity of the change that mattered.
00:37:18.000 --> 00:37:18.320
Right.
00:37:19.119 --> 00:37:22.079
Nothing he was saying, the bot was saying was anomalous.
00:37:22.320 --> 00:37:28.320
If you had had a traditional anomaly detection, everything sounded like him, it was his verbiage.
00:37:28.480 --> 00:37:34.079
You wouldn't have necessarily said, oh, somebody hacked in and it's his mom responding in our WhatsApp or something, right?
00:37:34.239 --> 00:37:34.559
Right.
00:37:34.719 --> 00:37:40.000
But it's the velocity of that change and the velocity of and the level of influence.
00:37:40.320 --> 00:37:46.719
So now what you saw is you saw a very quick change that was very influential.
00:37:47.039 --> 00:37:53.760
And those two things together, I think, is what we need to start understanding when we look at observability.
00:37:54.079 --> 00:38:07.679
Because, you know, I was talking to one potential client and they said, well, you know, you'll come in and you'll look at our systems, and you know, from there you'll be able to determine if something is anomalous.
00:38:08.159 --> 00:38:21.599
And I was like, Well, I don't see it that way because if I come in today and there's already been an infiltration or something's amiss, I am assuming when we show up day one, everything's okay.
00:38:22.000 --> 00:38:22.960
But we don't know that.
00:38:23.679 --> 00:38:24.000
Right.
00:38:24.239 --> 00:38:34.639
So by looking at influence, by looking at impact, it allows us as humans to figure out which parts of the system should I even be looking at.
00:38:35.599 --> 00:38:36.000
Right.
00:38:36.239 --> 00:38:44.239
We have, yes, we have AI to help us with all of those things, but without having that metric and that understanding, we're never going to necessarily find it.
00:38:44.480 --> 00:38:56.400
So some of that shift in understanding like what should we be thinking about from risk, that's one of the other pieces I think is really fascinating and an open conversation right now.
00:38:56.719 --> 00:38:58.480
It's like so much to digest.
00:38:58.639 --> 00:39:02.800
And you you say this like really good word, which is influence, right?
00:39:03.280 --> 00:39:08.639
And it's a new, I don't want to call it a new concept, even though it feels new, right?
00:39:08.800 --> 00:39:18.400
Like I think for as long as I've been in security, which is not very long in in the scale of how long computers have been around.
00:39:18.639 --> 00:39:26.559
I've always been used to um, again, computers being very much call and response.
00:39:26.719 --> 00:39:30.800
And I do one thing and I get the exact output that I expect, right?
00:39:31.199 --> 00:39:33.760
And it's just how we worked, right?
00:39:33.840 --> 00:39:40.400
As uh even as pen testers, you know exactly how you're gonna get the response you want.
00:39:40.559 --> 00:39:44.880
All you need to do is continue to ask the question in different ways.
00:39:45.119 --> 00:39:49.119
Um, you know, this is essentially how you test a service.
00:39:49.360 --> 00:39:53.679
Um and you you go and you now we have AI, right?
00:39:53.840 --> 00:39:56.159
And obviously we still have the same vector, right?
00:39:56.239 --> 00:39:59.119
With prompt injection and and all of these things.
00:39:59.280 --> 00:40:05.760
Um we do just go and you do call response and you figure it out, right?
00:40:06.000 --> 00:40:12.559
But then there's now this like other factor of influence, especially when you look at agenc, right?
00:40:13.119 --> 00:40:20.559
And where we are at now, where agentic um relationships are now just agents are influencing each other, right?
00:40:20.719 --> 00:40:33.119
You take it back to your space example, and now it's like, okay, well, this agent is doing this thing, and you know, it's basically peer pressure where you're getting pulled in one direction, you're getting pulled in another.
00:40:33.280 --> 00:40:34.719
And it's so difficult.
00:40:34.880 --> 00:40:48.079
Like when you think about environments that have a handful of agents, and then you think about that even larger when you've got an environment with hundreds of thousands of agents like the one we're currently living in, right?
00:40:48.800 --> 00:40:52.400
How are all these different things just impacting each other?
00:40:52.480 --> 00:41:00.159
And it's like we're in such an interesting territory right now when it comes to evaluating risk on the scale.
00:41:00.400 --> 00:41:11.519
And I think um, especially after the open AI um acquisition, or I don't know, yeah, I would call it an acquisition, um, of open claw, right?
00:41:11.599 --> 00:41:12.880
Now they're bringing it in.
00:41:13.119 --> 00:41:24.320
We're about to see, in my opinion, a very much quick movement of this technology into the hands of the everyday consumer really fast, right?
00:41:24.480 --> 00:41:30.159
You won't need a Mac Mini to run uh um, I call them multis, but I don't think that's what they're called.
00:41:30.239 --> 00:41:33.440
Um, but our little claw friends anymore, right?
00:41:33.599 --> 00:41:39.840
Now you can actually run your AI companion like likely from uh OpenAI's platform.
00:41:40.000 --> 00:41:43.280
And you know you can give it the same access and everything like that.
00:41:43.519 --> 00:41:47.280
I think we're like, man, it's like it's going real fast.
00:41:47.440 --> 00:41:52.480
Uh, but it's a really exciting time to be kind of tackling these problems.
00:41:52.800 --> 00:41:55.440
So moving away from agents for a second, right?
00:41:55.519 --> 00:41:59.039
Or maybe actually moving into the literal hands of agents.
00:41:59.679 --> 00:42:04.639
I know that you were literally just here to do a robotics competition.
00:42:04.800 --> 00:42:08.800
And robotics has kind of been one of those things that you've been involved with for a while.
00:42:09.039 --> 00:42:10.400
So, what was that kind of like?
00:42:10.480 --> 00:42:13.679
And then also like, what are you actually doing with robots and why?
00:42:13.840 --> 00:42:15.679
And when are you getting your neo?
00:42:16.639 --> 00:42:17.679
Oh, yeah, I know.
00:42:17.920 --> 00:42:26.639
Um, so I've been interested in robotics for the last couple of years just because I've got an interest in sovereign systems.
00:42:26.800 --> 00:42:30.639
I have an interest in data privacy and data protection.
00:42:31.119 --> 00:42:44.800
And the the meta glasses, as someone who feels very strongly about data sovereignty, were kind of hard for me because you couldn't opt out of your data being used to train.
00:42:44.960 --> 00:42:47.599
So I didn't love that as a policy.
00:42:47.920 --> 00:42:52.239
And as we know, everything's moving very quickly.
00:42:52.400 --> 00:42:54.880
Robotics is also moving very quickly.
00:42:55.119 --> 00:43:03.440
And we've had robot vacuums in our homes for years, but the humanoid robot is coming and it's coming very quickly.
00:43:03.679 --> 00:43:08.639
And so, as someone who's always looking to the future, I'm always thinking, okay, what does that mean?
00:43:08.800 --> 00:43:10.320
What do we need to prepare for?
00:43:10.480 --> 00:43:13.360
What infrastructure do we need to be thinking about?
00:43:13.440 --> 00:43:16.159
How can we be thoughtful about what's coming?
00:43:16.559 --> 00:43:23.119
And so I got really interested and had been talking to friends about inference at the edge and data and these things.
00:43:23.199 --> 00:43:28.800
But then I realized I was like, well, I gotta spend some time with some hardware in order to actually have an opinion.
00:43:28.960 --> 00:43:30.159
So what does that look like?
00:43:30.320 --> 00:43:35.599
So um a friend of mine, uh Andrea Turcou, who works at H2O, came out from France.
00:43:35.679 --> 00:43:39.119
I said, I'm going to go do this physical hackathon, come join me.
00:43:39.199 --> 00:43:40.320
So she flew out from France.
00:43:40.480 --> 00:43:46.079
I flew from the East Coast with this really wonderful fellow, Sidir Dadi, who was sitting at the table next to us.
00:43:46.320 --> 00:43:50.880
The three of us got together and we built a single claw arm, an SO101.
00:43:50.960 --> 00:43:56.480
It's got a camera on it, and it opens a book and it turns the pages and it reads aloud.
00:43:56.800 --> 00:43:59.840
Now, it doesn't seem like it's, you know, or it's shattering.
00:44:00.239 --> 00:44:00.719
Robotics.
00:44:00.960 --> 00:44:16.239
But as a mother of a son with autism who has a reading disability, who was served really well from audiobooks and different things like that, it just really got me thinking about what is now possible as a with this as an assistive device, right?
00:44:16.400 --> 00:44:33.519
So whether it's reading, um, whether it's, you know, learning to read in a foreign language or extra support for special education, or, you know, you're in a part of the world that doesn't have as much access to technology or education, you know, what could be possible?
00:44:33.840 --> 00:44:35.599
So it got me thinking about that.
00:44:35.920 --> 00:44:50.719
And what I really want is I'm the highest compliment I can give any builder, any founder, I mean, really any human, or probably AI as well, is is it thoughtful?
00:44:51.199 --> 00:44:51.519
Right?
00:44:51.679 --> 00:44:53.360
Is what I have built thoughtful?
00:44:53.519 --> 00:44:55.599
Is it thoughtful as far as risk?
00:44:55.760 --> 00:44:57.840
Is it thoughtful about impact?
00:44:58.000 --> 00:45:02.880
Have we have we figured out what it can do for good or evil?
00:45:03.119 --> 00:45:08.639
Um, and so I started thinking about, well, we've got to find a way to fund it, right?
00:45:08.719 --> 00:45:11.599
Because, you know, nonprofits are great, but it's a rough time.
00:45:11.760 --> 00:45:12.960
It's a rough time right now, right?
00:45:13.440 --> 00:45:16.320
So then I was thinking about robotics data sets.
00:45:16.480 --> 00:45:24.239
And what's really interesting to me on robotics data sets is that historically everything that we had built AI for was on a screen.
00:45:24.320 --> 00:45:27.119
So whether it's text or video, everything's two-dimensional.
00:45:27.920 --> 00:45:35.119
But we have to take into account force and we have to take into account weight and just physics in general.
00:45:36.159 --> 00:45:43.440
And so as we're looking at the data sets, um, when I was at the lab lab AI hackathon last weekend, people were doing really cool things.
00:45:43.599 --> 00:45:48.239
We were using a lot of um Isaac Sem and DeepMind.
00:45:48.400 --> 00:45:54.480
And a lot of what we were looking at was ways that people could use VR or iPhones to create these data sets.
00:45:54.639 --> 00:45:58.239
But for me, I'm like, but there's no there's no physics in it.
00:45:58.880 --> 00:46:04.000
But if we're in the simulation, we have all the physics, but no real world noise.
00:46:04.320 --> 00:46:10.800
So there's a gap in robotics training sets, which is real world collection.
00:46:10.880 --> 00:46:12.880
And historically, it's very expensive, right?
00:46:12.960 --> 00:46:15.760
It's very easy to do things in the lab, very hard to do things.
00:46:15.920 --> 00:46:19.760
So then, you know, a constraint is always just a puzzle to solve.
00:46:20.000 --> 00:46:24.639
So the the question became so we want to put this assistive technology out there.
00:46:24.880 --> 00:46:27.360
We know that there are these constraints in robotics.
00:46:27.519 --> 00:46:34.800
What would happen if we leverage these tools to really understand paper manipulation for robotics?
00:46:35.280 --> 00:46:48.719
So, our goal right now in the project that we're working on is our big lofty goal is to have 1 million SO101s across the globe serving 240 million children with learning disabilities.
00:46:48.880 --> 00:46:49.840
It's a big goal.
00:46:50.079 --> 00:46:50.960
I know we'll get there.
00:46:51.039 --> 00:46:52.400
I don't know how long it will take.
00:46:52.559 --> 00:47:02.000
And at the same time, take just the telemetry data, just the movement of the arm, just the understanding of, you know, we'll move into other hardware eventually, right?
00:47:02.239 --> 00:47:05.440
But the understanding of what it means to manipulate paper.
00:47:05.679 --> 00:47:13.920
So my personal goal is to participate in building robots that can perform origami.
00:47:14.239 --> 00:47:37.519
And all of this long story to say, because when the humanoid robot comes into my home and comes into your home, I want it to be able to be thoughtful and have that delicacy and that understanding of something very fine so that it can make good, intelligent movements to serve whatever it is that you want it to do.
00:47:37.840 --> 00:47:42.800
That is a lot, but sorry, um, no, no, no.
00:47:42.880 --> 00:47:44.000
I'm not saying no, no, no.
00:47:44.079 --> 00:47:45.920
I'm not saying what you're saying is a lot.
00:47:46.159 --> 00:47:49.280
What I'm actually saying is that paper is a lot, right?
00:47:49.599 --> 00:47:53.599
Um, paper in general is a very, it's a very delicate thing.
00:47:53.840 --> 00:48:08.239
And um, you know, I've been really obsessed with world models, not because of the world building aspect of them, but because um, again, from a security angle, I'm more interested in the physics piece, right?
00:48:08.639 --> 00:48:13.760
Not necessarily to think like, oh, um, you know, ball bumps into wall.
00:48:13.920 --> 00:48:16.639
Um, if ball is rubber, it bounces back.
00:48:16.880 --> 00:48:22.639
It's more like what kind of like chemicals can we actually like simulate there, right?
00:48:22.960 --> 00:48:27.599
Um, can we start to simulate chemical reactions against different substances?
00:48:27.920 --> 00:48:32.320
Can we go and like recreate the physical properties of things accurately, right?
00:48:32.639 --> 00:48:43.519
And if so, like what types of physical phenomena can we actually um again recreate in a fully simulated AI-driven environment?
00:48:43.760 --> 00:48:48.320
For example, what happens when paper touches water, right?
00:48:48.880 --> 00:48:52.480
Like, I don't think a robot will understand that, right?
00:48:52.559 --> 00:49:04.239
It'll just dip it in and then you know the paper is either super heavy paper and it can deal with it for a little while, or it's super soft paper and tears apart, or it's paper tissue and it just doesn't even come back, right?
00:49:04.559 --> 00:49:10.400
But like paper is so is a lot, is what I was I didn't say it right, but paper is a lot.
00:49:10.639 --> 00:49:20.400
And for me, it's like how do we like LLMs are not enough to actually get this information into a robot?
00:49:20.800 --> 00:49:37.440
I would say we need to go further and actually figure out how do we how do we up level the comprehension or the portability of a world model that has full understanding of objects in a world and how they interact with other objects in a world.
00:49:37.760 --> 00:49:41.440
And how do we come full circle with that kind of concept?
00:49:41.599 --> 00:49:51.760
Get it into robotics in a way where like you can have this robot turn the piece of paper and not necessarily think about it in um in measurements of like force, right?
00:49:51.840 --> 00:49:56.800
Like how hard do you need to like push on this or um you know, like these things, right?
00:49:56.960 --> 00:50:03.840
And instead have an understanding of like, okay, well, this is how thick it is, and this is what my fingers look like, right?
00:50:04.079 --> 00:50:06.239
Not, you know, right now it feels binary.
00:50:06.320 --> 00:50:07.760
It's like grab this, right?
00:50:07.840 --> 00:50:13.360
And grab it with X amount of force, but it doesn't have an understanding of like why, right?
00:50:13.679 --> 00:50:15.360
So that why is so important.
00:50:15.440 --> 00:50:17.760
It's the same reason why my cat, for example.
00:50:17.840 --> 00:50:20.559
Um, my cat did not grow up around other cats.
00:50:20.800 --> 00:50:26.480
So one of the biggest behaviors that cats have is when they when they scratch or bite, it's actually a playful thing.
00:50:26.719 --> 00:50:30.400
But if they don't have any indicators of like, oh, this is hurting you, guess what?
00:50:30.559 --> 00:50:36.400
Your cat is going to scratch you and you're gonna bleed because it doesn't understand, oh, this is actually causing you pain.
00:50:36.639 --> 00:50:39.280
But again, the same thing with machines, right?
00:50:39.440 --> 00:50:48.079
They don't really understand what the for what the output or the effect of the output of the force is on whatever object that they're interacting with.
00:50:48.159 --> 00:50:54.079
And you really have to be careful when you control or when you go and you teach it these things.
00:50:54.239 --> 00:51:06.320
But we're teaching it with words, we're teaching it with numbers, we're not teaching it with an experience, which is where I think world models kind of bridge that gap, or at least in theory, in my opinion.
00:51:06.480 --> 00:51:10.960
And again, not a scientist or a researcher, but I love reading that stuff.
00:51:11.039 --> 00:51:13.679
So that's kind of where where it excites me.
00:51:13.920 --> 00:51:17.599
So you've been in this space for a very long time, was the well, actually, you haven't.
00:51:17.920 --> 00:51:24.800
You have a really interesting background and you've jumped careers into this particular governance risk and compliance space.
00:51:24.960 --> 00:51:29.519
And I wish we had a whole other hour because I would have loved to dig into your background more.
00:51:29.840 --> 00:51:38.159
But um, you've been in governance risk and compliance for a while, and we are most certainly in a race with AI.
00:51:38.320 --> 00:51:40.239
But are we in the race that you think we're in?
00:51:40.320 --> 00:51:46.320
I think we're in a race between our ability to build agentic AI and our ability to govern it.
00:51:47.119 --> 00:51:53.679
Out of these two sides, our ability to build or our ability to govern, who's winning.
00:51:54.000 --> 00:51:55.039
Building is way ahead.
00:51:55.280 --> 00:51:56.159
Building is way ahead.
00:51:56.320 --> 00:52:02.400
I mean, partially for the reasons that I said that we have a very good idea of data governance.
00:52:02.559 --> 00:52:04.880
We know you can access this data, you can't.
00:52:05.039 --> 00:52:06.159
We're really good at that.
00:52:06.400 --> 00:52:10.000
We're really good at you can use this tool, you can't.
00:52:10.320 --> 00:52:17.840
We are not good at understanding the mechanisms of what these tools are actually doing.
00:52:18.079 --> 00:52:20.719
And we can't govern what we don't understand.
00:52:21.039 --> 00:52:21.440
Yep.
00:52:21.679 --> 00:52:26.159
And I think that is kind of like everything that we've been talking about, right?
00:52:26.480 --> 00:52:30.320
It's the builders are have a huge advantage in this space.
00:52:30.480 --> 00:52:37.599
And as as govern governing folks or people in GRC, um, it's all catch up.
00:52:37.760 --> 00:52:38.639
It's all catch up.
00:52:38.719 --> 00:52:44.480
And honestly, it's the exact same place red teamers have been for uh the entirety of the red teaming practice, right?
00:52:44.719 --> 00:52:46.800
So governance net is real, Mo.
00:52:47.119 --> 00:52:48.559
Governance debt is real.
00:52:48.800 --> 00:52:51.599
Yeah, I know, and that's another thing that we could have talked about again.
00:52:51.760 --> 00:52:52.800
Another hour.
00:52:52.960 --> 00:52:54.960
Like another time, another time.
00:52:55.199 --> 00:52:57.679
So thank you so much for joining us today.
00:52:57.840 --> 00:52:59.920
And where can the people find you?
00:53:00.000 --> 00:53:01.440
What are you working on?
00:53:01.679 --> 00:53:03.119
What's coming up for you?
00:53:03.360 --> 00:53:04.960
I'm always working on a lot of things.
00:53:05.119 --> 00:53:07.920
Uh, the best way to find me, honestly, is always on LinkedIn.
00:53:08.079 --> 00:53:08.719
I'm always there.
00:53:08.800 --> 00:53:10.880
I try to respond as much as I can.
00:53:11.119 --> 00:53:18.000
Um, I've got, you know, an algorithm paper coming out in the next hopefully two to three weeks.
00:53:18.159 --> 00:53:24.960
Um, some retrieval stuff, working on some permissioning ideas for agent-to-human understanding.
00:53:25.039 --> 00:53:28.079
And LinkedIn's always the best place to find me and see what I'm up to.
00:53:28.320 --> 00:53:31.039
Well, thanks so much again for stopping by.
00:53:31.119 --> 00:53:32.079
And we finally got it.
00:53:32.159 --> 00:53:33.039
We finally did a podcast.
00:53:33.599 --> 00:53:35.440
Thank you so much for having me again.
00:53:35.599 --> 00:53:40.480
I as soon as I met you, there was an instant intellectual chemistry.
00:53:40.559 --> 00:53:42.000
So I was super happy to be here.
00:53:42.079 --> 00:53:43.280
So thank you for having me.
00:53:43.519 --> 00:53:48.159
I'm always happy to host Curious Find as we do here on Curiouser and Curiouser.
00:53:48.320 --> 00:53:52.320
If this episode helped cut through the noise, like or subscribe so you don't miss what's next.
00:53:52.480 --> 00:53:53.760
Thanks for spending time with us.
00:53:53.920 --> 00:53:56.480
Until next time, stay curious.