Speaker:
00:00:00
- [Nicky] Hello, and welcome to
Speaker:
00:00:01
the Red Olive Fibonacci Podcast.
Speaker:
00:00:03
The podcast all about the brilliant world of data
Speaker:
00:00:06
covering future trends and topical tech.
Speaker:
00:00:08
We'll be joined by experts in the data sphere
Speaker:
00:00:10
to share their opinions and advice.
Speaker:
00:00:12
I'm your host, Nicky Rudd.
Speaker:
00:00:14
Today's episode is the first of a two-part interview
Speaker:
00:00:17
recorded during lockdown with Robin Hayden.
Speaker:
00:00:19
Robin has worked for over 20 years in data
Speaker:
00:00:22
and has a wealth of experience in machine learning,
Speaker:
00:00:24
AI, large-scale engineering, analytics,
Speaker:
00:00:27
and product development.
Speaker:
00:00:28
In various chief data officer roles,
Speaker:
00:00:30
he's delivered over 100 machine learning models
Speaker:
00:00:32
into production,
Speaker:
00:00:33
been responsible for tens of millions in additional revenue
Speaker:
00:00:36
and hundreds of millions in spend.
Speaker:
00:00:39
The first part of the conversation covers a lot of ground
Speaker:
00:00:41
from crypto and AI,
Speaker:
00:00:42
data ethics and Skynet,
Speaker:
00:00:43
to the best use of the cloud
Speaker:
00:00:45
for businesses looking to update,
Speaker:
00:00:46
and how to run a data migration project.
Speaker:
00:00:49
So let's go.
Speaker:
00:00:50
(introductory rock music)
Speaker:
00:01:02
- [Nicky] I thought we could start off
Speaker:
00:01:04
if you could just tell me a little bit about
Speaker:
00:01:05
how you got into the world of data.
Speaker:
00:01:08
- [Robin] It was a bit of a long journey.
Speaker:
00:01:10
I actually, I was quite into this sort of AI space
Speaker:
00:01:13
from very early on,
Speaker:
00:01:14
but you couldn't really get a job in that
Speaker:
00:01:15
unless you're a researcher and or something in that space.
Speaker:
00:01:18
And you know, that wasn't an option that was open to me
Speaker:
00:01:21
at the time.
Speaker:
00:01:21
So I went off and did
Speaker:
00:01:22
a lot of sort of early internet type work.
Speaker:
00:01:26
And it started to become apparent
Speaker:
00:01:27
that we were able to do really smart things.
Speaker:
00:01:30
It was basically all based off of big data
Speaker:
00:01:32
and it was starting to become apparent
Speaker:
00:01:34
that we now had enough data
Speaker:
00:01:36
that you could go beyond doing
Speaker:
00:01:39
a sort of ad hoc analysis.
Speaker:
00:01:41
You could really start to extract decisions.
Speaker:
00:01:44
And it was obvious that at some point
Speaker:
00:01:46
it was going to change the way we did all of computing.
Speaker:
00:01:48
I sort of moved my career to focus primarily on data
Speaker:
00:01:52
from that point it's 80% data engineering
Speaker:
00:01:55
and 20% intelligence.
Speaker:
00:01:57
But it is the 20% that's really fascinating.
Speaker:
00:02:00
- [Nicky] It's a really exciting industry
Speaker:
00:02:01
to be involved with,
Speaker:
00:02:02
even in the work that I've been doing
Speaker:
00:02:04
over the last couple of years.
Speaker:
00:02:05
The step changes really just have jumped.
Speaker:
00:02:08
What do you think of the most exciting changes of foot?
Speaker:
00:02:11
How do you see things developing over the next
Speaker:
00:02:13
two to three years.
Speaker:
00:02:15
- [Robin] In the space, I would say definitely.
Speaker:
00:02:17
I think the proliferation of edge computing
Speaker:
00:02:21
and the ability to push intelligence
Speaker:
00:02:23
all the way out to the edge
Speaker:
00:02:24
is something that is, some people know about it,
Speaker:
00:02:26
but it isn't as widely understood in this space.
Speaker:
00:02:29
People think of big data as big and expensive
Speaker:
00:02:31
and this sort of planet killer in many cases
Speaker:
00:02:34
because these big models,
Speaker:
00:02:35
and they are some big models
Speaker:
00:02:36
and they are expensive to build like GPT-3,
Speaker:
00:02:38
or Google's competitor to that recently,
Speaker:
00:02:40
which was,
Speaker:
00:02:41
I think GPT-3 was several billion,
Speaker:
00:02:43
a few hundred billion parameters or something.
Speaker:
00:02:45
And Google's equivalent, not that long ago,
Speaker:
00:02:48
was over a trillion parameters.
Speaker:
00:02:49
And those things really do take
Speaker:
00:02:50
enormous amounts of power and computing and stuff to build.
Speaker:
00:02:53
But the flip side is that the computing,
Speaker:
00:02:56
it takes orders of magnitude more energy
Speaker:
00:03:00
to process something,
Speaker:
00:03:01
to push something all the way to the cloud and back.
Speaker:
00:03:03
It actually, even just pushing data
Speaker:
00:03:05
from the processor into RAM on your machine
Speaker:
00:03:07
versus processor registers,
Speaker:
00:03:09
is it very expensive?
Speaker:
00:03:10
Like that distance,
Speaker:
00:03:11
that extra distance costs a lot in computing terms.
Speaker:
00:03:14
The idea that things like neural networks,
Speaker:
00:03:15
that basically they're doing matrix operations
Speaker:
00:03:18
over and over again,
Speaker:
00:03:19
that's basically taking the same data
Speaker:
00:03:21
and recycling it,
Speaker:
00:03:21
and reprocessing it over and over again in the processor.
Speaker:
00:03:24
So in many ways it's actually very power efficient
Speaker:
00:03:27
and this idea that once you have these in these models
Speaker:
00:03:29
and they can do inference
Speaker:
00:03:30
out at the edge on some very low power devices.
Speaker:
00:03:32
I mean, there's this real, you know,
Speaker:
00:03:37
as they're starting to
Speaker:
00:03:38
become embedded in all sorts of things,
Speaker:
00:03:39
cars obviously being the most prominent of those,
Speaker:
00:03:41
but in almost anything that's out in our environment
Speaker:
00:03:43
and all sorts of sensors and things,
Speaker:
00:03:45
you're getting really low power chips and things
Speaker:
00:03:47
that are optimised for this.
Speaker:
00:03:48
And I think our whole world
Speaker:
00:03:49
may well become much, much more intelligent.
Speaker:
00:03:52
And you combine that with things that are emerging,
Speaker:
00:03:54
like the crypto space,
Speaker:
00:03:55
which if you go beyond the currencies
Speaker:
00:03:57
is really just people exploring
Speaker:
00:03:58
how far can we take this distributed computing,
Speaker:
00:04:01
this sort of collaborative computing,
Speaker:
00:04:02
and perhaps that we've had a large amount of centralization,
Speaker:
00:04:05
which I suppose is a bit counter to
Speaker:
00:04:07
what was the original sort of spirit of the internet
Speaker:
00:04:09
when it was very open for everyone.
Speaker:
00:04:11
And now we've got this consolidation
Speaker:
00:04:12
around a few big powers.
Speaker:
00:04:14
And there's a lot of political backlash to that.
Speaker:
00:04:15
And all the rest at the same time,
Speaker:
00:04:17
you've got things like crypto space developing
Speaker:
00:04:18
and despite the sort of hype,
Speaker:
00:04:20
and I'm sure a lot of people may still get hurt
Speaker:
00:04:22
in the investment world there.
Speaker:
00:04:24
But I think the idea of being able to encrypt
Speaker:
00:04:27
and process things very widely
Speaker:
00:04:29
and developing those sorts of,
Speaker:
00:04:32
they're almost like democratic networks
Speaker:
00:04:33
that allow people to share processing of data and stuff,
Speaker:
00:04:36
and the file storage and stuff
Speaker:
00:04:38
in ways that it couldn't do before.
Speaker:
00:04:40
And you leverage that
Speaker:
00:04:42
with the sort of ubiquitous intelligence
Speaker:
00:04:44
and the ability to run intelligent algorithms on everything.
Speaker:
00:04:47
It's hard to actually rarely,
Speaker:
00:04:49
to even fully imagine what that might mean.
Speaker:
00:04:51
Honestly, if you think of an entire world
Speaker:
00:04:53
where everything is more intelligent,
Speaker:
00:04:54
where, you know,
Speaker:
00:04:55
also where there's not necessarily
Speaker:
00:04:57
one or two big controllers of that world.
Speaker:
00:04:59
That could be a very radically different world.
Speaker:
00:05:02
Then, of course, if you zoom forward even further
Speaker:
00:05:05
and you look at things like quantum computing,
Speaker:
00:05:06
which doesn't solve everything,
Speaker:
00:05:07
that particular class of problems,
Speaker:
00:05:09
but certainly then there are some very big problems
Speaker:
00:05:11
and things that were much, much harder to do.
Speaker:
00:05:12
And so if you zoom forward another decade or so,
Speaker:
00:05:15
maybe two,
Speaker:
00:05:16
I don't know however long it takes to do that.
Speaker:
00:05:18
You know, then we just have this
Speaker:
00:05:19
another sort of order of magnitude
Speaker:
00:05:21
for a certain class of problems
Speaker:
00:05:22
and other sort of order of magnitude improvement
Speaker:
00:05:24
in sophistication that we're able to achieve.
Speaker:
00:05:26
So it just feels like this idea that,
Speaker:
00:05:28
I don't really obsess much about generalised intelligence,
Speaker:
00:05:33
and those people who think about
Speaker:
00:05:34
Skynet's going to take over the world.
Speaker:
00:05:35
I think it's impossible to say how far we are from something
Speaker:
00:05:39
when we don't know, we don't have a map to getting there,
Speaker:
00:05:42
if you know what I mean.
Speaker:
00:05:43
We could be a decade away.
Speaker:
00:05:46
It could be a million years away.
Speaker:
00:05:47
Nobody really actually knows.
Speaker:
00:05:48
So I don't really spend much time thinking about that.
Speaker:
00:05:50
But I do think that it's obvious
Speaker:
00:05:52
that over the next decade even,
Speaker:
00:05:54
all of our products are going to be much more intelligent.
Speaker:
00:05:56
And not just the products we're used to using
Speaker:
00:05:58
on the internet.
Speaker:
00:05:59
All of our products.
Speaker:
00:06:00
The things that are in our house,
Speaker:
00:06:01
and in our cars,
Speaker:
00:06:02
and in our gardens,
Speaker:
00:06:03
and that sort of thing.
Speaker:
00:06:04
And that's really exciting.
Speaker:
00:06:07
- [Nicky] What would you say
Speaker:
00:06:08
when it comes to that kind of intelligence,
Speaker:
00:06:10
and what's your take on
Speaker:
00:06:11
the kind of the ethics around it,
Speaker:
00:06:13
and that kind of ethical data question?
Speaker:
00:06:15
- [Robin] Technology is neutral,
Speaker:
00:06:17
but nuclear efficient processes is just nuclear efficient.
Speaker:
00:06:21
When you put it into bombs,
Speaker:
00:06:22
it's obviously a bad thing.
Speaker:
00:06:23
And I think the same could be true of a lot of technology.
Speaker:
00:06:26
So there will be people who abuse them.
Speaker:
00:06:28
This is not dismissing that.
Speaker:
00:06:29
I think it's very important that as a society,
Speaker:
00:06:31
we think about ethics,
Speaker:
00:06:32
and we think about things like bias and all the rest.
Speaker:
00:06:35
I don't think we're actually worse off as a society.
Speaker:
00:06:37
I think it's that swings back and forth.
Speaker:
00:06:39
We get more sophisticated,
Speaker:
00:06:40
we keep progressing.
Speaker:
00:06:42
And I think there will be moments of disruption.
Speaker:
00:06:44
Maybe it's things like
Speaker:
00:06:45
the crypto space wrestle some of their power back
Speaker:
00:06:47
from the centralised control.
Speaker:
00:06:49
Some of the big cloud providers and things have now,
Speaker:
00:06:51
or maybe it's something else.
Speaker:
00:06:52
And in that moment, opportunities will be created.
Speaker:
00:06:54
And of course you will have
Speaker:
00:06:55
people who take advantage of that,
Speaker:
00:06:56
and there will be negative side effects,
Speaker:
00:06:58
but I've also no doubt that I was doing,
Speaker:
00:07:01
for example, when I did my master's,
Speaker:
00:07:02
I did quite a bit of stuff that was medically focused.
Speaker:
00:07:05
I'm not in that space now at all.
Speaker:
00:07:07
The things I was looking at like epilepsy,
Speaker:
00:07:09
and heart conditions,
Speaker:
00:07:11
and things like that.
Speaker:
00:07:12
And looking at trying to detect certain conditions,
Speaker:
00:07:14
or forecast them and that sort of thing.
Speaker:
00:07:16
So the one or two things I did,
Speaker:
00:07:17
what really drove this kind of home to me,
Speaker:
00:07:19
the power of this stuff,
Speaker:
00:07:20
is that today,
Speaker:
00:07:21
if you took like this particular sort of epilepsy condition
Speaker:
00:07:23
that we're looking at,
Speaker:
00:07:24
they would have five or six experts sit around,
Speaker:
00:07:26
a sneeze of people who takes well over a decade before
Speaker:
00:07:28
they can even at the level where they're qualified
Speaker:
00:07:30
to sit in front of a screen.
Speaker:
00:07:31
And then you've got like maybe five of these,
Speaker:
00:07:33
sometimes looking at someone with this condition.
Speaker:
00:07:35
That's a problem right now.
Speaker:
00:07:36
As humanity has gotten where there's seven billion of us,
Speaker:
00:07:38
and maybe we get to ten billion or more, who knows,
Speaker:
00:07:40
and we just have a critical shortage
Speaker:
00:07:42
of those kinds of skills.
Speaker:
00:07:44
And so that sort of analysis and diagnosis and stuff,
Speaker:
00:07:47
we're not displacing doctors and things entirely,
Speaker:
00:07:49
but we certainly augmenting them
Speaker:
00:07:50
and making it possible for a lot more people
Speaker:
00:07:53
to scale up things like healthcare.
Speaker:
00:07:55
And so you can't possibly see that as an all bad thing.
Speaker:
00:07:59
That is if it's seven billion people now,
Speaker:
00:08:01
probably one billion of us have okay health care.
Speaker:
00:08:04
Some people would argue how okay our healthcare is,
Speaker:
00:08:07
but it's certainly much better than the other,
Speaker:
00:08:09
than the vast majority of the population.
Speaker:
00:08:11
So I think those kinds of things,
Speaker:
00:08:14
automating intelligence,
Speaker:
00:08:15
scales up a lot of things which make it easier
Speaker:
00:08:18
to sustain the people that we have
Speaker:
00:08:21
with a lot better healthcare,
Speaker:
00:08:22
probably better food,
Speaker:
00:08:23
probably better legal representation.
Speaker:
00:08:25
That's another field where
Speaker:
00:08:26
legal representation is extremely expensive,
Speaker:
00:08:28
but you're getting all of this stuff emerging,
Speaker:
00:08:30
which is automating some of the analysis in the legal space,
Speaker:
00:08:33
which is probably going to democratise legal representation.
Speaker:
00:08:35
There's all those kinds of things
Speaker:
00:08:37
that I just can't help seeing the opportunity for good.
Speaker:
00:08:41
For sure, there are some issues,
Speaker:
00:08:43
there are things like bias
Speaker:
00:08:44
and other things in our data sets.
Speaker:
00:08:47
And I think on that particular thing, bias,
Speaker:
00:08:50
it's probably worth talking about.
Speaker:
00:08:52
Bias isn't,
Speaker:
00:08:53
there's a bad bias and there's a good bias, right?
Speaker:
00:08:55
When you hire somebody for a job,
Speaker:
00:08:57
in a sense, they've got a certain amount of bias
Speaker:
00:08:59
built into them,
Speaker:
00:09:00
and bias, in the sense that
Speaker:
00:09:00
they have come to know a certain domain
Speaker:
00:09:03
and they've come to get familiar with things.
Speaker:
00:09:06
So they lean towards, when something happens,
Speaker:
00:09:08
they lean towards saying,
Speaker:
00:09:09
oh, my experience tells me that this is probably that.
Speaker:
00:09:12
And that in the machine learning world,
Speaker:
00:09:14
there's this notion of bias-variance trade off.
Speaker:
00:09:17
And in fact, in that sense,
Speaker:
00:09:19
you don't want models that are completely unbiased
Speaker:
00:09:22
because every time they get a new data point,
Speaker:
00:09:24
they would just leap to a new conclusion
Speaker:
00:09:26
and they wouldn't actually be learning anything.
Speaker:
00:09:28
So they learn a bit of bias,
Speaker:
00:09:29
which is actually useful.
Speaker:
00:09:31
The kind of bias that we talk about
Speaker:
00:09:32
and we think is bad,
Speaker:
00:09:33
is when we learn biases that are not useful.
Speaker:
00:09:37
We learn things that are incorrect.
Speaker:
00:09:39
Like the fact that let's say the colour of someone's skin
Speaker:
00:09:41
determines whether they're good at a particular job,
Speaker:
00:09:43
which is latently incorrect.
Speaker:
00:09:45
And so that's not useful bias.
Speaker:
00:09:46
But all that's doing,
Speaker:
00:09:48
it's not that the technology has introduced the bias.
Speaker:
00:09:51
What it's doing,
Speaker:
00:09:52
is it's forcing us to confront the fact
Speaker:
00:09:55
that sort of bias has been implicit in all of our data,
Speaker:
00:09:59
and in all of our examples and things.
Speaker:
00:10:00
So we're training the stuff off of that
Speaker:
00:10:01
and it's learning the bias
Speaker:
00:10:02
and it's forcing us to confront that and say,
Speaker:
00:10:05
hold on, perhaps we need to correct that.
Speaker:
00:10:07
But it's not the algorithm that's causing it,
Speaker:
00:10:10
it's decades and millennia and years and years
Speaker:
00:10:13
of bias that we've developed as a society.
Speaker:
00:10:16
You could even say,
Speaker:
00:10:17
it's a positive that this is triggering that conversation.
Speaker:
00:10:19
And now that it's scaling that up
Speaker:
00:10:21
and people are saying,
Speaker:
00:10:22
hold on, you're scaling up algorithms
Speaker:
00:10:23
and things that are now re-entrenching
Speaker:
00:10:25
this bias we want to get rid of.
Speaker:
00:10:26
So it's amplifying the conversation,
Speaker:
00:10:28
which, I think it's a good thing.
Speaker:
00:10:30
- [Nicky] Let's talk a little bit about scale
Speaker:
00:10:33
because the work that you've done with Red Olive
Speaker:
00:10:35
on a couple of projects,
Speaker:
00:10:36
most recently,
Speaker:
00:10:37
it gave me a sense part of it was moving to the cloud,
Speaker:
00:10:40
and I'm guessing that's really where scale comes in.
Speaker:
00:10:43
But what would you say is the big deal with cloud?
Speaker:
00:10:45
What are the big business drivers?
Speaker:
00:10:46
Where does that enable you to go next?
Speaker:
00:10:49
- [Robin] Cloud, for me, it's about,
Speaker:
00:10:51
you could slide other things in here, not just cloud.
Speaker:
00:10:53
What you're trying to do all the time
Speaker:
00:10:54
is you're continually trying to stand
Speaker:
00:10:57
on the shoulders of giants.
Speaker:
00:10:58
So how we progress as a society,
Speaker:
00:11:00
as we build on the things
Speaker:
00:11:01
that other people have done before us,
Speaker:
00:11:03
this idea of don't reinvent the wheel.
Speaker:
00:11:06
We make progress by basically,
Speaker:
00:11:08
we do more than the generation before us
Speaker:
00:11:10
by using everything that they've done
Speaker:
00:11:12
and adding to it.
Speaker:
00:11:13
And when you build things, say build software,
Speaker:
00:11:16
or build algorithms,
Speaker:
00:11:17
and that sort of thing.
Speaker:
00:11:18
What you do find,
Speaker:
00:11:19
sometimes people at the earliest stages in their careers,
Speaker:
00:11:21
they're very interested in learning
Speaker:
00:11:22
the underlying mechanics of things.
Speaker:
00:11:23
So they go off and they want to build everything
Speaker:
00:11:25
from the ground up.
Speaker:
00:11:27
And sometimes what happens is you get people more senior.
Speaker:
00:11:30
Very often, they start to change their attitude,
Speaker:
00:11:33
or they start to become, has someone else built this?
Speaker:
00:11:35
And then what they look for
Speaker:
00:11:37
is the stuff that someone else hasn't built yet,
Speaker:
00:11:38
and then they build that.
Speaker:
00:11:39
And usually it's building on all of the things
Speaker:
00:11:41
people have built before.
Speaker:
00:11:42
Cloud aside, even things like a normal software development,
Speaker:
00:11:45
almost everyone now uses frameworks
Speaker:
00:11:47
and all sorts of things that other people have built,
Speaker:
00:11:50
and they build on top of them
Speaker:
00:11:51
and they wire them together.
Speaker:
00:11:52
And cloud is one of those things.
Speaker:
00:11:54
It's saying that you don't differentiate yourself by say,
Speaker:
00:11:58
installing a database.
Speaker:
00:12:00
So that's just something
Speaker:
00:12:01
that sort of has to be done to facilitate,
Speaker:
00:12:03
let's say, the analysis or the machine learning,
Speaker:
00:12:05
or whatever work that we then do on top of that.
Speaker:
00:12:08
And so what cloud is giving you,
Speaker:
00:12:09
it's giving you the ability to offload
Speaker:
00:12:11
all that non-differentiating stuff.
Speaker:
00:12:13
And economic terms,
Speaker:
00:12:14
one of the things that certainly served us well
Speaker:
00:12:17
is this certain amount of specialisation.
Speaker:
00:12:20
And what it's allowing you to do,
Speaker:
00:12:22
is it allowing you to take the experts,
Speaker:
00:12:24
the top specialists in those areas,
Speaker:
00:12:26
in setting up that infrastructure
Speaker:
00:12:28
and making it available to services
Speaker:
00:12:29
and that sort of thing,
Speaker:
00:12:30
concentrating them in a few places,
Speaker:
00:12:32
and then reusing their efforts
Speaker:
00:12:33
and everyone else reusing their efforts.
Speaker:
00:12:35
So that for me is that the point with cloud,
Speaker:
00:12:37
is you're not trying to save money,
Speaker:
00:12:39
you're trying to increase productivity.
Speaker:
00:12:41
You're trying to not build things again,
Speaker:
00:12:43
spend all your effort building
Speaker:
00:12:45
the stuff that's going to improve your business
Speaker:
00:12:48
and your domain in some way.
Speaker:
00:12:50
Rather than installing a database,
Speaker:
00:12:52
or installing, I don't know, it could be
Speaker:
00:12:53
a streaming platform,
Speaker:
00:12:55
or whatever the case may be.
Speaker:
00:12:56
And it really does increase your productivity.
Speaker:
00:12:58
If anyone has worked with,
Speaker:
00:13:01
let's say, in data space, like a big data warehouse,
Speaker:
00:13:05
it's really easy these days with something
Speaker:
00:13:07
like say, big query.
Speaker:
00:13:08
And you just use it, right?
Speaker:
00:13:10
It's obviously still things you have to do,
Speaker:
00:13:12
but there was a time when you had to spend a lot more time.
Speaker:
00:13:15
You have to have a lot more DBAs
Speaker:
00:13:17
and things optimising your system.
Speaker:
00:13:19
And that's, if you're a business,
Speaker:
00:13:20
that's got a team of, I don't know,
Speaker:
00:13:22
you might have a team of two people.
Speaker:
00:13:23
If one of them has to do all of the maintaining the database
Speaker:
00:13:25
and optimise that all the time, that's half the capacity.
Speaker:
00:13:28
But even bigger teams,
Speaker:
00:13:29
you've got a team of a hundred.
Speaker:
00:13:30
I can bet you that if you're doing it on-premise,
Speaker:
00:13:32
the 20% of those is permanently looking after
Speaker:
00:13:35
your environment.
Speaker:
00:13:36
And that's 20% you're not spending
Speaker:
00:13:37
on some sort of business differentiator.
Speaker:
00:13:39
- [Nicky] With that, you're talking about the teams,
Speaker:
00:13:41
how do you get it so that team and culture
Speaker:
00:13:44
are tied into that.
Speaker:
00:13:44
Because I'm guessing there is a sort of,
Speaker:
00:13:46
a bit of an education
Speaker:
00:13:47
that might come along with that kind of
Speaker:
00:13:49
moving to the cloud.
Speaker:
00:13:50
And how differently you might have to work
Speaker:
00:13:52
than in a traditional sort of data and IT projects.
Speaker:
00:13:56
How important is it to have the vision
Speaker:
00:13:58
and be able to communicate that internally
Speaker:
00:14:01
to the rest of the organisation?
Speaker:
00:14:03
- [Robin] It's not really a hard sell anymore
Speaker:
00:14:05
to tell everyone that everyone's moving to the cloud
Speaker:
00:14:07
and we got to do it too.
Speaker:
00:14:08
Now, people may not always recognise why.
Speaker:
00:14:10
I think that the people who, if you've got say,
Speaker:
00:14:14
technologists and people who are working on the platforms,
Speaker:
00:14:18
for example,
Speaker:
00:14:19
they're always very willing to do it
Speaker:
00:14:22
because they know it's where everyone else is going
Speaker:
00:14:24
and they want their careers to progress.
Speaker:
00:14:26
I think executives use those increasingly,
Speaker:
00:14:28
because everyone's doing it.
Speaker:
00:14:29
It just seems that's the kind of common wisdom, right?
Speaker:
00:14:32
So that's what you do now.
Speaker:
00:14:33
I think the important thing now,
Speaker:
00:14:35
it's not so hard to sell people on the idea of doing cloud,
Speaker:
00:14:38
I think it's just probably more important
Speaker:
00:14:40
just to make everyone understand,
Speaker:
00:14:42
what is it do you want from cloud?
Speaker:
00:14:43
Why are you doing cloud?
Speaker:
00:14:44
Why is everyone doing it?
Speaker:
00:14:45
You're getting those kinds of philosophies out there
Speaker:
00:14:47
and saying, look, all we're trying to get is,
Speaker:
00:14:50
we're trying to offload all that non-differentiating effort
Speaker:
00:14:53
because that's where you will get people
Speaker:
00:14:56
trying to build things themselves,
Speaker:
00:14:58
or you'll get,
Speaker:
00:14:59
it's a hard thing sometimes to move something
Speaker:
00:15:00
when you've got a complicated on-premise platform,
Speaker:
00:15:03
for example.
Speaker:
00:15:03
And it could be data,
Speaker:
00:15:04
it could be any other software.
Speaker:
00:15:06
And it's actually quite hard to move from one to the other,
Speaker:
00:15:08
so you had a lot of this sort of
Speaker:
00:15:09
lift and shift type mentality.
Speaker:
00:15:10
And the reason that doesn't work is because
Speaker:
00:15:12
it doesn't leverage this idea that you are floating,
Speaker:
00:15:14
all the non-differentiating work.
Speaker:
00:15:16
If you're fundamentally still doing most of the instal
Speaker:
00:15:18
and maintaining all the applications.
Speaker:
00:15:20
So a lot of startups,
Speaker:
00:15:21
but even bigger organisations
Speaker:
00:15:22
are getting there with sort of saying,
Speaker:
00:15:24
okay, actually, why don't we, you know,
Speaker:
00:15:25
shift all of this,
Speaker:
00:15:26
well a lot of stuff, to something that's more
Speaker:
00:15:30
server-less.
Speaker:
00:15:31
Not the idea that it's as a service,
Speaker:
00:15:33
I just use it to just write some code
Speaker:
00:15:35
and deploy that,
Speaker:
00:15:35
rather than have to manage all the intermediate steps.
Speaker:
00:15:39
That in itself is a lot of work to get there.
Speaker:
00:15:41
So it is actually sometimes easier for people
Speaker:
00:15:44
just to think in terms of, okay,
Speaker:
00:15:44
what if I just do what I was doing,
Speaker:
00:15:46
but to do it on the cloud.
Speaker:
00:15:48
And I think that's probably where people fail
Speaker:
00:15:50
and when they end up increasing their bowl,
Speaker:
00:15:52
because this is like renting or buying a house
Speaker:
00:15:54
in some respects.
Speaker:
00:15:55
If you're just going to do exactly what you did before,
Speaker:
00:15:58
then you're better off buying than renting
Speaker:
00:16:00
because they've got a margin on top
Speaker:
00:16:03
and they're doing a little bit of work underneath the hood.
Speaker:
00:16:05
But if you don't change your attitude,
Speaker:
00:16:06
you will just spend more money
Speaker:
00:16:08
and you won't get the benefits.
Speaker:
00:16:09
So I think that's the important part
Speaker:
00:16:12
is understanding what you're trying to get
Speaker:
00:16:13
is additional productivity,
Speaker:
00:16:15
and other things as well.
Speaker:
00:16:16
There's some philosophies there that I think people,
Speaker:
00:16:18
I don't know,
Speaker:
00:16:19
maybe it's because I've been doing it so long,
Speaker:
00:16:19
it doesn't seem new to me anymore,
Speaker:
00:16:20
but I'm not sensing that I'm getting a lot of resistance
Speaker:
00:16:23
to the ideas anymore.
Speaker:
00:16:24
But I think other things people don't always fully grasp,
Speaker:
00:16:27
if they're not in that world,
Speaker:
00:16:29
emphasis changes from things like
Speaker:
00:16:31
managing availability of service
Speaker:
00:16:33
to managing costs.
Speaker:
00:16:35
And what I mean by that is,
Speaker:
00:16:36
if you use kind of properly
Speaker:
00:16:37
and use things like auto scaling and all the rest,
Speaker:
00:16:39
and you use like chaos monkeys
Speaker:
00:16:41
and these things where you're actually
Speaker:
00:16:43
deliberately trying to take out services to see,
Speaker:
00:16:46
to make sure things are resilient.
Speaker:
00:16:47
You try to take out services,
Speaker:
00:16:48
say in the middle of the day,
Speaker:
00:16:50
and then because if they are going to break,
Speaker:
00:16:52
it's better for them to break while everyone's around,
Speaker:
00:16:54
basically is a philosophy with something like that.
Speaker:
00:16:55
But that's all facilitated by this idea
Speaker:
00:16:58
that as a consumer,
Speaker:
00:16:59
you can use the cloud as if it's an infinite resource.
Speaker:
00:17:02
And that's great,
Speaker:
00:17:03
because you can build services that never go down.
Speaker:
00:17:06
So the first thing,
Speaker:
00:17:06
you've got to philosophically get into that mindset.
Speaker:
00:17:09
We live in a different world
Speaker:
00:17:11
where things can be incredibly robust.
Speaker:
00:17:11
Where they can just scale up and down
Speaker:
00:17:13
as our market scales up and down.
Speaker:
00:17:15
And that is actually a great attribute
Speaker:
00:17:18
for any organisation to have.
Speaker:
00:17:19
To be able to adapt like that to circumstances.
Speaker:
00:17:22
The flip side, of course,
Speaker:
00:17:24
is that before your costs were then finite,
Speaker:
00:17:26
in the sense you bought some stuff
Speaker:
00:17:28
and your service might go down because it's overloaded,
Speaker:
00:17:31
but your costs were still fixed.
Speaker:
00:17:33
So that's where I think
Speaker:
00:17:35
you need to pay a bit more attention to cost management
Speaker:
00:17:37
and that sort of thing.
Speaker:
00:17:38
And that can be tricky.
Speaker:
00:17:39
Like I've been in a company where, you know,
Speaker:
00:17:41
early stages of the projects,
Speaker:
00:17:42
I was like, how much is going to cost?
Speaker:
00:17:44
Well we've just started doing it,
Speaker:
00:17:45
like we've just launched.
Speaker:
00:17:46
I have to spend a few months
Speaker:
00:17:49
figuring how greedy the service is.
Speaker:
00:17:51
And you kind of work through it for a bit in the beginning.
Speaker:
00:17:53
But you do have to spend some time,
Speaker:
00:17:55
work through that,
Speaker:
00:17:56
and then spend a lot of time putting together
Speaker:
00:17:59
those mechanisms where you actively trade off
Speaker:
00:18:01
that sort of cost versus availability and performance
Speaker:
00:18:05
for your customers, and that sort of thing.
Speaker:
00:18:06
So it's things like that,
Speaker:
00:18:07
it's those nuances I think,
Speaker:
00:18:09
is understanding you're trying to be more productive.
Speaker:
00:18:11
You can build a sort of bulletproof service,
Speaker:
00:18:14
very easy to port internationally
Speaker:
00:18:16
and all sorts of things if you do it right.
Speaker:
00:18:18
But understanding all those things
Speaker:
00:18:19
and specifically designing for that,
Speaker:
00:18:21
so that you actually get the value from cloud.
Speaker:
00:18:24
I think that's the bit which
Speaker:
00:18:25
you still have to spend time banging the drum,
Speaker:
00:18:27
making sure everybody understands that
Speaker:
00:18:29
it's not so hard to convince people to go to cloud anymore,
Speaker:
00:18:31
if they're not on cloud already.
Speaker:
00:18:33
- [Nicky] Do you think that
Speaker:
00:18:35
within the development side of things,
Speaker:
00:18:37
people are braver knowing that
Speaker:
00:18:38
if you're using a cloud platform,
Speaker:
00:18:41
that it gives you more scope to try things out
Speaker:
00:18:43
and if it doesn't work.
Speaker:
00:18:44
So you're kind of doing a bit more of that CI/CD
Speaker:
00:18:48
as you're moving along, that's how you develop them.
Speaker:
00:18:51
Do you think they sort of come together almost,
Speaker:
00:18:52
they fit together a little bit like hand in a glove.
Speaker:
00:18:55
- [Robin] They do,
Speaker:
00:18:56
in theory kind of CI/CD is, I don't know,
Speaker:
00:18:59
orthogonal to the cloud,
Speaker:
00:19:00
they compliment each other.
Speaker:
00:19:01
But you could do a lot of that stuff on-premise.
Speaker:
00:19:03
It makes it easier to do it on cloud
Speaker:
00:19:04
because you can just spin up and down instances on demand
Speaker:
00:19:08
and that sort of thing,
Speaker:
00:19:09
because you can treat this thing
Speaker:
00:19:10
as almost an infinite resource.
Speaker:
00:19:11
It makes it very easy
Speaker:
00:19:12
to build these very flexible pipelines and stuff.
Speaker:
00:19:14
But at its heart,
Speaker:
00:19:14
the idea of agile is about continuous feedback.
Speaker:
00:19:17
And it seems obvious that if you wait until,
Speaker:
00:19:19
if we look at traditional, much earlier philosophies
Speaker:
00:19:22
where you had waterfall type philosophies
Speaker:
00:19:24
in development and stuff,
Speaker:
00:19:25
where people would design whole big programmes
Speaker:
00:19:27
and projects and stuff,
Speaker:
00:19:29
and then you phase them.
Speaker:
00:19:30
You had design in the front,
Speaker:
00:19:31
and then you had this development,
Speaker:
00:19:32
and then you had testing,
Speaker:
00:19:33
and you had some sort of release.
Speaker:
00:19:34
And the problem with that stuff is all the UAT,
Speaker:
00:19:36
and all of that sort of thing was very late in the day.
Speaker:
00:19:39
And the projects took so long,
Speaker:
00:19:40
that lots changed in the meantime.
Speaker:
00:19:43
And you got to the end
Speaker:
00:19:44
and then you figured out that there were all sorts of things
Speaker:
00:19:46
that needed to change.
Speaker:
00:19:47
The world had changed,
Speaker:
00:19:47
Or there's a few, or things have been misinterpreted.
Speaker:
00:19:50
And there's a lot of people in the middle.
Speaker:
00:19:51
There's a lot of messages that have to be understood.
Speaker:
00:19:53
And there's a lot of misinterpretation that happens there.
Speaker:
00:19:56
So there's nothing quite like
Speaker:
00:19:57
putting actual product in front of people saying,
Speaker:
00:19:58
did I get it wrong,
Speaker:
00:19:59
or is it still in the context of my environment and stuff?
Speaker:
00:20:03
And so that's rarely,
Speaker:
00:20:04
agile,
Speaker:
00:20:06
a lot of it basically centres around that.
Speaker:
00:20:08
It's can we get feedback very early in the process?
Speaker:
00:20:11
Can we have short cycles?
Speaker:
00:20:13
Can we have lots of sort of show and tell type things?
Speaker:
00:20:15
Can we, and it depends on the product you're building,
Speaker:
00:20:17
but can we continuously,
Speaker:
00:20:19
can we encourage that continuous feedback
Speaker:
00:20:21
and then see those problems early?
Speaker:
00:20:24
Because it's a lot cheaper to fix something
Speaker:
00:20:26
when you've written three lines of code,
Speaker:
00:20:28
than at the end of the project
Speaker:
00:20:29
when you've written a whole complex thing,
Speaker:
00:20:30
several things all depending on each other,
Speaker:
00:20:32
and you have to unwind many things.
Speaker:
00:20:34
So if you say that's the heart of agile,
Speaker:
00:20:36
then the CI/CD thing just isn't natural fit to that.
Speaker:
00:20:40
I'm saying, so talking about to the cloud,
Speaker:
00:20:41
in the sense that that is about,
Speaker:
00:20:42
CI/CD is about really just putting things out often,
Speaker:
00:20:46
and quickly,
Speaker:
00:20:47
and efficiently, right?
Speaker:
00:20:48
Automating the stuff that you can automate.
Speaker:
00:20:50
The process from checking in your code
Speaker:
00:20:53
through to automating the testing.
Speaker:
00:20:54
And of course,
Speaker:
00:20:55
there's a discipline there about writing a lot of tests
Speaker:
00:20:57
and test first development, and that sort of thing,
Speaker:
00:20:59
which makes that work, right?
Speaker:
00:21:01
It gives you the confidence to look at
Speaker:
00:21:03
these large organisations that have
Speaker:
00:21:05
tens of thousands of releases a day and stuff.
Speaker:
00:21:07
They can only do that because
Speaker:
00:21:10
the process is designed so that every time
Speaker:
00:21:12
something is checked in and merged to main,
Speaker:
00:21:14
it's automatically tested and all the rest.
Speaker:
00:21:16
It's not entirely bulletproof.
Speaker:
00:21:18
You're much more confident in an environment
Speaker:
00:21:20
where something won't go all the way into production,
Speaker:
00:21:22
unless it gets thoroughly tested,
Speaker:
00:21:24
and this whole process is really automated.
Speaker:
00:21:26
That sort of gives you a confidence.
Speaker:
00:21:28
It means you can deploy very often.
Speaker:
00:21:29
And if you can deploy the full things very often like that,
Speaker:
00:21:32
then that fits that sort of feedback cycle.
Speaker:
00:21:35
You've got lots of little changes happening all the time,
Speaker:
00:21:37
and occasionally something will go wrong,
Speaker:
00:21:40
but then it's going to be one small thing
Speaker:
00:21:41
that you can roll back and you can learn from.
Speaker:
00:21:42
And you become a very adaptable.
Speaker:
00:21:45
And I'm not using the term agile with all the baggage
Speaker:
00:21:47
of what we think agile means in software.
Speaker:
00:21:49
I just mean you've truly become an agile organisation
Speaker:
00:21:51
in the sense that
Speaker:
00:21:53
you become very sensitive to little changes
Speaker:
00:21:54
in your environment.
Speaker:
00:21:55
And you change really quickly,
Speaker:
00:21:56
and you evolve fast.
Speaker:
00:21:57
And of course,
Speaker:
00:21:58
if you can evolve quickly,
Speaker:
00:22:00
then you've got an inherent advantage over something
Speaker:
00:22:03
that doesn't evolve quickly.
Speaker:
00:22:05
So that's where I think CI/CD fits in.
Speaker:
00:22:07
Again, it enables you to evolve really quickly.
Speaker:
00:22:10
How it relates to cloud,
Speaker:
00:22:11
is cloud has enabled us to, because
Speaker:
00:22:14
cloud has also allowed us,
Speaker:
00:22:15
so in the past, if you wanted to do something new,
Speaker:
00:22:17
you would have to go and get quite a large CapEx budget.
Speaker:
00:22:21
Let's say to buy some licence,
Speaker:
00:22:22
to instal some equipment,
Speaker:
00:22:23
to do all of those things upfront.
Speaker:
00:22:25
And so obviously in cloud, now,
Speaker:
00:22:26
if you want to do something small,
Speaker:
00:22:28
you can do something small.
Speaker:
00:22:29
And that's a key point, right?
Speaker:
00:22:31
You can go and try.
Speaker:
00:22:33
I talked about Google,
Speaker:
00:22:34
let's say Amazon,
Speaker:
00:22:35
I've got something like Kinesis or something.
Speaker:
00:22:39
There are products that, I actually prefer,
Speaker:
00:22:42
some of them it's the underlying product,
Speaker:
00:22:44
so let's say like Kafka or something.
Speaker:
00:22:46
That underlying product,
Speaker:
00:22:47
I think it was a really good product.
Speaker:
00:22:49
There's other things that have come along
Speaker:
00:22:51
in the streaming space that kind of improved on it a bit.
Speaker:
00:22:54
Historically, you've had to go and buy,
Speaker:
00:22:58
you have to go to a case to spend like
Speaker:
00:23:00
$100K+ with someone to get started.
Speaker:
00:23:05
And of course you can go into Amazon,
Speaker:
00:23:06
in something like Kinesis,
Speaker:
00:23:07
and you can just start.
Speaker:
00:23:08
And you can pay like a few quid just to do a POC, basically.
Speaker:
00:23:12
So that fits into the cycle of just being able to
Speaker:
00:23:15
iteratively
Speaker:
00:23:17
do the next thing.
Speaker:
00:23:18
See if it works.
Speaker:
00:23:19
It fits this whole picture of continuous evolution
Speaker:
00:23:21
in very gradual steps.
Speaker:
00:23:23
- [Nicky] Excellent advice from Robin there.
Speaker:
00:23:25
But we're going to have to leave our conversation
Speaker:
00:23:27
and pick up on it in the next episode
Speaker:
00:23:29
where he'll be talking about the sorts of skills
Speaker:
00:23:31
people need to develop to get ahead in the industry.
Speaker:
00:23:34
It's full of interesting and useful info,
Speaker:
00:23:36
so make sure you subscribe
Speaker:
00:23:37
to the Red Olive Fibonacci podcast,
Speaker:
00:23:39
from wherever you get your podcasts
Speaker:
00:23:41
to make sure you don't miss out.
Speaker:
00:23:43
That's all for today.
Speaker:
00:23:44
Thanks for listening.
Speaker:
00:23:45
And catch up next time.
00:00:00
- [Nicky] Hello, and welcome to
Speaker:
00:00:01
the Red Olive Fibonacci Podcast.
Speaker:
00:00:03
The podcast all about the brilliant world of data
Speaker:
00:00:06
covering future trends and topical tech.
Speaker:
00:00:08
We'll be joined by experts in the data sphere
Speaker:
00:00:10
to share their opinions and advice.
Speaker:
00:00:12
I'm your host, Nicky Rudd.
Speaker:
00:00:14
Today's episode is the first of a two-part interview
Speaker:
00:00:17
recorded during lockdown with Robin Hayden.
Speaker:
00:00:19
Robin has worked for over 20 years in data
Speaker:
00:00:22
and has a wealth of experience in machine learning,
Speaker:
00:00:24
AI, large-scale engineering, analytics,
Speaker:
00:00:27
and product development.
Speaker:
00:00:28
In various chief data officer roles,
Speaker:
00:00:30
he's delivered over 100 machine learning models
Speaker:
00:00:32
into production,
Speaker:
00:00:33
been responsible for tens of millions in additional revenue
Speaker:
00:00:36
and hundreds of millions in spend.
Speaker:
00:00:39
The first part of the conversation covers a lot of ground
Speaker:
00:00:41
from crypto and AI,
Speaker:
00:00:42
data ethics and Skynet,
Speaker:
00:00:43
to the best use of the cloud
Speaker:
00:00:45
for businesses looking to update,
Speaker:
00:00:46
and how to run a data migration project.
Speaker:
00:00:49
So let's go.
Speaker:
00:00:50
(introductory rock music)
Speaker:
00:01:02
- [Nicky] I thought we could start off
Speaker:
00:01:04
if you could just tell me a little bit about
Speaker:
00:01:05
how you got into the world of data.
Speaker:
00:01:08
- [Robin] It was a bit of a long journey.
Speaker:
00:01:10
I actually, I was quite into this sort of AI space
Speaker:
00:01:13
from very early on,
Speaker:
00:01:14
but you couldn't really get a job in that
Speaker:
00:01:15
unless you're a researcher and or something in that space.
Speaker:
00:01:18
And you know, that wasn't an option that was open to me
Speaker:
00:01:21
at the time.
Speaker:
00:01:21
So I went off and did
Speaker:
00:01:22
a lot of sort of early internet type work.
Speaker:
00:01:26
And it started to become apparent
Speaker:
00:01:27
that we were able to do really smart things.
Speaker:
00:01:30
It was basically all based off of big data
Speaker:
00:01:32
and it was starting to become apparent
Speaker:
00:01:34
that we now had enough data
Speaker:
00:01:36
that you could go beyond doing
Speaker:
00:01:39
a sort of ad hoc analysis.
Speaker:
00:01:41
You could really start to extract decisions.
Speaker:
00:01:44
And it was obvious that at some point
Speaker:
00:01:46
it was going to change the way we did all of computing.
Speaker:
00:01:48
I sort of moved my career to focus primarily on data
Speaker:
00:01:52
from that point it's 80% data engineering
Speaker:
00:01:55
and 20% intelligence.
Speaker:
00:01:57
But it is the 20% that's really fascinating.
Speaker:
00:02:00
- [Nicky] It's a really exciting industry
Speaker:
00:02:01
to be involved with,
Speaker:
00:02:02
even in the work that I've been doing
Speaker:
00:02:04
over the last couple of years.
Speaker:
00:02:05
The step changes really just have jumped.
Speaker:
00:02:08
What do you think of the most exciting changes of foot?
Speaker:
00:02:11
How do you see things developing over the next
Speaker:
00:02:13
two to three years.
Speaker:
00:02:15
- [Robin] In the space, I would say definitely.
Speaker:
00:02:17
I think the proliferation of edge computing
Speaker:
00:02:21
and the ability to push intelligence
Speaker:
00:02:23
all the way out to the edge
Speaker:
00:02:24
is something that is, some people know about it,
Speaker:
00:02:26
but it isn't as widely understood in this space.
Speaker:
00:02:29
People think of big data as big and expensive
Speaker:
00:02:31
and this sort of planet killer in many cases
Speaker:
00:02:34
because these big models,
Speaker:
00:02:35
and they are some big models
Speaker:
00:02:36
and they are expensive to build like GPT-3,
Speaker:
00:02:38
or Google's competitor to that recently,
Speaker:
00:02:40
which was,
Speaker:
00:02:41
I think GPT-3 was several billion,
Speaker:
00:02:43
a few hundred billion parameters or something.
Speaker:
00:02:45
And Google's equivalent, not that long ago,
Speaker:
00:02:48
was over a trillion parameters.
Speaker:
00:02:49
And those things really do take
Speaker:
00:02:50
enormous amounts of power and computing and stuff to build.
Speaker:
00:02:53
But the flip side is that the computing,
Speaker:
00:02:56
it takes orders of magnitude more energy
Speaker:
00:03:00
to process something,
Speaker:
00:03:01
to push something all the way to the cloud and back.
Speaker:
00:03:03
It actually, even just pushing data
Speaker:
00:03:05
from the processor into RAM on your machine
Speaker:
00:03:07
versus processor registers,
Speaker:
00:03:09
is it very expensive?
Speaker:
00:03:10
Like that distance,
Speaker:
00:03:11
that extra distance costs a lot in computing terms.
Speaker:
00:03:14
The idea that things like neural networks,
Speaker:
00:03:15
that basically they're doing matrix operations
Speaker:
00:03:18
over and over again,
Speaker:
00:03:19
that's basically taking the same data
Speaker:
00:03:21
and recycling it,
Speaker:
00:03:21
and reprocessing it over and over again in the processor.
Speaker:
00:03:24
So in many ways it's actually very power efficient
Speaker:
00:03:27
and this idea that once you have these in these models
Speaker:
00:03:29
and they can do inference
Speaker:
00:03:30
out at the edge on some very low power devices.
Speaker:
00:03:32
I mean, there's this real, you know,
Speaker:
00:03:37
as they're starting to
Speaker:
00:03:38
become embedded in all sorts of things,
Speaker:
00:03:39
cars obviously being the most prominent of those,
Speaker:
00:03:41
but in almost anything that's out in our environment
Speaker:
00:03:43
and all sorts of sensors and things,
Speaker:
00:03:45
you're getting really low power chips and things
Speaker:
00:03:47
that are optimised for this.
Speaker:
00:03:48
And I think our whole world
Speaker:
00:03:49
may well become much, much more intelligent.
Speaker:
00:03:52
And you combine that with things that are emerging,
Speaker:
00:03:54
like the crypto space,
Speaker:
00:03:55
which if you go beyond the currencies
Speaker:
00:03:57
is really just people exploring
Speaker:
00:03:58
how far can we take this distributed computing,
Speaker:
00:04:01
this sort of collaborative computing,
Speaker:
00:04:02
and perhaps that we've had a large amount of centralization,
Speaker:
00:04:05
which I suppose is a bit counter to
Speaker:
00:04:07
what was the original sort of spirit of the internet
Speaker:
00:04:09
when it was very open for everyone.
Speaker:
00:04:11
And now we've got this consolidation
Speaker:
00:04:12
around a few big powers.
Speaker:
00:04:14
And there's a lot of political backlash to that.
Speaker:
00:04:15
And all the rest at the same time,
Speaker:
00:04:17
you've got things like crypto space developing
Speaker:
00:04:18
and despite the sort of hype,
Speaker:
00:04:20
and I'm sure a lot of people may still get hurt
Speaker:
00:04:22
in the investment world there.
Speaker:
00:04:24
But I think the idea of being able to encrypt
Speaker:
00:04:27
and process things very widely
Speaker:
00:04:29
and developing those sorts of,
Speaker:
00:04:32
they're almost like democratic networks
Speaker:
00:04:33
that allow people to share processing of data and stuff,
Speaker:
00:04:36
and the file storage and stuff
Speaker:
00:04:38
in ways that it couldn't do before.
Speaker:
00:04:40
And you leverage that
Speaker:
00:04:42
with the sort of ubiquitous intelligence
Speaker:
00:04:44
and the ability to run intelligent algorithms on everything.
Speaker:
00:04:47
It's hard to actually rarely,
Speaker:
00:04:49
to even fully imagine what that might mean.
Speaker:
00:04:51
Honestly, if you think of an entire world
Speaker:
00:04:53
where everything is more intelligent,
Speaker:
00:04:54
where, you know,
Speaker:
00:04:55
also where there's not necessarily
Speaker:
00:04:57
one or two big controllers of that world.
Speaker:
00:04:59
That could be a very radically different world.
Speaker:
00:05:02
Then, of course, if you zoom forward even further
Speaker:
00:05:05
and you look at things like quantum computing,
Speaker:
00:05:06
which doesn't solve everything,
Speaker:
00:05:07
that particular class of problems,
Speaker:
00:05:09
but certainly then there are some very big problems
Speaker:
00:05:11
and things that were much, much harder to do.
Speaker:
00:05:12
And so if you zoom forward another decade or so,
Speaker:
00:05:15
maybe two,
Speaker:
00:05:16
I don't know however long it takes to do that.
Speaker:
00:05:18
You know, then we just have this
Speaker:
00:05:19
another sort of order of magnitude
Speaker:
00:05:21
for a certain class of problems
Speaker:
00:05:22
and other sort of order of magnitude improvement
Speaker:
00:05:24
in sophistication that we're able to achieve.
Speaker:
00:05:26
So it just feels like this idea that,
Speaker:
00:05:28
I don't really obsess much about generalised intelligence,
Speaker:
00:05:33
and those people who think about
Speaker:
00:05:34
Skynet's going to take over the world.
Speaker:
00:05:35
I think it's impossible to say how far we are from something
Speaker:
00:05:39
when we don't know, we don't have a map to getting there,
Speaker:
00:05:42
if you know what I mean.
Speaker:
00:05:43
We could be a decade away.
Speaker:
00:05:46
It could be a million years away.
Speaker:
00:05:47
Nobody really actually knows.
Speaker:
00:05:48
So I don't really spend much time thinking about that.
Speaker:
00:05:50
But I do think that it's obvious
Speaker:
00:05:52
that over the next decade even,
Speaker:
00:05:54
all of our products are going to be much more intelligent.
Speaker:
00:05:56
And not just the products we're used to using
Speaker:
00:05:58
on the internet.
Speaker:
00:05:59
All of our products.
Speaker:
00:06:00
The things that are in our house,
Speaker:
00:06:01
and in our cars,
Speaker:
00:06:02
and in our gardens,
Speaker:
00:06:03
and that sort of thing.
Speaker:
00:06:04
And that's really exciting.
Speaker:
00:06:07
- [Nicky] What would you say
Speaker:
00:06:08
when it comes to that kind of intelligence,
Speaker:
00:06:10
and what's your take on
Speaker:
00:06:11
the kind of the ethics around it,
Speaker:
00:06:13
and that kind of ethical data question?
Speaker:
00:06:15
- [Robin] Technology is neutral,
Speaker:
00:06:17
but nuclear efficient processes is just nuclear efficient.
Speaker:
00:06:21
When you put it into bombs,
Speaker:
00:06:22
it's obviously a bad thing.
Speaker:
00:06:23
And I think the same could be true of a lot of technology.
Speaker:
00:06:26
So there will be people who abuse them.
Speaker:
00:06:28
This is not dismissing that.
Speaker:
00:06:29
I think it's very important that as a society,
Speaker:
00:06:31
we think about ethics,
Speaker:
00:06:32
and we think about things like bias and all the rest.
Speaker:
00:06:35
I don't think we're actually worse off as a society.
Speaker:
00:06:37
I think it's that swings back and forth.
Speaker:
00:06:39
We get more sophisticated,
Speaker:
00:06:40
we keep progressing.
Speaker:
00:06:42
And I think there will be moments of disruption.
Speaker:
00:06:44
Maybe it's things like
Speaker:
00:06:45
the crypto space wrestle some of their power back
Speaker:
00:06:47
from the centralised control.
Speaker:
00:06:49
Some of the big cloud providers and things have now,
Speaker:
00:06:51
or maybe it's something else.
Speaker:
00:06:52
And in that moment, opportunities will be created.
Speaker:
00:06:54
And of course you will have
Speaker:
00:06:55
people who take advantage of that,
Speaker:
00:06:56
and there will be negative side effects,
Speaker:
00:06:58
but I've also no doubt that I was doing,
Speaker:
00:07:01
for example, when I did my master's,
Speaker:
00:07:02
I did quite a bit of stuff that was medically focused.
Speaker:
00:07:05
I'm not in that space now at all.
Speaker:
00:07:07
The things I was looking at like epilepsy,
Speaker:
00:07:09
and heart conditions,
Speaker:
00:07:11
and things like that.
Speaker:
00:07:12
And looking at trying to detect certain conditions,
Speaker:
00:07:14
or forecast them and that sort of thing.
Speaker:
00:07:16
So the one or two things I did,
Speaker:
00:07:17
what really drove this kind of home to me,
Speaker:
00:07:19
the power of this stuff,
Speaker:
00:07:20
is that today,
Speaker:
00:07:21
if you took like this particular sort of epilepsy condition
Speaker:
00:07:23
that we're looking at,
Speaker:
00:07:24
they would have five or six experts sit around,
Speaker:
00:07:26
a sneeze of people who takes well over a decade before
Speaker:
00:07:28
they can even at the level where they're qualified
Speaker:
00:07:30
to sit in front of a screen.
Speaker:
00:07:31
And then you've got like maybe five of these,
Speaker:
00:07:33
sometimes looking at someone with this condition.
Speaker:
00:07:35
That's a problem right now.
Speaker:
00:07:36
As humanity has gotten where there's seven billion of us,
Speaker:
00:07:38
and maybe we get to ten billion or more, who knows,
Speaker:
00:07:40
and we just have a critical shortage
Speaker:
00:07:42
of those kinds of skills.
Speaker:
00:07:44
And so that sort of analysis and diagnosis and stuff,
Speaker:
00:07:47
we're not displacing doctors and things entirely,
Speaker:
00:07:49
but we certainly augmenting them
Speaker:
00:07:50
and making it possible for a lot more people
Speaker:
00:07:53
to scale up things like healthcare.
Speaker:
00:07:55
And so you can't possibly see that as an all bad thing.
Speaker:
00:07:59
That is if it's seven billion people now,
Speaker:
00:08:01
probably one billion of us have okay health care.
Speaker:
00:08:04
Some people would argue how okay our healthcare is,
Speaker:
00:08:07
but it's certainly much better than the other,
Speaker:
00:08:09
than the vast majority of the population.
Speaker:
00:08:11
So I think those kinds of things,
Speaker:
00:08:14
automating intelligence,
Speaker:
00:08:15
scales up a lot of things which make it easier
Speaker:
00:08:18
to sustain the people that we have
Speaker:
00:08:21
with a lot better healthcare,
Speaker:
00:08:22
probably better food,
Speaker:
00:08:23
probably better legal representation.
Speaker:
00:08:25
That's another field where
Speaker:
00:08:26
legal representation is extremely expensive,
Speaker:
00:08:28
but you're getting all of this stuff emerging,
Speaker:
00:08:30
which is automating some of the analysis in the legal space,
Speaker:
00:08:33
which is probably going to democratise legal representation.
Speaker:
00:08:35
There's all those kinds of things
Speaker:
00:08:37
that I just can't help seeing the opportunity for good.
Speaker:
00:08:41
For sure, there are some issues,
Speaker:
00:08:43
there are things like bias
Speaker:
00:08:44
and other things in our data sets.
Speaker:
00:08:47
And I think on that particular thing, bias,
Speaker:
00:08:50
it's probably worth talking about.
Speaker:
00:08:52
Bias isn't,
Speaker:
00:08:53
there's a bad bias and there's a good bias, right?
Speaker:
00:08:55
When you hire somebody for a job,
Speaker:
00:08:57
in a sense, they've got a certain amount of bias
Speaker:
00:08:59
built into them,
Speaker:
00:09:00
and bias, in the sense that
Speaker:
00:09:00
they have come to know a certain domain
Speaker:
00:09:03
and they've come to get familiar with things.
Speaker:
00:09:06
So they lean towards, when something happens,
Speaker:
00:09:08
they lean towards saying,
Speaker:
00:09:09
oh, my experience tells me that this is probably that.
Speaker:
00:09:12
And that in the machine learning world,
Speaker:
00:09:14
there's this notion of bias-variance trade off.
Speaker:
00:09:17
And in fact, in that sense,
Speaker:
00:09:19
you don't want models that are completely unbiased
Speaker:
00:09:22
because every time they get a new data point,
Speaker:
00:09:24
they would just leap to a new conclusion
Speaker:
00:09:26
and they wouldn't actually be learning anything.
Speaker:
00:09:28
So they learn a bit of bias,
Speaker:
00:09:29
which is actually useful.
Speaker:
00:09:31
The kind of bias that we talk about
Speaker:
00:09:32
and we think is bad,
Speaker:
00:09:33
is when we learn biases that are not useful.
Speaker:
00:09:37
We learn things that are incorrect.
Speaker:
00:09:39
Like the fact that let's say the colour of someone's skin
Speaker:
00:09:41
determines whether they're good at a particular job,
Speaker:
00:09:43
which is latently incorrect.
Speaker:
00:09:45
And so that's not useful bias.
Speaker:
00:09:46
But all that's doing,
Speaker:
00:09:48
it's not that the technology has introduced the bias.
Speaker:
00:09:51
What it's doing,
Speaker:
00:09:52
is it's forcing us to confront the fact
Speaker:
00:09:55
that sort of bias has been implicit in all of our data,
Speaker:
00:09:59
and in all of our examples and things.
Speaker:
00:10:00
So we're training the stuff off of that
Speaker:
00:10:01
and it's learning the bias
Speaker:
00:10:02
and it's forcing us to confront that and say,
Speaker:
00:10:05
hold on, perhaps we need to correct that.
Speaker:
00:10:07
But it's not the algorithm that's causing it,
Speaker:
00:10:10
it's decades and millennia and years and years
Speaker:
00:10:13
of bias that we've developed as a society.
Speaker:
00:10:16
You could even say,
Speaker:
00:10:17
it's a positive that this is triggering that conversation.
Speaker:
00:10:19
And now that it's scaling that up
Speaker:
00:10:21
and people are saying,
Speaker:
00:10:22
hold on, you're scaling up algorithms
Speaker:
00:10:23
and things that are now re-entrenching
Speaker:
00:10:25
this bias we want to get rid of.
Speaker:
00:10:26
So it's amplifying the conversation,
Speaker:
00:10:28
which, I think it's a good thing.
Speaker:
00:10:30
- [Nicky] Let's talk a little bit about scale
Speaker:
00:10:33
because the work that you've done with Red Olive
Speaker:
00:10:35
on a couple of projects,
Speaker:
00:10:36
most recently,
Speaker:
00:10:37
it gave me a sense part of it was moving to the cloud,
Speaker:
00:10:40
and I'm guessing that's really where scale comes in.
Speaker:
00:10:43
But what would you say is the big deal with cloud?
Speaker:
00:10:45
What are the big business drivers?
Speaker:
00:10:46
Where does that enable you to go next?
Speaker:
00:10:49
- [Robin] Cloud, for me, it's about,
Speaker:
00:10:51
you could slide other things in here, not just cloud.
Speaker:
00:10:53
What you're trying to do all the time
Speaker:
00:10:54
is you're continually trying to stand
Speaker:
00:10:57
on the shoulders of giants.
Speaker:
00:10:58
So how we progress as a society,
Speaker:
00:11:00
as we build on the things
Speaker:
00:11:01
that other people have done before us,
Speaker:
00:11:03
this idea of don't reinvent the wheel.
Speaker:
00:11:06
We make progress by basically,
Speaker:
00:11:08
we do more than the generation before us
Speaker:
00:11:10
by using everything that they've done
Speaker:
00:11:12
and adding to it.
Speaker:
00:11:13
And when you build things, say build software,
Speaker:
00:11:16
or build algorithms,
Speaker:
00:11:17
and that sort of thing.
Speaker:
00:11:18
What you do find,
Speaker:
00:11:19
sometimes people at the earliest stages in their careers,
Speaker:
00:11:21
they're very interested in learning
Speaker:
00:11:22
the underlying mechanics of things.
Speaker:
00:11:23
So they go off and they want to build everything
Speaker:
00:11:25
from the ground up.
Speaker:
00:11:27
And sometimes what happens is you get people more senior.
Speaker:
00:11:30
Very often, they start to change their attitude,
Speaker:
00:11:33
or they start to become, has someone else built this?
Speaker:
00:11:35
And then what they look for
Speaker:
00:11:37
is the stuff that someone else hasn't built yet,
Speaker:
00:11:38
and then they build that.
Speaker:
00:11:39
And usually it's building on all of the things
Speaker:
00:11:41
people have built before.
Speaker:
00:11:42
Cloud aside, even things like a normal software development,
Speaker:
00:11:45
almost everyone now uses frameworks
Speaker:
00:11:47
and all sorts of things that other people have built,
Speaker:
00:11:50
and they build on top of them
Speaker:
00:11:51
and they wire them together.
Speaker:
00:11:52
And cloud is one of those things.
Speaker:
00:11:54
It's saying that you don't differentiate yourself by say,
Speaker:
00:11:58
installing a database.
Speaker:
00:12:00
So that's just something
Speaker:
00:12:01
that sort of has to be done to facilitate,
Speaker:
00:12:03
let's say, the analysis or the machine learning,
Speaker:
00:12:05
or whatever work that we then do on top of that.
Speaker:
00:12:08
And so what cloud is giving you,
Speaker:
00:12:09
it's giving you the ability to offload
Speaker:
00:12:11
all that non-differentiating stuff.
Speaker:
00:12:13
And economic terms,
Speaker:
00:12:14
one of the things that certainly served us well
Speaker:
00:12:17
is this certain amount of specialisation.
Speaker:
00:12:20
And what it's allowing you to do,
Speaker:
00:12:22
is it allowing you to take the experts,
Speaker:
00:12:24
the top specialists in those areas,
Speaker:
00:12:26
in setting up that infrastructure
Speaker:
00:12:28
and making it available to services
Speaker:
00:12:29
and that sort of thing,
Speaker:
00:12:30
concentrating them in a few places,
Speaker:
00:12:32
and then reusing their efforts
Speaker:
00:12:33
and everyone else reusing their efforts.
Speaker:
00:12:35
So that for me is that the point with cloud,
Speaker:
00:12:37
is you're not trying to save money,
Speaker:
00:12:39
you're trying to increase productivity.
Speaker:
00:12:41
You're trying to not build things again,
Speaker:
00:12:43
spend all your effort building
Speaker:
00:12:45
the stuff that's going to improve your business
Speaker:
00:12:48
and your domain in some way.
Speaker:
00:12:50
Rather than installing a database,
Speaker:
00:12:52
or installing, I don't know, it could be
Speaker:
00:12:53
a streaming platform,
Speaker:
00:12:55
or whatever the case may be.
Speaker:
00:12:56
And it really does increase your productivity.
Speaker:
00:12:58
If anyone has worked with,
Speaker:
00:13:01
let's say, in data space, like a big data warehouse,
Speaker:
00:13:05
it's really easy these days with something
Speaker:
00:13:07
like say, big query.
Speaker:
00:13:08
And you just use it, right?
Speaker:
00:13:10
It's obviously still things you have to do,
Speaker:
00:13:12
but there was a time when you had to spend a lot more time.
Speaker:
00:13:15
You have to have a lot more DBAs
Speaker:
00:13:17
and things optimising your system.
Speaker:
00:13:19
And that's, if you're a business,
Speaker:
00:13:20
that's got a team of, I don't know,
Speaker:
00:13:22
you might have a team of two people.
Speaker:
00:13:23
If one of them has to do all of the maintaining the database
Speaker:
00:13:25
and optimise that all the time, that's half the capacity.
Speaker:
00:13:28
But even bigger teams,
Speaker:
00:13:29
you've got a team of a hundred.
Speaker:
00:13:30
I can bet you that if you're doing it on-premise,
Speaker:
00:13:32
the 20% of those is permanently looking after
Speaker:
00:13:35
your environment.
Speaker:
00:13:36
And that's 20% you're not spending
Speaker:
00:13:37
on some sort of business differentiator.
Speaker:
00:13:39
- [Nicky] With that, you're talking about the teams,
Speaker:
00:13:41
how do you get it so that team and culture
Speaker:
00:13:44
are tied into that.
Speaker:
00:13:44
Because I'm guessing there is a sort of,
Speaker:
00:13:46
a bit of an education
Speaker:
00:13:47
that might come along with that kind of
Speaker:
00:13:49
moving to the cloud.
Speaker:
00:13:50
And how differently you might have to work
Speaker:
00:13:52
than in a traditional sort of data and IT projects.
Speaker:
00:13:56
How important is it to have the vision
Speaker:
00:13:58
and be able to communicate that internally
Speaker:
00:14:01
to the rest of the organisation?
Speaker:
00:14:03
- [Robin] It's not really a hard sell anymore
Speaker:
00:14:05
to tell everyone that everyone's moving to the cloud
Speaker:
00:14:07
and we got to do it too.
Speaker:
00:14:08
Now, people may not always recognise why.
Speaker:
00:14:10
I think that the people who, if you've got say,
Speaker:
00:14:14
technologists and people who are working on the platforms,
Speaker:
00:14:18
for example,
Speaker:
00:14:19
they're always very willing to do it
Speaker:
00:14:22
because they know it's where everyone else is going
Speaker:
00:14:24
and they want their careers to progress.
Speaker:
00:14:26
I think executives use those increasingly,
Speaker:
00:14:28
because everyone's doing it.
Speaker:
00:14:29
It just seems that's the kind of common wisdom, right?
Speaker:
00:14:32
So that's what you do now.
Speaker:
00:14:33
I think the important thing now,
Speaker:
00:14:35
it's not so hard to sell people on the idea of doing cloud,
Speaker:
00:14:38
I think it's just probably more important
Speaker:
00:14:40
just to make everyone understand,
Speaker:
00:14:42
what is it do you want from cloud?
Speaker:
00:14:43
Why are you doing cloud?
Speaker:
00:14:44
Why is everyone doing it?
Speaker:
00:14:45
You're getting those kinds of philosophies out there
Speaker:
00:14:47
and saying, look, all we're trying to get is,
Speaker:
00:14:50
we're trying to offload all that non-differentiating effort
Speaker:
00:14:53
because that's where you will get people
Speaker:
00:14:56
trying to build things themselves,
Speaker:
00:14:58
or you'll get,
Speaker:
00:14:59
it's a hard thing sometimes to move something
Speaker:
00:15:00
when you've got a complicated on-premise platform,
Speaker:
00:15:03
for example.
Speaker:
00:15:03
And it could be data,
Speaker:
00:15:04
it could be any other software.
Speaker:
00:15:06
And it's actually quite hard to move from one to the other,
Speaker:
00:15:08
so you had a lot of this sort of
Speaker:
00:15:09
lift and shift type mentality.
Speaker:
00:15:10
And the reason that doesn't work is because
Speaker:
00:15:12
it doesn't leverage this idea that you are floating,
Speaker:
00:15:14
all the non-differentiating work.
Speaker:
00:15:16
If you're fundamentally still doing most of the instal
Speaker:
00:15:18
and maintaining all the applications.
Speaker:
00:15:20
So a lot of startups,
Speaker:
00:15:21
but even bigger organisations
Speaker:
00:15:22
are getting there with sort of saying,
Speaker:
00:15:24
okay, actually, why don't we, you know,
Speaker:
00:15:25
shift all of this,
Speaker:
00:15:26
well a lot of stuff, to something that's more
Speaker:
00:15:30
server-less.
Speaker:
00:15:31
Not the idea that it's as a service,
Speaker:
00:15:33
I just use it to just write some code
Speaker:
00:15:35
and deploy that,
Speaker:
00:15:35
rather than have to manage all the intermediate steps.
Speaker:
00:15:39
That in itself is a lot of work to get there.
Speaker:
00:15:41
So it is actually sometimes easier for people
Speaker:
00:15:44
just to think in terms of, okay,
Speaker:
00:15:44
what if I just do what I was doing,
Speaker:
00:15:46
but to do it on the cloud.
Speaker:
00:15:48
And I think that's probably where people fail
Speaker:
00:15:50
and when they end up increasing their bowl,
Speaker:
00:15:52
because this is like renting or buying a house
Speaker:
00:15:54
in some respects.
Speaker:
00:15:55
If you're just going to do exactly what you did before,
Speaker:
00:15:58
then you're better off buying than renting
Speaker:
00:16:00
because they've got a margin on top
Speaker:
00:16:03
and they're doing a little bit of work underneath the hood.
Speaker:
00:16:05
But if you don't change your attitude,
Speaker:
00:16:06
you will just spend more money
Speaker:
00:16:08
and you won't get the benefits.
Speaker:
00:16:09
So I think that's the important part
Speaker:
00:16:12
is understanding what you're trying to get
Speaker:
00:16:13
is additional productivity,
Speaker:
00:16:15
and other things as well.
Speaker:
00:16:16
There's some philosophies there that I think people,
Speaker:
00:16:18
I don't know,
Speaker:
00:16:19
maybe it's because I've been doing it so long,
Speaker:
00:16:19
it doesn't seem new to me anymore,
Speaker:
00:16:20
but I'm not sensing that I'm getting a lot of resistance
Speaker:
00:16:23
to the ideas anymore.
Speaker:
00:16:24
But I think other things people don't always fully grasp,
Speaker:
00:16:27
if they're not in that world,
Speaker:
00:16:29
emphasis changes from things like
Speaker:
00:16:31
managing availability of service
Speaker:
00:16:33
to managing costs.
Speaker:
00:16:35
And what I mean by that is,
Speaker:
00:16:36
if you use kind of properly
Speaker:
00:16:37
and use things like auto scaling and all the rest,
Speaker:
00:16:39
and you use like chaos monkeys
Speaker:
00:16:41
and these things where you're actually
Speaker:
00:16:43
deliberately trying to take out services to see,
Speaker:
00:16:46
to make sure things are resilient.
Speaker:
00:16:47
You try to take out services,
Speaker:
00:16:48
say in the middle of the day,
Speaker:
00:16:50
and then because if they are going to break,
Speaker:
00:16:52
it's better for them to break while everyone's around,
Speaker:
00:16:54
basically is a philosophy with something like that.
Speaker:
00:16:55
But that's all facilitated by this idea
Speaker:
00:16:58
that as a consumer,
Speaker:
00:16:59
you can use the cloud as if it's an infinite resource.
Speaker:
00:17:02
And that's great,
Speaker:
00:17:03
because you can build services that never go down.
Speaker:
00:17:06
So the first thing,
Speaker:
00:17:06
you've got to philosophically get into that mindset.
Speaker:
00:17:09
We live in a different world
Speaker:
00:17:11
where things can be incredibly robust.
Speaker:
00:17:11
Where they can just scale up and down
Speaker:
00:17:13
as our market scales up and down.
Speaker:
00:17:15
And that is actually a great attribute
Speaker:
00:17:18
for any organisation to have.
Speaker:
00:17:19
To be able to adapt like that to circumstances.
Speaker:
00:17:22
The flip side, of course,
Speaker:
00:17:24
is that before your costs were then finite,
Speaker:
00:17:26
in the sense you bought some stuff
Speaker:
00:17:28
and your service might go down because it's overloaded,
Speaker:
00:17:31
but your costs were still fixed.
Speaker:
00:17:33
So that's where I think
Speaker:
00:17:35
you need to pay a bit more attention to cost management
Speaker:
00:17:37
and that sort of thing.
Speaker:
00:17:38
And that can be tricky.
Speaker:
00:17:39
Like I've been in a company where, you know,
Speaker:
00:17:41
early stages of the projects,
Speaker:
00:17:42
I was like, how much is going to cost?
Speaker:
00:17:44
Well we've just started doing it,
Speaker:
00:17:45
like we've just launched.
Speaker:
00:17:46
I have to spend a few months
Speaker:
00:17:49
figuring how greedy the service is.
Speaker:
00:17:51
And you kind of work through it for a bit in the beginning.
Speaker:
00:17:53
But you do have to spend some time,
Speaker:
00:17:55
work through that,
Speaker:
00:17:56
and then spend a lot of time putting together
Speaker:
00:17:59
those mechanisms where you actively trade off
Speaker:
00:18:01
that sort of cost versus availability and performance
Speaker:
00:18:05
for your customers, and that sort of thing.
Speaker:
00:18:06
So it's things like that,
Speaker:
00:18:07
it's those nuances I think,
Speaker:
00:18:09
is understanding you're trying to be more productive.
Speaker:
00:18:11
You can build a sort of bulletproof service,
Speaker:
00:18:14
very easy to port internationally
Speaker:
00:18:16
and all sorts of things if you do it right.
Speaker:
00:18:18
But understanding all those things
Speaker:
00:18:19
and specifically designing for that,
Speaker:
00:18:21
so that you actually get the value from cloud.
Speaker:
00:18:24
I think that's the bit which
Speaker:
00:18:25
you still have to spend time banging the drum,
Speaker:
00:18:27
making sure everybody understands that
Speaker:
00:18:29
it's not so hard to convince people to go to cloud anymore,
Speaker:
00:18:31
if they're not on cloud already.
Speaker:
00:18:33
- [Nicky] Do you think that
Speaker:
00:18:35
within the development side of things,
Speaker:
00:18:37
people are braver knowing that
Speaker:
00:18:38
if you're using a cloud platform,
Speaker:
00:18:41
that it gives you more scope to try things out
Speaker:
00:18:43
and if it doesn't work.
Speaker:
00:18:44
So you're kind of doing a bit more of that CI/CD
Speaker:
00:18:48
as you're moving along, that's how you develop them.
Speaker:
00:18:51
Do you think they sort of come together almost,
Speaker:
00:18:52
they fit together a little bit like hand in a glove.
Speaker:
00:18:55
- [Robin] They do,
Speaker:
00:18:56
in theory kind of CI/CD is, I don't know,
Speaker:
00:18:59
orthogonal to the cloud,
Speaker:
00:19:00
they compliment each other.
Speaker:
00:19:01
But you could do a lot of that stuff on-premise.
Speaker:
00:19:03
It makes it easier to do it on cloud
Speaker:
00:19:04
because you can just spin up and down instances on demand
Speaker:
00:19:08
and that sort of thing,
Speaker:
00:19:09
because you can treat this thing
Speaker:
00:19:10
as almost an infinite resource.
Speaker:
00:19:11
It makes it very easy
Speaker:
00:19:12
to build these very flexible pipelines and stuff.
Speaker:
00:19:14
But at its heart,
Speaker:
00:19:14
the idea of agile is about continuous feedback.
Speaker:
00:19:17
And it seems obvious that if you wait until,
Speaker:
00:19:19
if we look at traditional, much earlier philosophies
Speaker:
00:19:22
where you had waterfall type philosophies
Speaker:
00:19:24
in development and stuff,
Speaker:
00:19:25
where people would design whole big programmes
Speaker:
00:19:27
and projects and stuff,
Speaker:
00:19:29
and then you phase them.
Speaker:
00:19:30
You had design in the front,
Speaker:
00:19:31
and then you had this development,
Speaker:
00:19:32
and then you had testing,
Speaker:
00:19:33
and you had some sort of release.
Speaker:
00:19:34
And the problem with that stuff is all the UAT,
Speaker:
00:19:36
and all of that sort of thing was very late in the day.
Speaker:
00:19:39
And the projects took so long,
Speaker:
00:19:40
that lots changed in the meantime.
Speaker:
00:19:43
And you got to the end
Speaker:
00:19:44
and then you figured out that there were all sorts of things
Speaker:
00:19:46
that needed to change.
Speaker:
00:19:47
The world had changed,
Speaker:
00:19:47
Or there's a few, or things have been misinterpreted.
Speaker:
00:19:50
And there's a lot of people in the middle.
Speaker:
00:19:51
There's a lot of messages that have to be understood.
Speaker:
00:19:53
And there's a lot of misinterpretation that happens there.
Speaker:
00:19:56
So there's nothing quite like
Speaker:
00:19:57
putting actual product in front of people saying,
Speaker:
00:19:58
did I get it wrong,
Speaker:
00:19:59
or is it still in the context of my environment and stuff?
Speaker:
00:20:03
And so that's rarely,
Speaker:
00:20:04
agile,
Speaker:
00:20:06
a lot of it basically centres around that.
Speaker:
00:20:08
It's can we get feedback very early in the process?
Speaker:
00:20:11
Can we have short cycles?
Speaker:
00:20:13
Can we have lots of sort of show and tell type things?
Speaker:
00:20:15
Can we, and it depends on the product you're building,
Speaker:
00:20:17
but can we continuously,
Speaker:
00:20:19
can we encourage that continuous feedback
Speaker:
00:20:21
and then see those problems early?
Speaker:
00:20:24
Because it's a lot cheaper to fix something
Speaker:
00:20:26
when you've written three lines of code,
Speaker:
00:20:28
than at the end of the project
Speaker:
00:20:29
when you've written a whole complex thing,
Speaker:
00:20:30
several things all depending on each other,
Speaker:
00:20:32
and you have to unwind many things.
Speaker:
00:20:34
So if you say that's the heart of agile,
Speaker:
00:20:36
then the CI/CD thing just isn't natural fit to that.
Speaker:
00:20:40
I'm saying, so talking about to the cloud,
Speaker:
00:20:41
in the sense that that is about,
Speaker:
00:20:42
CI/CD is about really just putting things out often,
Speaker:
00:20:46
and quickly,
Speaker:
00:20:47
and efficiently, right?
Speaker:
00:20:48
Automating the stuff that you can automate.
Speaker:
00:20:50
The process from checking in your code
Speaker:
00:20:53
through to automating the testing.
Speaker:
00:20:54
And of course,
Speaker:
00:20:55
there's a discipline there about writing a lot of tests
Speaker:
00:20:57
and test first development, and that sort of thing,
Speaker:
00:20:59
which makes that work, right?
Speaker:
00:21:01
It gives you the confidence to look at
Speaker:
00:21:03
these large organisations that have
Speaker:
00:21:05
tens of thousands of releases a day and stuff.
Speaker:
00:21:07
They can only do that because
Speaker:
00:21:10
the process is designed so that every time
Speaker:
00:21:12
something is checked in and merged to main,
Speaker:
00:21:14
it's automatically tested and all the rest.
Speaker:
00:21:16
It's not entirely bulletproof.
Speaker:
00:21:18
You're much more confident in an environment
Speaker:
00:21:20
where something won't go all the way into production,
Speaker:
00:21:22
unless it gets thoroughly tested,
Speaker:
00:21:24
and this whole process is really automated.
Speaker:
00:21:26
That sort of gives you a confidence.
Speaker:
00:21:28
It means you can deploy very often.
Speaker:
00:21:29
And if you can deploy the full things very often like that,
Speaker:
00:21:32
then that fits that sort of feedback cycle.
Speaker:
00:21:35
You've got lots of little changes happening all the time,
Speaker:
00:21:37
and occasionally something will go wrong,
Speaker:
00:21:40
but then it's going to be one small thing
Speaker:
00:21:41
that you can roll back and you can learn from.
Speaker:
00:21:42
And you become a very adaptable.
Speaker:
00:21:45
And I'm not using the term agile with all the baggage
Speaker:
00:21:47
of what we think agile means in software.
Speaker:
00:21:49
I just mean you've truly become an agile organisation
Speaker:
00:21:51
in the sense that
Speaker:
00:21:53
you become very sensitive to little changes
Speaker:
00:21:54
in your environment.
Speaker:
00:21:55
And you change really quickly,
Speaker:
00:21:56
and you evolve fast.
Speaker:
00:21:57
And of course,
Speaker:
00:21:58
if you can evolve quickly,
Speaker:
00:22:00
then you've got an inherent advantage over something
Speaker:
00:22:03
that doesn't evolve quickly.
Speaker:
00:22:05
So that's where I think CI/CD fits in.
Speaker:
00:22:07
Again, it enables you to evolve really quickly.
Speaker:
00:22:10
How it relates to cloud,
Speaker:
00:22:11
is cloud has enabled us to, because
Speaker:
00:22:14
cloud has also allowed us,
Speaker:
00:22:15
so in the past, if you wanted to do something new,
Speaker:
00:22:17
you would have to go and get quite a large CapEx budget.
Speaker:
00:22:21
Let's say to buy some licence,
Speaker:
00:22:22
to instal some equipment,
Speaker:
00:22:23
to do all of those things upfront.
Speaker:
00:22:25
And so obviously in cloud, now,
Speaker:
00:22:26
if you want to do something small,
Speaker:
00:22:28
you can do something small.
Speaker:
00:22:29
And that's a key point, right?
Speaker:
00:22:31
You can go and try.
Speaker:
00:22:33
I talked about Google,
Speaker:
00:22:34
let's say Amazon,
Speaker:
00:22:35
I've got something like Kinesis or something.
Speaker:
00:22:39
There are products that, I actually prefer,
Speaker:
00:22:42
some of them it's the underlying product,
Speaker:
00:22:44
so let's say like Kafka or something.
Speaker:
00:22:46
That underlying product,
Speaker:
00:22:47
I think it was a really good product.
Speaker:
00:22:49
There's other things that have come along
Speaker:
00:22:51
in the streaming space that kind of improved on it a bit.
Speaker:
00:22:54
Historically, you've had to go and buy,
Speaker:
00:22:58
you have to go to a case to spend like
Speaker:
00:23:00
$100K+ with someone to get started.
Speaker:
00:23:05
And of course you can go into Amazon,
Speaker:
00:23:06
in something like Kinesis,
Speaker:
00:23:07
and you can just start.
Speaker:
00:23:08
And you can pay like a few quid just to do a POC, basically.
Speaker:
00:23:12
So that fits into the cycle of just being able to
Speaker:
00:23:15
iteratively
Speaker:
00:23:17
do the next thing.
Speaker:
00:23:18
See if it works.
Speaker:
00:23:19
It fits this whole picture of continuous evolution
Speaker:
00:23:21
in very gradual steps.
Speaker:
00:23:23
- [Nicky] Excellent advice from Robin there.
Speaker:
00:23:25
But we're going to have to leave our conversation
Speaker:
00:23:27
and pick up on it in the next episode
Speaker:
00:23:29
where he'll be talking about the sorts of skills
Speaker:
00:23:31
people need to develop to get ahead in the industry.
Speaker:
00:23:34
It's full of interesting and useful info,
Speaker:
00:23:36
so make sure you subscribe
Speaker:
00:23:37
to the Red Olive Fibonacci podcast,
Speaker:
00:23:39
from wherever you get your podcasts
Speaker:
00:23:41
to make sure you don't miss out.
Speaker:
00:23:43
That's all for today.
Speaker:
00:23:44
Thanks for listening.
Speaker:
00:23:45
And catch up next time.
