WEBVTT
00:00:00.000 --> 00:00:01.891
Okay, so it's 2024.
00:00:01.891 --> 00:00:07.056
Everyone's talking about generative AI and how it's going to change our jobs.
00:00:07.056 --> 00:00:25.274
For technical professionals, we hear about GitHub Copilot, other sorts of coding assistants, and there are quite a lot of them, but to make sense of how we can become more productive and use in the enterprise, I'm very pleased to welcome Uli Hitzel.
00:00:25.274 --> 00:00:26.596
Welcome, Uli.
00:00:26.838 --> 00:00:28.189
Hi Jon, thank you.
00:00:28.722 --> 00:00:31.635
So Uli, we've known each other for some time.
00:00:31.635 --> 00:00:50.453
You've had a very hands-on career in some quite established companies, Microsoft, Axway, more recently Dyson, and now you have a very strong focus on AI and particularly how to apply it.
00:00:50.453 --> 00:00:59.646
Do you want to share a little bit about why you decided to focus specifically on using the Generative AI tools.
00:01:00.750 --> 00:01:01.271
Thanks, Jon.
00:01:01.271 --> 00:01:03.051
Hey, it's so great to be here.
00:01:03.051 --> 00:01:06.212
Well, why did I decide to focus on that part?
00:01:07.733 --> 00:01:09.015
I was very fortunate.
00:01:09.015 --> 00:01:12.036
I mean, both of us, teach at NUS.
00:01:12.036 --> 00:01:20.399
And about four years, I started to get into the AI topic quite intensely also because of NUS.
00:01:20.739 --> 00:01:23.230
And at some point, I realized I need to do this full time.
00:01:23.230 --> 00:01:28.492
I cannot afford just to look at some of these AI things over the weekend.
00:01:28.587 --> 00:01:31.688
It's definitely to me, it's not blockchain, it's not crypto.
00:01:31.688 --> 00:01:36.391
AI is going to change everything and I need to be part of this.
00:01:36.932 --> 00:01:46.527
We've got AI since about 60 years, and GenAI is just the newest bit that we've started to get interested in.
00:01:46.677 --> 00:01:54.641
But yeah, this is just one way to look at making smart interconnected technology work.
00:01:54.641 --> 00:01:58.003
yeah, I know that I need to be part of this and need to look at this whole time.
00:01:58.102 --> 00:02:04.807
So there are lots of different aspects to data science generally and generative AI in particular.
00:02:04.807 --> 00:02:07.128
There are people who build large language models.
00:02:07.128 --> 00:02:11.512
There are people who build the RAG interfaces into large language models.
00:02:11.512 --> 00:02:27.016
You have decided specifically not so much to design a large language model, but to figure out how to use them best in producing code, diagrams, other sorts of things.
00:02:27.016 --> 00:02:51.786
So what are the sort of key things you've learned particularly on the last, I think you really started sort of two, three years ago to look at this intensely and we've seen an explosion of different tools from GitHub Copilot, Google has a tool, AWS, what are the things that you've seen that have really caught your attention?
00:02:51.786 --> 00:03:01.183
And if I'm looking to improve my own productivity as a technology professional, where should I look first?
00:03:02.008 --> 00:03:05.801
Okay, well there was quite a few nuggets in there what you just said.
00:03:05.901 --> 00:03:09.034
First of all, you mentioned data scientists and language models, right?
00:03:09.034 --> 00:03:14.449
I think many people are using the word LLM multiple times a day.
00:03:14.449 --> 00:03:17.352
They obviously use ChatGPT many times.
00:03:18.093 --> 00:03:22.817
To me, one of the key parts that I keep emphasizing is we're talking about language models.
00:03:23.098 --> 00:03:25.580
The key is language.
00:03:25.580 --> 00:03:29.133
Then, excuse me for emphasizing this so much.
00:03:29.354 --> 00:03:30.664
I've worked with data scientists.
00:03:30.664 --> 00:03:39.057
I know technical people who, well, I for one, I don't really understand too much detail about what's going on inside the language model.
00:03:39.057 --> 00:04:00.442
And quite honestly, I don't think I have to, but there was many, many very technical people, data scientists who struggle with language models because they don't exactly know how, while they work internally, it's impossible to, I will say reverse engineer and to debug why certain inputs produce certain outputs.
00:04:00.442 --> 00:04:05.652
So for a hardcore data scientist, that's actually disturbing.
00:04:05.652 --> 00:04:10.594
So I actually think that's the first part I wanted to make, the first point I wanted to make.
00:04:10.594 --> 00:04:20.716
If you're good with language, if you are able to structure information, we always talk about prompting, asking the right questions.
00:04:20.857 --> 00:04:23.708
You can actually get a lot out of language models.
00:04:23.708 --> 00:04:28.564
So you don't have to be a technical person to get superpowers with AI.
00:04:28.564 --> 00:04:29.865
I think that's the first part.
00:04:29.865 --> 00:04:42.975
And the second part is to apply this in your context, whether you're a technical person, whether you're a writer, a legal person, a doctor, that you learn how to use a variety of tools.
00:04:42.975 --> 00:04:46.779
So it's never just one tool that solves everything for you.
00:04:46.779 --> 00:04:49.762
So you have many specialized tools.
00:04:49.762 --> 00:04:56.036
There are so many rapidly advancing number of models of tools, of platforms.
00:04:56.581 --> 00:05:02.899
There can never be only one, that you learn how to work with them to be much more productive.
00:05:02.899 --> 00:05:05.291
I know that was a very long answer, but...
00:05:05.291 --> 00:05:16.127
Yeah, well, so I can see there have been huge advances in the last couple of years and proliferation of tools and large language models.
00:05:16.127 --> 00:05:20.581
But I also see a lot of people thinking, I need to learn all of these things.
00:05:20.581 --> 00:05:23.714
I need to learn exactly how the large language model works.
00:05:23.714 --> 00:05:40.021
And that seems like you never catch up trying to do that because the companies that are building the large language models, are spending a lot of money to do that.
00:05:40.261 --> 00:05:47.031
What I'm impressed with about your approach is I'm not going to try and create my own large language model.
00:05:47.031 --> 00:05:50.307
I'm going to figure out what I can do in my job.
00:05:50.307 --> 00:06:04.021
So when it comes to coding, for example, software development, I think there's been a lot of excitement about the tools that can help accelerate the productivity of the coding part.
00:06:04.021 --> 00:06:10.442
And I've seen studies that suggest there's a 30 % productivity improvement.
00:06:10.442 --> 00:06:33.557
We can get into what are the other things that software development professionals do in a little bit, but to start with the coding part, if I want to improve my productivity with writing a Python or a Java or JavaScript program, what should I really look at starting with?
00:06:34.117 --> 00:06:36.290
OK, well, there is obviously multiple tools.
00:06:36.290 --> 00:06:37.791
You mentioned Co-Pilot already.
00:06:37.791 --> 00:06:40.504
Yes, there is Google Code Assist.
00:06:40.504 --> 00:06:43.286
There is Claude, which works in the browser.
00:06:43.286 --> 00:06:47.370
There is enterprise-grade tools like Tabnine, for example.
00:06:47.370 --> 00:06:49.732
There is a multiple variety out there.
00:06:49.732 --> 00:06:59.279
I think for me, one of the key parts is, and you just mentioned the most important part, increasing your productivity.
00:06:59.279 --> 00:07:01.473
It doesn't mean taking lazy shortcuts.
00:07:01.473 --> 00:07:07.836
It's the same like somebody using chatGPT to create an essay about something they have no clue about.
00:07:07.836 --> 00:07:08.716
It's the same here.
00:07:08.716 --> 00:07:13.653
I use these tools every day, and I use a variety of tools every day.
00:07:13.653 --> 00:07:16.019
I need to figure out what works for me.
00:07:16.240 --> 00:07:27.485
And second, my technical stack and the programming languages I deal with are shell scripting, Python, APIs, connectivity.
00:07:28.146 --> 00:07:29.435
I'm not good at JavaScript.
00:07:29.435 --> 00:07:33.987
I can use some of these tools to make impressive frontends.
00:07:34.086 --> 00:07:36.127
Sometimes they work, sometimes they don't.
00:07:36.127 --> 00:07:38.908
But I can't debug them, I can't fix them.
00:07:38.908 --> 00:07:43.310
And even with the most advanced tools, it would be a constant back and forth.
00:07:43.310 --> 00:07:53.333
So it gets especially tricky for enterprise companies, because the platforms, the tools are changing so quickly.
00:07:53.673 --> 00:07:58.646
So when you do vendor selection, privacy and how do you integrate this.
00:07:58.646 --> 00:08:03.040
By the time you finish that process, that tool may not exist anymore.
00:08:03.040 --> 00:08:05.201
It may not be in the same shape anymore.
00:08:05.201 --> 00:08:13.048
That's where something, Tabnine is offering that stability for since, I think, they're in business since 10 years already.
00:08:13.048 --> 00:08:17.872
But as an individual, I think it depends on where you are.
00:08:18.293 --> 00:08:23.208
If you are early in your career, if you are starting to be a software developer, it's...
00:08:24.168 --> 00:08:31.019
I want to say it's almost like back in school where you're not supposed to use a calculator because you need to learn basic algebra yourself.
00:08:31.019 --> 00:08:35.410
And when you're more advanced, then you pick up these tools.
00:08:35.410 --> 00:08:41.202
So many of these tools like Cursor, for example, that I use every day, it's just one example.
00:08:41.202 --> 00:08:44.062
They are very good for senior developers.
00:08:44.624 --> 00:09:00.178
If you're just getting started, would say look at Co-Pilot, which is what many people say is an auto-complete on steroids, look at something like Google Code Assist and start to explore how these work.
00:09:00.259 --> 00:09:05.264
again, the danger is that you keep producing code that you don't understand.
00:09:05.264 --> 00:09:11.779
So maybe besides the actual code production, there may be other use cases where these things are interested.
00:09:12.846 --> 00:09:19.243
So there's a saying that in order to acquire knowledge, you need to have knowledge.
00:09:19.283 --> 00:09:28.589
So you can learn a lot from books, but if you can't read, then those books aren't accessible to you.
00:09:28.589 --> 00:09:47.246
I think that this is to your point about if you're a junior developer or you don't know a language especially well, you still need to learn elements of the language in order to validate what the tool is going to create for you.
00:09:47.246 --> 00:09:50.691
Because if it doesn't work, you have to debug it.
00:09:50.691 --> 00:09:58.057
The tools may help you to debug it, but again, you still have to understand the debugging process.
00:09:58.057 --> 00:10:02.402
There are, I guess there are different levels of developer.
00:10:02.402 --> 00:10:29.693
There's the junior one who's learned the basic structure of a language and maybe done some small projects and entering the workforce, then there's the more intermediate level they may have been coding with a language for two or three or four years, and they're quite proficient at it, but don't necessarily know much about other languages or how things fit in the big picture.
00:10:29.693 --> 00:10:54.217
And then there are people who have been around for a decade or more, they may not necessarily know the newer languages in detail, but they've seen a lot of cycles of renewal of the technology landscape and they sort of understand that there are lots of moving parts to this and they're probably more concerned about the architecture of how these things fit together.
00:10:54.217 --> 00:11:21.248
But to pick up on the producing the code part and validating it as an enterprise, as a manager, I'd be concerned about if I have a team of 10 people and they're all using a different tool, or even if they're using the same tool, that tool isn't necessarily producing code that is in line with my organization's style guide, standards, best practices.
00:11:21.248 --> 00:11:44.356
So what What would you suggest to a team leader or manager who wants to improve the productivity of their team but still has to comply or is in their interest to have maintainable code and they want to encourage a certain type of practice?
00:11:45.990 --> 00:11:56.307
Okay, well, I think one of the key parts is if you have a team of developers and style guide is only one thing, right?
00:11:56.307 --> 00:12:00.951
You have things like data governance that needs to be taken care of.
00:12:00.951 --> 00:12:03.182
You have something like IP protection.
00:12:03.182 --> 00:12:14.897
What if we get sued because somebody says the code that you produce is actually part of my GitHub repository that you shouldn't have been using, vendor due diligence and those sorts of things.
00:12:14.897 --> 00:12:15.957
That's only one aspect.
00:12:15.957 --> 00:12:20.951
The other one is we all know that software developers are very opinionated people.
00:12:21.331 --> 00:12:24.104
I still use VI to this very day to code.
00:12:24.104 --> 00:12:26.115
Yes, I also use Visual Studio.
00:12:26.115 --> 00:12:29.769
Other people are using a whole bunch.
00:12:29.769 --> 00:12:31.711
They use Eclipse, whatever it is.
00:12:31.711 --> 00:12:33.261
They use PyCharm.
00:12:33.261 --> 00:12:42.009
So I think one of the key parts, as you mentioned, architecture, is to give developers a choice to use whichever tool.
00:12:42.788 --> 00:12:57.028
And I will say the graphical user interface, the editor that they use, many of those coding assistants, whether it's Gemini, whether it's Tabnine, a few others, they actually integrate into those editors as plugins.
00:12:57.289 --> 00:13:12.153
So that means while the tool that you're using may look different, the language model, which can be a privately hosted one, it doesn't have to be one on the cloud because that's one very important aspect for enterprise companies.
00:13:12.153 --> 00:13:14.095
can be the same for everyone.
00:13:14.596 --> 00:13:23.160
And you can obviously connect this with data protection tools and all the other integrations that you need as an enterprise.
00:13:23.562 --> 00:13:25.023
As a team, you'll have to figure out.
00:13:25.023 --> 00:13:30.155
mean, if we are only having Python developers, they all make use of the same language model.
00:13:30.155 --> 00:13:33.067
But imagine you have somebody who produces front-end things.
00:13:33.067 --> 00:13:36.811
In the mobile app development world, everything changes constantly.
00:13:36.811 --> 00:13:39.292
And you can't always just train these models.
00:13:39.812 --> 00:13:41.827
Well, you mentioned that earlier.
00:13:41.827 --> 00:13:47.440
I don't have the budget to build my own language models, but I can obviously fine tune and train them on code.
00:13:47.440 --> 00:13:49.860
Companies may have to do this themselves.
00:13:50.340 --> 00:13:51.541
They have to start somewhere.
00:13:51.541 --> 00:14:08.125
And I think the key part is that we are not expecting a tool that does everything for everybody, but you need to build up your own practice as a team to discover which tools are available, which tools fit into the frameworks, the policies, the preferences that developers have.
00:14:08.125 --> 00:14:11.549
Something like Python or C++ are very different.
00:14:11.549 --> 00:14:13.049
very different languages from each other.
00:14:13.049 --> 00:14:14.821
So it may not be the same model.
00:14:14.821 --> 00:14:22.013
It could be still the same platform, but you use a selection of different models based on the use case.
00:14:22.013 --> 00:14:27.294
And the other thing, and you probably didn't expect that, but I was talking about language earlier, right?
00:14:27.294 --> 00:14:31.125
I know a lot of developers who are, well, how do I say this?
00:14:31.125 --> 00:14:33.196
They are not so good at asking for help.
00:14:33.395 --> 00:14:39.481
If you really, really start from the beginning and say, I know you've all used ChatGPT.
00:14:39.481 --> 00:14:46.984
You've all done amazing stuff with data science, but can we go back to the fundamentals of LLMs, the capabilities?
00:14:46.984 --> 00:14:48.234
Can we talk about the risks?
00:14:48.234 --> 00:14:53.285
Can we talk about the exposure of data to the outside world?
00:14:53.285 --> 00:14:56.486
Can we really give you examples on how do you ask for help?
00:14:56.486 --> 00:15:07.590
How do you, you know, and you have to sometimes when you are, when you have a question in your head and you ask your colleagues sitting next to you and you said, you know what, forget about it.
00:15:07.590 --> 00:15:09.129
I already have my answer.
00:15:09.423 --> 00:15:18.418
When you're forced to structure your thoughts, when you're forced to articulate out a question, that may already be 50%.
00:15:18.418 --> 00:15:25.783
And then you can use AI maybe not to produce code, but to produce test data, to say there was some really old code from 20 years ago.
00:15:25.783 --> 00:15:27.825
I have no idea how to understand it.
00:15:27.825 --> 00:15:30.346
There was some code that I wrote five years ago.
00:15:30.346 --> 00:15:31.886
Can you make a markdown?
00:15:31.886 --> 00:15:37.740
Can you create a document, piece of API swagger documentation for me?
00:15:37.740 --> 00:15:40.291
Or I'm supposed to pick up this new language.
00:15:40.351 --> 00:15:41.283
I'm so embarrassed.
00:15:41.283 --> 00:15:44.395
I wouldn't be able to tell a human, but I can tell a machine.
00:15:44.395 --> 00:15:45.547
I don't know what this code does.
00:15:45.547 --> 00:15:50.610
And you have a chat with this thing, and ultimately, you come up much better.
00:15:50.610 --> 00:15:55.865
But it's far away from this whole, you you just put in some requirements into a machine and produces code.
00:15:55.865 --> 00:16:01.591
It's really that, and that's why I think co-pilot is actually a beautiful term.
00:16:01.591 --> 00:16:06.163
So whoever came up with that brand, it's very good.
00:16:06.304 --> 00:16:09.187
So I think what you described is probably the second step.
00:16:09.187 --> 00:16:27.657
I mentioned in order to acquire knowledge you need to have knowledge, you need to know how to read, but you made the point, you need to know how to ask questions, you need to think about and ask for help and frame what your thinking is in order to get a result.
00:16:28.437 --> 00:16:31.336
So that's...
00:16:31.336 --> 00:16:33.647
That's probably the second step there.
00:16:33.868 --> 00:16:50.897
What we talked about so far is about the coding part, but many people will say, well, actually software developers, even people with software development in their title only spend about 10, 20 % of their time actually coding and the rest of the time they're doing other things.
00:16:50.897 --> 00:17:06.592
there's even before they start coding, there's the architecture and the design and After they've done some coding, there's the debugging part and then the testing and which has multiple levels.
00:17:06.592 --> 00:17:12.515
There's unit testing, system testing, user acceptance testing before it goes into production.
00:17:12.515 --> 00:17:24.153
where can the, what did enterprises need to think about when they look at the tools across those different stages of the software development life cycle.
00:17:25.392 --> 00:17:36.156
Okay, well, so the whole ideation part, building proof of concepts, building architectural blueprints, helping architects to make decisions.
00:17:36.156 --> 00:17:46.480
I think this has become so much part of the general knowledge of the language models that you may not even need those specialized tools for this anymore.
00:17:46.721 --> 00:17:54.213
Probably even with ChatGPT, the free version, definitely with Claude.ai in the browser.
00:17:54.375 --> 00:18:06.203
I use these things on my phone actually, where you can, even when you walk to the bus, you can have a conversation using voice and you say, in fact, you know, I'm looking to build a distributed system and I have this headache.
00:18:06.203 --> 00:18:07.996
How do these things communicate with each other?
00:18:07.996 --> 00:18:09.467
How do I achieve the scale?
00:18:09.467 --> 00:18:15.881
And at some point you get ideas to use a pops up, to use asynchronous messaging frameworks.
00:18:15.881 --> 00:18:23.230
And so this whole ideation part that you would typically do in your head or you have meetings the next day with people.
00:18:23.230 --> 00:18:25.882
putting this on the whiteboard.
00:18:25.882 --> 00:18:36.150
You still have to do this obviously, but I mean the sort of prep work that you can do just by yourself, just to throw ideas around and to validate certain aspects.
00:18:36.150 --> 00:18:41.093
And if you don't stop those models, they will even start to produce code which you don't want.
00:18:43.015 --> 00:18:44.318
it comes back to this whole...
00:18:44.318 --> 00:19:00.038
found for example, that when I'm writing something, an article or something else that using chat GPT or something else can help me get over the blank page problem that I'm not sure where to start.
00:19:00.038 --> 00:19:10.055
So I'll ask some questions and it won't produce what I, a final product, but it'll give me a sort of a rough draft with some.
00:19:10.058 --> 00:19:16.154
with an outline and then I realized, okay, it's missing that bit and I can improve that.
00:19:16.154 --> 00:19:48.732
I found you can also ask these models, well, list the major components in a data pipeline and it'll produce some sort of answer and then you can start to become more specific and make sure, okay, it's going to be real-time event-driven, what are the alternatives to Apache Kafka and it can start to give you some greater detail and then you start to use your own expert knowledge to drill down a little further than that.
00:19:48.973 --> 00:20:01.214
The challenge I've seen so far is not great for diagramming but you showed me some things earlier that show you can even produce a sequence diagram with the right sort of plugin.
00:20:02.234 --> 00:20:13.250
Yes, well, so I was going to say, yes, it's actually fantastic to do these exercises with, well, not with the language models, but with these, I'm always very specific.
00:20:13.250 --> 00:20:18.201
It's these things build around language models, like chat GPT, Cloud.
00:20:18.201 --> 00:20:20.982
People actually don't interact with the language model itself.
00:20:21.123 --> 00:20:23.483
It's a complicated chatbot.
00:20:23.483 --> 00:20:36.473
But anyway, the point what I wanted to make is, Yes, it delivers you this raw material and there is a lot of, if I use a technical term, lot of garbage in there sometimes, something that absolutely doesn't make sense.
00:20:36.473 --> 00:20:47.097
you have to know, again, absolutely, it gives me 80 % and there is some things that I have to fix and that are missing.
00:20:47.097 --> 00:20:50.378
But I think that's one of the key bits.
00:20:50.640 --> 00:20:52.140
Now, it...
00:20:52.601 --> 00:20:56.592
Sorry, you just have to remind me, at the end, what was your question?
00:20:56.623 --> 00:21:02.988
So we're talking about the whole system development life cycle, and we started with architecture, but then...
00:21:03.519 --> 00:21:03.880
yes.
00:21:03.880 --> 00:21:05.201
Yes, I have it.
00:21:06.461 --> 00:21:12.063
Ultimately, even music these days, Spotify, MP3s, are digital.
00:21:13.084 --> 00:21:22.667
Diagrams that we draw in Draw.io, in diagram.net, even SVG files are structured bits.
00:21:22.667 --> 00:21:29.904
And if you can teach a language model how one is constructed, you can also teach it to say, give me a...
00:21:29.904 --> 00:21:34.199
a sequence diagram, give me a flow diagram using this format.
00:21:34.199 --> 00:21:41.325
And you will have something that renders nicely in the browser so humans can see it, but it's also structured almost like code.
00:21:41.325 --> 00:21:44.787
So you can also create a version of this, put it in Git.
00:21:45.248 --> 00:21:50.551
And you can basically use language models to produce diagrams.
00:21:50.813 --> 00:21:55.857
I've not seen it much yet for architecture diagrams, but you know what, I'm going to try this out.
00:21:55.857 --> 00:21:57.965
I'm sure somebody has done this already.
00:21:57.965 --> 00:21:58.747
Yeah.
00:21:58.747 --> 00:21:59.157
Okay.
00:21:59.157 --> 00:22:04.074
So we go from architecture and then we already talked about the coding part.
00:22:04.154 --> 00:22:07.618
So then we get into testing.
00:22:07.618 --> 00:22:14.430
Now where, where can the tools help us with, with the testing part?
00:22:16.096 --> 00:22:26.291
Some of the tools that we mentioned like CodeAssist, like Cursor, like Tabnine, they can help you write test cases in certain formats.
00:22:26.291 --> 00:22:27.199
That's one thing.
00:22:27.199 --> 00:22:33.032
is always a very, I wouldn't say always, some people really love writing test cases.
00:22:33.032 --> 00:22:37.713
I can't say I'm enthusiastic.
00:22:38.315 --> 00:22:54.487
Writing one or two is okay, but writing an entire set that covers lots of different possibilities is quite, it gets repetitive and we don't, like doing repetitive things as humans.
00:22:54.487 --> 00:22:57.669
if we can get a machine to do it, that's great.
00:22:58.390 --> 00:23:03.855
But to your point, well, we discussed how you need to validate.
00:23:03.855 --> 00:23:11.740
How do you validate that what the test cases the tool has produced are going to be fit for purpose?
00:23:13.130 --> 00:23:17.990
Well, like I said earlier, you'll still have to understand.
00:23:17.990 --> 00:23:24.373
Sorry, I just wanted to say the number one thing that developers need is documentation.
00:23:24.373 --> 00:23:29.255
And the number one thing that developers don't enjoy doing is writing documentation.
00:23:29.255 --> 00:23:31.434
So it's similar here.
00:23:33.096 --> 00:23:36.336
When we write test cases, they can be for a manual tester.
00:23:36.336 --> 00:23:39.698
And I've worked with people who do manual QA.
00:23:40.342 --> 00:23:41.972
It's unfortunately very repetitive.
00:23:41.972 --> 00:23:52.644
You may have seen the Claude folks have just released a few days ago this thing where in fact the language model can look at the screen, click it buttons, which raises all kinds of issues.
00:23:52.644 --> 00:23:54.385
Is it going to be able to solve captures?
00:23:54.385 --> 00:23:58.987
Is it able to do bank account transactions on my behalf?
00:23:59.386 --> 00:24:00.686
Do we really want this?
00:24:00.686 --> 00:24:06.828
But if we do step by step, there is still a lot of need for manual QA testing.
00:24:06.890 --> 00:24:10.971
These things are unfortunately repetitive, so it needs a specific character.
00:24:10.971 --> 00:24:18.075
I've worked, I've hired with these sorts of people, and they are able to look at these things step by step.
00:24:18.075 --> 00:24:28.048
And with a language model, with a proper tool that understands your code, it can produce certain test cases for manual tests.
00:24:28.048 --> 00:24:29.509
And they are repetitive.
00:24:29.509 --> 00:24:33.721
They look a bit boring, but sometimes that's something you have to do.
00:24:33.721 --> 00:24:38.784
But obviously, there is ways to say which parts of these things can be automated.
00:24:38.784 --> 00:24:46.846
especially if they are front-end related, there may be some Selenium pieces that you can use.
00:24:46.866 --> 00:24:51.268
There may be some legal requirements why certain things cannot be tested automatically.
00:24:51.268 --> 00:24:54.349
especially, let's say we're testing APIs.
00:24:54.490 --> 00:25:02.574
At API Days a few years ago, I've already introduced a Python framework that you can...
00:25:03.241 --> 00:25:07.782
use to do testing, semantic testing on APIs.
00:25:07.782 --> 00:25:09.365
It's not just, the endpoint available?
00:25:09.365 --> 00:25:11.365
But I'm expecting a JSON payload.
00:25:11.365 --> 00:25:13.567
If I put this in, I'm expecting an error.
00:25:13.567 --> 00:25:14.977
So you can actually script this.
00:25:14.977 --> 00:25:24.330
And if you teach this to a language model, and if you build an agent around it, you can also use this to run automated tests, happy path.
00:25:25.432 --> 00:25:29.173
For example, there's many opportunities.
00:25:29.193 --> 00:25:30.013
Yeah.
00:25:30.806 --> 00:25:52.835
so align with the testing part is also security vulnerability scanning there are I guess there have been for some time automated tools for doing this for doing security vulnerability scanning but perhaps this space is also changing because that security space is constantly changing.
00:25:52.835 --> 00:25:55.207
There are new threats that come up all the time.
00:25:55.367 --> 00:26:05.616
When it comes to deploying to production, what's your sort of level of faith for how you can get the AI to deploy to production?
00:26:05.616 --> 00:26:10.560
Or is that something that you always want to have someone with a finger on the button before it goes?
00:26:10.721 --> 00:26:14.102
you're talking about CI, CD done by AI?
00:26:14.818 --> 00:26:16.220
Well, I mean, it depends.
00:26:16.220 --> 00:26:18.851
And like I said, AI is not language models.
00:26:18.851 --> 00:26:20.401
Language models isn't AI.
00:26:20.401 --> 00:26:22.741
We have a lot of smart tools.
00:26:23.382 --> 00:26:32.726
We don't have to talk about Jenkins or so on, but just GitHub Actions are a wonderful, mostly reliable tool to do automatic deployment.
00:26:33.266 --> 00:26:41.397
of course, you can have certain steps inside those bits being taken care of by AI, that's not a problem.
00:26:41.397 --> 00:26:51.700
think generally what I've seen companies saying that we know nothing about AI and can we deploy something in production in three weeks from now.
00:26:51.840 --> 00:26:55.682
It reminds me of what we had in the early days of cloud computing.
00:26:55.882 --> 00:27:03.463
So when you look at cloud adoption, you don't put your production database of your core system into the cloud within a few months.
00:27:03.463 --> 00:27:05.164
You start...
00:27:06.217 --> 00:27:12.099
while training your AI muscles and building up that knowledge and see how much you can rely on this thing.
00:27:12.099 --> 00:27:31.827
And ultimately, I mean, it's similar to building customer facing chatbots with no AI experience versus making your existing customer service agents, which are human, more successful with AI so they can hang up the phone in five minutes instead of 30 minutes.
00:27:31.827 --> 00:27:34.490
And I would always look at these things like...
00:27:34.490 --> 00:27:45.146
start with something non-critical, then even if it is critical, something that's not customer-facing, something that you can catch.
00:27:45.712 --> 00:27:49.233
Well, I would say it's this whole autonomous part that people always get.
00:27:49.233 --> 00:27:51.354
mean, we have incredible demos.
00:27:51.595 --> 00:27:59.297
They're very impressive, but they are not as reliable as the, everybody talks about the triple nine, the five nines and availability.
00:27:59.457 --> 00:28:06.118
We now have these alien intelligent bits that we use and I can't debug the code.
00:28:06.118 --> 00:28:13.000
So sometimes even if I engage with these things and I say, Hey, you put, I gave you this one and you produce this.
00:28:13.000 --> 00:28:13.810
doesn't make any sense.
00:28:13.810 --> 00:28:14.971
Why did you do that?
00:28:14.971 --> 00:28:18.463
And I can't look at the code and say, yes, it's because of the switch and such.
00:28:18.463 --> 00:28:22.728
It'll say, I'm sorry, you're right, I shouldn't have done this, I won't do this again.
00:28:22.728 --> 00:28:28.053
And I can make it very restrictive, but then it's not as flexible as I want.
00:28:28.053 --> 00:28:29.663
it takes a lot of experimentation.
00:28:29.663 --> 00:28:42.744
And it's really just setting the right expectations with senior people, with business people, that it's not a switch on download and you have, you know...
00:28:42.845 --> 00:28:45.226
90 % cost savings within weeks.
00:28:45.226 --> 00:28:57.490
It's a journey and you need to build up your own team of cross-functional players who figure out how to deal with this as a company, as a team, as an organization.
00:28:58.490 --> 00:29:06.393
But most of all, you don't wait for something to be ready because things are never going to be, you you have to start now.
00:29:06.872 --> 00:29:07.771
Yeah.
00:29:07.771 --> 00:29:10.432
Well, thanks for sharing that, Uli.
00:29:10.432 --> 00:29:24.012
I appreciate you sharing your own journey and then the different elements across the system development life cycle where tools can help us, but where we also need to watch and be careful.
00:29:24.012 --> 00:29:36.231
And I like your analogy about building the muscles where you need to get into training and realize that it's not a sprint, it's a marathon.
00:29:36.231 --> 00:29:38.528
So thanks very much, really, I appreciate it.
00:00:00.000 --> 00:00:01.891
Okay, so it's 2024.
00:00:01.891 --> 00:00:07.056
Everyone's talking about generative AI and how it's going to change our jobs.
00:00:07.056 --> 00:00:25.274
For technical professionals, we hear about GitHub Copilot, other sorts of coding assistants, and there are quite a lot of them, but to make sense of how we can become more productive and use in the enterprise, I'm very pleased to welcome Uli Hitzel.
00:00:25.274 --> 00:00:26.596
Welcome, Uli.
00:00:26.838 --> 00:00:28.189
Hi Jon, thank you.
00:00:28.722 --> 00:00:31.635
So Uli, we've known each other for some time.
00:00:31.635 --> 00:00:50.453
You've had a very hands-on career in some quite established companies, Microsoft, Axway, more recently Dyson, and now you have a very strong focus on AI and particularly how to apply it.
00:00:50.453 --> 00:00:59.646
Do you want to share a little bit about why you decided to focus specifically on using the Generative AI tools.
00:01:00.750 --> 00:01:01.271
Thanks, Jon.
00:01:01.271 --> 00:01:03.051
Hey, it's so great to be here.
00:01:03.051 --> 00:01:06.212
Well, why did I decide to focus on that part?
00:01:07.733 --> 00:01:09.015
I was very fortunate.
00:01:09.015 --> 00:01:12.036
I mean, both of us, teach at NUS.
00:01:12.036 --> 00:01:20.399
And about four years, I started to get into the AI topic quite intensely also because of NUS.
00:01:20.739 --> 00:01:23.230
And at some point, I realized I need to do this full time.
00:01:23.230 --> 00:01:28.492
I cannot afford just to look at some of these AI things over the weekend.
00:01:28.587 --> 00:01:31.688
It's definitely to me, it's not blockchain, it's not crypto.
00:01:31.688 --> 00:01:36.391
AI is going to change everything and I need to be part of this.
00:01:36.932 --> 00:01:46.527
We've got AI since about 60 years, and GenAI is just the newest bit that we've started to get interested in.
00:01:46.677 --> 00:01:54.641
But yeah, this is just one way to look at making smart interconnected technology work.
00:01:54.641 --> 00:01:58.003
yeah, I know that I need to be part of this and need to look at this whole time.
00:01:58.102 --> 00:02:04.807
So there are lots of different aspects to data science generally and generative AI in particular.
00:02:04.807 --> 00:02:07.128
There are people who build large language models.
00:02:07.128 --> 00:02:11.512
There are people who build the RAG interfaces into large language models.
00:02:11.512 --> 00:02:27.016
You have decided specifically not so much to design a large language model, but to figure out how to use them best in producing code, diagrams, other sorts of things.
00:02:27.016 --> 00:02:51.786
So what are the sort of key things you've learned particularly on the last, I think you really started sort of two, three years ago to look at this intensely and we've seen an explosion of different tools from GitHub Copilot, Google has a tool, AWS, what are the things that you've seen that have really caught your attention?
00:02:51.786 --> 00:03:01.183
And if I'm looking to improve my own productivity as a technology professional, where should I look first?
00:03:02.008 --> 00:03:05.801
Okay, well there was quite a few nuggets in there what you just said.
00:03:05.901 --> 00:03:09.034
First of all, you mentioned data scientists and language models, right?
00:03:09.034 --> 00:03:14.449
I think many people are using the word LLM multiple times a day.
00:03:14.449 --> 00:03:17.352
They obviously use ChatGPT many times.
00:03:18.093 --> 00:03:22.817
To me, one of the key parts that I keep emphasizing is we're talking about language models.
00:03:23.098 --> 00:03:25.580
The key is language.
00:03:25.580 --> 00:03:29.133
Then, excuse me for emphasizing this so much.
00:03:29.354 --> 00:03:30.664
I've worked with data scientists.
00:03:30.664 --> 00:03:39.057
I know technical people who, well, I for one, I don't really understand too much detail about what's going on inside the language model.
00:03:39.057 --> 00:04:00.442
And quite honestly, I don't think I have to, but there was many, many very technical people, data scientists who struggle with language models because they don't exactly know how, while they work internally, it's impossible to, I will say reverse engineer and to debug why certain inputs produce certain outputs.
00:04:00.442 --> 00:04:05.652
So for a hardcore data scientist, that's actually disturbing.
00:04:05.652 --> 00:04:10.594
So I actually think that's the first part I wanted to make, the first point I wanted to make.
00:04:10.594 --> 00:04:20.716
If you're good with language, if you are able to structure information, we always talk about prompting, asking the right questions.
00:04:20.857 --> 00:04:23.708
You can actually get a lot out of language models.
00:04:23.708 --> 00:04:28.564
So you don't have to be a technical person to get superpowers with AI.
00:04:28.564 --> 00:04:29.865
I think that's the first part.
00:04:29.865 --> 00:04:42.975
And the second part is to apply this in your context, whether you're a technical person, whether you're a writer, a legal person, a doctor, that you learn how to use a variety of tools.
00:04:42.975 --> 00:04:46.779
So it's never just one tool that solves everything for you.
00:04:46.779 --> 00:04:49.762
So you have many specialized tools.
00:04:49.762 --> 00:04:56.036
There are so many rapidly advancing number of models of tools, of platforms.
00:04:56.581 --> 00:05:02.899
There can never be only one, that you learn how to work with them to be much more productive.
00:05:02.899 --> 00:05:05.291
I know that was a very long answer, but...
00:05:05.291 --> 00:05:16.127
Yeah, well, so I can see there have been huge advances in the last couple of years and proliferation of tools and large language models.
00:05:16.127 --> 00:05:20.581
But I also see a lot of people thinking, I need to learn all of these things.
00:05:20.581 --> 00:05:23.714
I need to learn exactly how the large language model works.
00:05:23.714 --> 00:05:40.021
And that seems like you never catch up trying to do that because the companies that are building the large language models, are spending a lot of money to do that.
00:05:40.261 --> 00:05:47.031
What I'm impressed with about your approach is I'm not going to try and create my own large language model.
00:05:47.031 --> 00:05:50.307
I'm going to figure out what I can do in my job.
00:05:50.307 --> 00:06:04.021
So when it comes to coding, for example, software development, I think there's been a lot of excitement about the tools that can help accelerate the productivity of the coding part.
00:06:04.021 --> 00:06:10.442
And I've seen studies that suggest there's a 30 % productivity improvement.
00:06:10.442 --> 00:06:33.557
We can get into what are the other things that software development professionals do in a little bit, but to start with the coding part, if I want to improve my productivity with writing a Python or a Java or JavaScript program, what should I really look at starting with?
00:06:34.117 --> 00:06:36.290
OK, well, there is obviously multiple tools.
00:06:36.290 --> 00:06:37.791
You mentioned Co-Pilot already.
00:06:37.791 --> 00:06:40.504
Yes, there is Google Code Assist.
00:06:40.504 --> 00:06:43.286
There is Claude, which works in the browser.
00:06:43.286 --> 00:06:47.370
There is enterprise-grade tools like Tabnine, for example.
00:06:47.370 --> 00:06:49.732
There is a multiple variety out there.
00:06:49.732 --> 00:06:59.279
I think for me, one of the key parts is, and you just mentioned the most important part, increasing your productivity.
00:06:59.279 --> 00:07:01.473
It doesn't mean taking lazy shortcuts.
00:07:01.473 --> 00:07:07.836
It's the same like somebody using chatGPT to create an essay about something they have no clue about.
00:07:07.836 --> 00:07:08.716
It's the same here.
00:07:08.716 --> 00:07:13.653
I use these tools every day, and I use a variety of tools every day.
00:07:13.653 --> 00:07:16.019
I need to figure out what works for me.
00:07:16.240 --> 00:07:27.485
And second, my technical stack and the programming languages I deal with are shell scripting, Python, APIs, connectivity.
00:07:28.146 --> 00:07:29.435
I'm not good at JavaScript.
00:07:29.435 --> 00:07:33.987
I can use some of these tools to make impressive frontends.
00:07:34.086 --> 00:07:36.127
Sometimes they work, sometimes they don't.
00:07:36.127 --> 00:07:38.908
But I can't debug them, I can't fix them.
00:07:38.908 --> 00:07:43.310
And even with the most advanced tools, it would be a constant back and forth.
00:07:43.310 --> 00:07:53.333
So it gets especially tricky for enterprise companies, because the platforms, the tools are changing so quickly.
00:07:53.673 --> 00:07:58.646
So when you do vendor selection, privacy and how do you integrate this.
00:07:58.646 --> 00:08:03.040
By the time you finish that process, that tool may not exist anymore.
00:08:03.040 --> 00:08:05.201
It may not be in the same shape anymore.
00:08:05.201 --> 00:08:13.048
That's where something, Tabnine is offering that stability for since, I think, they're in business since 10 years already.
00:08:13.048 --> 00:08:17.872
But as an individual, I think it depends on where you are.
00:08:18.293 --> 00:08:23.208
If you are early in your career, if you are starting to be a software developer, it's...
00:08:24.168 --> 00:08:31.019
I want to say it's almost like back in school where you're not supposed to use a calculator because you need to learn basic algebra yourself.
00:08:31.019 --> 00:08:35.410
And when you're more advanced, then you pick up these tools.
00:08:35.410 --> 00:08:41.202
So many of these tools like Cursor, for example, that I use every day, it's just one example.
00:08:41.202 --> 00:08:44.062
They are very good for senior developers.
00:08:44.624 --> 00:09:00.178
If you're just getting started, would say look at Co-Pilot, which is what many people say is an auto-complete on steroids, look at something like Google Code Assist and start to explore how these work.
00:09:00.259 --> 00:09:05.264
again, the danger is that you keep producing code that you don't understand.
00:09:05.264 --> 00:09:11.779
So maybe besides the actual code production, there may be other use cases where these things are interested.
00:09:12.846 --> 00:09:19.243
So there's a saying that in order to acquire knowledge, you need to have knowledge.
00:09:19.283 --> 00:09:28.589
So you can learn a lot from books, but if you can't read, then those books aren't accessible to you.
00:09:28.589 --> 00:09:47.246
I think that this is to your point about if you're a junior developer or you don't know a language especially well, you still need to learn elements of the language in order to validate what the tool is going to create for you.
00:09:47.246 --> 00:09:50.691
Because if it doesn't work, you have to debug it.
00:09:50.691 --> 00:09:58.057
The tools may help you to debug it, but again, you still have to understand the debugging process.
00:09:58.057 --> 00:10:02.402
There are, I guess there are different levels of developer.
00:10:02.402 --> 00:10:29.693
There's the junior one who's learned the basic structure of a language and maybe done some small projects and entering the workforce, then there's the more intermediate level they may have been coding with a language for two or three or four years, and they're quite proficient at it, but don't necessarily know much about other languages or how things fit in the big picture.
00:10:29.693 --> 00:10:54.217
And then there are people who have been around for a decade or more, they may not necessarily know the newer languages in detail, but they've seen a lot of cycles of renewal of the technology landscape and they sort of understand that there are lots of moving parts to this and they're probably more concerned about the architecture of how these things fit together.
00:10:54.217 --> 00:11:21.248
But to pick up on the producing the code part and validating it as an enterprise, as a manager, I'd be concerned about if I have a team of 10 people and they're all using a different tool, or even if they're using the same tool, that tool isn't necessarily producing code that is in line with my organization's style guide, standards, best practices.
00:11:21.248 --> 00:11:44.356
So what What would you suggest to a team leader or manager who wants to improve the productivity of their team but still has to comply or is in their interest to have maintainable code and they want to encourage a certain type of practice?
00:11:45.990 --> 00:11:56.307
Okay, well, I think one of the key parts is if you have a team of developers and style guide is only one thing, right?
00:11:56.307 --> 00:12:00.951
You have things like data governance that needs to be taken care of.
00:12:00.951 --> 00:12:03.182
You have something like IP protection.
00:12:03.182 --> 00:12:14.897
What if we get sued because somebody says the code that you produce is actually part of my GitHub repository that you shouldn't have been using, vendor due diligence and those sorts of things.
00:12:14.897 --> 00:12:15.957
That's only one aspect.
00:12:15.957 --> 00:12:20.951
The other one is we all know that software developers are very opinionated people.
00:12:21.331 --> 00:12:24.104
I still use VI to this very day to code.
00:12:24.104 --> 00:12:26.115
Yes, I also use Visual Studio.
00:12:26.115 --> 00:12:29.769
Other people are using a whole bunch.
00:12:29.769 --> 00:12:31.711
They use Eclipse, whatever it is.
00:12:31.711 --> 00:12:33.261
They use PyCharm.
00:12:33.261 --> 00:12:42.009
So I think one of the key parts, as you mentioned, architecture, is to give developers a choice to use whichever tool.
00:12:42.788 --> 00:12:57.028
And I will say the graphical user interface, the editor that they use, many of those coding assistants, whether it's Gemini, whether it's Tabnine, a few others, they actually integrate into those editors as plugins.
00:12:57.289 --> 00:13:12.153
So that means while the tool that you're using may look different, the language model, which can be a privately hosted one, it doesn't have to be one on the cloud because that's one very important aspect for enterprise companies.
00:13:12.153 --> 00:13:14.095
can be the same for everyone.
00:13:14.596 --> 00:13:23.160
And you can obviously connect this with data protection tools and all the other integrations that you need as an enterprise.
00:13:23.562 --> 00:13:25.023
As a team, you'll have to figure out.
00:13:25.023 --> 00:13:30.155
mean, if we are only having Python developers, they all make use of the same language model.
00:13:30.155 --> 00:13:33.067
But imagine you have somebody who produces front-end things.
00:13:33.067 --> 00:13:36.811
In the mobile app development world, everything changes constantly.
00:13:36.811 --> 00:13:39.292
And you can't always just train these models.
00:13:39.812 --> 00:13:41.827
Well, you mentioned that earlier.
00:13:41.827 --> 00:13:47.440
I don't have the budget to build my own language models, but I can obviously fine tune and train them on code.
00:13:47.440 --> 00:13:49.860
Companies may have to do this themselves.
00:13:50.340 --> 00:13:51.541
They have to start somewhere.
00:13:51.541 --> 00:14:08.125
And I think the key part is that we are not expecting a tool that does everything for everybody, but you need to build up your own practice as a team to discover which tools are available, which tools fit into the frameworks, the policies, the preferences that developers have.
00:14:08.125 --> 00:14:11.549
Something like Python or C++ are very different.
00:14:11.549 --> 00:14:13.049
very different languages from each other.
00:14:13.049 --> 00:14:14.821
So it may not be the same model.
00:14:14.821 --> 00:14:22.013
It could be still the same platform, but you use a selection of different models based on the use case.
00:14:22.013 --> 00:14:27.294
And the other thing, and you probably didn't expect that, but I was talking about language earlier, right?
00:14:27.294 --> 00:14:31.125
I know a lot of developers who are, well, how do I say this?
00:14:31.125 --> 00:14:33.196
They are not so good at asking for help.
00:14:33.395 --> 00:14:39.481
If you really, really start from the beginning and say, I know you've all used ChatGPT.
00:14:39.481 --> 00:14:46.984
You've all done amazing stuff with data science, but can we go back to the fundamentals of LLMs, the capabilities?
00:14:46.984 --> 00:14:48.234
Can we talk about the risks?
00:14:48.234 --> 00:14:53.285
Can we talk about the exposure of data to the outside world?
00:14:53.285 --> 00:14:56.486
Can we really give you examples on how do you ask for help?
00:14:56.486 --> 00:15:07.590
How do you, you know, and you have to sometimes when you are, when you have a question in your head and you ask your colleagues sitting next to you and you said, you know what, forget about it.
00:15:07.590 --> 00:15:09.129
I already have my answer.
00:15:09.423 --> 00:15:18.418
When you're forced to structure your thoughts, when you're forced to articulate out a question, that may already be 50%.
00:15:18.418 --> 00:15:25.783
And then you can use AI maybe not to produce code, but to produce test data, to say there was some really old code from 20 years ago.
00:15:25.783 --> 00:15:27.825
I have no idea how to understand it.
00:15:27.825 --> 00:15:30.346
There was some code that I wrote five years ago.
00:15:30.346 --> 00:15:31.886
Can you make a markdown?
00:15:31.886 --> 00:15:37.740
Can you create a document, piece of API swagger documentation for me?
00:15:37.740 --> 00:15:40.291
Or I'm supposed to pick up this new language.
00:15:40.351 --> 00:15:41.283
I'm so embarrassed.
00:15:41.283 --> 00:15:44.395
I wouldn't be able to tell a human, but I can tell a machine.
00:15:44.395 --> 00:15:45.547
I don't know what this code does.
00:15:45.547 --> 00:15:50.610
And you have a chat with this thing, and ultimately, you come up much better.
00:15:50.610 --> 00:15:55.865
But it's far away from this whole, you you just put in some requirements into a machine and produces code.
00:15:55.865 --> 00:16:01.591
It's really that, and that's why I think co-pilot is actually a beautiful term.
00:16:01.591 --> 00:16:06.163
So whoever came up with that brand, it's very good.
00:16:06.304 --> 00:16:09.187
So I think what you described is probably the second step.
00:16:09.187 --> 00:16:27.657
I mentioned in order to acquire knowledge you need to have knowledge, you need to know how to read, but you made the point, you need to know how to ask questions, you need to think about and ask for help and frame what your thinking is in order to get a result.
00:16:28.437 --> 00:16:31.336
So that's...
00:16:31.336 --> 00:16:33.647
That's probably the second step there.
00:16:33.868 --> 00:16:50.897
What we talked about so far is about the coding part, but many people will say, well, actually software developers, even people with software development in their title only spend about 10, 20 % of their time actually coding and the rest of the time they're doing other things.
00:16:50.897 --> 00:17:06.592
there's even before they start coding, there's the architecture and the design and After they've done some coding, there's the debugging part and then the testing and which has multiple levels.
00:17:06.592 --> 00:17:12.515
There's unit testing, system testing, user acceptance testing before it goes into production.
00:17:12.515 --> 00:17:24.153
where can the, what did enterprises need to think about when they look at the tools across those different stages of the software development life cycle.
00:17:25.392 --> 00:17:36.156
Okay, well, so the whole ideation part, building proof of concepts, building architectural blueprints, helping architects to make decisions.
00:17:36.156 --> 00:17:46.480
I think this has become so much part of the general knowledge of the language models that you may not even need those specialized tools for this anymore.
00:17:46.721 --> 00:17:54.213
Probably even with ChatGPT, the free version, definitely with Claude.ai in the browser.
00:17:54.375 --> 00:18:06.203
I use these things on my phone actually, where you can, even when you walk to the bus, you can have a conversation using voice and you say, in fact, you know, I'm looking to build a distributed system and I have this headache.
00:18:06.203 --> 00:18:07.996
How do these things communicate with each other?
00:18:07.996 --> 00:18:09.467
How do I achieve the scale?
00:18:09.467 --> 00:18:15.881
And at some point you get ideas to use a pops up, to use asynchronous messaging frameworks.
00:18:15.881 --> 00:18:23.230
And so this whole ideation part that you would typically do in your head or you have meetings the next day with people.
00:18:23.230 --> 00:18:25.882
putting this on the whiteboard.
00:18:25.882 --> 00:18:36.150
You still have to do this obviously, but I mean the sort of prep work that you can do just by yourself, just to throw ideas around and to validate certain aspects.
00:18:36.150 --> 00:18:41.093
And if you don't stop those models, they will even start to produce code which you don't want.
00:18:43.015 --> 00:18:44.318
it comes back to this whole...
00:18:44.318 --> 00:19:00.038
found for example, that when I'm writing something, an article or something else that using chat GPT or something else can help me get over the blank page problem that I'm not sure where to start.
00:19:00.038 --> 00:19:10.055
So I'll ask some questions and it won't produce what I, a final product, but it'll give me a sort of a rough draft with some.
00:19:10.058 --> 00:19:16.154
with an outline and then I realized, okay, it's missing that bit and I can improve that.
00:19:16.154 --> 00:19:48.732
I found you can also ask these models, well, list the major components in a data pipeline and it'll produce some sort of answer and then you can start to become more specific and make sure, okay, it's going to be real-time event-driven, what are the alternatives to Apache Kafka and it can start to give you some greater detail and then you start to use your own expert knowledge to drill down a little further than that.
00:19:48.973 --> 00:20:01.214
The challenge I've seen so far is not great for diagramming but you showed me some things earlier that show you can even produce a sequence diagram with the right sort of plugin.
00:20:02.234 --> 00:20:13.250
Yes, well, so I was going to say, yes, it's actually fantastic to do these exercises with, well, not with the language models, but with these, I'm always very specific.
00:20:13.250 --> 00:20:18.201
It's these things build around language models, like chat GPT, Cloud.
00:20:18.201 --> 00:20:20.982
People actually don't interact with the language model itself.
00:20:21.123 --> 00:20:23.483
It's a complicated chatbot.
00:20:23.483 --> 00:20:36.473
But anyway, the point what I wanted to make is, Yes, it delivers you this raw material and there is a lot of, if I use a technical term, lot of garbage in there sometimes, something that absolutely doesn't make sense.
00:20:36.473 --> 00:20:47.097
you have to know, again, absolutely, it gives me 80 % and there is some things that I have to fix and that are missing.
00:20:47.097 --> 00:20:50.378
But I think that's one of the key bits.
00:20:50.640 --> 00:20:52.140
Now, it...
00:20:52.601 --> 00:20:56.592
Sorry, you just have to remind me, at the end, what was your question?
00:20:56.623 --> 00:21:02.988
So we're talking about the whole system development life cycle, and we started with architecture, but then...
00:21:03.519 --> 00:21:03.880
yes.
00:21:03.880 --> 00:21:05.201
Yes, I have it.
00:21:06.461 --> 00:21:12.063
Ultimately, even music these days, Spotify, MP3s, are digital.
00:21:13.084 --> 00:21:22.667
Diagrams that we draw in Draw.io, in diagram.net, even SVG files are structured bits.
00:21:22.667 --> 00:21:29.904
And if you can teach a language model how one is constructed, you can also teach it to say, give me a...
00:21:29.904 --> 00:21:34.199
a sequence diagram, give me a flow diagram using this format.
00:21:34.199 --> 00:21:41.325
And you will have something that renders nicely in the browser so humans can see it, but it's also structured almost like code.
00:21:41.325 --> 00:21:44.787
So you can also create a version of this, put it in Git.
00:21:45.248 --> 00:21:50.551
And you can basically use language models to produce diagrams.
00:21:50.813 --> 00:21:55.857
I've not seen it much yet for architecture diagrams, but you know what, I'm going to try this out.
00:21:55.857 --> 00:21:57.965
I'm sure somebody has done this already.
00:21:57.965 --> 00:21:58.747
Yeah.
00:21:58.747 --> 00:21:59.157
Okay.
00:21:59.157 --> 00:22:04.074
So we go from architecture and then we already talked about the coding part.
00:22:04.154 --> 00:22:07.618
So then we get into testing.
00:22:07.618 --> 00:22:14.430
Now where, where can the tools help us with, with the testing part?
00:22:16.096 --> 00:22:26.291
Some of the tools that we mentioned like CodeAssist, like Cursor, like Tabnine, they can help you write test cases in certain formats.
00:22:26.291 --> 00:22:27.199
That's one thing.
00:22:27.199 --> 00:22:33.032
is always a very, I wouldn't say always, some people really love writing test cases.
00:22:33.032 --> 00:22:37.713
I can't say I'm enthusiastic.
00:22:38.315 --> 00:22:54.487
Writing one or two is okay, but writing an entire set that covers lots of different possibilities is quite, it gets repetitive and we don't, like doing repetitive things as humans.
00:22:54.487 --> 00:22:57.669
if we can get a machine to do it, that's great.
00:22:58.390 --> 00:23:03.855
But to your point, well, we discussed how you need to validate.
00:23:03.855 --> 00:23:11.740
How do you validate that what the test cases the tool has produced are going to be fit for purpose?
00:23:13.130 --> 00:23:17.990
Well, like I said earlier, you'll still have to understand.
00:23:17.990 --> 00:23:24.373
Sorry, I just wanted to say the number one thing that developers need is documentation.
00:23:24.373 --> 00:23:29.255
And the number one thing that developers don't enjoy doing is writing documentation.
00:23:29.255 --> 00:23:31.434
So it's similar here.
00:23:33.096 --> 00:23:36.336
When we write test cases, they can be for a manual tester.
00:23:36.336 --> 00:23:39.698
And I've worked with people who do manual QA.
00:23:40.342 --> 00:23:41.972
It's unfortunately very repetitive.
00:23:41.972 --> 00:23:52.644
You may have seen the Claude folks have just released a few days ago this thing where in fact the language model can look at the screen, click it buttons, which raises all kinds of issues.
00:23:52.644 --> 00:23:54.385
Is it going to be able to solve captures?
00:23:54.385 --> 00:23:58.987
Is it able to do bank account transactions on my behalf?
00:23:59.386 --> 00:24:00.686
Do we really want this?
00:24:00.686 --> 00:24:06.828
But if we do step by step, there is still a lot of need for manual QA testing.
00:24:06.890 --> 00:24:10.971
These things are unfortunately repetitive, so it needs a specific character.
00:24:10.971 --> 00:24:18.075
I've worked, I've hired with these sorts of people, and they are able to look at these things step by step.
00:24:18.075 --> 00:24:28.048
And with a language model, with a proper tool that understands your code, it can produce certain test cases for manual tests.
00:24:28.048 --> 00:24:29.509
And they are repetitive.
00:24:29.509 --> 00:24:33.721
They look a bit boring, but sometimes that's something you have to do.
00:24:33.721 --> 00:24:38.784
But obviously, there is ways to say which parts of these things can be automated.
00:24:38.784 --> 00:24:46.846
especially if they are front-end related, there may be some Selenium pieces that you can use.
00:24:46.866 --> 00:24:51.268
There may be some legal requirements why certain things cannot be tested automatically.
00:24:51.268 --> 00:24:54.349
especially, let's say we're testing APIs.
00:24:54.490 --> 00:25:02.574
At API Days a few years ago, I've already introduced a Python framework that you can...
00:25:03.241 --> 00:25:07.782
use to do testing, semantic testing on APIs.
00:25:07.782 --> 00:25:09.365
It's not just, the endpoint available?
00:25:09.365 --> 00:25:11.365
But I'm expecting a JSON payload.
00:25:11.365 --> 00:25:13.567
If I put this in, I'm expecting an error.
00:25:13.567 --> 00:25:14.977
So you can actually script this.
00:25:14.977 --> 00:25:24.330
And if you teach this to a language model, and if you build an agent around it, you can also use this to run automated tests, happy path.
00:25:25.432 --> 00:25:29.173
For example, there's many opportunities.
00:25:29.193 --> 00:25:30.013
Yeah.
00:25:30.806 --> 00:25:52.835
so align with the testing part is also security vulnerability scanning there are I guess there have been for some time automated tools for doing this for doing security vulnerability scanning but perhaps this space is also changing because that security space is constantly changing.
00:25:52.835 --> 00:25:55.207
There are new threats that come up all the time.
00:25:55.367 --> 00:26:05.616
When it comes to deploying to production, what's your sort of level of faith for how you can get the AI to deploy to production?
00:26:05.616 --> 00:26:10.560
Or is that something that you always want to have someone with a finger on the button before it goes?
00:26:10.721 --> 00:26:14.102
you're talking about CI, CD done by AI?
00:26:14.818 --> 00:26:16.220
Well, I mean, it depends.
00:26:16.220 --> 00:26:18.851
And like I said, AI is not language models.
00:26:18.851 --> 00:26:20.401
Language models isn't AI.
00:26:20.401 --> 00:26:22.741
We have a lot of smart tools.
00:26:23.382 --> 00:26:32.726
We don't have to talk about Jenkins or so on, but just GitHub Actions are a wonderful, mostly reliable tool to do automatic deployment.
00:26:33.266 --> 00:26:41.397
of course, you can have certain steps inside those bits being taken care of by AI, that's not a problem.
00:26:41.397 --> 00:26:51.700
think generally what I've seen companies saying that we know nothing about AI and can we deploy something in production in three weeks from now.
00:26:51.840 --> 00:26:55.682
It reminds me of what we had in the early days of cloud computing.
00:26:55.882 --> 00:27:03.463
So when you look at cloud adoption, you don't put your production database of your core system into the cloud within a few months.
00:27:03.463 --> 00:27:05.164
You start...
00:27:06.217 --> 00:27:12.099
while training your AI muscles and building up that knowledge and see how much you can rely on this thing.
00:27:12.099 --> 00:27:31.827
And ultimately, I mean, it's similar to building customer facing chatbots with no AI experience versus making your existing customer service agents, which are human, more successful with AI so they can hang up the phone in five minutes instead of 30 minutes.
00:27:31.827 --> 00:27:34.490
And I would always look at these things like...
00:27:34.490 --> 00:27:45.146
start with something non-critical, then even if it is critical, something that's not customer-facing, something that you can catch.
00:27:45.712 --> 00:27:49.233
Well, I would say it's this whole autonomous part that people always get.
00:27:49.233 --> 00:27:51.354
mean, we have incredible demos.
00:27:51.595 --> 00:27:59.297
They're very impressive, but they are not as reliable as the, everybody talks about the triple nine, the five nines and availability.
00:27:59.457 --> 00:28:06.118
We now have these alien intelligent bits that we use and I can't debug the code.
00:28:06.118 --> 00:28:13.000
So sometimes even if I engage with these things and I say, Hey, you put, I gave you this one and you produce this.
00:28:13.000 --> 00:28:13.810
doesn't make any sense.
00:28:13.810 --> 00:28:14.971
Why did you do that?
00:28:14.971 --> 00:28:18.463
And I can't look at the code and say, yes, it's because of the switch and such.
00:28:18.463 --> 00:28:22.728
It'll say, I'm sorry, you're right, I shouldn't have done this, I won't do this again.
00:28:22.728 --> 00:28:28.053
And I can make it very restrictive, but then it's not as flexible as I want.
00:28:28.053 --> 00:28:29.663
it takes a lot of experimentation.
00:28:29.663 --> 00:28:42.744
And it's really just setting the right expectations with senior people, with business people, that it's not a switch on download and you have, you know...
00:28:42.845 --> 00:28:45.226
90 % cost savings within weeks.
00:28:45.226 --> 00:28:57.490
It's a journey and you need to build up your own team of cross-functional players who figure out how to deal with this as a company, as a team, as an organization.
00:28:58.490 --> 00:29:06.393
But most of all, you don't wait for something to be ready because things are never going to be, you you have to start now.
00:29:06.872 --> 00:29:07.771
Yeah.
00:29:07.771 --> 00:29:10.432
Well, thanks for sharing that, Uli.
00:29:10.432 --> 00:29:24.012
I appreciate you sharing your own journey and then the different elements across the system development life cycle where tools can help us, but where we also need to watch and be careful.
00:29:24.012 --> 00:29:36.231
And I like your analogy about building the muscles where you need to get into training and realize that it's not a sprint, it's a marathon.
00:29:36.231 --> 00:29:38.528
So thanks very much, really, I appreciate it.
