S8E8 The Content Architecture Beneath AI with Patrick Bosek의 전체 녹취록

00:02:15.598 Yeah. 00:02:15.598 --> 00:02:17.520 And then I'm a developer by background. 00:02:17.520 --> 00:02:21.758 So I was one of the developers and architects of the first version of our software. 00:02:21.758 --> 00:02:26.318 So I'm I'm fairly technical, but I don't cause problems for our development team anymore. 00:02:26.318 --> 00:02:30.800 Nowadays, I focus on what's going to make customers happy and those kinds of things. 00:02:31.360 --> 00:02:35.360 What spawned this idea to look into structured content as a developer? 00:02:35.360 --> 00:02:36.479 I'm curious. 00:02:37.360 --> 00:02:37.598 Oh boy. 00:02:37.598 --> 00:02:41.360 So that's actually going back a lot further than I'd like to admit. 00:02:41.360 --> 00:03:04.318 But my co-founder and I, we got into structured content, not DITA, which is where we are today, but XML content with an actually with a custom schema when we were in college, actually, and we just saw that it was a superior way to store and manage content and that it was going to make some of the publishing activities we were working on easier and more capable. 00:03:04.318 --> 00:03:10.878 At that point in time, the technologies surrounding digital were just far, far less well developed than they are today. 00:03:10.878 --> 00:03:18.878 So structured content provided it actually provided something quite different at that moment in time, now that I'm reflecting on it, than it does today. 00:03:18.878 --> 00:03:25.520 At that moment in time, it it basically gave you the ability to put, like, a little bit of a database in a file. 00:03:25.520 --> 00:03:40.240 The amount of things you had to manage and the flexibility you got when you were using structured content at that point in time, if you're looking to apply that to like a very structured digital publishing environment, there was a lot of advantages there. 00:03:40.240 --> 00:03:42.158 Um so that was why we picked it up originally. 00:03:42.158 --> 00:04:02.080 And then over the years, we eventually drifted into, or I guess found our way, is maybe a better way of putting it, into technical content where you still get a lot of those benefits that we we were originally looking at with DITA and in the digital publishing around uh technical content and documentation, but there's a ton more today. 00:04:02.080 --> 00:04:04.639 So there's there's so much more that you get from it. 00:04:05.199 --> 00:04:08.158 Yeah, we could probably have a whole episode on structured content. 00:04:08.158 --> 00:04:12.960 And I have, as I mentioned, had some episodes covering what that is. 00:04:12.960 --> 00:04:16.160 And it seems to me like the context can be different. 00:04:16.160 --> 00:04:28.319 Of course, DITA comes close to mind, but for those of us that don't use DITA, structured content still follows a certain framework, a certain guidelines, if you will. 00:04:28.319 --> 00:04:41.439 But for this conversation, I want to ground what that definition means to you, and then we can, I guess, start talking about how we can deep dive into how that lends to AI. 00:04:42.160 --> 00:04:48.560 Yeah, I should think that is really important to start with the definition of structured content because everybody does define it a little bit differently. 00:04:48.560 --> 00:04:52.480 And I think the individual definition matters. 00:04:52.480 --> 00:04:55.040 So I'll give you my definition. 00:04:55.040 --> 00:04:57.278 I'll also tell you why I think mine is right. 00:04:57.278 --> 00:05:01.199 But that doesn't mean that other people's don't uh can't be right too. 00:05:01.199 --> 00:05:21.920 So technically speaking, if you put all of your content into Excel or Google Sheets and you put it into, you know, let's say every topic was a column, or maybe better a row, and every portion of that topic or that piece of information was divided up into different columns. 00:05:21.920 --> 00:05:25.120 So you have the pieces broken up into the different cells. 00:05:25.120 --> 00:05:31.120 That would be structured content because you would have the content broken up into a structure. 00:05:31.120 --> 00:05:35.519 Like in the most basic definition, you have a structure for your content. 00:05:35.519 --> 00:05:41.519 So you're you're labeling the different pieces of it in a methodology, and that's structured content. 00:05:41.519 --> 00:05:43.519 Now, obviously, that's a terrible choice. 00:05:43.519 --> 00:05:48.000 Like that wouldn't be a thing that you'd want to do in almost any production environment. 00:05:48.000 --> 00:05:56.160 Although I will say that we have had customers migrate from that exact environment just like that to our tool, which is DITA. 00:05:56.160 --> 00:06:09.759 But when you think about like how this comes together, the reason I start there is because there are a lot of systems that are they're effectively that, because if you think of what a form is, so like a digital form, it is just actually a table. 00:06:09.759 --> 00:06:12.879 You have your question one, question two. 00:06:12.879 --> 00:06:15.278 So your question one might be what is the heading for this file? 00:06:15.278 --> 00:06:19.519 Your question two might be who authored this file, question three might be the body. 00:06:19.519 --> 00:06:24.720 And people will call that structured content because, you know, you can put metadata in one place. 00:06:24.720 --> 00:06:29.439 And and in certain circumstances, that's what headless CMSs are, for instance. 00:06:29.439 --> 00:06:32.560 So, like things like Contentful, those kinds, that's that's what they are. 00:06:32.560 --> 00:06:35.278 In certain circumstances, that actually works quite well. 00:06:35.278 --> 00:06:38.000 Um, that's a really effective form of structured content. 00:06:38.000 --> 00:06:47.040 It's backed by a database, uh, it has the ability to create links between those pieces of information, uh, it has the metadata separated out. 00:06:47.040 --> 00:07:00.079 Like you have like there are

certain things, but it in the background, it is functionally a table that is fundamentally different than what we work with, which is XML, because XML is a tree, which means that you can have structure below those big boxes. 00:07:00.079 --> 00:07:04.639 So when you go into a topic, there's going to be structure at multiple levels. 00:07:04.639 --> 00:07:10.240 So you'll have a piece, like let's say we'll use a procedure or a task topic, for instance. 00:07:10.240 --> 00:07:18.639 When you start creating that, you'll have a title, and then you'll have metadata around it, and then you'll have the prerequisites for running this procedure. 00:07:18.639 --> 00:07:24.079 And then the individual procedure, instead of being an ordered list, it would be steps, right? 00:07:24.079 --> 00:07:28.079 And then there's a step inside of it, there's structure inside of the steps. 00:07:28.079 --> 00:07:38.480 And so by having this multiple layers of structure and semantics, it means that you're able to manipulate that in a more granular way beyond just formatting. 00:07:38.480 --> 00:07:42.959 It's not just HTML and boxes, effectively, but they're both structured. 00:07:42.959 --> 00:07:47.759 Like there is a, like, both things are valid in terms of the content label. 00:07:47.759 --> 00:07:54.480 So, what you really need to ask yourself when you're thinking about, okay, well, we're moving to structure, it's beyond just the term structure. 00:07:54.480 --> 00:08:00.000 Like you should really be thinking about what is the form of that's going to serve our mission? 00:08:00.000 --> 00:08:02.720 What do we care about in terms of interoperability? 00:08:02.720 --> 00:08:07.199 What forms of linking are really important to us? 00:08:07.199 --> 00:08:10.240 What forms of output are going to be really important to us? 00:08:10.240 --> 00:08:18.560 Because those types of questions will guide you towards the type of structure that's going to be best for your particular implementation and your particular needs. 00:08:19.360 --> 00:08:24.000 I have to say that I thought my definition was going to be very different from yours or my understanding. 00:08:24.000 --> 00:08:28.160 But at some level, I think we are on the same page. 00:08:28.160 --> 00:08:28.800 Yeah. 00:08:28.800 --> 00:08:30.639 I mean, I think my definition is pretty broad. 00:08:30.639 --> 00:08:31.920 Like I said, I think it's right too. 00:08:32.399 --> 00:08:33.278 Yes, it's very inclusive. 00:08:33.278 --> 00:08:39.120 So that allows me to be kind of okay, that definition sounds close to what I have in my mind. 00:08:39.120 --> 00:08:40.000 Fantastic. 00:08:40.000 --> 00:08:41.120 Yes. 00:08:41.120 --> 00:08:48.960 I have not worked with DITA, and you mentioned DITA, and DITA comes up a lot when we talk about structured content. 00:08:48.960 --> 00:08:57.918 We've touched upon this before, but I think for the benefit of the audience, it would be nice to quickly do a shout out to what DITA is. 00:08:58.399

--> 00:08:58.798 Yeah, for sure. 00:08:58.798 --> 00:09:02.480 So DITA, in the most basic definition, it's it's a form of XML. 00:09:02.480 --> 00:09:07.840 So in that way, you can actually think of it as like a cousin to HTML in some ways. 00:09:07.840 --> 00:09:11.120 So it actually shares a lot of the same elements that HTML has. 00:09:11.120 --> 00:09:20.480 The thing to know about DITA is that it has built in many of the patterns that you need to scale up technical content. 00:09:20.480 --> 00:09:31.678 So there's a ton of stuff that we've pretty much figured out over the last call it 15 or 20 years, are really critical to larger or more complex sets of technical content. 00:09:31.678 --> 00:09:39.519 And the DITA committee, so the people who work on the open source standard, they put all of those things into the standard. 00:09:39.519 --> 00:09:41.360 So you don't have to recreate them. 00:09:41.360 --> 00:09:48.399 And then perhaps just as importantly, beyond not having to recreate them, you don't have any vendor opinion or lock-in. 00:09:48.399 --> 00:10:04.798 So one of the challenges that the that DITA was trying to solve, and I think has solved pretty effectively, is that when you have vendor-specific formats, it means that okay, well, we've reused content, but we've reused content in this particular system. 00:10:04.798 --> 00:10:14.080 And once you get to a certain level of reuse, if you didn't have a portable technology underneath it, your ability to migrate someplace else is I mean, it's it's almost impossible. 00:10:14.080 --> 00:10:17.759 It just becomes the lock in there, it's just so high. 00:10:17.759 --> 00:10:29.120 So what DITA really solves is it solves having good patterns for the core things that you really need to do with technical content, and then it also solves an interoperability problem. 00:10:29.120 --> 00:10:34.158 And you know, there's some of the criticisms of DITA are that it comes with you know like a high learning curve. 00:10:34.158 --> 00:10:36.320 Yeah, I don't know if I actually agree with that. 00:10:36.320 --> 00:10:38.240 Like I've seen a lot of people learn it pretty quickly. 00:10:38.240 --> 00:10:41.759 There's people feel that it's too heavy for today's uses. 00:10:41.759 --> 00:10:42.720 That's another criticism. 00:10:42.720 --> 00:10:45.440 Those people tend to come from like the Markdown world. 00:10:45.440 --> 00:10:50.639 So, like in comparison to Markdown, you know, Markdown is a text format, DITA is an is an XML format. 00:10:50.639 --> 00:10:52.558 I think that's just a use case thing. 00:10:52.558 --> 00:10:58.798 So like DITA is not a great, is not like you never write the README files for your vibe coded project in DITA. 00:10:58.798 --> 00:10:59.918 I don't do that. 00:10:59.918 --> 00:11:04.960 Like when I have small apps and things like that that I build, I don't write any of the documentation in DITA. 00:11:04.960 --> 00:11:06.158 It doesn't make any sense. 00:11:06.158 --> 00:11:13.360 DITA is a format that you use when you arrive at a sufficient level of complexity and a sufficient scale. 00:11:13.360 --> 00:11:24.320 So if you want like really rough and tough numbers, what we've found over the years is that if you're below a thousand pages of content, DITA probably doesn't make sense. 00:11:24.320 --> 00:11:39.200 Um, there are some exceptions, but even at a thousand pages of content, like it can be questionable there depending on your And so where it really starts to make sense is when you have multiple variations of products or variations of content, audiences. 00:11:39.200 --> 00:11:49.519 So you have anything that's like personalization, and there's a bunch of different terms that people use for the content changes when somebody looks at it for a different person looks at it for a different reason. 00:11:49.519 --> 00:11:53.120 Localization can be a driver, although I think less so today. 00:11:53.120 --> 00:11:55.759 And then just sheer scale with like reuse. 00:11:55.759 --> 00:12:02.000 So if you have a lot of content that you is used in multiple places, that's when you know DITA starts to really shine. 00:12:11.278 --> 00:12:12.399 In a simplified way. 00:12:12.399 --> 00:12:30.720 Since we are going to do a deep dive into how structured content lends itself to AI, does DITA lend itself better, or just this the generic definition of the structured content that we ground it, that we are trying to ground this conversation in also lends itself to better AI interactions? 00:12:31.840 --> 00:12:32.798 That is such a good question. 00:12:32.798 --> 00:12:36.798 So the answer is yes, it depends in here. 00:12:36.798 --> 00:12:41.678 And I feel like I'm among the right people to give like engineering style, it depends answers. 00:12:41.678 --> 00:12:46.558 One of the jokes in our leadership team is that when you ask the business people a question, they give you an answer. 00:12:46.558 --> 00:12:49.200 When you ask the technical people a question, they say it depends. 00:12:49.200 --> 00:12:53.360 And so I think that I'm I'm filling that role in this particular moment. 00:12:53.360 --> 00:12:57.120 So, you know, this is kind of like what we we wanted to cover today. 00:12:57.120 --> 00:12:59.678 So it is a it's a very, very long answer. 00:12:59.678 --> 00:13:10.320 At the very core of it, if, like, you wanted, like, the too long, didn't listen version of it, structured content is going to give you better structured contents with the broad definition. 00:13:10.320 --> 00:13:20.240 So like anything that is broken up into labeled pieces that are then stored consistently, like that definition of structured content. 00:13:20.240 --> 00:13:28.720 So that's inclusive of DITA, inclusive of, uh, headless CMSs, inclusive of anything that has a well-defined schema applied to content. 00:13:28.720 --> 00:13:31.759 So, why would that be better for AI? 00:13:31.759 --> 00:13:37.278 And it really just comes down to chunking,

like that's the fundamental difference between those things. 00:13:37.278 --> 00:13:46.558 So the problem with that statement is that when you get to certain forms of structured content, and I'll like I'll give you like an example from a headless CMS. 00:13:46.558 --> 00:13:50.320 The headless CMS is every headless CMS is gonna have a different schema. 00:13:50.320 --> 00:13:55.918 Like part of the the appeal of a headless CMS is that you can create your own schema. 00:13:55.918 --> 00:14:06.000 So you go in and you get a blanks blank page, and all you have to have is a title, and then you can add a body and an author, and you could add you know links to other things. 00:14:06.000 --> 00:14:13.200 And like the standard thing is like, you know, how would you lay out a one of the standard examples is how would you lay that out for like a music venue? 00:14:13.200 --> 00:14:18.720 And you'd have one thing that was a band, and then you'd have an artist, and then you'd have like those kinds of things, right? 00:14:18.720 --> 00:14:20.000 And you'd you build those schemas. 00:14:20.000 --> 00:14:31.038 Well, in a really well-designed schema, it's gonna function really well as giving you good sized pieces of content for the chunking. 00:14:31.038 --> 00:14:40.720 In a poorly designed schema, what's gonna happen is you're gonna get people basically putting in a title and a body, and like that's really not any different than a Google Doc, right? 00:14:40.720 --> 00:14:45.759 So at that point in time, people are just gonna throw huge quantities of content into that body field. 00:14:45.759 --> 00:14:54.960 So even though technically you have two fields, the structure is two boxes, like it's one box, it's it's two boxes with unstructured content in one of them. 00:14:54.960 --> 00:14:57.278 And so then you lose the benefits of chunking. 00:14:57.278 --> 00:15:19.200 So now there's a bunch of other things, but if you like if you want the biggest bang for your buck, so like the I don't know, the 10% that gives you 60% outcome, it's having really well, clean, defined borders around your content, which means smaller pieces, more intentional pieces, pieces of content that contain ideally one piece of information. 00:15:19.200 --> 00:15:23.038 So in DITA, like a topic is supposed to be about one thing. 00:15:23.038 --> 00:15:28.240 It like if it's a concept topic, it's supposed to teach you one thing or explain one thing. 00:15:28.240 --> 00:15:33.200 If it's a task, it's supposed to give you a procedure that does one thing. 00:15:33.200 --> 00:15:39.678 Um, so it takes you from from the beginning to end of one thing you have to complete, and there's other versions of that as well. 00:15:39.678 --> 00:16:05.038 So in the AI world, when you go to produce results with an AI system, being able to provide it very clean chunking, so very clean set like pieces of content for the classification on when you're putting it into a RAG system, and then also for retrieval, there's a bunch of other things, but that's the thing that gets you the biggest bang for your buck. 00:16:05.759 --> 00:16:11.918 Modern language models are pretty good at extracting meaning from unstructured content. 00:16:11.918 --> 00:16:15.840 Does that reduce the need for structured content? 00:16:15.840 --> 00:16:21.200 Because we're trying to make the case that you need to have chunking with structured content. 00:16:21.200 --> 00:16:32.399 But I'm kind of trying to flip that question and see if the LLMs have gotten so bad so good, or is there still an advantage to use structured content as we have established? 00:16:33.360 --> 00:16:35.038 Yeah, so I think that is also a great question. 00:16:35.038 --> 00:16:41.600 It's one of the places that there's I don't want to say because I think that's the wrong way to think about it. 00:16:41.600 --> 00:16:45.519 But I think that there's so maybe this is a better way to put it. 00:16:45.519 --> 00:16:52.158 You're right that the the modern LLMs have gotten very good at extracting information when the question is basic. 00:16:52.158 --> 00:17:03.120 And the reality is that most of the time when people are interacting with these LLMs, their their modalities of interaction fall into a couple of categories. 00:17:03.120 --> 00:17:06.558 One is coding, and coding is a verifiable tree. 00:17:06.558 --> 00:17:19.278 So like part of the reason coding has gotten so good is not because the LLMs have gotten so good, it's because the harnesses have gotten so good at managing something that is verifiable, like that. 00:17:19.278 --> 00:17:20.318 It's logically verifiable. 00:17:20.318 --> 00:17:26.480 The other modality that we use, we use these in is effectively a replacement for search, right? 00:17:26.480 --> 00:17:29.038 So you go to Google and like you look at the AI result. 00:17:29.038 --> 00:17:35.519 Again, a lot of the advancement there is actually in the deterministic harness under the hood. 00:17:35.519 --> 00:17:51.920 Like the amount of engineering that Google and Anthropic and Chat or and OpenAI have done so that what is being delivered to the AI for consideration is high quality, is probably equivalent to the amount of like engineering they've done in the AI model. 00:17:51.920 --> 00:17:54.960 Well, maybe not, I don't know, but like it's a lot is the point. 00:17:54.960 --> 00:18:05.278 And then when you're but when you're thinking about interacting with these systems in a more general sense, typically people are putting very small quantities of information into them. 00:18:05.278 --> 00:18:14.798 So like if I go to NotebookLM and I throw in a couple of PDFs, it does work quite well because the quantity of information is very small. 00:18:14.798 --> 00:18:21.038 Now, in the background, it is still chopping that PDF up and it is still indexing it. 00:18:21.038 --> 00:18:25.440 Like it is not just feeding that PDF full stock to the to the LLM. 00:18:25.440 --> 00:18:46.558 But what's happening

when you ask that the question, or you're working with the system to get information out of that, is that because it's a small number of chunks, and because the system is then reducing down what it's sending to the LLM to a smaller set, it's not that big of a deal. 00:18:46.558 --> 00:18:48.960 The problem is when you scale up. 00:18:48.960 --> 00:18:55.680 So there's a reason you can only put, I don't know what it is, it's not a huge quantity of content into NotebookLM. 00:18:55.680 --> 00:19:10.640 When you start talking about thousands of pages or tens of thousands of pages or more, at that point in time, especially if you're asking more human questions as opposed to like search queries, that's when it absolutely still breaks down. 00:19:10.640 --> 00:19:12.880 And like we have research that shows this. 00:19:12.880 --> 00:19:16.240 So like this isn't this isn't just me hypothesizing these things. 00:19:16.240 --> 00:19:18.400 Like this is this is stuff that we can show. 00:19:18.400 --> 00:19:35.759 And in those worlds where having really strong RAG retrieval, which is built on the foundation of having good tokenization of the content, you start to see structured content outperform unstructured content by about 50%. 00:19:35.759 --> 00:19:38.078 Now it's use case dependent. 00:19:38.078 --> 00:19:43.920 Like there are like simpler questions will do better in both cases, more complex questions will do better. 00:19:43.920 --> 00:19:46.318 So there's there's a lot of it depends in there. 00:19:46.318 --> 00:19:55.598 But for things that really matter, especially in like regulated industries, or I mean, I guess everybody should care about their customers getting the right answer, but you know, like some people care more than others, I suppose. 00:19:55.598 --> 00:20:01.519 It still is a substantial difference in overall quality of retrieval and accuracy. 00:20:03.038 --> 00:20:11.200 So I think this is a great, I guess, segue into the RAG, uh, the retrieve and generate model that we start looking into. 00:20:11.200 --> 00:20:19.759 Under the hood, retrieval is evolving from a fairly linear model towards more agents that can decide what information they need to go and find. 00:20:19.759 --> 00:20:24.880 Where does structured content fit into this newer model of agentic retrieval and context engineering? 00:20:25.680 --> 00:20:29.358 Yeah, I so I think that's an interesting area of evolution right now. 00:20:29.358 --> 00:20:35.519 And I guess the first thing I want to call out here is that the models are themselves getting better. 00:20:35.519 --> 00:20:37.200 Like there's no doubt about that. 00:20:37.200 --> 00:20:46.240 But I actually think the advancement in RAG systems is more in the harness sophistication than it actually is in the models right now. 00:20:46.240 --> 00:20:56.318 So when you think about like how does an AI work with a a RAG like content operations or RAG architecture such that it's going to produce a better outcome. 00:20:56.318 --> 00:21:00.318 Well, actually, should we stop and like give a definition of what of like what RAG is before? 00:21:00.960 --> 00:21:01.200 Yes, please. 00:21:01.200 --> 00:21:01.598 Absolutely. 00:21:01.598 --> 00:21:02.720 Go for it. 00:21:03.038 --> 00:21:03.358 Yeah, okay. 00:21:03.358 --> 00:21:11.278 So RAG is retrieval augmented generation, which is technical people trying to confuse the world with with fancy words. 00:21:11.278 --> 00:21:15.680 All it means is that you're running a deterministic on the well, you're running a search. 00:21:15.680 --> 00:21:18.480 I was about to say deterministic search, but that's not necessarily true. 00:21:18.480 --> 00:21:26.720 You're running a search, the output of that search is then being put into the context for an AI agent to generate a result. 00:21:26.720 --> 00:21:27.598 That's all it means. 00:21:27.598 --> 00:21:31.118 So it's it's just search and then generate, is what it comes down to. 00:21:31.118 --> 00:21:35.118 Now, most RAG systems are built on vector databases. 00:21:35.118 --> 00:21:52.880 And why it matters to call that out is because fundamentally, vector databases, what they are is they're just they're calculations around a piece of information as to like kind of where it sits in space, and they what they do is they retrieve things that are close together. 00:21:52.880 --> 00:21:56.078 Now, those calculations are done by LLMs typically. 00:21:56.078 --> 00:22:04.160 When you look at like some of the research that we've done, like When you look at, like, some of the research that we've done, like of different ways of calculating the embeddings. 00:22:04.160 --> 00:22:08.640 So the embeddings are like the position, I guess, if you want to think about it that way. 00:22:08.640 --> 00:22:16.160 And the way that we we do that is that you can calculate the embeddings with a bunch of different with any of the AI models that are out there. 00:22:16.160 --> 00:22:24.640 So what we've found is that in our testing, OpenAI tends to be the one that's most cost effective and easiest to do. 00:22:24.640 --> 00:22:26.318 And so that's the one that we've used. 00:22:26.318 --> 00:22:32.160 We've tried a bunch, but that's the one we've kind of settled on as like our baseline, which is funny because we use Anthropic everywhere else. 00:22:32.160 --> 00:22:33.759 But like that's how that comes together. 00:22:33.759 --> 00:22:45.519 So then you have the content in the vector database, the way you retrieve it is through these vector searches, and then when you retrieve sets of content, that's going to be put into the LLM. 00:22:45.519 --> 00:22:53.519 So the major difference here between like the first versions of these and the versions that we deal with today, the models are better, yes. 00:22:53.519 --> 00:23:10.400 But now you may see AI being in a place where you send the question to the LLM, the LLM modifies the question or it breaks it down into some way, or it clarifies the

question, changes the question, and then it passes the query to the vector database. 00:23:10.400 --> 00:23:14.160 The vector database returns results, and that loop can continue. 00:23:14.160 --> 00:23:16.960 So reasoning is largely just a loop, right? 00:23:16.960 --> 00:23:19.038 Like they basically feed the results back in. 00:23:19.038 --> 00:23:25.519 There could be some deterministic logic that does some parsing or extraction between the LLM and the database. 00:23:25.519 --> 00:23:32.558 Like there's other ways that they, you know, they can basically upgrade the overall loop, but it really fundamentally is a loop. 00:23:32.558 --> 00:23:38.400 So I think the misconception that a lot of people have is they think all of this is happening inside of the LLM. 00:23:38.400 --> 00:23:44.960 Like the LLM is making a decision about what information it's retrieving, but like the information is in the LLM. 00:23:44.960 --> 00:23:48.880 The information that you're when you're talking about RAG is not in the LLM. 00:23:48.880 --> 00:23:55.278 It's in a separate piece of technology that the LLM is making, is having an interaction with? 00:23:55.920 --> 00:24:04.640 When I started working with structured content, metadata became a centerpiece to how a cornerstone to how search was done. 00:24:04.640 --> 00:24:09.759 So the the richer the metadata, the better the output was. 00:24:09.759 --> 00:24:12.558 So organizing information was super critical. 00:24:12.558 --> 00:24:24.400 What changes when a machine is using that metadata to decide what information belongs in an AI model's context with all the technology breakdown that you've given us? 00:24:24.400 --> 00:24:24.960 Yeah. 00:24:24.960 --> 00:24:29.358 So this could be a whole series of podcasts in and of itself. 00:24:30.078 --> 00:24:30.240 Of course. 00:24:30.798 --> 00:24:31.038 What changes? 00:24:31.038 --> 00:24:32.798 Well, oh boy. 00:24:32.798 --> 00:24:41.038 The first thing I think that's important to think about here is that metadata gives you a lot of options in how you process something. 00:24:41.038 --> 00:24:58.400 So one of the key aspects of having really high quality metadata on your content is it has hooks and, like, levers that you can pull in deterministic processes that decide how content ends up and how it shows up. 00:24:58.400 --> 00:24:58.960 Right. 00:24:58.960 --> 00:25:13.118 So when you're thinking about what you're gonna end up providing to an LLM, you almost always have some type of a pipeline that's going to be coming into whatever your RAG system is or whatever, wherever the LLM is actually going to be touching that content. 00:25:13.118 --> 00:25:22.640 And metadata means that you can be more specific about what and how those things go into the uh the LLM or the AI-based system. 00:25:22.640 --> 00:25:26.960 So it gives you a lot more control with traditional code. 00:25:26.960 --> 00:25:31.358 And I'm emphasizing that because it really shouldn't be undervalued. 00:25:31.358 --> 00:25:33.838 That is still enormously powerful. 00:25:33.838 --> 00:25:50.078 The more specific you can be in a deterministic way in the systems that you build, the more confidence you can have and the more sustainable those systems are going to be, because you can write unit tests, right? 00:25:50.078 --> 00:25:53.200 Like things in deterministic systems have deterministic outcomes. 00:25:53.200 --> 00:25:56.078 Like that's fundamentally how they're different than AI systems. 00:25:56.078 --> 00:25:59.598 So metadata still has a ton of value in that way. 00:25:59.598 --> 00:26:03.118 This is kind of a the other side of that coin, but I think also valuable. 00:26:03.118 --> 00:26:15.358 There are circumstances where traditional search, if metadata is effectively applied, will perform equally well to vector search, and it's a lot less expensive. 00:26:15.358 --> 00:26:20.078 So there's certain applications where the old stuff works just as well. 00:26:20.078 --> 00:26:26.000 And there's actually cases where it works better, but then you know, we don't have to go, we don't have to dig too deeply into those things. 00:26:26.000 --> 00:26:29.519 But the point is that metadata also gives you options. 00:26:29.519 --> 00:26:46.160 So if you're in a place where you're completely reliant on AI systems and you don't have really strong metadata around how you're trying to build a retrieval pipeline, um, then you're totally reliant on the inference of context from the AI into a vector database. 00:26:46.160 --> 00:26:57.598 Whereas if you had that metadata, you might be in a place where you could have a deterministic system or a traditional search that was just as just as good or better, um, and also gave you some other options. 00:26:57.598 --> 00:27:04.078 So metadata can be really valuable in terms of comparison of not just an optionality for your system. 00:27:04.078 --> 00:27:07.038 So you have optionality, you have comparison, like you have those types of things. 00:27:07.038 --> 00:27:32.880 And then when you think about like how does it actually benefit the LLMs themselves, this is where you get into like a very interesting discussion because there are aspects of it that can be really Typically, what it means is that you're transitioning structured metadata into, I would say, like statements for the LLM, such that it is still context, right? 00:27:32.880 --> 00:27:38.880 But like LLMs receive context differently than like humans and traditional machines to some extent. 00:27:38.880 --> 00:27:42.480 But if you have everything tagged for, so I'll give you an example. 00:27:42.480 --> 00:27:44.640 Let's say you have two products. 00:27:44.640 --> 00:27:47.838 You have a we'll use washing machines, right? 00:27:47.838 --> 00:27:55.200 Let's say you have a 1000 version and a 1001 version, and those are very, very similar systems, right? 00:27:55.200 --> 00:27:56.960 They're the same line of washing machine. 00:27:56.960 --> 00:27:58.240

There's a few differences. 00:27:58.240 --> 00:28:06.960 Well, you've got a hundred topics for each of those systems, and let's say 95 of them are the same. 00:28:06.960 --> 00:28:12.400 So they all carry the 1000 and the 1001 tag, and then you have five that diverge. 00:28:12.400 --> 00:28:18.960 So you have 105 topics in total, and you know, five with a thousand, five with a thousand and one. 00:28:18.960 --> 00:28:26.640 If you're just stuffing all of that into an LLM, and remember, like this is only 105 topics, so you can probably get away with that. 00:28:26.640 --> 00:28:33.200 The way you would use metadata there, and notice I'm specifically avoiding RAG and search in this particular example. 00:28:33.200 --> 00:28:56.798 Like I'm just talking about how you put the whole 105 in, is instead of having a tag on that, you would probably be better off actually having a statement at the top of it where before you put it in, you'd use a deterministic system to take the tag from you know product and then it had the tag name to this is the 1001 version of this washing machine. 00:28:56.798 --> 00:28:59.680 This is the 1000 version of this washing machine. 00:28:59.680 --> 00:29:01.358 You just put that at the top of every topic. 00:29:01.358 --> 00:29:07.920 And an LLM is going to interpret that better than having like the actual tag on there. 00:29:07.920 --> 00:29:10.078 And so I've done a bunch of testing around this. 00:29:10.078 --> 00:29:12.480 Like, this also is the same with like glossary terms. 00:29:12.480 --> 00:29:21.440 So if you have like structured glossaries, um, different LLMs will receive the pieces of that glossary, which are tags, right? 00:29:21.440 --> 00:29:22.000 Like it's metadata. 00:29:22.000 --> 00:29:26.400 Like you're like, you know, this is an abbreviation, this is another way of saying it. 00:29:26.400 --> 00:29:29.920 Like, there's like right, it that's just metadata that describes different pieces of it. 00:29:29.920 --> 00:29:40.160 If you write it out as prose, even if it's the same prose over and over again, the LLMs tend to work with that version of that content better. 00:29:40.160 --> 00:29:57.278 So transitioning the structured metadata into clear and simple statements, even if they're repetitive for the LLM, tends to produce a better result today with the LLMs we have today, because of how they've been trained and you know the biases and like the internal weightings that they have. 00:29:57.278 --> 00:30:00.318 Like you could train one that works better with the actual structure. 00:30:00.318 --> 00:30:02.400 That's just not the ones that we have today. 00:30:02.400 --> 00:30:06.000 So metadata touches a ton of different pieces of it. 00:30:06.000 --> 00:30:13.440 And I haven't even gone into like how metadata can then be used to build graphs and to create like more sophisticated like selection mechanisms. 00:30:13.440 --> 00:30:31.200 But the point is that once you get to any level of complexity, having the metadata on the content gives you so much more optionality in terms of how you build the systems around the LLM, and then also how you actually instruct the LLM when you're putting inputs into it. 00:30:31.680 --> 00:30:36.880 I think this was such a great overview because my goal here was to kind of step back. 00:30:36.880 --> 00:30:57.598 And for those amongst us who have very, I guess the as a technical writer, one would have that structured content, the concept But if you don't, if you just have a Google Doc and you're starting off, then adding metadata to that, you've given us a why, a solid why. 00:30:57.598 --> 00:30:59.519 Why should you still consider it? 00:30:59.519 --> 00:31:04.160 And how that would kind of work under the hood that I keep going back to. 00:31:04.160 --> 00:31:11.519 And I would love for us to maybe even dive into the knowledge graphs and how they would relate to metadata. 00:31:11.519 --> 00:31:25.038 With more than 70 sessions and workshops featuring speakers from companies like Salesforce, TikTok, T-Mobile, and Amazon. 00:31:25.038 --> 00:31:32.880 LavaCon brings together content strategists, documentation leaders, and content professionals to share practical ideas you can apply right away. 00:31:32.880 --> 00:31:39.920 And yes, it's known as the Fun Conference, too with networking events, therapy dogs, comfort llamas, storytelling, and karaoke. 00:31:39.920 --> 00:31:44.160 Use discount code ITC26 to save$200 on registration. 00:31:44.160 --> 00:31:47.920 You did mention graphs here. 00:31:47.920 --> 00:31:52.400 So I think it's worth going into that a little bit, if not more. 00:31:53.118 --> 00:31:53.440 Yeah, sure. 00:31:53.440 --> 00:31:54.798 So I'm happy to go. 00:31:54.798 --> 00:31:58.640 Well, there are people who are are experts in knowledge graph. 00:31:58.640 --> 00:32:01.358 I wouldn't consider myself an expert in knowledge graphs. 00:32:01.358 --> 00:32:04.960 I do know I I would say a pretty good amount about them. 00:32:04.960 --> 00:32:10.880 So there's probably a limit to how far I can go, but I'm very happy to talk about them in relation to structured content metadata. 00:32:10.880 --> 00:32:15.519 So maybe uh start with just the definition of what a knowledge graph is. 00:32:15.519 --> 00:32:24.400 So so a knowledge graph is if you want to think about it in technical terms, it's a database that has typed nodes and typed connections between those nodes. 00:32:24.400 --> 00:32:33.440 So you could have people and then you could have, like, a 'knows' or like this person knows this person or is married to. 00:32:33.440 --> 00:32:36.400 Obviously, that would be a one-to-one connection in most cases. 00:32:36.400 --> 00:32:39.200 But like, you know, so this is like the friend of a friend stuff. 00:32:39.200 --> 00:32:41.920 Like, so like ontologies, knowledge graphs, those kinds of things. 00:32:41.920 --> 00:32:52.160 When you put this information into graph databases, what it means is you can

start asking multi-level questions of the database. 00:32:52.160 --> 00:33:04.480 So the thing that people really like about them is if you go back to like the people example, you could ask a question of a knowledge knowledge graph of like, give me all the companies for people that Patrick has been on podcasts with. 00:33:04.480 --> 00:33:34.078 And a good knowledge graph, like a good knowledge graph database would be able to go and say, Okay, well, Patrick is a podcaster, and then there would be a field that's like has been on, or something like that, and that would link out to a bunch of different podcasts, and then each of those podcasts would have guests and hosts, and each of those guests and hosts would be people, and all the people would have as employment or you know, works for something along those lines, right? 00:33:34.078 --> 00:33:41.038 And so you could have a query that would say like doop, doop, doop, doop, doop, and hop across all those things and select all of those companies. 00:33:41.038 --> 00:33:53.759 That's why people like knowledge graphs, is because it allows you to ask those kinds of questions of an information set, which are traditionally very difficult to ask in the context of like relational databases. 00:33:53.759 --> 00:34:00.000 So you can do some of that stuff in relational databases, but it's much less flexible, I guess is the best way to put it. 00:34:00.000 --> 00:34:09.039 So you're dealing with very large and you know well tagged or very semantic sets of content, like you're just gonna do better in a knowledge graph. 00:34:09.039 --> 00:34:20.079 So, how this collides with structured content and metadata is that it's not always that you're typically you're not actually keep maintaining your content in the knowledge graph, right? 00:34:20.079 --> 00:34:22.400 So you're maintaining your content in another location. 00:34:22.400 --> 00:34:29.599 So the way that you you correlate, you tie the content to the knowledge graph is through the metadata in many cases. 00:34:29.599 --> 00:34:37.280 So the metadata that becomes the selection mechanism that the the knowledge graph can use to pull that out. 00:34:37.280 --> 00:34:51.920 You can also use direct linking, but you oftentimes like the metadata is going to become like a major piece of like how those two things come together, or how the knowledge graph forms the nodes in the graph out of the content, like it contributes to those things. 00:34:52.960 --> 00:35:00.800 I'm thinking in a typical environment, you have documentation coming from product, from engineering, from support. 00:35:00.800 --> 00:35:11.760 How should the infrastructure establish authority and provenance and a source of truth within the context of that uh of structured content? 00:35:11.760 --> 00:35:14.719 Because that content may be structured differently. 00:35:14.719 --> 00:35:17.280 What's your take on that? 00:35:17.920 --> 00:35:19.920 Yeah, so I think this is a really timely question. 00:35:19.920 --> 00:35:36.079 So the the environment you've described, you know, very briefly, where like technical content comes from support, product engineering, um, the technical writers, like that is one version of companies where the the technical content is kind of like the result of all these contributors. 00:35:36.079 --> 00:35:39.039 That's how you get to those corpuses of content. 00:35:39.039 --> 00:35:50.480 The other structure of companies is where you have the technical writers is more of like a clearinghouse, like they're the ones that manage most of the content, and they probably write a large portion of it. 00:35:50.480 --> 00:36:10.639 And in those environments, you do tend to have knowledge bases that grow up around that, but there's a clear, very clear delineation between like the docs are gold plated, the knowledge bases can be a source, it can be leveraged, but it's not like the level of governance and quality that's expected is is very, very different. 00:36:10.639 --> 00:36:17.840 And I think it's very situational as to which which is going to work for any one individual company. 00:36:17.840 --> 00:36:28.000 You know, some companies can work really well in the more almost crowdsourced version of documentation, and then other companies really can't go that route. 00:36:28.000 --> 00:36:36.159 Like they have to have technical writers as the authorities on truth, more or less, like is the way to think about it. 00:36:36.159 --> 00:36:39.360 Um, and that's like a that's like um a center of excellence approach. 00:36:39.360 --> 00:36:46.559 I have a lot more experience in that approach because our technology lends itself to that approach. 00:36:46.559 --> 00:36:53.360 Like if you have the highly like crowdsourced collaborative version of it, like that's just not where we fit. 00:36:53.360 --> 00:36:55.599 So I don't work in that world as much. 00:36:55.599 --> 00:37:06.320 When you're trying to establish authority and governance across multiple groups like that, so again, I guess there's kind of two situations there. 00:37:06.320 --> 00:37:12.559 One is they're all using the same thing, which in my experience there is almost always a wiki or Google Docs. 00:37:12.559 --> 00:37:15.440 Like I haven't seen very many variations on that. 00:37:15.440 --> 00:37:24.559 Um, and that's just because when you have a large group of people who are not writers writing the content, you have to give them a tool that's not a writer's tool. 00:37:24.559 --> 00:37:35.360 It's like it's super uncommon that you'll see a circumstance where you have engineering product support and technical, and like maybe you know, a couple of technical writers working in technical writing tools. 00:37:35.360 --> 00:37:41.840 Like I think I've maybe seen that once in my entire career doing this for like I'm not even gonna say how many years, but more than a decade. 00:37:41.840 --> 00:37:46.000 It's much more common that you'll see you hit you go to the lowest

common denominator. 00:37:46.000 --> 00:37:48.320 Maybe that's that's not exactly what I mean. 00:37:48.320 --> 00:37:52.480 You go to the most available system, which is almost always a wiki. 00:37:52.480 --> 00:37:57.039 Wikis are not built for governance, like there's just no way that. 00:37:57.039 --> 00:38:12.159 Like wikis are built for availability, and governance is an that imposes structure, it needfully imposes roadblocks and workflow and things like that, because that's where quality comes from. 00:38:12.159 --> 00:38:19.599 It comes from people doing the same things, knowing the patterns, checking themselves in predictable ways, that kind of stuff. 00:38:19.599 --> 00:38:21.199 Like that's governance, you know. 00:38:21.199 --> 00:38:24.159 Like you look at any like Six Sigma or Six Sigma, right? 00:38:24.159 --> 00:38:26.159 Like it's it's a set it's a set of patterns and practices. 00:38:26.159 --> 00:38:34.000 You can't apply those to a group of people that are not professionals in patterns and practices around that particular environment. 00:38:34.000 --> 00:38:37.280 So it becomes very challenging, is what it comes down to. 00:38:37.280 --> 00:38:41.440 You can't really have like broad inputs and high governance. 00:38:41.440 --> 00:38:52.079 And if you don't have high governance in your content, the work of like reverse engineering authority and like source of truth becomes much more complicated. 00:38:52.079 --> 00:38:54.960 It's not impossible in all cases. 00:38:54.960 --> 00:38:57.199 So I can give you an example of this. 00:38:57.199 --> 00:39:01.360 So like we have technical writers here, obviously, like that would be crazy if we didn't. 00:39:01.360 --> 00:39:10.400 But like our product team doesn't do the product specifications in Heretto, which is you know, Heretto is a documentation system. 00:39:10.400 --> 00:39:23.199 Like we do our product specifications in Confluence, and we do that because like we like product specifications are the sorry, these are the the design specs, not the actual, like these are these are not the final specs. 00:39:23.199 --> 00:39:25.199 We don't do the design specs in Heretto. 00:39:25.199 --> 00:39:28.559 So the design specs, because like they are working spaces, right? 00:39:28.559 --> 00:39:38.800 Like you've got a bunch of people in there and like they're throwing stuff at the wall, they're pulling things around, like they're just they're kind of screwing around in the content to figure out what's going to work while they're going through the process, the design process. 00:39:38.800 --> 00:39:48.480 It's a very creative, free-flowing process of like getting together, and that's just not the right place to apply like high structure and high governance. 00:39:48.480 --> 00:39:58.960 Like, you need to give people the minimum amount of those things to give them the most creative freedom to get to the end design results they're looking for. 00:39:58.960 --> 00:40:06.880 Now, at the same time, the outcome of these initial design documents that they do, they tend to be very high quality. 00:40:06.880 --> 00:40:08.880 So they're totally inconsistent. 00:40:08.880 --> 00:40:21.440 Like there's no like like they're not, you know, they're not in the same format of the same structure, but like they have really, really good information in them because they are for the initial design of something, like they kind of are the single source of truth. 00:40:21.440 --> 00:40:29.840 So in our world, what we do there, this is before the thing is built, before the documentation is written, all those kinds of things, is you just lock it down. 00:40:29.840 --> 00:40:35.599 Like once it's done, you just reduce the number of people that can change it, and the people that can change it are much more mindful. 00:40:35.599 --> 00:40:44.800 Now, we also have the benefit of the people who are working on this, are people who are who have all of the benefits of knowing structured content and documentation because that's a tool that we work on. 00:40:44.800 --> 00:40:59.599 So, like inherently, they're gonna be better about some of these practices, and you do see that bleed into you know building just frankly, probably more information architecture, you know, kind of by osmosis in some of these documents. 00:40:59.599 --> 00:41:02.639 So, like we see a lot of content come from Confluence. 00:41:02.639 --> 00:41:14.239 It's very common that people move from Confluence to Heretto, and it tends it comes in a wide variety of of qualities and like like positions from the structure and like a governance perspective. 00:41:14.239 --> 00:41:21.599 So, but we use those as inputs to to different LLM systems that we have, those those Confluence pages. 00:41:21.599 --> 00:41:38.320 So before a feature is built and before the content is migrated into Heretto and it's put on our internal like our internal point of reference is a thing we have like an internal name for it, that is all designed and built in Heretto, and that is considered the highest order of truth in the organization. 00:41:38.320 --> 00:41:40.159 Like if it's there, it's true. 00:41:40.159 --> 00:41:49.599 But before, you know, in the design process, if people have questions about things, like they can ask an AI agent and it will source from those Confluence pages. 00:41:49.599 --> 00:41:59.280 And it works quite well because it's a very small set of content, it's maintained by a very small team, it has a very known position in the cycle of information. 00:41:59.280 --> 00:42:03.679 Like it was designed at this point in time by these people for this purpose. 00:42:03.679 --> 00:42:11.360 So like there is a set of metadata that's assumed about that content simply by the definition of what it is. 00:42:11.360 --> 00:42:16.159 Like there's all these attributes that come with it that you can build into a system to get good results from. 00:42:16.960 --> 00:42:21.518 I was also thinking about lone writers and

systems where they've not evolved. 00:42:21.518 --> 00:42:25.920 You gave them great examples of how to think about it. 00:42:25.920 --> 00:42:29.518 So if you don't have those systems in place, you can start small. 00:42:29.518 --> 00:42:32.480 Where can I start if I don't have it? 00:42:32.480 --> 00:42:38.480 How do I think about this where you don't have established processes or uh governance frameworks? 00:42:38.480 --> 00:42:48.960 How do you establish that so that you can get to the source of truth and locking it down and having the right people, get the right gatekeepers, so that eventually your content is gold-plated? 00:42:49.760 --> 00:42:53.199 Um Yeah, I can actually give one other piece of AI-related advice to that. 00:42:53.199 --> 00:43:02.960 So if you're a lone writer, like, so if I was gonna go be a lone writer in an organization today, almost no chance that you have access to like true structured content. 00:43:02.960 --> 00:43:05.518 Like that's just not common for lone writers. 00:43:05.518 --> 00:43:11.280 So I think the most common circumstance for those individuals today is going to be that they're writing in Markdown. 00:43:11.280 --> 00:43:14.800 There's a bunch of other situations, but like that's probably the most common. 00:43:14.800 --> 00:43:16.159 So I'll focus on that one. 00:43:16.159 --> 00:43:38.480 One of the things that I would do after I wrote a very basic style guide, I'm not even gonna call it an information architecture at that point in time, but a set of rules and patterns around how it is that we created content, is I would feed that into an agent and I would say, you're now going to be my editor, and you're gonna hold me to these rules. 00:43:38.480 --> 00:43:50.400 And so then every time, so again, we're in Markdown, we're probably checking this content into GitHub, which at a you know, at if I'm maintaining a thousand topics or less, that's probably gonna work pretty well. 00:43:50.400 --> 00:43:56.400 Every time uh it was checked in, it just had the agent go and look look it over and have it give me feedback. 00:43:56.400 --> 00:43:59.199 Oh, you didn't put enough metadata on the top of this topic. 00:43:59.199 --> 00:44:01.039 Oh, this topic is too long. 00:44:01.039 --> 00:44:03.518 Oh, it it this topic contains too many claims. 00:44:03.518 --> 00:44:11.280 Like you should break this up into more more Markdown files because you know you're mixing concepts and and tasks here. 00:44:11.280 --> 00:44:16.800 Like this is clearly, you know, this section is clearly a task, and this section contains a bunch of claims. 00:44:16.800 --> 00:44:19.599 Like there's uh claims facts, sorry. 00:44:19.599 --> 00:44:31.039 So there's a bunch of things that you could do with a very simple configuration of Claude, or I've actually not done this in uh in Gemini or OpenAI, so like I'm just gonna say Claude, but I'm assuming you can do this in other LLMs. 00:44:31.039 --> 00:44:32.079 Yeah, yeah. 00:44:32.079 --> 00:44:48.559 So you could set all those things up and it could basically just help you enforce a number of rules that are going to get you a lot further down the road than you would be if you were you know just writing content as text and Markdown or in YAML files or things like that. 00:44:48.559 --> 00:45:01.840 And I think you would find that if it did scale up and you hired more people and you did need to transition to a true structured format, you know, for personalization or reuse, those types of things, that you'd still have a lot of work to do, but you'd be in a better place. 00:45:02.559 --> 00:45:07.518 Yeah, I think that's a great suggestion to kind of lean into those systems available to you. 00:45:07.518 --> 00:45:10.800 If you were designing a content architecture from scratch today. 00:45:10.800 --> 00:45:17.440 Let's say you've been hired, there are no systems, and you are going to be designing content architecture from scratch. 00:45:17.440 --> 00:45:26.719 Uh specifically for AI and agentic systems, what would you build from the source content all the way through retrieval context, uh governance, and execution? 00:45:26.719 --> 00:45:28.559 Oh my god. 00:45:28.559 --> 00:45:36.800 But you you do you you've been you've done great so far with all the I mean geez. 00:45:38.000 --> 00:45:43.039 Well, so it would depend a ton on there's so many variables that go into that. 00:45:43.039 --> 00:45:45.599 So I'll give you one version of it. 00:45:45.599 --> 00:46:03.360 Okay, so if I'm coming into an organization that has, let's say 10 proper technical writers maintaining a hundred thousand pages of content, or well, let's just say fifty thousand pages of content. 00:46:03.360 --> 00:46:06.559 It's that's a little more on the on the medians there. 00:46:06.559 --> 00:46:16.000 And that content is you know, has any level of complexity, which it almost always does at that size, like you can almost never get around that. 00:46:16.000 --> 00:46:21.039 The first thing I'm going to do, if they're not already, if so, like you said, the nothing exists. 00:46:21.039 --> 00:46:26.239 The first thing I'm gonna do is I'm gonna introduce structured content and a proper CCMS. 00:46:26.239 --> 00:46:28.320 Like that would be step one. 00:46:28.320 --> 00:46:31.518 And then from there, I'm going to look at the delivery requirements. 00:46:31.518 --> 00:46:39.679 So one of the things that I think people undervalue is the importance of having a really high-quality website today. 00:46:39.679 --> 00:46:43.440 Like that is where most of your AI answers are coming from. 00:46:43.440 --> 00:46:46.800 Like before you implement any AI, you need to feed the public AI. 00:46:46.800 --> 00:47:12.159 And to do that, you need a website that has all the different Many of them are just SEO things, but all the different aspects that are going to give you the proper authority with Google, OpenAI, Anthropic, and the other major models, so that when people are asking

questions of those systems, they're properly retrieving information from your set of information for your customers. 00:47:12.159 --> 00:47:16.960 Like that's gonna be 90% of your customers' AI interaction. 00:47:16.960 --> 00:47:20.320 From there, it's a very it's gonna get very situational. 00:47:20.320 --> 00:47:35.360 If I've got a field service team, I'm gonna look at how I'm gonna deploy an agent to that field service team that covers the particular issues that they're having a hard time solving that are also their information issues. 00:47:35.360 --> 00:47:41.039 I would start deploying agents and connections to our support channel. 00:47:41.039 --> 00:47:43.119 So like there was a feedback loop there. 00:47:43.119 --> 00:47:46.960 You know, those things are maybe not simple, but fairly straightforward, right? 00:47:46.960 --> 00:47:49.679 Like that's kind of like, you know, that's known work at this point in time. 00:47:49.679 --> 00:47:53.039 I'm not gonna call it a solved problem, but it's known work at this point in time. 00:47:53.039 --> 00:48:03.440 I think if you you kind of started with that path in that you're going to get yourself so far down the road that you're probably gonna be ahead of most people already. 00:48:03.440 --> 00:48:05.440 That I think that that is a blueprint. 00:48:05.440 --> 00:48:19.360 I guess the other thing I should say there too is when I said implement a CCMS, I meant do the information architecture work on the content, come up with an information model, get the content into really good shape, and then put it into a CCMS. 00:48:19.360 --> 00:48:21.760 So that's like the process of implementing structure. 00:48:21.760 --> 00:48:28.320 One of the biggest mistakes that people make is that they think they're gonna buy a tool and that's gonna solve their problems, and that's not the—not the case. 00:48:28.320 --> 00:48:37.280 Like the work it takes to go from having a ton of content that's unstructured to good structured content that's actually going to solve problems. 00:48:37.280 --> 00:48:43.440 That's an information modeling and information architecture and conversion process, and that is work. 00:48:43.440 --> 00:48:47.679 It has a huge payoff, it's the foundation for everything else. 00:48:47.679 --> 00:48:52.000 But if you don't do that work, you don't get the benefit out of structured content. 00:48:52.000 --> 00:48:54.000 So, like that would be the first step. 00:48:54.719 --> 00:48:56.559 First of all, my apologies, Patrick. 00:48:56.559 --> 00:49:00.480 I did not mean to quote unquote assault you with that question. 00:49:01.199 --> 00:49:01.920 No, no, no, no. 00:49:01.920 --> 00:49:03.039 It's a great question. 00:49:03.039 --> 00:49:04.880 It's just it's such a big question. 00:49:05.599 --> 00:49:12.159 So I was on a team where the mindset was that the tool would solve all the problems. 00:49:12.159 --> 00:49:20.880 And thinking about information architecture and mod—well, let's say that we had to revisit how our content was modeled and architected. 00:49:20.880 --> 00:49:23.518 The tool did not solve the problem. 00:49:24.239 --> 00:49:24.800 You think about it. 00:49:24.800 --> 00:49:32.559 If you were gonna go build a house and you showed up at the build site with a bunch of tools but no blueprints, yeah, what would you get? 00:49:32.559 --> 00:49:33.599 What would you end up with? 00:49:34.320 --> 00:49:35.039 Thank you so much for watching. 00:49:35.039 --> 00:49:38.320 This was a great conversation, and I hope you had fun as well. 00:49:38.320 --> 00:49:38.960 I did. 00:49:39.440 --> 00:49:39.920 Thanks again. 00:49:40.559 --> 00:49:41.119 Thank you. 00:49:41.119 --> 00:49:44.480 Thank you for listening to Inside Tech Comm. 00:49:44.480 --> 00:49:49.039 If there's a guest you would love to hear on the show, please let me know. 00:49:49.039 --> 00:49:54.320 And don't forget to follow and catch every episode on your favorite podcast app. 00:49:54.320 --> 00:49:55.599 See you next time.