WEBVTT
00:00:02.240 --> 00:00:06.878
Hello friends and welcome to season eight of Inside Tech Comm with Zohra Mutabanna.
00:00:06.878 --> 00:00:16.160
This season features a collection of conversations that explore the ideas, challenges, and opportunities shaping technical today.
00:00:16.160 --> 00:00:21.760
From AI and content strategy to leadership, product thinking, and the future of our profession.
00:00:21.760 --> 00:00:26.160
Each episode offers a fresh perspective from people doing the work.
00:00:26.160 --> 00:00:27.120
Let's get started.
00:00:27.120 --> 00:00:35.679
Season eight is proudly sponsored by LavaCon.
00:00:35.679 --> 00:00:42.640
Use discount code ITC26 to save$200 on registration for this year's conference.
00:00:42.640 --> 00:00:45.920
I'll share more about LavaCon later in this episode.
00:00:45.920 --> 00:00:47.200
Hello listeners.
00:00:47.200 --> 00:00:49.920
Welcome to another episode of Inside Tech Comm.
00:00:49.920 --> 00:00:53.759
Today we'll be talking with Patrick Bosek from Heretto.
00:00:53.759 --> 00:01:03.039
And in the past, we've talked about structured content and its impact on how AI ingests that framework.
00:01:03.039 --> 00:01:05.358
Today we're going to do a deep dive.
00:01:05.358 --> 00:01:14.400
Patrick is going to be talking to us and educating us on how structured content contributes useful context under the hood and how it works.
00:01:14.400 --> 00:01:16.640
With that, Patrick, welcome to my show.
00:01:17.280 --> 00:01:18.000
Great to be here, Zohra.
00:01:18.000 --> 00:01:18.959
It's very nice.
00:01:19.599 --> 00:01:23.120
Patrick, first tell us a little about yourself, what you do, and yourself.
00:01:24.000 --> 00:01:24.640
Sure, happy to.
00:01:24.640 --> 00:01:25.760
So I'm Patrick Bosek.
00:01:25.760 --> 00:01:28.879
I am one of the founders of Heretto.
00:01:28.879 --> 00:01:30.239
I'm also the CEO.
00:01:30.239 --> 00:01:34.000
So I've been in the structured content industry for quite a while.
00:01:34.000 --> 00:01:39.200
As the uh some I guess I could be considered the face of the company sometimes.
00:01:39.200 --> 00:01:43.519
So in that role, I also do quite a bit of media things.
00:01:43.519 --> 00:01:46.000
So I have a talk show I do with Scott Abel.
00:01:46.000 --> 00:01:48.078
So that's called Coffee and Content.
00:01:48.078 --> 00:01:54.319
We've been on a little bit of a hiatus due to a technical issue, but we typically do Coffee and Content every other week.
00:01:54.319 --> 00:01:58.319
So those are a lot of fun, cover a lot of different topics from the content world.
00:01:58.319 --> 00:02:02.000
We also have a podcast called Content Components.
00:02:02.000 --> 00:02:06.400
If you look it up on Spotify, that's got four or five seasons.
00:02:06.400 --> 00:02:10.240
It's all about documentation, structured content, those kinds of things.
00:02:10.240 --> 00:02:15.120
So if you're wanting nerd out about structured content, that's a great place to go.
00:02:15.120 --> 00:02:15.598
Yeah.
00:02:15.598 --> 00:02:17.520
And then I'm a developer by background.
00:02:17.520 --> 00:02:21.758
So I was one of the developers and architects of the first version of our software.
00:02:21.758 --> 00:02:26.318
So I'm I'm fairly technical, but I don't cause problems for our development team anymore.
00:02:26.318 --> 00:02:30.800
Nowadays, I focus on what's going to make customers happy and those kinds of things.
00:02:31.360 --> 00:02:35.360
What spawned this idea to look into structured content as a developer?
00:02:35.360 --> 00:02:36.479
I'm curious.
00:02:37.360 --> 00:02:37.598
Oh boy.
00:02:37.598 --> 00:02:41.360
So that's actually going back a lot further than I'd like to admit.
00:02:41.360 --> 00:03:04.318
But my co-founder and I, we got into structured content, not DITA, which is where we are today, but XML content with an actually with a custom schema when we were in college, actually, and we just saw that it was a superior way to store and manage content and that it was going to make some of the publishing activities we were working on easier and more capable.
00:03:04.318 --> 00:03:10.878
At that point in time, the technologies surrounding digital were just far, far less well developed than they are today.
00:03:10.878 --> 00:03:18.878
So structured content provided it actually provided something quite different at that moment in time, now that I'm reflecting on it, than it does today.
00:03:18.878 --> 00:03:25.520
At that moment in time, it it basically gave you the ability to put, like, a little bit of a database in a file.
00:03:25.520 --> 00:03:40.240
The amount of things you had to manage and the flexibility you got when you were using structured content at that point in time, if you're looking to apply that to like a very structured digital publishing environment, there was a lot of advantages there.
00:03:40.240 --> 00:03:42.158
Um so that was why we picked it up originally.
00:03:42.158 --> 00:04:02.080
And then over the years, we eventually drifted into, or I guess found our way, is maybe a better way of putting it, into technical content where you still get a lot of those benefits that we we were originally looking at with DITA and in the digital publishing around uh technical content and documentation, but there's a ton more today.
00:04:02.080 --> 00:04:04.639
So there's there's so much more that you get from it.
00:04:05.199 --> 00:04:08.158
Yeah, we could probably have a whole episode on structured content.
00:04:08.158 --> 00:04:12.960
And I have, as I mentioned, had some episodes covering what that is.
00:04:12.960 --> 00:04:16.160
And it seems to me like the context can be different.
00:04:16.160 --> 00:04:28.319
Of course, DITA comes close to mind, but for those of us that don't use DITA, structured content still follows a certain framework, a certain guidelines, if you will.
00:04:28.319 --> 00:04:41.439
But for this conversation, I want to ground what that definition means to you, and then we can, I guess, start talking about how we can deep dive into how that lends to AI.
00:04:42.160 --> 00:04:48.560
Yeah, I should think that is really important to start with the definition of structured content because everybody does define it a little bit differently.
00:04:48.560 --> 00:04:52.480
And I think the individual definition matters.
00:04:52.480 --> 00:04:55.040
So I'll give you my definition.
00:04:55.040 --> 00:04:57.278
I'll also tell you why I think mine is right.
00:04:57.278 --> 00:05:01.199
But that doesn't mean that other people's don't uh can't be right too.
00:05:01.199 --> 00:05:21.920
So technically speaking, if you put all of your content into Excel or Google Sheets and you put it into, you know, let's say every topic was a column, or maybe better a row, and every portion of that topic or that piece of information was divided up into different columns.
00:05:21.920 --> 00:05:25.120
So you have the pieces broken up into the different cells.
00:05:25.120 --> 00:05:31.120
That would be structured content because you would have the content broken up into a structure.
00:05:31.120 --> 00:05:35.519
Like in the most basic definition, you have a structure for your content.
00:05:35.519 --> 00:05:41.519
So you're you're labeling the different pieces of it in a methodology, and that's structured content.
00:05:41.519 --> 00:05:43.519
Now, obviously, that's a terrible choice.
00:05:43.519 --> 00:05:48.000
Like that wouldn't be a thing that you'd want to do in almost any production environment.
00:05:48.000 --> 00:05:56.160
Although I will say that we have had customers migrate from that exact environment just like that to our tool, which is DITA.
00:05:56.160 --> 00:06:09.759
But when you think about like how this comes together, the reason I start there is because there are a lot of systems that are they're effectively that, because if you think of what a form is, so like a digital form, it is just actually a table.
00:06:09.759 --> 00:06:12.879
You have your question one, question two.
00:06:12.879 --> 00:06:15.278
So your question one might be what is the heading for this file?
00:06:15.278 --> 00:06:19.519
Your question two might be who authored this file, question three might be the body.
00:06:19.519 --> 00:06:24.720
And people will call that structured content because, you know, you can put metadata in one place.
00:06:24.720 --> 00:06:29.439
And and in certain circumstances, that's what headless CMSs are, for instance.
00:06:29.439 --> 00:06:32.560
So, like things like Contentful, those kinds, that's that's what they are.
00:06:32.560 --> 00:06:35.278
In certain circumstances, that actually works quite well.
00:06:35.278 --> 00:06:38.000
Um, that's a really effective form of structured content.
00:06:38.000 --> 00:06:47.040
It's backed by a database, uh, it has the ability to create links between those pieces of information, uh, it has the metadata separated out.
00:06:47.040 --> 00:07:00.079
Like you have like there are certain things, but it in the background, it is functionally a table that is fundamentally different than what we work with, which is XML, because XML is a tree, which means that you can have structure below those big boxes.
00:07:00.079 --> 00:07:04.639
So when you go into a topic, there's going to be structure at multiple levels.
00:07:04.639 --> 00:07:10.240
So you'll have a piece, like let's say we'll use a procedure or a task topic, for instance.
00:07:10.240 --> 00:07:18.639
When you start creating that, you'll have a title, and then you'll have metadata around it, and then you'll have the prerequisites for running this procedure.
00:07:18.639 --> 00:07:24.079
And then the individual procedure, instead of being an ordered list, it would be steps, right?
00:07:24.079 --> 00:07:28.079
And then there's a step inside of it, there's structure inside of the steps.
00:07:28.079 --> 00:07:38.480
And so by having this multiple layers of structure and semantics, it means that you're able to manipulate that in a more granular way beyond just formatting.
00:07:38.480 --> 00:07:42.959
It's not just HTML and boxes, effectively, but they're both structured.
00:07:42.959 --> 00:07:47.759
Like there is a, like, both things are valid in terms of the content label.
00:07:47.759 --> 00:07:54.480
So, what you really need to ask yourself when you're thinking about, okay, well, we're moving to structure, it's beyond just the term structure.
00:07:54.480 --> 00:08:00.000
Like you should really be thinking about what is the form of that's going to serve our mission?
00:08:00.000 --> 00:08:02.720
What do we care about in terms of interoperability?
00:08:02.720 --> 00:08:07.199
What forms of linking are really important to us?
00:08:07.199 --> 00:08:10.240
What forms of output are going to be really important to us?
00:08:10.240 --> 00:08:18.560
Because those types of questions will guide you towards the type of structure that's going to be best for your particular implementation and your particular needs.
00:08:19.360 --> 00:08:24.000
I have to say that I thought my definition was going to be very different from yours or my understanding.
00:08:24.000 --> 00:08:28.160
But at some level, I think we are on the same page.
00:08:28.160 --> 00:08:28.800
Yeah.
00:08:28.800 --> 00:08:30.639
I mean, I think my definition is pretty broad.
00:08:30.639 --> 00:08:31.920
Like I said, I think it's right too.
00:08:32.399 --> 00:08:33.278
Yes, it's very inclusive.
00:08:33.278 --> 00:08:39.120
So that allows me to be kind of okay, that definition sounds close to what I have in my mind.
00:08:39.120 --> 00:08:40.000
Fantastic.
00:08:40.000 --> 00:08:41.120
Yes.
00:08:41.120 --> 00:08:48.960
I have not worked with DITA, and you mentioned DITA, and DITA comes up a lot when we talk about structured content.
00:08:48.960 --> 00:08:57.918
We've touched upon this before, but I think for the benefit of the audience, it would be nice to quickly do a shout out to what DITA is.
00:08:58.399 --> 00:08:58.798
Yeah, for sure.
00:08:58.798 --> 00:09:02.480
So DITA, in the most basic definition, it's it's a form of XML.
00:09:02.480 --> 00:09:07.840
So in that way, you can actually think of it as like a cousin to HTML in some ways.
00:09:07.840 --> 00:09:11.120
So it actually shares a lot of the same elements that HTML has.
00:09:11.120 --> 00:09:20.480
The thing to know about DITA is that it has built in many of the patterns that you need to scale up technical content.
00:09:20.480 --> 00:09:31.678
So there's a ton of stuff that we've pretty much figured out over the last call it 15 or 20 years, are really critical to larger or more complex sets of technical content.
00:09:31.678 --> 00:09:39.519
And the DITA committee, so the people who work on the open source standard, they put all of those things into the standard.
00:09:39.519 --> 00:09:41.360
So you don't have to recreate them.
00:09:41.360 --> 00:09:48.399
And then perhaps just as importantly, beyond not having to recreate them, you don't have any vendor opinion or lock-in.
00:09:48.399 --> 00:10:04.798
So one of the challenges that the that DITA was trying to solve, and I think has solved pretty effectively, is that when you have vendor-specific formats, it means that okay, well, we've reused content, but we've reused content in this particular system.
00:10:04.798 --> 00:10:14.080
And once you get to a certain level of reuse, if you didn't have a portable technology underneath it, your ability to migrate someplace else is I mean, it's it's almost impossible.
00:10:14.080 --> 00:10:17.759
It just becomes the lock in there, it's just so high.
00:10:17.759 --> 00:10:29.120
So what DITA really solves is it solves having good patterns for the core things that you really need to do with technical content, and then it also solves an interoperability problem.
00:10:29.120 --> 00:10:34.158
And you know, there's some of the criticisms of DITA are that it comes with you know like a high learning curve.
00:10:34.158 --> 00:10:36.320
Yeah, I don't know if I actually agree with that.
00:10:36.320 --> 00:10:38.240
Like I've seen a lot of people learn it pretty quickly.
00:10:38.240 --> 00:10:41.759
There's people feel that it's too heavy for today's uses.
00:10:41.759 --> 00:10:42.720
That's another criticism.
00:10:42.720 --> 00:10:45.440
Those people tend to come from like the Markdown world.
00:10:45.440 --> 00:10:50.639
So, like in comparison to Markdown, you know, Markdown is a text format, DITA is an is an XML format.
00:10:50.639 --> 00:10:52.558
I think that's just a use case thing.
00:10:52.558 --> 00:10:58.798
So like DITA is not a great, is not like you never write the README files for your vibe coded project in DITA.
00:10:58.798 --> 00:10:59.918
I don't do that.
00:10:59.918 --> 00:11:04.960
Like when I have small apps and things like that that I build, I don't write any of the documentation in DITA.
00:11:04.960 --> 00:11:06.158
It doesn't make any sense.
00:11:06.158 --> 00:11:13.360
DITA is a format that you use when you arrive at a sufficient level of complexity and a sufficient scale.
00:11:13.360 --> 00:11:24.320
So if you want like really rough and tough numbers, what we've found over the years is that if you're below a thousand pages of content, DITA probably doesn't make sense.
00:11:24.320 --> 00:11:39.200
Um, there are some exceptions, but even at a thousand pages of content, like it can be questionable there depending on your And so where it really starts to make sense is when you have multiple variations of products or variations of content, audiences.
00:11:39.200 --> 00:11:49.519
So you have anything that's like personalization, and there's a bunch of different terms that people use for the content changes when somebody looks at it for a different person looks at it for a different reason.
00:11:49.519 --> 00:11:53.120
Localization can be a driver, although I think less so today.
00:11:53.120 --> 00:11:55.759
And then just sheer scale with like reuse.
00:11:55.759 --> 00:12:02.000
So if you have a lot of content that you is used in multiple places, that's when you know DITA starts to really shine.
00:12:11.278 --> 00:12:12.399
In a simplified way.
00:12:12.399 --> 00:12:30.720
Since we are going to do a deep dive into how structured content lends itself to AI, does DITA lend itself better, or just this the generic definition of the structured content that we ground it, that we are trying to ground this conversation in also lends itself to better AI interactions?
00:12:31.840 --> 00:12:32.798
That is such a good question.
00:12:32.798 --> 00:12:36.798
So the answer is yes, it depends in here.
00:12:36.798 --> 00:12:41.678
And I feel like I'm among the right people to give like engineering style, it depends answers.
00:12:41.678 --> 00:12:46.558
One of the jokes in our leadership team is that when you ask the business people a question, they give you an answer.
00:12:46.558 --> 00:12:49.200
When you ask the technical people a question, they say it depends.
00:12:49.200 --> 00:12:53.360
And so I think that I'm I'm filling that role in this particular moment.
00:12:53.360 --> 00:12:57.120
So, you know, this is kind of like what we we wanted to cover today.
00:12:57.120 --> 00:12:59.678
So it is a it's a very, very long answer.
00:12:59.678 --> 00:13:10.320
At the very core of it, if, like, you wanted, like, the too long, didn't listen version of it, structured content is going to give you better structured contents with the broad definition.
00:13:10.320 --> 00:13:20.240
So like anything that is broken up into labeled pieces that are then stored consistently, like that definition of structured content.
00:13:20.240 --> 00:13:28.720
So that's inclusive of DITA, inclusive of, uh, headless CMSs, inclusive of anything that has a well-defined schema applied to content.
00:13:28.720 --> 00:13:31.759
So, why would that be better for AI?
00:13:31.759 --> 00:13:37.278
And it really just comes down to chunking, like that's the fundamental difference between those things.
00:13:37.278 --> 00:13:46.558
So the problem with that statement is that when you get to certain forms of structured content, and I'll like I'll give you like an example from a headless CMS.
00:13:46.558 --> 00:13:50.320
The headless CMS is every headless CMS is gonna have a different schema.
00:13:50.320 --> 00:13:55.918
Like part of the the appeal of a headless CMS is that you can create your own schema.
00:13:55.918 --> 00:14:06.000
So you go in and you get a blanks blank page, and all you have to have is a title, and then you can add a body and an author, and you could add you know links to other things.
00:14:06.000 --> 00:14:13.200
And like the standard thing is like, you know, how would you lay out a one of the standard examples is how would you lay that out for like a music venue?
00:14:13.200 --> 00:14:18.720
And you'd have one thing that was a band, and then you'd have an artist, and then you'd have like those kinds of things, right?
00:14:18.720 --> 00:14:20.000
And you'd you build those schemas.
00:14:20.000 --> 00:14:31.038
Well, in a really well-designed schema, it's gonna function really well as giving you good sized pieces of content for the chunking.
00:14:31.038 --> 00:14:40.720
In a poorly designed schema, what's gonna happen is you're gonna get people basically putting in a title and a body, and like that's really not any different than a Google Doc, right?
00:14:40.720 --> 00:14:45.759
So at that point in time, people are just gonna throw huge quantities of content into that body field.
00:14:45.759 --> 00:14:54.960
So even though technically you have two fields, the structure is two boxes, like it's one box, it's it's two boxes with unstructured content in one of them.
00:14:54.960 --> 00:14:57.278
And so then you lose the benefits of chunking.
00:14:57.278 --> 00:15:19.200
So now there's a bunch of other things, but if you like if you want the biggest bang for your buck, so like the I don't know, the 10% that gives you 60% outcome, it's having really well, clean, defined borders around your content, which means smaller pieces, more intentional pieces, pieces of content that contain ideally one piece of information.
00:15:19.200 --> 00:15:23.038
So in DITA, like a topic is supposed to be about one thing.
00:15:23.038 --> 00:15:28.240
It like if it's a concept topic, it's supposed to teach you one thing or explain one thing.
00:15:28.240 --> 00:15:33.200
If it's a task, it's supposed to give you a procedure that does one thing.
00:15:33.200 --> 00:15:39.678
Um, so it takes you from from the beginning to end of one thing you have to complete, and there's other versions of that as well.
00:15:39.678 --> 00:16:05.038
So in the AI world, when you go to produce results with an AI system, being able to provide it very clean chunking, so very clean set like pieces of content for the classification on when you're putting it into a RAG system, and then also for retrieval, there's a bunch of other things, but that's the thing that gets you the biggest bang for your buck.
00:16:05.759 --> 00:16:11.918
Modern language models are pretty good at extracting meaning from unstructured content.
00:16:11.918 --> 00:16:15.840
Does that reduce the need for structured content?
00:16:15.840 --> 00:16:21.200
Because we're trying to make the case that you need to have chunking with structured content.
00:16:21.200 --> 00:16:32.399
But I'm kind of trying to flip that question and see if the LLMs have gotten so bad so good, or is there still an advantage to use structured content as we have established?
00:16:33.360 --> 00:16:35.038
Yeah, so I think that is also a great question.
00:16:35.038 --> 00:16:41.600
It's one of the places that there's I don't want to say because I think that's the wrong way to think about it.
00:16:41.600 --> 00:16:45.519
But I think that there's so maybe this is a better way to put it.
00:16:45.519 --> 00:16:52.158
You're right that the the modern LLMs have gotten very good at extracting information when the question is basic.
00:16:52.158 --> 00:17:03.120
And the reality is that most of the time when people are interacting with these LLMs, their their modalities of interaction fall into a couple of categories.
00:17:03.120 --> 00:17:06.558
One is coding, and coding is a verifiable tree.
00:17:06.558 --> 00:17:19.278
So like part of the reason coding has gotten so good is not because the LLMs have gotten so good, it's because the harnesses have gotten so good at managing something that is verifiable, like that.
00:17:19.278 --> 00:17:20.318
It's logically verifiable.
00:17:20.318 --> 00:17:26.480
The other modality that we use, we use these in is effectively a replacement for search, right?
00:17:26.480 --> 00:17:29.038
So you go to Google and like you look at the AI result.
00:17:29.038 --> 00:17:35.519
Again, a lot of the advancement there is actually in the deterministic harness under the hood.
00:17:35.519 --> 00:17:51.920
Like the amount of engineering that Google and Anthropic and Chat or and OpenAI have done so that what is being delivered to the AI for consideration is high quality, is probably equivalent to the amount of like engineering they've done in the AI model.
00:17:51.920 --> 00:17:54.960
Well, maybe not, I don't know, but like it's a lot is the point.
00:17:54.960 --> 00:18:05.278
And then when you're but when you're thinking about interacting with these systems in a more general sense, typically people are putting very small quantities of information into them.
00:18:05.278 --> 00:18:14.798
So like if I go to NotebookLM and I throw in a couple of PDFs, it does work quite well because the quantity of information is very small.
00:18:14.798 --> 00:18:21.038
Now, in the background, it is still chopping that PDF up and it is still indexing it.
00:18:21.038 --> 00:18:25.440
Like it is not just feeding that PDF full stock to the to the LLM.
00:18:25.440 --> 00:18:46.558
But what's happening when you ask that the question, or you're working with the system to get information out of that, is that because it's a small number of chunks, and because the system is then reducing down what it's sending to the LLM to a smaller set, it's not that big of a deal.
00:18:46.558 --> 00:18:48.960
The problem is when you scale up.
00:18:48.960 --> 00:18:55.680
So there's a reason you can only put, I don't know what it is, it's not a huge quantity of content into NotebookLM.
00:18:55.680 --> 00:19:10.640
When you start talking about thousands of pages or tens of thousands of pages or more, at that point in time, especially if you're asking more human questions as opposed to like search queries, that's when it absolutely still breaks down.
00:19:10.640 --> 00:19:12.880
And like we have research that shows this.
00:19:12.880 --> 00:19:16.240
So like this isn't this isn't just me hypothesizing these things.
00:19:16.240 --> 00:19:18.400
Like this is this is stuff that we can show.
00:19:18.400 --> 00:19:35.759
And in those worlds where having really strong RAG retrieval, which is built on the foundation of having good tokenization of the content, you start to see structured content outperform unstructured content by about 50%.
00:19:35.759 --> 00:19:38.078
Now it's use case dependent.
00:19:38.078 --> 00:19:43.920
Like there are like simpler questions will do better in both cases, more complex questions will do better.
00:19:43.920 --> 00:19:46.318
So there's there's a lot of it depends in there.
00:19:46.318 --> 00:19:55.598
But for things that really matter, especially in like regulated industries, or I mean, I guess everybody should care about their customers getting the right answer, but you know, like some people care more than others, I suppose.
00:19:55.598 --> 00:20:01.519
It still is a substantial difference in overall quality of retrieval and accuracy.
00:20:03.038 --> 00:20:11.200
So I think this is a great, I guess, segue into the RAG, uh, the retrieve and generate model that we start looking into.
00:20:11.200 --> 00:20:19.759
Under the hood, retrieval is evolving from a fairly linear model towards more agents that can decide what information they need to go and find.
00:20:19.759 --> 00:20:24.880
Where does structured content fit into this newer model of agentic retrieval and context engineering?
00:20:25.680 --> 00:20:29.358
Yeah, I so I think that's an interesting area of evolution right now.
00:20:29.358 --> 00:20:35.519
And I guess the first thing I want to call out here is that the models are themselves getting better.
00:20:35.519 --> 00:20:37.200
Like there's no doubt about that.
00:20:37.200 --> 00:20:46.240
But I actually think the advancement in RAG systems is more in the harness sophistication than it actually is in the models right now.
00:20:46.240 --> 00:20:56.318
So when you think about like how does an AI work with a a RAG like content operations or RAG architecture such that it's going to produce a better outcome.
00:20:56.318 --> 00:21:00.318
Well, actually, should we stop and like give a definition of what of like what RAG is before?
00:21:00.960 --> 00:21:01.200
Yes, please.
00:21:01.200 --> 00:21:01.598
Absolutely.
00:21:01.598 --> 00:21:02.720
Go for it.
00:21:03.038 --> 00:21:03.358
Yeah, okay.
00:21:03.358 --> 00:21:11.278
So RAG is retrieval augmented generation, which is technical people trying to confuse the world with with fancy words.
00:21:11.278 --> 00:21:15.680
All it means is that you're running a deterministic on the well, you're running a search.
00:21:15.680 --> 00:21:18.480
I was about to say deterministic search, but that's not necessarily true.
00:21:18.480 --> 00:21:26.720
You're running a search, the output of that search is then being put into the context for an AI agent to generate a result.
00:21:26.720 --> 00:21:27.598
That's all it means.
00:21:27.598 --> 00:21:31.118
So it's it's just search and then generate, is what it comes down to.
00:21:31.118 --> 00:21:35.118
Now, most RAG systems are built on vector databases.
00:21:35.118 --> 00:21:52.880
And why it matters to call that out is because fundamentally, vector databases, what they are is they're just they're calculations around a piece of information as to like kind of where it sits in space, and they what they do is they retrieve things that are close together.
00:21:52.880 --> 00:21:56.078
Now, those calculations are done by LLMs typically.
00:21:56.078 --> 00:22:04.160
When you look at like some of the research that we've done, like When you look at, like, some of the research that we've done, like of different ways of calculating the embeddings.
00:22:04.160 --> 00:22:08.640
So the embeddings are like the position, I guess, if you want to think about it that way.
00:22:08.640 --> 00:22:16.160
And the way that we we do that is that you can calculate the embeddings with a bunch of different with any of the AI models that are out there.
00:22:16.160 --> 00:22:24.640
So what we've found is that in our testing, OpenAI tends to be the one that's most cost effective and easiest to do.
00:22:24.640 --> 00:22:26.318
And so that's the one that we've used.
00:22:26.318 --> 00:22:32.160
We've tried a bunch, but that's the one we've kind of settled on as like our baseline, which is funny because we use Anthropic everywhere else.
00:22:32.160 --> 00:22:33.759
But like that's how that comes together.
00:22:33.759 --> 00:22:45.519
So then you have the content in the vector database, the way you retrieve it is through these vector searches, and then when you retrieve sets of content, that's going to be put into the LLM.
00:22:45.519 --> 00:22:53.519
So the major difference here between like the first versions of these and the versions that we deal with today, the models are better, yes.
00:22:53.519 --> 00:23:10.400
But now you may see AI being in a place where you send the question to the LLM, the LLM modifies the question or it breaks it down into some way, or it clarifies the question, changes the question, and then it passes the query to the vector database.
00:23:10.400 --> 00:23:14.160
The vector database returns results, and that loop can continue.
00:23:14.160 --> 00:23:16.960
So reasoning is largely just a loop, right?
00:23:16.960 --> 00:23:19.038
Like they basically feed the results back in.
00:23:19.038 --> 00:23:25.519
There could be some deterministic logic that does some parsing or extraction between the LLM and the database.
00:23:25.519 --> 00:23:32.558
Like there's other ways that they, you know, they can basically upgrade the overall loop, but it really fundamentally is a loop.
00:23:32.558 --> 00:23:38.400
So I think the misconception that a lot of people have is they think all of this is happening inside of the LLM.
00:23:38.400 --> 00:23:44.960
Like the LLM is making a decision about what information it's retrieving, but like the information is in the LLM.
00:23:44.960 --> 00:23:48.880
The information that you're when you're talking about RAG is not in the LLM.
00:23:48.880 --> 00:23:55.278
It's in a separate piece of technology that the LLM is making, is having an interaction with?
00:23:55.920 --> 00:24:04.640
When I started working with structured content, metadata became a centerpiece to how a cornerstone to how search was done.
00:24:04.640 --> 00:24:09.759
So the the richer the metadata, the better the output was.
00:24:09.759 --> 00:24:12.558
So organizing information was super critical.
00:24:12.558 --> 00:24:24.400
What changes when a machine is using that metadata to decide what information belongs in an AI model's context with all the technology breakdown that you've given us?
00:24:24.400 --> 00:24:24.960
Yeah.
00:24:24.960 --> 00:24:29.358
So this could be a whole series of podcasts in and of itself.
00:24:30.078 --> 00:24:30.240
Of course.
00:24:30.798 --> 00:24:31.038
What changes?
00:24:31.038 --> 00:24:32.798
Well, oh boy.
00:24:32.798 --> 00:24:41.038
The first thing I think that's important to think about here is that metadata gives you a lot of options in how you process something.
00:24:41.038 --> 00:24:58.400
So one of the key aspects of having really high quality metadata on your content is it has hooks and, like, levers that you can pull in deterministic processes that decide how content ends up and how it shows up.
00:24:58.400 --> 00:24:58.960
Right.
00:24:58.960 --> 00:25:13.118
So when you're thinking about what you're gonna end up providing to an LLM, you almost always have some type of a pipeline that's going to be coming into whatever your RAG system is or whatever, wherever the LLM is actually going to be touching that content.
00:25:13.118 --> 00:25:22.640
And metadata means that you can be more specific about what and how those things go into the uh the LLM or the AI-based system.
00:25:22.640 --> 00:25:26.960
So it gives you a lot more control with traditional code.
00:25:26.960 --> 00:25:31.358
And I'm emphasizing that because it really shouldn't be undervalued.
00:25:31.358 --> 00:25:33.838
That is still enormously powerful.
00:25:33.838 --> 00:25:50.078
The more specific you can be in a deterministic way in the systems that you build, the more confidence you can have and the more sustainable those systems are going to be, because you can write unit tests, right?
00:25:50.078 --> 00:25:53.200
Like things in deterministic systems have deterministic outcomes.
00:25:53.200 --> 00:25:56.078
Like that's fundamentally how they're different than AI systems.
00:25:56.078 --> 00:25:59.598
So metadata still has a ton of value in that way.
00:25:59.598 --> 00:26:03.118
This is kind of a the other side of that coin, but I think also valuable.
00:26:03.118 --> 00:26:15.358
There are circumstances where traditional search, if metadata is effectively applied, will perform equally well to vector search, and it's a lot less expensive.
00:26:15.358 --> 00:26:20.078
So there's certain applications where the old stuff works just as well.
00:26:20.078 --> 00:26:26.000
And there's actually cases where it works better, but then you know, we don't have to go, we don't have to dig too deeply into those things.
00:26:26.000 --> 00:26:29.519
But the point is that metadata also gives you options.
00:26:29.519 --> 00:26:46.160
So if you're in a place where you're completely reliant on AI systems and you don't have really strong metadata around how you're trying to build a retrieval pipeline, um, then you're totally reliant on the inference of context from the AI into a vector database.
00:26:46.160 --> 00:26:57.598
Whereas if you had that metadata, you might be in a place where you could have a deterministic system or a traditional search that was just as just as good or better, um, and also gave you some other options.
00:26:57.598 --> 00:27:04.078
So metadata can be really valuable in terms of comparison of not just an optionality for your system.
00:27:04.078 --> 00:27:07.038
So you have optionality, you have comparison, like you have those types of things.
00:27:07.038 --> 00:27:32.880
And then when you think about like how does it actually benefit the LLMs themselves, this is where you get into like a very interesting discussion because there are aspects of it that can be really Typically, what it means is that you're transitioning structured metadata into, I would say, like statements for the LLM, such that it is still context, right?
00:27:32.880 --> 00:27:38.880
But like LLMs receive context differently than like humans and traditional machines to some extent.
00:27:38.880 --> 00:27:42.480
But if you have everything tagged for, so I'll give you an example.
00:27:42.480 --> 00:27:44.640
Let's say you have two products.
00:27:44.640 --> 00:27:47.838
You have a we'll use washing machines, right?
00:27:47.838 --> 00:27:55.200
Let's say you have a 1000 version and a 1001 version, and those are very, very similar systems, right?
00:27:55.200 --> 00:27:56.960
They're the same line of washing machine.
00:27:56.960 --> 00:27:58.240
There's a few differences.
00:27:58.240 --> 00:28:06.960
Well, you've got a hundred topics for each of those systems, and let's say 95 of them are the same.
00:28:06.960 --> 00:28:12.400
So they all carry the 1000 and the 1001 tag, and then you have five that diverge.
00:28:12.400 --> 00:28:18.960
So you have 105 topics in total, and you know, five with a thousand, five with a thousand and one.
00:28:18.960 --> 00:28:26.640
If you're just stuffing all of that into an LLM, and remember, like this is only 105 topics, so you can probably get away with that.
00:28:26.640 --> 00:28:33.200
The way you would use metadata there, and notice I'm specifically avoiding RAG and search in this particular example.
00:28:33.200 --> 00:28:56.798
Like I'm just talking about how you put the whole 105 in, is instead of having a tag on that, you would probably be better off actually having a statement at the top of it where before you put it in, you'd use a deterministic system to take the tag from you know product and then it had the tag name to this is the 1001 version of this washing machine.
00:28:56.798 --> 00:28:59.680
This is the 1000 version of this washing machine.
00:28:59.680 --> 00:29:01.358
You just put that at the top of every topic.
00:29:01.358 --> 00:29:07.920
And an LLM is going to interpret that better than having like the actual tag on there.
00:29:07.920 --> 00:29:10.078
And so I've done a bunch of testing around this.
00:29:10.078 --> 00:29:12.480
Like, this also is the same with like glossary terms.
00:29:12.480 --> 00:29:21.440
So if you have like structured glossaries, um, different LLMs will receive the pieces of that glossary, which are tags, right?
00:29:21.440 --> 00:29:22.000
Like it's metadata.
00:29:22.000 --> 00:29:26.400
Like you're like, you know, this is an abbreviation, this is another way of saying it.
00:29:26.400 --> 00:29:29.920
Like, there's like right, it that's just metadata that describes different pieces of it.
00:29:29.920 --> 00:29:40.160
If you write it out as prose, even if it's the same prose over and over again, the LLMs tend to work with that version of that content better.
00:29:40.160 --> 00:29:57.278
So transitioning the structured metadata into clear and simple statements, even if they're repetitive for the LLM, tends to produce a better result today with the LLMs we have today, because of how they've been trained and you know the biases and like the internal weightings that they have.
00:29:57.278 --> 00:30:00.318
Like you could train one that works better with the actual structure.
00:30:00.318 --> 00:30:02.400
That's just not the ones that we have today.
00:30:02.400 --> 00:30:06.000
So metadata touches a ton of different pieces of it.
00:30:06.000 --> 00:30:13.440
And I haven't even gone into like how metadata can then be used to build graphs and to create like more sophisticated like selection mechanisms.
00:30:13.440 --> 00:30:31.200
But the point is that once you get to any level of complexity, having the metadata on the content gives you so much more optionality in terms of how you build the systems around the LLM, and then also how you actually instruct the LLM when you're putting inputs into it.
00:30:31.680 --> 00:30:36.880
I think this was such a great overview because my goal here was to kind of step back.
00:30:36.880 --> 00:30:57.598
And for those amongst us who have very, I guess the as a technical writer, one would have that structured content, the concept But if you don't, if you just have a Google Doc and you're starting off, then adding metadata to that, you've given us a why, a solid why.
00:30:57.598 --> 00:30:59.519
Why should you still consider it?
00:30:59.519 --> 00:31:04.160
And how that would kind of work under the hood that I keep going back to.
00:31:04.160 --> 00:31:11.519
And I would love for us to maybe even dive into the knowledge graphs and how they would relate to metadata.
00:31:11.519 --> 00:31:25.038
With more than 70 sessions and workshops featuring speakers from companies like Salesforce, TikTok, T-Mobile, and Amazon.
00:31:25.038 --> 00:31:32.880
LavaCon brings together content strategists, documentation leaders, and content professionals to share practical ideas you can apply right away.
00:31:32.880 --> 00:31:39.920
And yes, it's known as the Fun Conference, too with networking events, therapy dogs, comfort llamas, storytelling, and karaoke.
00:31:39.920 --> 00:31:44.160
Use discount code ITC26 to save$200 on registration.
00:31:44.160 --> 00:31:47.920
You did mention graphs here.
00:31:47.920 --> 00:31:52.400
So I think it's worth going into that a little bit, if not more.
00:31:53.118 --> 00:31:53.440
Yeah, sure.
00:31:53.440 --> 00:31:54.798
So I'm happy to go.
00:31:54.798 --> 00:31:58.640
Well, there are people who are are experts in knowledge graph.
00:31:58.640 --> 00:32:01.358
I wouldn't consider myself an expert in knowledge graphs.
00:32:01.358 --> 00:32:04.960
I do know I I would say a pretty good amount about them.
00:32:04.960 --> 00:32:10.880
So there's probably a limit to how far I can go, but I'm very happy to talk about them in relation to structured content metadata.
00:32:10.880 --> 00:32:15.519
So maybe uh start with just the definition of what a knowledge graph is.
00:32:15.519 --> 00:32:24.400
So so a knowledge graph is if you want to think about it in technical terms, it's a database that has typed nodes and typed connections between those nodes.
00:32:24.400 --> 00:32:33.440
So you could have people and then you could have, like, a 'knows' or like this person knows this person or is married to.
00:32:33.440 --> 00:32:36.400
Obviously, that would be a one-to-one connection in most cases.
00:32:36.400 --> 00:32:39.200
But like, you know, so this is like the friend of a friend stuff.
00:32:39.200 --> 00:32:41.920
Like, so like ontologies, knowledge graphs, those kinds of things.
00:32:41.920 --> 00:32:52.160
When you put this information into graph databases, what it means is you can start asking multi-level questions of the database.
00:32:52.160 --> 00:33:04.480
So the thing that people really like about them is if you go back to like the people example, you could ask a question of a knowledge knowledge graph of like, give me all the companies for people that Patrick has been on podcasts with.
00:33:04.480 --> 00:33:34.078
And a good knowledge graph, like a good knowledge graph database would be able to go and say, Okay, well, Patrick is a podcaster, and then there would be a field that's like has been on, or something like that, and that would link out to a bunch of different podcasts, and then each of those podcasts would have guests and hosts, and each of those guests and hosts would be people, and all the people would have as employment or you know, works for something along those lines, right?
00:33:34.078 --> 00:33:41.038
And so you could have a query that would say like doop, doop, doop, doop, doop, and hop across all those things and select all of those companies.
00:33:41.038 --> 00:33:53.759
That's why people like knowledge graphs, is because it allows you to ask those kinds of questions of an information set, which are traditionally very difficult to ask in the context of like relational databases.
00:33:53.759 --> 00:34:00.000
So you can do some of that stuff in relational databases, but it's much less flexible, I guess is the best way to put it.
00:34:00.000 --> 00:34:09.039
So you're dealing with very large and you know well tagged or very semantic sets of content, like you're just gonna do better in a knowledge graph.
00:34:09.039 --> 00:34:20.079
So, how this collides with structured content and metadata is that it's not always that you're typically you're not actually keep maintaining your content in the knowledge graph, right?
00:34:20.079 --> 00:34:22.400
So you're maintaining your content in another location.
00:34:22.400 --> 00:34:29.599
So the way that you you correlate, you tie the content to the knowledge graph is through the metadata in many cases.
00:34:29.599 --> 00:34:37.280
So the metadata that becomes the selection mechanism that the the knowledge graph can use to pull that out.
00:34:37.280 --> 00:34:51.920
You can also use direct linking, but you oftentimes like the metadata is going to become like a major piece of like how those two things come together, or how the knowledge graph forms the nodes in the graph out of the content, like it contributes to those things.
00:34:52.960 --> 00:35:00.800
I'm thinking in a typical environment, you have documentation coming from product, from engineering, from support.
00:35:00.800 --> 00:35:11.760
How should the infrastructure establish authority and provenance and a source of truth within the context of that uh of structured content?
00:35:11.760 --> 00:35:14.719
Because that content may be structured differently.
00:35:14.719 --> 00:35:17.280
What's your take on that?
00:35:17.920 --> 00:35:19.920
Yeah, so I think this is a really timely question.
00:35:19.920 --> 00:35:36.079
So the the environment you've described, you know, very briefly, where like technical content comes from support, product engineering, um, the technical writers, like that is one version of companies where the the technical content is kind of like the result of all these contributors.
00:35:36.079 --> 00:35:39.039
That's how you get to those corpuses of content.
00:35:39.039 --> 00:35:50.480
The other structure of companies is where you have the technical writers is more of like a clearinghouse, like they're the ones that manage most of the content, and they probably write a large portion of it.
00:35:50.480 --> 00:36:10.639
And in those environments, you do tend to have knowledge bases that grow up around that, but there's a clear, very clear delineation between like the docs are gold plated, the knowledge bases can be a source, it can be leveraged, but it's not like the level of governance and quality that's expected is is very, very different.
00:36:10.639 --> 00:36:17.840
And I think it's very situational as to which which is going to work for any one individual company.
00:36:17.840 --> 00:36:28.000
You know, some companies can work really well in the more almost crowdsourced version of documentation, and then other companies really can't go that route.
00:36:28.000 --> 00:36:36.159
Like they have to have technical writers as the authorities on truth, more or less, like is the way to think about it.
00:36:36.159 --> 00:36:39.360
Um, and that's like a that's like um a center of excellence approach.
00:36:39.360 --> 00:36:46.559
I have a lot more experience in that approach because our technology lends itself to that approach.
00:36:46.559 --> 00:36:53.360
Like if you have the highly like crowdsourced collaborative version of it, like that's just not where we fit.
00:36:53.360 --> 00:36:55.599
So I don't work in that world as much.
00:36:55.599 --> 00:37:06.320
When you're trying to establish authority and governance across multiple groups like that, so again, I guess there's kind of two situations there.
00:37:06.320 --> 00:37:12.559
One is they're all using the same thing, which in my experience there is almost always a wiki or Google Docs.
00:37:12.559 --> 00:37:15.440
Like I haven't seen very many variations on that.
00:37:15.440 --> 00:37:24.559
Um, and that's just because when you have a large group of people who are not writers writing the content, you have to give them a tool that's not a writer's tool.
00:37:24.559 --> 00:37:35.360
It's like it's super uncommon that you'll see a circumstance where you have engineering product support and technical, and like maybe you know, a couple of technical writers working in technical writing tools.
00:37:35.360 --> 00:37:41.840
Like I think I've maybe seen that once in my entire career doing this for like I'm not even gonna say how many years, but more than a decade.
00:37:41.840 --> 00:37:46.000
It's much more common that you'll see you hit you go to the lowest common denominator.
00:37:46.000 --> 00:37:48.320
Maybe that's that's not exactly what I mean.
00:37:48.320 --> 00:37:52.480
You go to the most available system, which is almost always a wiki.
00:37:52.480 --> 00:37:57.039
Wikis are not built for governance, like there's just no way that.
00:37:57.039 --> 00:38:12.159
Like wikis are built for availability, and governance is an that imposes structure, it needfully imposes roadblocks and workflow and things like that, because that's where quality comes from.
00:38:12.159 --> 00:38:19.599
It comes from people doing the same things, knowing the patterns, checking themselves in predictable ways, that kind of stuff.
00:38:19.599 --> 00:38:21.199
Like that's governance, you know.
00:38:21.199 --> 00:38:24.159
Like you look at any like Six Sigma or Six Sigma, right?
00:38:24.159 --> 00:38:26.159
Like it's it's a set it's a set of patterns and practices.
00:38:26.159 --> 00:38:34.000
You can't apply those to a group of people that are not professionals in patterns and practices around that particular environment.
00:38:34.000 --> 00:38:37.280
So it becomes very challenging, is what it comes down to.
00:38:37.280 --> 00:38:41.440
You can't really have like broad inputs and high governance.
00:38:41.440 --> 00:38:52.079
And if you don't have high governance in your content, the work of like reverse engineering authority and like source of truth becomes much more complicated.
00:38:52.079 --> 00:38:54.960
It's not impossible in all cases.
00:38:54.960 --> 00:38:57.199
So I can give you an example of this.
00:38:57.199 --> 00:39:01.360
So like we have technical writers here, obviously, like that would be crazy if we didn't.
00:39:01.360 --> 00:39:10.400
But like our product team doesn't do the product specifications in Heretto, which is you know, Heretto is a documentation system.
00:39:10.400 --> 00:39:23.199
Like we do our product specifications in Confluence, and we do that because like we like product specifications are the sorry, these are the the design specs, not the actual, like these are these are not the final specs.
00:39:23.199 --> 00:39:25.199
We don't do the design specs in Heretto.
00:39:25.199 --> 00:39:28.559
So the design specs, because like they are working spaces, right?
00:39:28.559 --> 00:39:38.800
Like you've got a bunch of people in there and like they're throwing stuff at the wall, they're pulling things around, like they're just they're kind of screwing around in the content to figure out what's going to work while they're going through the process, the design process.
00:39:38.800 --> 00:39:48.480
It's a very creative, free-flowing process of like getting together, and that's just not the right place to apply like high structure and high governance.
00:39:48.480 --> 00:39:58.960
Like, you need to give people the minimum amount of those things to give them the most creative freedom to get to the end design results they're looking for.
00:39:58.960 --> 00:40:06.880
Now, at the same time, the outcome of these initial design documents that they do, they tend to be very high quality.
00:40:06.880 --> 00:40:08.880
So they're totally inconsistent.
00:40:08.880 --> 00:40:21.440
Like there's no like like they're not, you know, they're not in the same format of the same structure, but like they have really, really good information in them because they are for the initial design of something, like they kind of are the single source of truth.
00:40:21.440 --> 00:40:29.840
So in our world, what we do there, this is before the thing is built, before the documentation is written, all those kinds of things, is you just lock it down.
00:40:29.840 --> 00:40:35.599
Like once it's done, you just reduce the number of people that can change it, and the people that can change it are much more mindful.
00:40:35.599 --> 00:40:44.800
Now, we also have the benefit of the people who are working on this, are people who are who have all of the benefits of knowing structured content and documentation because that's a tool that we work on.
00:40:44.800 --> 00:40:59.599
So, like inherently, they're gonna be better about some of these practices, and you do see that bleed into you know building just frankly, probably more information architecture, you know, kind of by osmosis in some of these documents.
00:40:59.599 --> 00:41:02.639
So, like we see a lot of content come from Confluence.
00:41:02.639 --> 00:41:14.239
It's very common that people move from Confluence to Heretto, and it tends it comes in a wide variety of of qualities and like like positions from the structure and like a governance perspective.
00:41:14.239 --> 00:41:21.599
So, but we use those as inputs to to different LLM systems that we have, those those Confluence pages.
00:41:21.599 --> 00:41:38.320
So before a feature is built and before the content is migrated into Heretto and it's put on our internal like our internal point of reference is a thing we have like an internal name for it, that is all designed and built in Heretto, and that is considered the highest order of truth in the organization.
00:41:38.320 --> 00:41:40.159
Like if it's there, it's true.
00:41:40.159 --> 00:41:49.599
But before, you know, in the design process, if people have questions about things, like they can ask an AI agent and it will source from those Confluence pages.
00:41:49.599 --> 00:41:59.280
And it works quite well because it's a very small set of content, it's maintained by a very small team, it has a very known position in the cycle of information.
00:41:59.280 --> 00:42:03.679
Like it was designed at this point in time by these people for this purpose.
00:42:03.679 --> 00:42:11.360
So like there is a set of metadata that's assumed about that content simply by the definition of what it is.
00:42:11.360 --> 00:42:16.159
Like there's all these attributes that come with it that you can build into a system to get good results from.
00:42:16.960 --> 00:42:21.518
I was also thinking about lone writers and systems where they've not evolved.
00:42:21.518 --> 00:42:25.920
You gave them great examples of how to think about it.
00:42:25.920 --> 00:42:29.518
So if you don't have those systems in place, you can start small.
00:42:29.518 --> 00:42:32.480
Where can I start if I don't have it?
00:42:32.480 --> 00:42:38.480
How do I think about this where you don't have established processes or uh governance frameworks?
00:42:38.480 --> 00:42:48.960
How do you establish that so that you can get to the source of truth and locking it down and having the right people, get the right gatekeepers, so that eventually your content is gold-plated?
00:42:49.760 --> 00:42:53.199
Um Yeah, I can actually give one other piece of AI-related advice to that.
00:42:53.199 --> 00:43:02.960
So if you're a lone writer, like, so if I was gonna go be a lone writer in an organization today, almost no chance that you have access to like true structured content.
00:43:02.960 --> 00:43:05.518
Like that's just not common for lone writers.
00:43:05.518 --> 00:43:11.280
So I think the most common circumstance for those individuals today is going to be that they're writing in Markdown.
00:43:11.280 --> 00:43:14.800
There's a bunch of other situations, but like that's probably the most common.
00:43:14.800 --> 00:43:16.159
So I'll focus on that one.
00:43:16.159 --> 00:43:38.480
One of the things that I would do after I wrote a very basic style guide, I'm not even gonna call it an information architecture at that point in time, but a set of rules and patterns around how it is that we created content, is I would feed that into an agent and I would say, you're now going to be my editor, and you're gonna hold me to these rules.
00:43:38.480 --> 00:43:50.400
And so then every time, so again, we're in Markdown, we're probably checking this content into GitHub, which at a you know, at if I'm maintaining a thousand topics or less, that's probably gonna work pretty well.
00:43:50.400 --> 00:43:56.400
Every time uh it was checked in, it just had the agent go and look look it over and have it give me feedback.
00:43:56.400 --> 00:43:59.199
Oh, you didn't put enough metadata on the top of this topic.
00:43:59.199 --> 00:44:01.039
Oh, this topic is too long.
00:44:01.039 --> 00:44:03.518
Oh, it it this topic contains too many claims.
00:44:03.518 --> 00:44:11.280
Like you should break this up into more more Markdown files because you know you're mixing concepts and and tasks here.
00:44:11.280 --> 00:44:16.800
Like this is clearly, you know, this section is clearly a task, and this section contains a bunch of claims.
00:44:16.800 --> 00:44:19.599
Like there's uh claims facts, sorry.
00:44:19.599 --> 00:44:31.039
So there's a bunch of things that you could do with a very simple configuration of Claude, or I've actually not done this in uh in Gemini or OpenAI, so like I'm just gonna say Claude, but I'm assuming you can do this in other LLMs.
00:44:31.039 --> 00:44:32.079
Yeah, yeah.
00:44:32.079 --> 00:44:48.559
So you could set all those things up and it could basically just help you enforce a number of rules that are going to get you a lot further down the road than you would be if you were you know just writing content as text and Markdown or in YAML files or things like that.
00:44:48.559 --> 00:45:01.840
And I think you would find that if it did scale up and you hired more people and you did need to transition to a true structured format, you know, for personalization or reuse, those types of things, that you'd still have a lot of work to do, but you'd be in a better place.
00:45:02.559 --> 00:45:07.518
Yeah, I think that's a great suggestion to kind of lean into those systems available to you.
00:45:07.518 --> 00:45:10.800
If you were designing a content architecture from scratch today.
00:45:10.800 --> 00:45:17.440
Let's say you've been hired, there are no systems, and you are going to be designing content architecture from scratch.
00:45:17.440 --> 00:45:26.719
Uh specifically for AI and agentic systems, what would you build from the source content all the way through retrieval context, uh governance, and execution?
00:45:26.719 --> 00:45:28.559
Oh my god.
00:45:28.559 --> 00:45:36.800
But you you do you you've been you've done great so far with all the I mean geez.
00:45:38.000 --> 00:45:43.039
Well, so it would depend a ton on there's so many variables that go into that.
00:45:43.039 --> 00:45:45.599
So I'll give you one version of it.
00:45:45.599 --> 00:46:03.360
Okay, so if I'm coming into an organization that has, let's say 10 proper technical writers maintaining a hundred thousand pages of content, or well, let's just say fifty thousand pages of content.
00:46:03.360 --> 00:46:06.559
It's that's a little more on the on the medians there.
00:46:06.559 --> 00:46:16.000
And that content is you know, has any level of complexity, which it almost always does at that size, like you can almost never get around that.
00:46:16.000 --> 00:46:21.039
The first thing I'm going to do, if they're not already, if so, like you said, the nothing exists.
00:46:21.039 --> 00:46:26.239
The first thing I'm gonna do is I'm gonna introduce structured content and a proper CCMS.
00:46:26.239 --> 00:46:28.320
Like that would be step one.
00:46:28.320 --> 00:46:31.518
And then from there, I'm going to look at the delivery requirements.
00:46:31.518 --> 00:46:39.679
So one of the things that I think people undervalue is the importance of having a really high-quality website today.
00:46:39.679 --> 00:46:43.440
Like that is where most of your AI answers are coming from.
00:46:43.440 --> 00:46:46.800
Like before you implement any AI, you need to feed the public AI.
00:46:46.800 --> 00:47:12.159
And to do that, you need a website that has all the different Many of them are just SEO things, but all the different aspects that are going to give you the proper authority with Google, OpenAI, Anthropic, and the other major models, so that when people are asking questions of those systems, they're properly retrieving information from your set of information for your customers.
00:47:12.159 --> 00:47:16.960
Like that's gonna be 90% of your customers' AI interaction.
00:47:16.960 --> 00:47:20.320
From there, it's a very it's gonna get very situational.
00:47:20.320 --> 00:47:35.360
If I've got a field service team, I'm gonna look at how I'm gonna deploy an agent to that field service team that covers the particular issues that they're having a hard time solving that are also their information issues.
00:47:35.360 --> 00:47:41.039
I would start deploying agents and connections to our support channel.
00:47:41.039 --> 00:47:43.119
So like there was a feedback loop there.
00:47:43.119 --> 00:47:46.960
You know, those things are maybe not simple, but fairly straightforward, right?
00:47:46.960 --> 00:47:49.679
Like that's kind of like, you know, that's known work at this point in time.
00:47:49.679 --> 00:47:53.039
I'm not gonna call it a solved problem, but it's known work at this point in time.
00:47:53.039 --> 00:48:03.440
I think if you you kind of started with that path in that you're going to get yourself so far down the road that you're probably gonna be ahead of most people already.
00:48:03.440 --> 00:48:05.440
That I think that that is a blueprint.
00:48:05.440 --> 00:48:19.360
I guess the other thing I should say there too is when I said implement a CCMS, I meant do the information architecture work on the content, come up with an information model, get the content into really good shape, and then put it into a CCMS.
00:48:19.360 --> 00:48:21.760
So that's like the process of implementing structure.
00:48:21.760 --> 00:48:28.320
One of the biggest mistakes that people make is that they think they're gonna buy a tool and that's gonna solve their problems, and that's not the—not the case.
00:48:28.320 --> 00:48:37.280
Like the work it takes to go from having a ton of content that's unstructured to good structured content that's actually going to solve problems.
00:48:37.280 --> 00:48:43.440
That's an information modeling and information architecture and conversion process, and that is work.
00:48:43.440 --> 00:48:47.679
It has a huge payoff, it's the foundation for everything else.
00:48:47.679 --> 00:48:52.000
But if you don't do that work, you don't get the benefit out of structured content.
00:48:52.000 --> 00:48:54.000
So, like that would be the first step.
00:48:54.719 --> 00:48:56.559
First of all, my apologies, Patrick.
00:48:56.559 --> 00:49:00.480
I did not mean to quote unquote assault you with that question.
00:49:01.199 --> 00:49:01.920
No, no, no, no.
00:49:01.920 --> 00:49:03.039
It's a great question.
00:49:03.039 --> 00:49:04.880
It's just it's such a big question.
00:49:05.599 --> 00:49:12.159
So I was on a team where the mindset was that the tool would solve all the problems.
00:49:12.159 --> 00:49:20.880
And thinking about information architecture and mod—well, let's say that we had to revisit how our content was modeled and architected.
00:49:20.880 --> 00:49:23.518
The tool did not solve the problem.
00:49:24.239 --> 00:49:24.800
You think about it.
00:49:24.800 --> 00:49:32.559
If you were gonna go build a house and you showed up at the build site with a bunch of tools but no blueprints, yeah, what would you get?
00:49:32.559 --> 00:49:33.599
What would you end up with?
00:49:34.320 --> 00:49:35.039
Thank you so much for watching.
00:49:35.039 --> 00:49:38.320
This was a great conversation, and I hope you had fun as well.
00:49:38.320 --> 00:49:38.960
I did.
00:49:39.440 --> 00:49:39.920
Thanks again.
00:49:40.559 --> 00:49:41.119
Thank you.
00:49:41.119 --> 00:49:44.480
Thank you for listening to Inside Tech Comm.
00:49:44.480 --> 00:49:49.039
If there's a guest you would love to hear on the show, please let me know.
00:49:49.039 --> 00:49:54.320
And don't forget to follow and catch every episode on your favorite podcast app.
00:49:54.320 --> 00:49:55.599
See you next time.
00:00:02.240 --> 00:00:06.878
Hello friends and welcome to season eight of Inside Tech Comm with Zohra Mutabanna.
00:00:06.878 --> 00:00:16.160
This season features a collection of conversations that explore the ideas, challenges, and opportunities shaping technical today.
00:00:16.160 --> 00:00:21.760
From AI and content strategy to leadership, product thinking, and the future of our profession.
00:00:21.760 --> 00:00:26.160
Each episode offers a fresh perspective from people doing the work.
00:00:26.160 --> 00:00:27.120
Let's get started.
00:00:27.120 --> 00:00:35.679
Season eight is proudly sponsored by LavaCon.
00:00:35.679 --> 00:00:42.640
Use discount code ITC26 to save$200 on registration for this year's conference.
00:00:42.640 --> 00:00:45.920
I'll share more about LavaCon later in this episode.
00:00:45.920 --> 00:00:47.200
Hello listeners.
00:00:47.200 --> 00:00:49.920
Welcome to another episode of Inside Tech Comm.
00:00:49.920 --> 00:00:53.759
Today we'll be talking with Patrick Bosek from Heretto.
00:00:53.759 --> 00:01:03.039
And in the past, we've talked about structured content and its impact on how AI ingests that framework.
00:01:03.039 --> 00:01:05.358
Today we're going to do a deep dive.
00:01:05.358 --> 00:01:14.400
Patrick is going to be talking to us and educating us on how structured content contributes useful context under the hood and how it works.
00:01:14.400 --> 00:01:16.640
With that, Patrick, welcome to my show.
00:01:17.280 --> 00:01:18.000
Great to be here, Zohra.
00:01:18.000 --> 00:01:18.959
It's very nice.
00:01:19.599 --> 00:01:23.120
Patrick, first tell us a little about yourself, what you do, and yourself.
00:01:24.000 --> 00:01:24.640
Sure, happy to.
00:01:24.640 --> 00:01:25.760
So I'm Patrick Bosek.
00:01:25.760 --> 00:01:28.879
I am one of the founders of Heretto.
00:01:28.879 --> 00:01:30.239
I'm also the CEO.
00:01:30.239 --> 00:01:34.000
So I've been in the structured content industry for quite a while.
00:01:34.000 --> 00:01:39.200
As the uh some I guess I could be considered the face of the company sometimes.
00:01:39.200 --> 00:01:43.519
So in that role, I also do quite a bit of media things.
00:01:43.519 --> 00:01:46.000
So I have a talk show I do with Scott Abel.
00:01:46.000 --> 00:01:48.078
So that's called Coffee and Content.
00:01:48.078 --> 00:01:54.319
We've been on a little bit of a hiatus due to a technical issue, but we typically do Coffee and Content every other week.
00:01:54.319 --> 00:01:58.319
So those are a lot of fun, cover a lot of different topics from the content world.
00:01:58.319 --> 00:02:02.000
We also have a podcast called Content Components.
00:02:02.000 --> 00:02:06.400
If you look it up on Spotify, that's got four or five seasons.
00:02:06.400 --> 00:02:10.240
It's all about documentation, structured content, those kinds of things.
00:02:10.240 --> 00:02:15.120
So if you're wanting nerd out about structured content, that's a great place to go.
00:02:15.120 --> 00:02:15.598
Yeah.
00:02:15.598 --> 00:02:17.520
And then I'm a developer by background.
00:02:17.520 --> 00:02:21.758
So I was one of the developers and architects of the first version of our software.
00:02:21.758 --> 00:02:26.318
So I'm I'm fairly technical, but I don't cause problems for our development team anymore.
00:02:26.318 --> 00:02:30.800
Nowadays, I focus on what's going to make customers happy and those kinds of things.
00:02:31.360 --> 00:02:35.360
What spawned this idea to look into structured content as a developer?
00:02:35.360 --> 00:02:36.479
I'm curious.
00:02:37.360 --> 00:02:37.598
Oh boy.
00:02:37.598 --> 00:02:41.360
So that's actually going back a lot further than I'd like to admit.
00:02:41.360 --> 00:03:04.318
But my co-founder and I, we got into structured content, not DITA, which is where we are today, but XML content with an actually with a custom schema when we were in college, actually, and we just saw that it was a superior way to store and manage content and that it was going to make some of the publishing activities we were working on easier and more capable.
00:03:04.318 --> 00:03:10.878
At that point in time, the technologies surrounding digital were just far, far less well developed than they are today.
00:03:10.878 --> 00:03:18.878
So structured content provided it actually provided something quite different at that moment in time, now that I'm reflecting on it, than it does today.
00:03:18.878 --> 00:03:25.520
At that moment in time, it it basically gave you the ability to put, like, a little bit of a database in a file.
00:03:25.520 --> 00:03:40.240
The amount of things you had to manage and the flexibility you got when you were using structured content at that point in time, if you're looking to apply that to like a very structured digital publishing environment, there was a lot of advantages there.
00:03:40.240 --> 00:03:42.158
Um so that was why we picked it up originally.
00:03:42.158 --> 00:04:02.080
And then over the years, we eventually drifted into, or I guess found our way, is maybe a better way of putting it, into technical content where you still get a lot of those benefits that we we were originally looking at with DITA and in the digital publishing around uh technical content and documentation, but there's a ton more today.
00:04:02.080 --> 00:04:04.639
So there's there's so much more that you get from it.
00:04:05.199 --> 00:04:08.158
Yeah, we could probably have a whole episode on structured content.
00:04:08.158 --> 00:04:12.960
And I have, as I mentioned, had some episodes covering what that is.
00:04:12.960 --> 00:04:16.160
And it seems to me like the context can be different.
00:04:16.160 --> 00:04:28.319
Of course, DITA comes close to mind, but for those of us that don't use DITA, structured content still follows a certain framework, a certain guidelines, if you will.
00:04:28.319 --> 00:04:41.439
But for this conversation, I want to ground what that definition means to you, and then we can, I guess, start talking about how we can deep dive into how that lends to AI.
00:04:42.160 --> 00:04:48.560
Yeah, I should think that is really important to start with the definition of structured content because everybody does define it a little bit differently.
00:04:48.560 --> 00:04:52.480
And I think the individual definition matters.
00:04:52.480 --> 00:04:55.040
So I'll give you my definition.
00:04:55.040 --> 00:04:57.278
I'll also tell you why I think mine is right.
00:04:57.278 --> 00:05:01.199
But that doesn't mean that other people's don't uh can't be right too.
00:05:01.199 --> 00:05:21.920
So technically speaking, if you put all of your content into Excel or Google Sheets and you put it into, you know, let's say every topic was a column, or maybe better a row, and every portion of that topic or that piece of information was divided up into different columns.
00:05:21.920 --> 00:05:25.120
So you have the pieces broken up into the different cells.
00:05:25.120 --> 00:05:31.120
That would be structured content because you would have the content broken up into a structure.
00:05:31.120 --> 00:05:35.519
Like in the most basic definition, you have a structure for your content.
00:05:35.519 --> 00:05:41.519
So you're you're labeling the different pieces of it in a methodology, and that's structured content.
00:05:41.519 --> 00:05:43.519
Now, obviously, that's a terrible choice.
00:05:43.519 --> 00:05:48.000
Like that wouldn't be a thing that you'd want to do in almost any production environment.
00:05:48.000 --> 00:05:56.160
Although I will say that we have had customers migrate from that exact environment just like that to our tool, which is DITA.
00:05:56.160 --> 00:06:09.759
But when you think about like how this comes together, the reason I start there is because there are a lot of systems that are they're effectively that, because if you think of what a form is, so like a digital form, it is just actually a table.
00:06:09.759 --> 00:06:12.879
You have your question one, question two.
00:06:12.879 --> 00:06:15.278
So your question one might be what is the heading for this file?
00:06:15.278 --> 00:06:19.519
Your question two might be who authored this file, question three might be the body.
00:06:19.519 --> 00:06:24.720
And people will call that structured content because, you know, you can put metadata in one place.
00:06:24.720 --> 00:06:29.439
And and in certain circumstances, that's what headless CMSs are, for instance.
00:06:29.439 --> 00:06:32.560
So, like things like Contentful, those kinds, that's that's what they are.
00:06:32.560 --> 00:06:35.278
In certain circumstances, that actually works quite well.
00:06:35.278 --> 00:06:38.000
Um, that's a really effective form of structured content.
00:06:38.000 --> 00:06:47.040
It's backed by a database, uh, it has the ability to create links between those pieces of information, uh, it has the metadata separated out.
00:06:47.040 --> 00:07:00.079
Like you have like there are certain things, but it in the background, it is functionally a table that is fundamentally different than what we work with, which is XML, because XML is a tree, which means that you can have structure below those big boxes.
00:07:00.079 --> 00:07:04.639
So when you go into a topic, there's going to be structure at multiple levels.
00:07:04.639 --> 00:07:10.240
So you'll have a piece, like let's say we'll use a procedure or a task topic, for instance.
00:07:10.240 --> 00:07:18.639
When you start creating that, you'll have a title, and then you'll have metadata around it, and then you'll have the prerequisites for running this procedure.
00:07:18.639 --> 00:07:24.079
And then the individual procedure, instead of being an ordered list, it would be steps, right?
00:07:24.079 --> 00:07:28.079
And then there's a step inside of it, there's structure inside of the steps.
00:07:28.079 --> 00:07:38.480
And so by having this multiple layers of structure and semantics, it means that you're able to manipulate that in a more granular way beyond just formatting.
00:07:38.480 --> 00:07:42.959
It's not just HTML and boxes, effectively, but they're both structured.
00:07:42.959 --> 00:07:47.759
Like there is a, like, both things are valid in terms of the content label.
00:07:47.759 --> 00:07:54.480
So, what you really need to ask yourself when you're thinking about, okay, well, we're moving to structure, it's beyond just the term structure.
00:07:54.480 --> 00:08:00.000
Like you should really be thinking about what is the form of that's going to serve our mission?
00:08:00.000 --> 00:08:02.720
What do we care about in terms of interoperability?
00:08:02.720 --> 00:08:07.199
What forms of linking are really important to us?
00:08:07.199 --> 00:08:10.240
What forms of output are going to be really important to us?
00:08:10.240 --> 00:08:18.560
Because those types of questions will guide you towards the type of structure that's going to be best for your particular implementation and your particular needs.
00:08:19.360 --> 00:08:24.000
I have to say that I thought my definition was going to be very different from yours or my understanding.
00:08:24.000 --> 00:08:28.160
But at some level, I think we are on the same page.
00:08:28.160 --> 00:08:28.800
Yeah.
00:08:28.800 --> 00:08:30.639
I mean, I think my definition is pretty broad.
00:08:30.639 --> 00:08:31.920
Like I said, I think it's right too.
00:08:32.399 --> 00:08:33.278
Yes, it's very inclusive.
00:08:33.278 --> 00:08:39.120
So that allows me to be kind of okay, that definition sounds close to what I have in my mind.
00:08:39.120 --> 00:08:40.000
Fantastic.
00:08:40.000 --> 00:08:41.120
Yes.
00:08:41.120 --> 00:08:48.960
I have not worked with DITA, and you mentioned DITA, and DITA comes up a lot when we talk about structured content.
00:08:48.960 --> 00:08:57.918
We've touched upon this before, but I think for the benefit of the audience, it would be nice to quickly do a shout out to what DITA is.
00:08:58.399 --> 00:08:58.798
Yeah, for sure.
00:08:58.798 --> 00:09:02.480
So DITA, in the most basic definition, it's it's a form of XML.
00:09:02.480 --> 00:09:07.840
So in that way, you can actually think of it as like a cousin to HTML in some ways.
00:09:07.840 --> 00:09:11.120
So it actually shares a lot of the same elements that HTML has.
00:09:11.120 --> 00:09:20.480
The thing to know about DITA is that it has built in many of the patterns that you need to scale up technical content.
00:09:20.480 --> 00:09:31.678
So there's a ton of stuff that we've pretty much figured out over the last call it 15 or 20 years, are really critical to larger or more complex sets of technical content.
00:09:31.678 --> 00:09:39.519
And the DITA committee, so the people who work on the open source standard, they put all of those things into the standard.
00:09:39.519 --> 00:09:41.360
So you don't have to recreate them.
00:09:41.360 --> 00:09:48.399
And then perhaps just as importantly, beyond not having to recreate them, you don't have any vendor opinion or lock-in.
00:09:48.399 --> 00:10:04.798
So one of the challenges that the that DITA was trying to solve, and I think has solved pretty effectively, is that when you have vendor-specific formats, it means that okay, well, we've reused content, but we've reused content in this particular system.
00:10:04.798 --> 00:10:14.080
And once you get to a certain level of reuse, if you didn't have a portable technology underneath it, your ability to migrate someplace else is I mean, it's it's almost impossible.
00:10:14.080 --> 00:10:17.759
It just becomes the lock in there, it's just so high.
00:10:17.759 --> 00:10:29.120
So what DITA really solves is it solves having good patterns for the core things that you really need to do with technical content, and then it also solves an interoperability problem.
00:10:29.120 --> 00:10:34.158
And you know, there's some of the criticisms of DITA are that it comes with you know like a high learning curve.
00:10:34.158 --> 00:10:36.320
Yeah, I don't know if I actually agree with that.
00:10:36.320 --> 00:10:38.240
Like I've seen a lot of people learn it pretty quickly.
00:10:38.240 --> 00:10:41.759
There's people feel that it's too heavy for today's uses.
00:10:41.759 --> 00:10:42.720
That's another criticism.
00:10:42.720 --> 00:10:45.440
Those people tend to come from like the Markdown world.
00:10:45.440 --> 00:10:50.639
So, like in comparison to Markdown, you know, Markdown is a text format, DITA is an is an XML format.
00:10:50.639 --> 00:10:52.558
I think that's just a use case thing.
00:10:52.558 --> 00:10:58.798
So like DITA is not a great, is not like you never write the README files for your vibe coded project in DITA.
00:10:58.798 --> 00:10:59.918
I don't do that.
00:10:59.918 --> 00:11:04.960
Like when I have small apps and things like that that I build, I don't write any of the documentation in DITA.
00:11:04.960 --> 00:11:06.158
It doesn't make any sense.
00:11:06.158 --> 00:11:13.360
DITA is a format that you use when you arrive at a sufficient level of complexity and a sufficient scale.
00:11:13.360 --> 00:11:24.320
So if you want like really rough and tough numbers, what we've found over the years is that if you're below a thousand pages of content, DITA probably doesn't make sense.
00:11:24.320 --> 00:11:39.200
Um, there are some exceptions, but even at a thousand pages of content, like it can be questionable there depending on your And so where it really starts to make sense is when you have multiple variations of products or variations of content, audiences.
00:11:39.200 --> 00:11:49.519
So you have anything that's like personalization, and there's a bunch of different terms that people use for the content changes when somebody looks at it for a different person looks at it for a different reason.
00:11:49.519 --> 00:11:53.120
Localization can be a driver, although I think less so today.
00:11:53.120 --> 00:11:55.759
And then just sheer scale with like reuse.
00:11:55.759 --> 00:12:02.000
So if you have a lot of content that you is used in multiple places, that's when you know DITA starts to really shine.
00:12:11.278 --> 00:12:12.399
In a simplified way.
00:12:12.399 --> 00:12:30.720
Since we are going to do a deep dive into how structured content lends itself to AI, does DITA lend itself better, or just this the generic definition of the structured content that we ground it, that we are trying to ground this conversation in also lends itself to better AI interactions?
00:12:31.840 --> 00:12:32.798
That is such a good question.
00:12:32.798 --> 00:12:36.798
So the answer is yes, it depends in here.
00:12:36.798 --> 00:12:41.678
And I feel like I'm among the right people to give like engineering style, it depends answers.
00:12:41.678 --> 00:12:46.558
One of the jokes in our leadership team is that when you ask the business people a question, they give you an answer.
00:12:46.558 --> 00:12:49.200
When you ask the technical people a question, they say it depends.
00:12:49.200 --> 00:12:53.360
And so I think that I'm I'm filling that role in this particular moment.
00:12:53.360 --> 00:12:57.120
So, you know, this is kind of like what we we wanted to cover today.
00:12:57.120 --> 00:12:59.678
So it is a it's a very, very long answer.
00:12:59.678 --> 00:13:10.320
At the very core of it, if, like, you wanted, like, the too long, didn't listen version of it, structured content is going to give you better structured contents with the broad definition.
00:13:10.320 --> 00:13:20.240
So like anything that is broken up into labeled pieces that are then stored consistently, like that definition of structured content.
00:13:20.240 --> 00:13:28.720
So that's inclusive of DITA, inclusive of, uh, headless CMSs, inclusive of anything that has a well-defined schema applied to content.
00:13:28.720 --> 00:13:31.759
So, why would that be better for AI?
00:13:31.759 --> 00:13:37.278
And it really just comes down to chunking, like that's the fundamental difference between those things.
00:13:37.278 --> 00:13:46.558
So the problem with that statement is that when you get to certain forms of structured content, and I'll like I'll give you like an example from a headless CMS.
00:13:46.558 --> 00:13:50.320
The headless CMS is every headless CMS is gonna have a different schema.
00:13:50.320 --> 00:13:55.918
Like part of the the appeal of a headless CMS is that you can create your own schema.
00:13:55.918 --> 00:14:06.000
So you go in and you get a blanks blank page, and all you have to have is a title, and then you can add a body and an author, and you could add you know links to other things.
00:14:06.000 --> 00:14:13.200
And like the standard thing is like, you know, how would you lay out a one of the standard examples is how would you lay that out for like a music venue?
00:14:13.200 --> 00:14:18.720
And you'd have one thing that was a band, and then you'd have an artist, and then you'd have like those kinds of things, right?
00:14:18.720 --> 00:14:20.000
And you'd you build those schemas.
00:14:20.000 --> 00:14:31.038
Well, in a really well-designed schema, it's gonna function really well as giving you good sized pieces of content for the chunking.
00:14:31.038 --> 00:14:40.720
In a poorly designed schema, what's gonna happen is you're gonna get people basically putting in a title and a body, and like that's really not any different than a Google Doc, right?
00:14:40.720 --> 00:14:45.759
So at that point in time, people are just gonna throw huge quantities of content into that body field.
00:14:45.759 --> 00:14:54.960
So even though technically you have two fields, the structure is two boxes, like it's one box, it's it's two boxes with unstructured content in one of them.
00:14:54.960 --> 00:14:57.278
And so then you lose the benefits of chunking.
00:14:57.278 --> 00:15:19.200
So now there's a bunch of other things, but if you like if you want the biggest bang for your buck, so like the I don't know, the 10% that gives you 60% outcome, it's having really well, clean, defined borders around your content, which means smaller pieces, more intentional pieces, pieces of content that contain ideally one piece of information.
00:15:19.200 --> 00:15:23.038
So in DITA, like a topic is supposed to be about one thing.
00:15:23.038 --> 00:15:28.240
It like if it's a concept topic, it's supposed to teach you one thing or explain one thing.
00:15:28.240 --> 00:15:33.200
If it's a task, it's supposed to give you a procedure that does one thing.
00:15:33.200 --> 00:15:39.678
Um, so it takes you from from the beginning to end of one thing you have to complete, and there's other versions of that as well.
00:15:39.678 --> 00:16:05.038
So in the AI world, when you go to produce results with an AI system, being able to provide it very clean chunking, so very clean set like pieces of content for the classification on when you're putting it into a RAG system, and then also for retrieval, there's a bunch of other things, but that's the thing that gets you the biggest bang for your buck.
00:16:05.759 --> 00:16:11.918
Modern language models are pretty good at extracting meaning from unstructured content.
00:16:11.918 --> 00:16:15.840
Does that reduce the need for structured content?
00:16:15.840 --> 00:16:21.200
Because we're trying to make the case that you need to have chunking with structured content.
00:16:21.200 --> 00:16:32.399
But I'm kind of trying to flip that question and see if the LLMs have gotten so bad so good, or is there still an advantage to use structured content as we have established?
00:16:33.360 --> 00:16:35.038
Yeah, so I think that is also a great question.
00:16:35.038 --> 00:16:41.600
It's one of the places that there's I don't want to say because I think that's the wrong way to think about it.
00:16:41.600 --> 00:16:45.519
But I think that there's so maybe this is a better way to put it.
00:16:45.519 --> 00:16:52.158
You're right that the the modern LLMs have gotten very good at extracting information when the question is basic.
00:16:52.158 --> 00:17:03.120
And the reality is that most of the time when people are interacting with these LLMs, their their modalities of interaction fall into a couple of categories.
00:17:03.120 --> 00:17:06.558
One is coding, and coding is a verifiable tree.
00:17:06.558 --> 00:17:19.278
So like part of the reason coding has gotten so good is not because the LLMs have gotten so good, it's because the harnesses have gotten so good at managing something that is verifiable, like that.
00:17:19.278 --> 00:17:20.318
It's logically verifiable.
00:17:20.318 --> 00:17:26.480
The other modality that we use, we use these in is effectively a replacement for search, right?
00:17:26.480 --> 00:17:29.038
So you go to Google and like you look at the AI result.
00:17:29.038 --> 00:17:35.519
Again, a lot of the advancement there is actually in the deterministic harness under the hood.
00:17:35.519 --> 00:17:51.920
Like the amount of engineering that Google and Anthropic and Chat or and OpenAI have done so that what is being delivered to the AI for consideration is high quality, is probably equivalent to the amount of like engineering they've done in the AI model.
00:17:51.920 --> 00:17:54.960
Well, maybe not, I don't know, but like it's a lot is the point.
00:17:54.960 --> 00:18:05.278
And then when you're but when you're thinking about interacting with these systems in a more general sense, typically people are putting very small quantities of information into them.
00:18:05.278 --> 00:18:14.798
So like if I go to NotebookLM and I throw in a couple of PDFs, it does work quite well because the quantity of information is very small.
00:18:14.798 --> 00:18:21.038
Now, in the background, it is still chopping that PDF up and it is still indexing it.
00:18:21.038 --> 00:18:25.440
Like it is not just feeding that PDF full stock to the to the LLM.
00:18:25.440 --> 00:18:46.558
But what's happening when you ask that the question, or you're working with the system to get information out of that, is that because it's a small number of chunks, and because the system is then reducing down what it's sending to the LLM to a smaller set, it's not that big of a deal.
00:18:46.558 --> 00:18:48.960
The problem is when you scale up.
00:18:48.960 --> 00:18:55.680
So there's a reason you can only put, I don't know what it is, it's not a huge quantity of content into NotebookLM.
00:18:55.680 --> 00:19:10.640
When you start talking about thousands of pages or tens of thousands of pages or more, at that point in time, especially if you're asking more human questions as opposed to like search queries, that's when it absolutely still breaks down.
00:19:10.640 --> 00:19:12.880
And like we have research that shows this.
00:19:12.880 --> 00:19:16.240
So like this isn't this isn't just me hypothesizing these things.
00:19:16.240 --> 00:19:18.400
Like this is this is stuff that we can show.
00:19:18.400 --> 00:19:35.759
And in those worlds where having really strong RAG retrieval, which is built on the foundation of having good tokenization of the content, you start to see structured content outperform unstructured content by about 50%.
00:19:35.759 --> 00:19:38.078
Now it's use case dependent.
00:19:38.078 --> 00:19:43.920
Like there are like simpler questions will do better in both cases, more complex questions will do better.
00:19:43.920 --> 00:19:46.318
So there's there's a lot of it depends in there.
00:19:46.318 --> 00:19:55.598
But for things that really matter, especially in like regulated industries, or I mean, I guess everybody should care about their customers getting the right answer, but you know, like some people care more than others, I suppose.
00:19:55.598 --> 00:20:01.519
It still is a substantial difference in overall quality of retrieval and accuracy.
00:20:03.038 --> 00:20:11.200
So I think this is a great, I guess, segue into the RAG, uh, the retrieve and generate model that we start looking into.
00:20:11.200 --> 00:20:19.759
Under the hood, retrieval is evolving from a fairly linear model towards more agents that can decide what information they need to go and find.
00:20:19.759 --> 00:20:24.880
Where does structured content fit into this newer model of agentic retrieval and context engineering?
00:20:25.680 --> 00:20:29.358
Yeah, I so I think that's an interesting area of evolution right now.
00:20:29.358 --> 00:20:35.519
And I guess the first thing I want to call out here is that the models are themselves getting better.
00:20:35.519 --> 00:20:37.200
Like there's no doubt about that.
00:20:37.200 --> 00:20:46.240
But I actually think the advancement in RAG systems is more in the harness sophistication than it actually is in the models right now.
00:20:46.240 --> 00:20:56.318
So when you think about like how does an AI work with a a RAG like content operations or RAG architecture such that it's going to produce a better outcome.
00:20:56.318 --> 00:21:00.318
Well, actually, should we stop and like give a definition of what of like what RAG is before?
00:21:00.960 --> 00:21:01.200
Yes, please.
00:21:01.200 --> 00:21:01.598
Absolutely.
00:21:01.598 --> 00:21:02.720
Go for it.
00:21:03.038 --> 00:21:03.358
Yeah, okay.
00:21:03.358 --> 00:21:11.278
So RAG is retrieval augmented generation, which is technical people trying to confuse the world with with fancy words.
00:21:11.278 --> 00:21:15.680
All it means is that you're running a deterministic on the well, you're running a search.
00:21:15.680 --> 00:21:18.480
I was about to say deterministic search, but that's not necessarily true.
00:21:18.480 --> 00:21:26.720
You're running a search, the output of that search is then being put into the context for an AI agent to generate a result.
00:21:26.720 --> 00:21:27.598
That's all it means.
00:21:27.598 --> 00:21:31.118
So it's it's just search and then generate, is what it comes down to.
00:21:31.118 --> 00:21:35.118
Now, most RAG systems are built on vector databases.
00:21:35.118 --> 00:21:52.880
And why it matters to call that out is because fundamentally, vector databases, what they are is they're just they're calculations around a piece of information as to like kind of where it sits in space, and they what they do is they retrieve things that are close together.
00:21:52.880 --> 00:21:56.078
Now, those calculations are done by LLMs typically.
00:21:56.078 --> 00:22:04.160
When you look at like some of the research that we've done, like When you look at, like, some of the research that we've done, like of different ways of calculating the embeddings.
00:22:04.160 --> 00:22:08.640
So the embeddings are like the position, I guess, if you want to think about it that way.
00:22:08.640 --> 00:22:16.160
And the way that we we do that is that you can calculate the embeddings with a bunch of different with any of the AI models that are out there.
00:22:16.160 --> 00:22:24.640
So what we've found is that in our testing, OpenAI tends to be the one that's most cost effective and easiest to do.
00:22:24.640 --> 00:22:26.318
And so that's the one that we've used.
00:22:26.318 --> 00:22:32.160
We've tried a bunch, but that's the one we've kind of settled on as like our baseline, which is funny because we use Anthropic everywhere else.
00:22:32.160 --> 00:22:33.759
But like that's how that comes together.
00:22:33.759 --> 00:22:45.519
So then you have the content in the vector database, the way you retrieve it is through these vector searches, and then when you retrieve sets of content, that's going to be put into the LLM.
00:22:45.519 --> 00:22:53.519
So the major difference here between like the first versions of these and the versions that we deal with today, the models are better, yes.
00:22:53.519 --> 00:23:10.400
But now you may see AI being in a place where you send the question to the LLM, the LLM modifies the question or it breaks it down into some way, or it clarifies the question, changes the question, and then it passes the query to the vector database.
00:23:10.400 --> 00:23:14.160
The vector database returns results, and that loop can continue.
00:23:14.160 --> 00:23:16.960
So reasoning is largely just a loop, right?
00:23:16.960 --> 00:23:19.038
Like they basically feed the results back in.
00:23:19.038 --> 00:23:25.519
There could be some deterministic logic that does some parsing or extraction between the LLM and the database.
00:23:25.519 --> 00:23:32.558
Like there's other ways that they, you know, they can basically upgrade the overall loop, but it really fundamentally is a loop.
00:23:32.558 --> 00:23:38.400
So I think the misconception that a lot of people have is they think all of this is happening inside of the LLM.
00:23:38.400 --> 00:23:44.960
Like the LLM is making a decision about what information it's retrieving, but like the information is in the LLM.
00:23:44.960 --> 00:23:48.880
The information that you're when you're talking about RAG is not in the LLM.
00:23:48.880 --> 00:23:55.278
It's in a separate piece of technology that the LLM is making, is having an interaction with?
00:23:55.920 --> 00:24:04.640
When I started working with structured content, metadata became a centerpiece to how a cornerstone to how search was done.
00:24:04.640 --> 00:24:09.759
So the the richer the metadata, the better the output was.
00:24:09.759 --> 00:24:12.558
So organizing information was super critical.
00:24:12.558 --> 00:24:24.400
What changes when a machine is using that metadata to decide what information belongs in an AI model's context with all the technology breakdown that you've given us?
00:24:24.400 --> 00:24:24.960
Yeah.
00:24:24.960 --> 00:24:29.358
So this could be a whole series of podcasts in and of itself.
00:24:30.078 --> 00:24:30.240
Of course.
00:24:30.798 --> 00:24:31.038
What changes?
00:24:31.038 --> 00:24:32.798
Well, oh boy.
00:24:32.798 --> 00:24:41.038
The first thing I think that's important to think about here is that metadata gives you a lot of options in how you process something.
00:24:41.038 --> 00:24:58.400
So one of the key aspects of having really high quality metadata on your content is it has hooks and, like, levers that you can pull in deterministic processes that decide how content ends up and how it shows up.
00:24:58.400 --> 00:24:58.960
Right.
00:24:58.960 --> 00:25:13.118
So when you're thinking about what you're gonna end up providing to an LLM, you almost always have some type of a pipeline that's going to be coming into whatever your RAG system is or whatever, wherever the LLM is actually going to be touching that content.
00:25:13.118 --> 00:25:22.640
And metadata means that you can be more specific about what and how those things go into the uh the LLM or the AI-based system.
00:25:22.640 --> 00:25:26.960
So it gives you a lot more control with traditional code.
00:25:26.960 --> 00:25:31.358
And I'm emphasizing that because it really shouldn't be undervalued.
00:25:31.358 --> 00:25:33.838
That is still enormously powerful.
00:25:33.838 --> 00:25:50.078
The more specific you can be in a deterministic way in the systems that you build, the more confidence you can have and the more sustainable those systems are going to be, because you can write unit tests, right?
00:25:50.078 --> 00:25:53.200
Like things in deterministic systems have deterministic outcomes.
00:25:53.200 --> 00:25:56.078
Like that's fundamentally how they're different than AI systems.
00:25:56.078 --> 00:25:59.598
So metadata still has a ton of value in that way.
00:25:59.598 --> 00:26:03.118
This is kind of a the other side of that coin, but I think also valuable.
00:26:03.118 --> 00:26:15.358
There are circumstances where traditional search, if metadata is effectively applied, will perform equally well to vector search, and it's a lot less expensive.
00:26:15.358 --> 00:26:20.078
So there's certain applications where the old stuff works just as well.
00:26:20.078 --> 00:26:26.000
And there's actually cases where it works better, but then you know, we don't have to go, we don't have to dig too deeply into those things.
00:26:26.000 --> 00:26:29.519
But the point is that metadata also gives you options.
00:26:29.519 --> 00:26:46.160
So if you're in a place where you're completely reliant on AI systems and you don't have really strong metadata around how you're trying to build a retrieval pipeline, um, then you're totally reliant on the inference of context from the AI into a vector database.
00:26:46.160 --> 00:26:57.598
Whereas if you had that metadata, you might be in a place where you could have a deterministic system or a traditional search that was just as just as good or better, um, and also gave you some other options.
00:26:57.598 --> 00:27:04.078
So metadata can be really valuable in terms of comparison of not just an optionality for your system.
00:27:04.078 --> 00:27:07.038
So you have optionality, you have comparison, like you have those types of things.
00:27:07.038 --> 00:27:32.880
And then when you think about like how does it actually benefit the LLMs themselves, this is where you get into like a very interesting discussion because there are aspects of it that can be really Typically, what it means is that you're transitioning structured metadata into, I would say, like statements for the LLM, such that it is still context, right?
00:27:32.880 --> 00:27:38.880
But like LLMs receive context differently than like humans and traditional machines to some extent.
00:27:38.880 --> 00:27:42.480
But if you have everything tagged for, so I'll give you an example.
00:27:42.480 --> 00:27:44.640
Let's say you have two products.
00:27:44.640 --> 00:27:47.838
You have a we'll use washing machines, right?
00:27:47.838 --> 00:27:55.200
Let's say you have a 1000 version and a 1001 version, and those are very, very similar systems, right?
00:27:55.200 --> 00:27:56.960
They're the same line of washing machine.
00:27:56.960 --> 00:27:58.240
There's a few differences.
00:27:58.240 --> 00:28:06.960
Well, you've got a hundred topics for each of those systems, and let's say 95 of them are the same.
00:28:06.960 --> 00:28:12.400
So they all carry the 1000 and the 1001 tag, and then you have five that diverge.
00:28:12.400 --> 00:28:18.960
So you have 105 topics in total, and you know, five with a thousand, five with a thousand and one.
00:28:18.960 --> 00:28:26.640
If you're just stuffing all of that into an LLM, and remember, like this is only 105 topics, so you can probably get away with that.
00:28:26.640 --> 00:28:33.200
The way you would use metadata there, and notice I'm specifically avoiding RAG and search in this particular example.
00:28:33.200 --> 00:28:56.798
Like I'm just talking about how you put the whole 105 in, is instead of having a tag on that, you would probably be better off actually having a statement at the top of it where before you put it in, you'd use a deterministic system to take the tag from you know product and then it had the tag name to this is the 1001 version of this washing machine.
00:28:56.798 --> 00:28:59.680
This is the 1000 version of this washing machine.
00:28:59.680 --> 00:29:01.358
You just put that at the top of every topic.
00:29:01.358 --> 00:29:07.920
And an LLM is going to interpret that better than having like the actual tag on there.
00:29:07.920 --> 00:29:10.078
And so I've done a bunch of testing around this.
00:29:10.078 --> 00:29:12.480
Like, this also is the same with like glossary terms.
00:29:12.480 --> 00:29:21.440
So if you have like structured glossaries, um, different LLMs will receive the pieces of that glossary, which are tags, right?
00:29:21.440 --> 00:29:22.000
Like it's metadata.
00:29:22.000 --> 00:29:26.400
Like you're like, you know, this is an abbreviation, this is another way of saying it.
00:29:26.400 --> 00:29:29.920
Like, there's like right, it that's just metadata that describes different pieces of it.
00:29:29.920 --> 00:29:40.160
If you write it out as prose, even if it's the same prose over and over again, the LLMs tend to work with that version of that content better.
00:29:40.160 --> 00:29:57.278
So transitioning the structured metadata into clear and simple statements, even if they're repetitive for the LLM, tends to produce a better result today with the LLMs we have today, because of how they've been trained and you know the biases and like the internal weightings that they have.
00:29:57.278 --> 00:30:00.318
Like you could train one that works better with the actual structure.
00:30:00.318 --> 00:30:02.400
That's just not the ones that we have today.
00:30:02.400 --> 00:30:06.000
So metadata touches a ton of different pieces of it.
00:30:06.000 --> 00:30:13.440
And I haven't even gone into like how metadata can then be used to build graphs and to create like more sophisticated like selection mechanisms.
00:30:13.440 --> 00:30:31.200
But the point is that once you get to any level of complexity, having the metadata on the content gives you so much more optionality in terms of how you build the systems around the LLM, and then also how you actually instruct the LLM when you're putting inputs into it.
00:30:31.680 --> 00:30:36.880
I think this was such a great overview because my goal here was to kind of step back.
00:30:36.880 --> 00:30:57.598
And for those amongst us who have very, I guess the as a technical writer, one would have that structured content, the concept But if you don't, if you just have a Google Doc and you're starting off, then adding metadata to that, you've given us a why, a solid why.
00:30:57.598 --> 00:30:59.519
Why should you still consider it?
00:30:59.519 --> 00:31:04.160
And how that would kind of work under the hood that I keep going back to.
00:31:04.160 --> 00:31:11.519
And I would love for us to maybe even dive into the knowledge graphs and how they would relate to metadata.
00:31:11.519 --> 00:31:25.038
With more than 70 sessions and workshops featuring speakers from companies like Salesforce, TikTok, T-Mobile, and Amazon.
00:31:25.038 --> 00:31:32.880
LavaCon brings together content strategists, documentation leaders, and content professionals to share practical ideas you can apply right away.
00:31:32.880 --> 00:31:39.920
And yes, it's known as the Fun Conference, too with networking events, therapy dogs, comfort llamas, storytelling, and karaoke.
00:31:39.920 --> 00:31:44.160
Use discount code ITC26 to save$200 on registration.
00:31:44.160 --> 00:31:47.920
You did mention graphs here.
00:31:47.920 --> 00:31:52.400
So I think it's worth going into that a little bit, if not more.
00:31:53.118 --> 00:31:53.440
Yeah, sure.
00:31:53.440 --> 00:31:54.798
So I'm happy to go.
00:31:54.798 --> 00:31:58.640
Well, there are people who are are experts in knowledge graph.
00:31:58.640 --> 00:32:01.358
I wouldn't consider myself an expert in knowledge graphs.
00:32:01.358 --> 00:32:04.960
I do know I I would say a pretty good amount about them.
00:32:04.960 --> 00:32:10.880
So there's probably a limit to how far I can go, but I'm very happy to talk about them in relation to structured content metadata.
00:32:10.880 --> 00:32:15.519
So maybe uh start with just the definition of what a knowledge graph is.
00:32:15.519 --> 00:32:24.400
So so a knowledge graph is if you want to think about it in technical terms, it's a database that has typed nodes and typed connections between those nodes.
00:32:24.400 --> 00:32:33.440
So you could have people and then you could have, like, a 'knows' or like this person knows this person or is married to.
00:32:33.440 --> 00:32:36.400
Obviously, that would be a one-to-one connection in most cases.
00:32:36.400 --> 00:32:39.200
But like, you know, so this is like the friend of a friend stuff.
00:32:39.200 --> 00:32:41.920
Like, so like ontologies, knowledge graphs, those kinds of things.
00:32:41.920 --> 00:32:52.160
When you put this information into graph databases, what it means is you can start asking multi-level questions of the database.
00:32:52.160 --> 00:33:04.480
So the thing that people really like about them is if you go back to like the people example, you could ask a question of a knowledge knowledge graph of like, give me all the companies for people that Patrick has been on podcasts with.
00:33:04.480 --> 00:33:34.078
And a good knowledge graph, like a good knowledge graph database would be able to go and say, Okay, well, Patrick is a podcaster, and then there would be a field that's like has been on, or something like that, and that would link out to a bunch of different podcasts, and then each of those podcasts would have guests and hosts, and each of those guests and hosts would be people, and all the people would have as employment or you know, works for something along those lines, right?
00:33:34.078 --> 00:33:41.038
And so you could have a query that would say like doop, doop, doop, doop, doop, and hop across all those things and select all of those companies.
00:33:41.038 --> 00:33:53.759
That's why people like knowledge graphs, is because it allows you to ask those kinds of questions of an information set, which are traditionally very difficult to ask in the context of like relational databases.
00:33:53.759 --> 00:34:00.000
So you can do some of that stuff in relational databases, but it's much less flexible, I guess is the best way to put it.
00:34:00.000 --> 00:34:09.039
So you're dealing with very large and you know well tagged or very semantic sets of content, like you're just gonna do better in a knowledge graph.
00:34:09.039 --> 00:34:20.079
So, how this collides with structured content and metadata is that it's not always that you're typically you're not actually keep maintaining your content in the knowledge graph, right?
00:34:20.079 --> 00:34:22.400
So you're maintaining your content in another location.
00:34:22.400 --> 00:34:29.599
So the way that you you correlate, you tie the content to the knowledge graph is through the metadata in many cases.
00:34:29.599 --> 00:34:37.280
So the metadata that becomes the selection mechanism that the the knowledge graph can use to pull that out.
00:34:37.280 --> 00:34:51.920
You can also use direct linking, but you oftentimes like the metadata is going to become like a major piece of like how those two things come together, or how the knowledge graph forms the nodes in the graph out of the content, like it contributes to those things.
00:34:52.960 --> 00:35:00.800
I'm thinking in a typical environment, you have documentation coming from product, from engineering, from support.
00:35:00.800 --> 00:35:11.760
How should the infrastructure establish authority and provenance and a source of truth within the context of that uh of structured content?
00:35:11.760 --> 00:35:14.719
Because that content may be structured differently.
00:35:14.719 --> 00:35:17.280
What's your take on that?
00:35:17.920 --> 00:35:19.920
Yeah, so I think this is a really timely question.
00:35:19.920 --> 00:35:36.079
So the the environment you've described, you know, very briefly, where like technical content comes from support, product engineering, um, the technical writers, like that is one version of companies where the the technical content is kind of like the result of all these contributors.
00:35:36.079 --> 00:35:39.039
That's how you get to those corpuses of content.
00:35:39.039 --> 00:35:50.480
The other structure of companies is where you have the technical writers is more of like a clearinghouse, like they're the ones that manage most of the content, and they probably write a large portion of it.
00:35:50.480 --> 00:36:10.639
And in those environments, you do tend to have knowledge bases that grow up around that, but there's a clear, very clear delineation between like the docs are gold plated, the knowledge bases can be a source, it can be leveraged, but it's not like the level of governance and quality that's expected is is very, very different.
00:36:10.639 --> 00:36:17.840
And I think it's very situational as to which which is going to work for any one individual company.
00:36:17.840 --> 00:36:28.000
You know, some companies can work really well in the more almost crowdsourced version of documentation, and then other companies really can't go that route.
00:36:28.000 --> 00:36:36.159
Like they have to have technical writers as the authorities on truth, more or less, like is the way to think about it.
00:36:36.159 --> 00:36:39.360
Um, and that's like a that's like um a center of excellence approach.
00:36:39.360 --> 00:36:46.559
I have a lot more experience in that approach because our technology lends itself to that approach.
00:36:46.559 --> 00:36:53.360
Like if you have the highly like crowdsourced collaborative version of it, like that's just not where we fit.
00:36:53.360 --> 00:36:55.599
So I don't work in that world as much.
00:36:55.599 --> 00:37:06.320
When you're trying to establish authority and governance across multiple groups like that, so again, I guess there's kind of two situations there.
00:37:06.320 --> 00:37:12.559
One is they're all using the same thing, which in my experience there is almost always a wiki or Google Docs.
00:37:12.559 --> 00:37:15.440
Like I haven't seen very many variations on that.
00:37:15.440 --> 00:37:24.559
Um, and that's just because when you have a large group of people who are not writers writing the content, you have to give them a tool that's not a writer's tool.
00:37:24.559 --> 00:37:35.360
It's like it's super uncommon that you'll see a circumstance where you have engineering product support and technical, and like maybe you know, a couple of technical writers working in technical writing tools.
00:37:35.360 --> 00:37:41.840
Like I think I've maybe seen that once in my entire career doing this for like I'm not even gonna say how many years, but more than a decade.
00:37:41.840 --> 00:37:46.000
It's much more common that you'll see you hit you go to the lowest common denominator.
00:37:46.000 --> 00:37:48.320
Maybe that's that's not exactly what I mean.
00:37:48.320 --> 00:37:52.480
You go to the most available system, which is almost always a wiki.
00:37:52.480 --> 00:37:57.039
Wikis are not built for governance, like there's just no way that.
00:37:57.039 --> 00:38:12.159
Like wikis are built for availability, and governance is an that imposes structure, it needfully imposes roadblocks and workflow and things like that, because that's where quality comes from.
00:38:12.159 --> 00:38:19.599
It comes from people doing the same things, knowing the patterns, checking themselves in predictable ways, that kind of stuff.
00:38:19.599 --> 00:38:21.199
Like that's governance, you know.
00:38:21.199 --> 00:38:24.159
Like you look at any like Six Sigma or Six Sigma, right?
00:38:24.159 --> 00:38:26.159
Like it's it's a set it's a set of patterns and practices.
00:38:26.159 --> 00:38:34.000
You can't apply those to a group of people that are not professionals in patterns and practices around that particular environment.
00:38:34.000 --> 00:38:37.280
So it becomes very challenging, is what it comes down to.
00:38:37.280 --> 00:38:41.440
You can't really have like broad inputs and high governance.
00:38:41.440 --> 00:38:52.079
And if you don't have high governance in your content, the work of like reverse engineering authority and like source of truth becomes much more complicated.
00:38:52.079 --> 00:38:54.960
It's not impossible in all cases.
00:38:54.960 --> 00:38:57.199
So I can give you an example of this.
00:38:57.199 --> 00:39:01.360
So like we have technical writers here, obviously, like that would be crazy if we didn't.
00:39:01.360 --> 00:39:10.400
But like our product team doesn't do the product specifications in Heretto, which is you know, Heretto is a documentation system.
00:39:10.400 --> 00:39:23.199
Like we do our product specifications in Confluence, and we do that because like we like product specifications are the sorry, these are the the design specs, not the actual, like these are these are not the final specs.
00:39:23.199 --> 00:39:25.199
We don't do the design specs in Heretto.
00:39:25.199 --> 00:39:28.559
So the design specs, because like they are working spaces, right?
00:39:28.559 --> 00:39:38.800
Like you've got a bunch of people in there and like they're throwing stuff at the wall, they're pulling things around, like they're just they're kind of screwing around in the content to figure out what's going to work while they're going through the process, the design process.
00:39:38.800 --> 00:39:48.480
It's a very creative, free-flowing process of like getting together, and that's just not the right place to apply like high structure and high governance.
00:39:48.480 --> 00:39:58.960
Like, you need to give people the minimum amount of those things to give them the most creative freedom to get to the end design results they're looking for.
00:39:58.960 --> 00:40:06.880
Now, at the same time, the outcome of these initial design documents that they do, they tend to be very high quality.
00:40:06.880 --> 00:40:08.880
So they're totally inconsistent.
00:40:08.880 --> 00:40:21.440
Like there's no like like they're not, you know, they're not in the same format of the same structure, but like they have really, really good information in them because they are for the initial design of something, like they kind of are the single source of truth.
00:40:21.440 --> 00:40:29.840
So in our world, what we do there, this is before the thing is built, before the documentation is written, all those kinds of things, is you just lock it down.
00:40:29.840 --> 00:40:35.599
Like once it's done, you just reduce the number of people that can change it, and the people that can change it are much more mindful.
00:40:35.599 --> 00:40:44.800
Now, we also have the benefit of the people who are working on this, are people who are who have all of the benefits of knowing structured content and documentation because that's a tool that we work on.
00:40:44.800 --> 00:40:59.599
So, like inherently, they're gonna be better about some of these practices, and you do see that bleed into you know building just frankly, probably more information architecture, you know, kind of by osmosis in some of these documents.
00:40:59.599 --> 00:41:02.639
So, like we see a lot of content come from Confluence.
00:41:02.639 --> 00:41:14.239
It's very common that people move from Confluence to Heretto, and it tends it comes in a wide variety of of qualities and like like positions from the structure and like a governance perspective.
00:41:14.239 --> 00:41:21.599
So, but we use those as inputs to to different LLM systems that we have, those those Confluence pages.
00:41:21.599 --> 00:41:38.320
So before a feature is built and before the content is migrated into Heretto and it's put on our internal like our internal point of reference is a thing we have like an internal name for it, that is all designed and built in Heretto, and that is considered the highest order of truth in the organization.
00:41:38.320 --> 00:41:40.159
Like if it's there, it's true.
00:41:40.159 --> 00:41:49.599
But before, you know, in the design process, if people have questions about things, like they can ask an AI agent and it will source from those Confluence pages.
00:41:49.599 --> 00:41:59.280
And it works quite well because it's a very small set of content, it's maintained by a very small team, it has a very known position in the cycle of information.
00:41:59.280 --> 00:42:03.679
Like it was designed at this point in time by these people for this purpose.
00:42:03.679 --> 00:42:11.360
So like there is a set of metadata that's assumed about that content simply by the definition of what it is.
00:42:11.360 --> 00:42:16.159
Like there's all these attributes that come with it that you can build into a system to get good results from.
00:42:16.960 --> 00:42:21.518
I was also thinking about lone writers and systems where they've not evolved.
00:42:21.518 --> 00:42:25.920
You gave them great examples of how to think about it.
00:42:25.920 --> 00:42:29.518
So if you don't have those systems in place, you can start small.
00:42:29.518 --> 00:42:32.480
Where can I start if I don't have it?
00:42:32.480 --> 00:42:38.480
How do I think about this where you don't have established processes or uh governance frameworks?
00:42:38.480 --> 00:42:48.960
How do you establish that so that you can get to the source of truth and locking it down and having the right people, get the right gatekeepers, so that eventually your content is gold-plated?
00:42:49.760 --> 00:42:53.199
Um Yeah, I can actually give one other piece of AI-related advice to that.
00:42:53.199 --> 00:43:02.960
So if you're a lone writer, like, so if I was gonna go be a lone writer in an organization today, almost no chance that you have access to like true structured content.
00:43:02.960 --> 00:43:05.518
Like that's just not common for lone writers.
00:43:05.518 --> 00:43:11.280
So I think the most common circumstance for those individuals today is going to be that they're writing in Markdown.
00:43:11.280 --> 00:43:14.800
There's a bunch of other situations, but like that's probably the most common.
00:43:14.800 --> 00:43:16.159
So I'll focus on that one.
00:43:16.159 --> 00:43:38.480
One of the things that I would do after I wrote a very basic style guide, I'm not even gonna call it an information architecture at that point in time, but a set of rules and patterns around how it is that we created content, is I would feed that into an agent and I would say, you're now going to be my editor, and you're gonna hold me to these rules.
00:43:38.480 --> 00:43:50.400
And so then every time, so again, we're in Markdown, we're probably checking this content into GitHub, which at a you know, at if I'm maintaining a thousand topics or less, that's probably gonna work pretty well.
00:43:50.400 --> 00:43:56.400
Every time uh it was checked in, it just had the agent go and look look it over and have it give me feedback.
00:43:56.400 --> 00:43:59.199
Oh, you didn't put enough metadata on the top of this topic.
00:43:59.199 --> 00:44:01.039
Oh, this topic is too long.
00:44:01.039 --> 00:44:03.518
Oh, it it this topic contains too many claims.
00:44:03.518 --> 00:44:11.280
Like you should break this up into more more Markdown files because you know you're mixing concepts and and tasks here.
00:44:11.280 --> 00:44:16.800
Like this is clearly, you know, this section is clearly a task, and this section contains a bunch of claims.
00:44:16.800 --> 00:44:19.599
Like there's uh claims facts, sorry.
00:44:19.599 --> 00:44:31.039
So there's a bunch of things that you could do with a very simple configuration of Claude, or I've actually not done this in uh in Gemini or OpenAI, so like I'm just gonna say Claude, but I'm assuming you can do this in other LLMs.
00:44:31.039 --> 00:44:32.079
Yeah, yeah.
00:44:32.079 --> 00:44:48.559
So you could set all those things up and it could basically just help you enforce a number of rules that are going to get you a lot further down the road than you would be if you were you know just writing content as text and Markdown or in YAML files or things like that.
00:44:48.559 --> 00:45:01.840
And I think you would find that if it did scale up and you hired more people and you did need to transition to a true structured format, you know, for personalization or reuse, those types of things, that you'd still have a lot of work to do, but you'd be in a better place.
00:45:02.559 --> 00:45:07.518
Yeah, I think that's a great suggestion to kind of lean into those systems available to you.
00:45:07.518 --> 00:45:10.800
If you were designing a content architecture from scratch today.
00:45:10.800 --> 00:45:17.440
Let's say you've been hired, there are no systems, and you are going to be designing content architecture from scratch.
00:45:17.440 --> 00:45:26.719
Uh specifically for AI and agentic systems, what would you build from the source content all the way through retrieval context, uh governance, and execution?
00:45:26.719 --> 00:45:28.559
Oh my god.
00:45:28.559 --> 00:45:36.800
But you you do you you've been you've done great so far with all the I mean geez.
00:45:38.000 --> 00:45:43.039
Well, so it would depend a ton on there's so many variables that go into that.
00:45:43.039 --> 00:45:45.599
So I'll give you one version of it.
00:45:45.599 --> 00:46:03.360
Okay, so if I'm coming into an organization that has, let's say 10 proper technical writers maintaining a hundred thousand pages of content, or well, let's just say fifty thousand pages of content.
00:46:03.360 --> 00:46:06.559
It's that's a little more on the on the medians there.
00:46:06.559 --> 00:46:16.000
And that content is you know, has any level of complexity, which it almost always does at that size, like you can almost never get around that.
00:46:16.000 --> 00:46:21.039
The first thing I'm going to do, if they're not already, if so, like you said, the nothing exists.
00:46:21.039 --> 00:46:26.239
The first thing I'm gonna do is I'm gonna introduce structured content and a proper CCMS.
00:46:26.239 --> 00:46:28.320
Like that would be step one.
00:46:28.320 --> 00:46:31.518
And then from there, I'm going to look at the delivery requirements.
00:46:31.518 --> 00:46:39.679
So one of the things that I think people undervalue is the importance of having a really high-quality website today.
00:46:39.679 --> 00:46:43.440
Like that is where most of your AI answers are coming from.
00:46:43.440 --> 00:46:46.800
Like before you implement any AI, you need to feed the public AI.
00:46:46.800 --> 00:47:12.159
And to do that, you need a website that has all the different Many of them are just SEO things, but all the different aspects that are going to give you the proper authority with Google, OpenAI, Anthropic, and the other major models, so that when people are asking questions of those systems, they're properly retrieving information from your set of information for your customers.
00:47:12.159 --> 00:47:16.960
Like that's gonna be 90% of your customers' AI interaction.
00:47:16.960 --> 00:47:20.320
From there, it's a very it's gonna get very situational.
00:47:20.320 --> 00:47:35.360
If I've got a field service team, I'm gonna look at how I'm gonna deploy an agent to that field service team that covers the particular issues that they're having a hard time solving that are also their information issues.
00:47:35.360 --> 00:47:41.039
I would start deploying agents and connections to our support channel.
00:47:41.039 --> 00:47:43.119
So like there was a feedback loop there.
00:47:43.119 --> 00:47:46.960
You know, those things are maybe not simple, but fairly straightforward, right?
00:47:46.960 --> 00:47:49.679
Like that's kind of like, you know, that's known work at this point in time.
00:47:49.679 --> 00:47:53.039
I'm not gonna call it a solved problem, but it's known work at this point in time.
00:47:53.039 --> 00:48:03.440
I think if you you kind of started with that path in that you're going to get yourself so far down the road that you're probably gonna be ahead of most people already.
00:48:03.440 --> 00:48:05.440
That I think that that is a blueprint.
00:48:05.440 --> 00:48:19.360
I guess the other thing I should say there too is when I said implement a CCMS, I meant do the information architecture work on the content, come up with an information model, get the content into really good shape, and then put it into a CCMS.
00:48:19.360 --> 00:48:21.760
So that's like the process of implementing structure.
00:48:21.760 --> 00:48:28.320
One of the biggest mistakes that people make is that they think they're gonna buy a tool and that's gonna solve their problems, and that's not the—not the case.
00:48:28.320 --> 00:48:37.280
Like the work it takes to go from having a ton of content that's unstructured to good structured content that's actually going to solve problems.
00:48:37.280 --> 00:48:43.440
That's an information modeling and information architecture and conversion process, and that is work.
00:48:43.440 --> 00:48:47.679
It has a huge payoff, it's the foundation for everything else.
00:48:47.679 --> 00:48:52.000
But if you don't do that work, you don't get the benefit out of structured content.
00:48:52.000 --> 00:48:54.000
So, like that would be the first step.
00:48:54.719 --> 00:48:56.559
First of all, my apologies, Patrick.
00:48:56.559 --> 00:49:00.480
I did not mean to quote unquote assault you with that question.
00:49:01.199 --> 00:49:01.920
No, no, no, no.
00:49:01.920 --> 00:49:03.039
It's a great question.
00:49:03.039 --> 00:49:04.880
It's just it's such a big question.
00:49:05.599 --> 00:49:12.159
So I was on a team where the mindset was that the tool would solve all the problems.
00:49:12.159 --> 00:49:20.880
And thinking about information architecture and mod—well, let's say that we had to revisit how our content was modeled and architected.
00:49:20.880 --> 00:49:23.518
The tool did not solve the problem.
00:49:24.239 --> 00:49:24.800
You think about it.
00:49:24.800 --> 00:49:32.559
If you were gonna go build a house and you showed up at the build site with a bunch of tools but no blueprints, yeah, what would you get?
00:49:32.559 --> 00:49:33.599
What would you end up with?
00:49:34.320 --> 00:49:35.039
Thank you so much for watching.
00:49:35.039 --> 00:49:38.320
This was a great conversation, and I hope you had fun as well.
00:49:38.320 --> 00:49:38.960
I did.
00:49:39.440 --> 00:49:39.920
Thanks again.
00:49:40.559 --> 00:49:41.119
Thank you.
00:49:41.119 --> 00:49:44.480
Thank you for listening to Inside Tech Comm.
00:49:44.480 --> 00:49:49.039
If there's a guest you would love to hear on the show, please let me know.
00:49:49.039 --> 00:49:54.320
And don't forget to follow and catch every episode on your favorite podcast app.
00:49:54.320 --> 00:49:55.599
See you next time.
