WEBVTT
00:00:02.240 --> 00:00:07.599
Hello friends and welcome to season eight of Inside Techcom with Zahra Mutabana.
00:00:07.919 --> 00:00:16.559
This season features a collection of conversations that explore the ideas, challenges, and opportunities shaping technical communication today.
00:00:16.719 --> 00:00:22.320
From AI and content strategy to leadership, product thinking, and the future of our profession.
00:00:22.559 --> 00:00:26.320
Each episode offers a fresh perspective from people doing the work.
00:00:26.480 --> 00:00:27.760
Let's get started.
00:00:31.839 --> 00:00:36.479
Season eight is proudly sponsored by LavaCon.
00:00:36.560 --> 00:00:43.200
Use discount code ITC26 to save$200 on registration for this year's conference.
00:00:43.359 --> 00:00:46.320
I'll share more about LavaCon later in this episode.
00:00:46.799 --> 00:00:47.840
Hello listeners.
00:00:47.920 --> 00:00:50.479
Welcome to another episode of Inside TechCon.
00:00:50.960 --> 00:00:54.240
Today we'll be talking with Patrick Gozek from Herero.
00:00:54.960 --> 00:01:03.600
And in the past, we've talked about structured content and its impact on how AI ingests that framework.
00:01:04.000 --> 00:01:05.680
Today we're going to do a deep dive.
00:01:06.159 --> 00:01:14.799
Patrick is going to be talking to us and educating us on how structured content contributes useful to the context under the hood and how it works.
00:01:15.120 --> 00:01:16.879
With that, Patrick, welcome to my show.
00:01:17.280 --> 00:01:18.480
Great to be here, Sarah.
00:01:18.560 --> 00:01:19.200
It's very nice.
00:01:19.599 --> 00:01:23.599
Patrick, first tell us a little about yourself, what you do, and yourself.
00:01:24.000 --> 00:01:24.799
Sure, happy to.
00:01:24.959 --> 00:01:26.480
So I'm Patrick Bozick.
00:01:26.799 --> 00:01:29.599
I am one of the founders of Heretto.
00:01:29.680 --> 00:01:30.799
I'm also the CEO.
00:01:31.040 --> 00:01:34.319
So I've been in the structured content industry for quite a while.
00:01:34.560 --> 00:01:39.760
As the uh some I guess I could be considered the face of the company sometimes.
00:01:40.000 --> 00:01:43.920
So in that role, I also do quite a bit of media things.
00:01:44.000 --> 00:01:46.319
So I have a talk show I do with Scott Abel.
00:01:46.640 --> 00:01:48.400
So that's called Coffee and Content.
00:01:48.480 --> 00:01:54.640
We've been on a little bit of a hiatus due to a technical issue, but we typically do coffee and contents every other week.
00:01:54.799 --> 00:01:58.640
So those are a lot of fun, cover a lot of different topics from the content world.
00:01:58.879 --> 00:02:02.480
We also have a podcast called Components, a content components.
00:02:02.560 --> 00:02:06.799
If you look it up on Spotify, that's got four or five seasons.
00:02:06.959 --> 00:02:10.479
It's all about documentation, structured content, those kinds of things.
00:02:10.560 --> 00:02:15.280
So if you're wanting nerd out about structured content, that's a great place to go.
00:02:15.599 --> 00:02:15.840
Yeah.
00:02:15.919 --> 00:02:18.080
And then I'm a developer by background.
00:02:18.159 --> 00:02:22.159
So I was one of the developers and architects of the first version of our software.
00:02:22.240 --> 00:02:26.639
So I'm I'm fairly technical, but I don't cause problems for our development team anymore.
00:02:26.719 --> 00:02:31.039
Nowadays, I focus on what's going to make customers happy and those kinds of things.
00:02:31.360 --> 00:02:36.000
What spawned this idea to look into structured content as a developer?
00:02:36.159 --> 00:02:37.039
I'm curious.
00:02:37.360 --> 00:02:37.919
Oh boy.
00:02:38.080 --> 00:02:41.759
So that's actually going back a lot further than I'd like to admit.
00:02:42.000 --> 00:03:04.800
But my co-founder and I, we got into structured content, not DITA, which is where we are today, but XML content with an actually with a custom schema when we were in college, actually, and we just saw that it was a superior way to store and manage content and that it was going to make some of the publishing activities we were working on easier and more capable.
00:03:05.039 --> 00:03:11.280
At that point in time, the technologies surrounding digital publishing were just far, far less well developed than they are today.
00:03:11.680 --> 00:03:19.280
So structured content provided it actually provided something quite different at that moment in time, now that I'm reflecting on it, than it does today.
00:03:19.520 --> 00:03:25.919
At that moment in time, it it basically gave you the ability to put a like a little bit of a database and a file.
00:03:26.159 --> 00:03:40.479
The amount of things you had to manage and the flexibility you got when you were using structured content at that point in time, if you're looking to apply that to like a very structured digital publishing environment, there was a lot of advantages there.
00:03:40.560 --> 00:03:42.719
Um so that was why we picked it up originally.
00:03:42.879 --> 00:04:02.400
And then over the years, we eventually drifted into, or I guess found our way, is maybe a better way of putting it, into technical content where you still get a lot of those benefits that we we were originally looking at with DIDA and in the digital publishing around uh technical content and documentation, but there's a ton more today.
00:04:02.560 --> 00:04:04.800
So there's there's so much more that you get from it.
00:04:05.199 --> 00:04:08.560
Yeah, we could probably have a whole episode on structured content.
00:04:08.639 --> 00:04:13.199
And I have, as I mentioned, had some episodes covering what that is.
00:04:13.360 --> 00:04:16.560
And it seems to me like the context can be different.
00:04:16.959 --> 00:04:28.560
Of course, data comes close to mind, but for those of us that don't use data, structured content still follows a certain framework, a certain guidelines, if you will.
00:04:28.800 --> 00:04:41.839
But for this conversation, I want to ground what that definition means to you, and then we can, I guess, start talking about how we can deep dive into how that lends to AI.
00:04:42.160 --> 00:04:49.199
Yeah, I should think that is really important to start with the definition of structured content because everybody does define it a little bit differently.
00:04:49.439 --> 00:04:53.360
And I think the individual definition matters.
00:04:53.759 --> 00:04:55.439
So I'll give you my definition.
00:04:55.519 --> 00:04:57.600
I'll also tell you why I think mine is right.
00:04:58.480 --> 00:05:01.519
But that doesn't mean that other people's don't uh can't be right too.
00:05:01.839 --> 00:05:22.639
So technically speaking, if you put all of your content into Excel or Google Sheets and you put it into, you know, let's say every topic was a column, or maybe better a row, and every portion of that topic or that piece of information was divided up into different columns.
00:05:23.040 --> 00:05:25.839
So you have the pieces broken up into the different cells.
00:05:26.079 --> 00:05:31.680
That would be structured content because you would have the content broken up into a structure.
00:05:31.839 --> 00:05:35.920
Like in the most basic definition, you have a structure for your content.
00:05:36.079 --> 00:05:41.920
So you're you're labeling the different pieces of it in a methodology, and that's structured content.
00:05:42.079 --> 00:05:43.839
Now, obviously, that's a terrible choice.
00:05:44.000 --> 00:05:48.399
Like that wouldn't be a thing that you'd want to do in almost any production environment.
00:05:48.480 --> 00:05:56.879
Although I will say that we have had customers migrate from that exact environment just like that to our tool, which is DIDA.
00:05:57.040 --> 00:06:10.319
But when you think about like how this comes together, the reason I start there is because there are a lot of systems that are they're effectively that, because if you think of what a form is, so like a digital form, it is just actually a table.
00:06:10.560 --> 00:06:12.959
You have your question one, question two.
00:06:13.199 --> 00:06:15.600
So your question one might be what is the heading for this file?
00:06:15.839 --> 00:06:19.920
Your question two might be who authored this file, question three might be the body.
00:06:20.160 --> 00:06:24.879
And people will call that structure content because you know you can put metadata in one place.
00:06:24.959 --> 00:06:29.680
And and in certain circumstances, that's what headless CMSs are, for instance.
00:06:29.759 --> 00:06:32.720
So, like things like contemptful, those kinds, that's that's what they are.
00:06:32.879 --> 00:06:35.519
In certain circumstances, that actually works quite well.
00:06:35.680 --> 00:06:38.319
Um, that's a really effective form of structured content.
00:06:38.480 --> 00:06:47.199
It's backed by a database, uh, it has the ability to create links between those pieces of information, uh, it has the metadata properly separated out.
00:06:47.279 --> 00:07:00.720
Like you have like there are certain things, but it in the background, it is functionally a table that is fundamentally different than what we work with, which is XML, because XML is a tree, which means that you can have structure below those big boxes.
00:07:00.959 --> 00:07:05.120
So when you go into a topic, there's going to be structure at multiple levels.
00:07:05.360 --> 00:07:10.639
So you'll have a piece, like let's say we'll use a procedure or a task topic, for instance.
00:07:10.879 --> 00:07:19.279
When you start creating that, you'll have a title, and then you'll have metadata around it, and then you'll have the prerequisites for running this procedure.
00:07:19.439 --> 00:07:24.319
And then the individual procedure, instead of being an ordered list, it would be steps, right?
00:07:24.879 --> 00:07:28.560
And then there's a step inside of it, there's structure inside of the steps.
00:07:28.800 --> 00:07:39.040
And so by having this multiple layers of structure and semantics, it means that you're able to manipulate that in a more granular way beyond just formatting.
00:07:39.199 --> 00:07:43.439
It's not just HTML and boxes, effectively, but they're both structure.
00:07:43.519 --> 00:07:48.160
Like there is a like both things are valid in terms of the structure content label.
00:07:48.399 --> 00:07:54.959
So, what you really need to ask yourself when you're thinking about, okay, well, we're moving to structure, it's beyond just the term structure.
00:07:55.040 --> 00:08:00.399
Like you should really be thinking about what is the form of structure that's going to serve our mission?
00:08:00.560 --> 00:08:04.079
What do we care about in terms of interoperability?
00:08:04.399 --> 00:08:07.439
What forms of linking are really important to us?
00:08:07.680 --> 00:08:10.480
What forms of output are going to be really important to us?
00:08:10.639 --> 00:08:18.959
Because those types of questions will guide you towards the type of structure that's going to be best for your particular implementation and your particular needs.
00:08:19.360 --> 00:08:24.800
I have to say that I thought my definition was going to be very different from yours or my understanding.
00:08:25.120 --> 00:08:28.480
But at some level, I think we are on the same page.
00:08:28.800 --> 00:08:29.040
Yeah.
00:08:29.279 --> 00:08:30.879
I mean, I think my definition is pretty broad.
00:08:30.959 --> 00:08:32.080
Like I said, I think it's right too.
00:08:32.399 --> 00:08:34.000
Yes, it's very inclusive.
00:08:34.320 --> 00:08:39.200
So that allows me to be kind of okay, that definition sounds close to what I have in my mind.
00:08:40.000 --> 00:08:40.639
Fantastic.
00:08:41.120 --> 00:08:41.440
Yes.
00:08:41.759 --> 00:08:49.519
I have not worked with Ditta, and you mentioned Ditta, and Dita comes up a lot when we talk about structured content.
00:08:49.840 --> 00:08:58.080
We've touched upon this before, but I think for the benefit of the audience, it would be nice to quickly do a shout out to what Dita is.
00:08:58.399 --> 00:08:58.960
Yeah, for sure.
00:08:59.120 --> 00:09:03.200
So Dita, in the most basic definition, it's it's a form of XML.
00:09:03.360 --> 00:09:08.000
So in that way, you can actually think of it as like a cousin to HTML in some ways.
00:09:08.080 --> 00:09:11.279
So it actually shares a lot of the same elements that HTML has.
00:09:11.519 --> 00:09:20.960
The thing to know about DITA is that it has built in many of the patterns that you need to scale up technical content.
00:09:21.039 --> 00:09:32.240
So there's a ton of stuff that we've pretty much figured out over the last call it 15 or 20 years, are really critical to larger or more complex sets of technical content.
00:09:32.559 --> 00:09:39.840
And the DITA committee, so the people who work on the open source standard, they put all of those things into the standard.
00:09:39.919 --> 00:09:41.600
So you don't have to recreate them.
00:09:41.759 --> 00:09:48.879
And then perhaps just as importantly, beyond not having to recreate them, you don't have any vendor opinion or lock-in.
00:09:49.279 --> 00:10:05.120
So one of the challenges that the that DITA was trying to solve, and I think has solved pretty effectively, is that when you have vendor-specific formats, it means that okay, well, we've reused content, but we've reused content in this particular system.
00:10:05.360 --> 00:10:14.639
And once you get to a certain level of reuse, if you didn't have a portable technology underneath it, your ability to migrate someplace else is I mean, it's it's almost impossible.
00:10:14.960 --> 00:10:18.320
It just becomes the lock in there, it's just so high.
00:10:18.639 --> 00:10:29.440
So what Ditter really solves is it solves having good patterns for the core things that you really need to do with technical content, and then it also solves an interoperability problem.
00:10:29.600 --> 00:10:34.480
And you know, there's some of the criticisms of Ditto are that it comes with you know like a high learning curve.
00:10:34.559 --> 00:10:36.480
Yeah, I don't know if I actually agree with that.
00:10:36.559 --> 00:10:38.879
Like I've seen a lot of people learn it pretty quickly.
00:10:39.120 --> 00:10:42.080
There's people feel that it's too heavy for today's uses.
00:10:42.159 --> 00:10:43.360
That's another criticism.
00:10:43.600 --> 00:10:45.679
Those people tend to come from like the markdown world.
00:10:45.840 --> 00:10:51.039
So, like in comparison to Markdown, you know, markdown is a text format, DITA is an is an XML format.
00:10:51.279 --> 00:10:52.799
I think that's just a use case thing.
00:10:52.879 --> 00:10:59.279
So like DITA is not a great, is not like you never write the README files for your vibe coded project in DITA.
00:10:59.360 --> 00:11:00.159
I don't do that.
00:11:00.320 --> 00:11:05.360
Like when I have small apps and things like that that I build, I don't write any of the documentation in DITA.
00:11:05.440 --> 00:11:06.399
It doesn't make any sense.
00:11:06.639 --> 00:11:13.840
Did it is a format that you use when you arrive at a sufficient level of complexity and a sufficient scale.
00:11:14.080 --> 00:11:24.720
So if you want like really rough and tough numbers, what we've found over the years is that if you're below a thousand pages of content, DITA probably doesn't make sense.
00:11:24.879 --> 00:11:31.919
Um, there are some exceptions, but even at a thousand pages of content, like it can be questionable there depending on your needs.
00:11:32.159 --> 00:11:40.000
And so where it really starts to make sense is when you have multiple variations of products or variations of content, audiences.
00:11:40.320 --> 00:11:49.919
So you have anything that's like personalization, and there's a bunch of different terms that people use for the content changes when somebody looks at it for a different person looks at it for a different reason.
00:11:50.159 --> 00:11:53.519
Localization can be a driver, although I think less so today.
00:11:53.679 --> 00:11:56.159
And then just sheer scale with like reuse.
00:11:56.240 --> 00:12:02.240
So if you have a lot of content that you is used in multiple places, that's when you know data starts to really shine.
00:12:11.279 --> 00:12:12.639
In a simplified way.
00:12:13.039 --> 00:12:31.519
Since we are going to do a deep dive into how structured content lends itself to AI, does data lend itself better, or just this the generic definition of the structured content that we ground it, that we are trying to ground this conversation in also lends itself to better AI interactions?
00:12:31.840 --> 00:12:33.279
That is such a good question.
00:12:33.600 --> 00:12:36.960
So the answer is yes, it depends in here.
00:12:37.120 --> 00:12:42.080
And I feel like I'm among the right people to give like engineering style, it depends answers.
00:12:42.240 --> 00:12:46.720
One of the jokes in our leadership team is that when you ask the business people a question, they give you an answer.
00:12:46.960 --> 00:12:49.759
When you ask the technical people a question, they say it depends.
00:12:50.000 --> 00:12:53.600
And so I think that I'm I'm filling that role in this particular moment.
00:12:53.840 --> 00:12:57.360
So, you know, this is kind of like what we we wanted to cover today.
00:12:57.440 --> 00:13:00.320
So it is a it's a very, very long answer.
00:13:00.720 --> 00:13:10.879
At the very core of it, if like he wanted like the too long didn't listen version of it, structured content is going to give you better structured contents with the broad definition.
00:13:10.960 --> 00:13:20.559
So like anything that is broken up into labeled pieces that are then stored consistently, like that definition of structured content.
00:13:20.639 --> 00:13:29.200
So did inclusive of data, inclusive of uh headless CMSs, inclusive of anything that has a well-defined schema applied to content.
00:13:29.360 --> 00:13:32.240
So, why would that be better for AI?
00:13:32.320 --> 00:13:37.759
And it really just comes down to chunking, like that's the fundamental difference between those things.
00:13:38.080 --> 00:13:47.039
So the problem with that statement is that when you get to certain forms of structured content, and I'll like I'll give you like an example from a headless CMS.
00:13:47.200 --> 00:13:50.799
The headless CMS is every headless CMS is gonna have a different schema.
00:13:50.960 --> 00:13:56.240
Like part of the the appeal of a headless CMS is that you can create your own schema.
00:13:56.399 --> 00:14:06.240
So you go in and you get a blanks blank page, and all you have to have is a title, and then you can add a body and an afterthoughts, and you could add you know links to other things.
00:14:06.399 --> 00:14:13.600
And like the standard thing is like, you know, how would you lay out a one of the standard examples is how would you lay that out for like a music venue?
00:14:13.759 --> 00:14:18.879
And you'd have one thing that was a band, and then you'd have an artist, and then you'd have like those kinds of things, right?
00:14:18.960 --> 00:14:20.879
And you'd you build those schemas.
00:14:21.120 --> 00:14:31.679
Well, in a really well-designed schema, it's gonna function really well as giving you good sized pieces of content for the chunking.
00:14:31.919 --> 00:14:40.879
In a poorly designed schema, what's gonna happen is you're gonna get people basically putting in a title and a body, and like that's really not any different than a Google Doc, right?
00:14:41.039 --> 00:14:46.159
So at that point in time, people are just gonna throw huge quantities of content into that body field.
00:14:46.320 --> 00:14:55.200
So even though technically you have two fields, the structure is two boxes, like it's one box, it's it's two boxes with unstructured content in one of them.
00:14:55.360 --> 00:14:57.840
And so then you lose the benefits of chunking.
00:14:57.919 --> 00:15:19.679
So now there's a bunch of other things, but if you like if you want the biggest bang for your buck, so like the I don't know, the 10% that gives you 60% outcome, it's having really well clean defined borders around your content, which means smaller pieces, more intentional pieces, pieces of content that contain ideally one piece of information.
00:15:19.919 --> 00:15:23.440
So in DITA, like a topic is supposed to be about one thing.
00:15:23.679 --> 00:15:28.639
It like if it's a concept topic, it's supposed to teach you one thing or explain one thing.
00:15:28.799 --> 00:15:33.600
If it's a task, it's supposed to give you a procedure that does one thing.
00:15:33.759 --> 00:15:39.919
Um, so it takes you from from the beginning to end of one thing you have to complete, and there's other versions of that as well.
00:15:40.159 --> 00:16:05.360
So in the AI world, when you go to produce results with an AI system, being able to provide it very clean chunking, so very clean set like pieces of content for the classification on when you're putting it into a rag system, and then also for retrieval, there's a bunch of other things, but that's the thing that gets you the biggest bang for your buck.
00:16:05.759 --> 00:16:12.480
Modern language models are pretty good at extracting meaning from unstructured content.
00:16:13.440 --> 00:16:16.159
Does that reduce the need for structured content?
00:16:16.559 --> 00:16:21.759
Because we're trying to make the case that you need to have chunking with structured content.
00:16:22.320 --> 00:16:33.039
But I'm kind of trying to flip that question and see if the LLMs have gotten so bad so good, or is there still an advantage to use structured content as we have established?
00:16:33.360 --> 00:16:35.279
Yeah, so I think that is also a great question.
00:16:35.440 --> 00:16:41.759
It's one of the places that there's I don't want to say misunderstanding because I think that's the wrong way to think about it.
00:16:42.000 --> 00:16:45.679
But I think that there's so maybe this is a better way to put it.
00:16:46.000 --> 00:16:52.720
You're right that the the modern LLMs have gotten very good at extracting information when the question is basic.
00:16:52.960 --> 00:17:03.759
And the reality is that most of the time when people are interacting with these LLMs, their their modalities of interaction fall into a couple of categories.
00:17:04.000 --> 00:17:06.960
One is coding, and coding is a verifiable tree.
00:17:07.039 --> 00:17:19.440
So like part of the reason coding has gotten so good is not because the LLMs have gotten so good, it's because the harnesses have gotten so good at managing something that is verifiable, like that.
00:17:19.599 --> 00:17:20.960
It's logically verifiable.
00:17:21.200 --> 00:17:26.720
The other modality that we use, we use these in is effectively a replacement for search, right?
00:17:26.799 --> 00:17:29.200
So you go to Google and like you look at the AI result.
00:17:29.759 --> 00:17:35.839
Again, a lot of the advancement there is actually in the deterministic harness under the hood.
00:17:36.000 --> 00:17:52.160
Like the amount of engineering that Google and Anthropic and Chat or and OpenAI have done so that what is being delivered to the AI for consideration is high quality, is probably equivalent to the amount of like engineering they've done in the AI model.
00:17:52.240 --> 00:17:55.200
Well, maybe not, I don't know, but like it's a lot is the point.
00:17:55.440 --> 00:18:05.519
And then when you're but when you're thinking about interacting with these systems in a more general sense, typically people are putting very small quantities of information into them.
00:18:05.759 --> 00:18:15.200
So like if I go to notebook LLM and I throw in a couple of PDFs, it does work quite well because the quantity of information is very small.
00:18:15.359 --> 00:18:21.200
Now, in the background, it is still chopping that PDF up and it is still indexing it.
00:18:21.359 --> 00:18:26.160
Like it is not just feeding that PDF full stock to the to the LLM.
00:18:26.720 --> 00:18:46.960
But what's happening when you ask that the question, or you're working with the system to get information out of that, is that because it's a small number of chunks, and because the system is then reducing down what it's sending to the LLM to a smaller set, it's not that big of a deal.
00:18:47.200 --> 00:18:49.200
The problem is when you scale up.
00:18:49.519 --> 00:18:56.240
So there's a reason you can only put, I don't know what it is, it's not a huge quantity of content into notebook LLM.
00:18:56.480 --> 00:19:10.880
When you start talking about thousands of pages or tens of thousands of pages or more, at that point in time, especially if you're asking more human questions as opposed to like search queries, that's when it absolutely still breaks down.
00:19:11.119 --> 00:19:13.039
And like we have research that shows this.
00:19:13.119 --> 00:19:16.480
So like this isn't this isn't just me hypothesizing these things.
00:19:16.559 --> 00:19:18.799
Like this is this is stuff that we can show.
00:19:19.039 --> 00:19:36.319
And in those worlds where having really strong rag retrieval, which is built on the foundation of having good tokenization of the content, you start to see structured content outperform unstructured content by about 50%.
00:19:36.960 --> 00:19:38.480
Now it's use case dependent.
00:19:38.559 --> 00:19:44.160
Like there are like simpler questions will do better in both cases, more complex questions will do better.
00:19:44.319 --> 00:19:46.559
So there's there's a lot of it depends in there.
00:19:46.720 --> 00:19:56.079
But for things that really matter, especially in like regulated industries, or I mean, I guess everybody should care about their customers getting the right answer, but you know, like some people care more than others, I suppose.
00:19:56.240 --> 00:20:02.640
It still is a substantial difference in overall quality of retrieval and accuracy.
00:20:03.039 --> 00:20:11.519
So I think this is a great, I guess, segue into the RAG, uh, the retrieve and generate model that we start looking into.
00:20:11.920 --> 00:20:20.160
Under the hood, retrieval is evolving from a fairly linear model towards more agents that can decide what information they need to go and find.
00:20:20.319 --> 00:20:25.440
Where does structured content fit into this newer model of agentic retrieval and context engineering?
00:20:25.680 --> 00:20:29.599
Yeah, I so I think that's an interesting area of evolution right now.
00:20:29.759 --> 00:20:35.839
And I guess the first thing I want to call out here is that the models are themselves getting better.
00:20:36.000 --> 00:20:37.440
Like there's no doubt about that.
00:20:37.599 --> 00:20:46.400
But I actually think the advancement in rag systems is more in the harness sophistication than it actually is in the models right now.
00:20:46.640 --> 00:20:56.559
So when you think about like how does an AI work with a a rag like content operations or rag architecture such that it's going to produce a better outcome.
00:20:56.720 --> 00:21:00.480
Well, actually, should we stop and like give a definition of what of like what rag is before?
00:21:00.960 --> 00:21:01.519
Yes, please.
00:21:01.599 --> 00:21:02.079
Absolutely.
00:21:02.400 --> 00:21:02.799
Go for it.
00:21:03.039 --> 00:21:03.519
Yeah, okay.
00:21:03.680 --> 00:21:11.680
So rag is retrieval augmented generation, which is technical people trying to confuse the world with with fancy words.
00:21:11.839 --> 00:21:15.920
All it means is that you're running a deterministic on the well, you're running a search.
00:21:16.079 --> 00:21:18.720
I was about to say deterministic search, but that's not necessarily true.
00:21:18.960 --> 00:21:27.039
You're running a search, the output of that search is then being put into the context for an AI agent to generate a result.
00:21:27.119 --> 00:21:27.839
That's all it means.
00:21:27.920 --> 00:21:31.279
So it's it's just search and then generate, is what it comes down to.
00:21:31.519 --> 00:21:36.160
Now, most rag systems are built on vector databases.
00:21:36.400 --> 00:21:53.359
And why it matters to call that out is because fundamentally, vector databases, what they are is they're just they're calculations around a piece of information as to like kind of where it sits in space, and they what they do is they retrieve things that are close together.
00:21:53.519 --> 00:21:56.720
Now, those calculations are done by LLMs typically.
00:21:57.039 --> 00:22:04.720
When you look at like some of the research that we've done, like we've Tried a bunch of different vector databases and a bunch of different ways of calculating the embeddings.
00:22:04.880 --> 00:22:08.880
So the embeddings are like the position, I guess, if you want to think about it that way.
00:22:09.119 --> 00:22:16.400
And the way that we we do that is that you can calculate the embeddings with a bunch of different with any of the AI models that are out there.
00:22:16.640 --> 00:22:24.799
So what we've found is that in our testing, open AI tends to be the one that's most cost effective and easiest to do.
00:22:24.880 --> 00:22:26.559
And so that's the one that we've used.
00:22:26.720 --> 00:22:32.480
We've tried a bunch, but that's the one we've kind of settled on as like our baseline, which is funny because we use anthropic everywhere else.
00:22:32.640 --> 00:22:34.240
But like that's how that comes together.
00:22:34.400 --> 00:22:46.160
So then you have the content in the vector database, the way you retrieve it is through these vector searches, and then when you retrieve sets of content, that's going to be put into the LLM.
00:22:46.400 --> 00:22:53.920
So the major difference here between like the first versions of these and the versions that we deal with today, the models are better, yes.
00:22:54.079 --> 00:23:10.960
But now you may see being a in a place where you send the question to the LLM, the LLM modifies the question or it breaks it down into some way, or it clarifies the question, changes the question, and then it passes the query to the vector database.
00:23:11.119 --> 00:23:14.640
The vector database returns results, and that loop can continue.
00:23:14.880 --> 00:23:17.119
So reasoning is largely just a loop, right?
00:23:17.200 --> 00:23:19.279
Like they basically feed the results back in.
00:23:19.359 --> 00:23:26.160
There could be some deterministic logic that does some parsing or extraction between the LLM and the database.
00:23:26.319 --> 00:23:32.880
Like there's other ways that they, you know, they can basically upgrade the overall loop, but it really fundamentally is a loop.
00:23:33.039 --> 00:23:39.119
So I think the misconception that a lot of people have is they think all of this is happening inside of the LLM.
00:23:39.279 --> 00:23:45.519
Like the LLM is making a decision about what information it's retrieving, but like the information is in the LLM.
00:23:45.599 --> 00:23:49.359
The information that you're when you're talking about RAG is not in the LLM.
00:23:49.440 --> 00:23:55.440
It's in a separate piece of technology that the LLM is making, is having an interaction with?
00:23:55.920 --> 00:24:04.880
When I started working with structured content, metadata became a centerpiece to how a cornerstone to how search was done.
00:24:04.960 --> 00:24:10.079
So the the richer the metadata, the better the output was.
00:24:10.720 --> 00:24:12.960
So organizing information was super critical.
00:24:13.200 --> 00:24:24.559
What changes when a machine is using that metadata to decide what information belongs in an AI models context with all the technology breakdown that you've given us?
00:24:24.960 --> 00:24:25.279
Yeah.
00:24:26.240 --> 00:24:29.519
So this could be a whole series of podcasts in and of itself.
00:24:30.079 --> 00:24:30.319
Of course.
00:24:30.799 --> 00:24:31.440
What changes?
00:24:31.680 --> 00:24:33.200
Well, oh boy.
00:24:33.519 --> 00:24:41.599
The first thing I think that's important to think about here is that metadata gives you a lot of options in how you process something.
00:24:42.079 --> 00:24:58.640
So one of the key aspects of having really high quality metadata on your content is it's hooks and like levers that you can pull in deterministic processes that decide how content ends up and how it shows up.
00:24:58.960 --> 00:24:59.519
Right.
00:24:59.759 --> 00:25:13.599
So when you're thinking about what you're gonna end up providing to an LLM, you almost always have some type of a pipeline that's going to be coming into whatever your rag system is or whatever, wherever the LLM is actually going to be touching that content.
00:25:13.839 --> 00:25:22.960
And metadata means that you can be more specific about what and how those things go into the uh the LLM or the AI-based system.
00:25:23.119 --> 00:25:27.359
So it gives you a lot more control with traditional code.
00:25:27.599 --> 00:25:32.160
And I'm emphasizing that because it really shouldn't be undervalued.
00:25:32.319 --> 00:25:34.559
That is still enormously powerful.
00:25:34.799 --> 00:25:50.400
The more specific you can be in a deterministic way in the systems that you build, the more confidence you can have and the more sustainability those systems are going to have, because you can write unit tests, right?
00:25:50.480 --> 00:25:53.519
Like things in deterministic systems have deterministic outcomes.
00:25:53.680 --> 00:25:56.480
Like that's fundamentally how they're different than AI systems.
00:25:56.720 --> 00:25:59.839
So metadata still has a ton of value in that way.
00:26:00.079 --> 00:26:03.759
This is kind of a the other side of that coin, but I think also valuable.
00:26:04.000 --> 00:26:15.920
There are circumstances where traditional search, if metadata is effectively applied, will perform equally well to vector search, and it's a lot less expensive.
00:26:16.160 --> 00:26:20.240
So there's certain applications where the old stuff works just as well.
00:26:20.400 --> 00:26:26.400
And there's actually cases where it works better, but then you know, we don't have to go, we don't have to dig too deeply into those things.
00:26:26.559 --> 00:26:30.079
But the point is that metadata also gives you options.
00:26:30.240 --> 00:26:46.720
So if you're in a place where you're completely reliant on AI systems and you don't have really strong metadata around how you're trying to build a retrieval pipeline, um, then you're totally reliant on the inference of context from the AI into a vector database.
00:26:46.960 --> 00:26:58.000
Whereas if you had that metadata, you might be in a place where you could have a deterministic system or a traditional search that was just as just as good or better, um, and also gave you some other options.
00:26:58.160 --> 00:27:04.240
So metadata can be really valuable in terms of comparison of systems, not just an optionality for your system.
00:27:04.319 --> 00:27:07.359
So you have optionality, you have comparison, like you have those types of things.
00:27:07.519 --> 00:27:19.839
And then when you think about like how does it actually benefit the LLMs themselves, this is where you get into like a very interesting discussion because there are aspects of it that can be really beneficial.
00:27:20.079 --> 00:27:33.119
Typically, what it means is that you're transitioning a structured metadata into, I would say, like statements for the LLM, such that it is still context, right?
00:27:33.200 --> 00:27:39.279
But like LLMs receive context differently than like humans and traditional machines to some extent.
00:27:39.440 --> 00:27:42.960
But if you have everything tagged for, so I'll give you an example.
00:27:43.119 --> 00:27:45.200
Let's say you have two products.
00:27:45.359 --> 00:27:48.000
You have a we'll use washing machines, right?
00:27:48.079 --> 00:27:55.440
Let's say you have a 1000 version and a 1001 version, and those are very, very similar systems, right?
00:27:55.519 --> 00:27:57.359
They're the same line of washing machine.
00:27:57.519 --> 00:27:58.880
There's a few differences.
00:27:59.440 --> 00:28:07.359
Well, you've got a hundred topics for each of those systems, and let's say 95 of them are the same.
00:28:07.519 --> 00:28:12.799
So they all carry the 1000 and the 1001 tag, and then you have five that diverge.
00:28:12.880 --> 00:28:19.279
So you have 105 topics in total, and you know, five with a thousand, five with a thousand and one.
00:28:19.599 --> 00:28:26.880
If you're just stuffing all of that into an LLM, and remember, like this is only 105 topics, so you can probably get away with that.
00:28:27.119 --> 00:28:33.599
The way you would use metadata there, and notice I'm specifically avoiding rag and search in this particular example.
00:28:33.680 --> 00:28:57.200
Like I'm just talking about how you put the whole 105 in, is instead of having a tag on that, you would probably be better off actually having a statement at the top of it where before you put it in, you'd use a deterministic system to take the tag from you know product and then it had the tag name to this is the 1001 version of this washing machine.
00:28:57.440 --> 00:29:00.000
This is the 1000 version of this washing machine.
00:29:00.079 --> 00:29:01.839
You just put that at the top of every topic.
00:29:02.079 --> 00:29:08.160
And an LLM is going to interpret that better than having like the actual tag on there.
00:29:08.319 --> 00:29:10.240
And so I've done a bunch of testing around this.
00:29:10.319 --> 00:29:12.640
Like, this also is the same with like glossary terms.
00:29:12.720 --> 00:29:21.599
So if you have like structured glossaries, um, different LLMs will receive the pieces of that glossary, which they're tags, right?
00:29:21.680 --> 00:29:22.480
Like it's metadata.
00:29:22.559 --> 00:29:26.559
Like you're like, you know, this is an abbreviation, this is another way of saying it.
00:29:26.640 --> 00:29:30.160
Like, there's like right, it that's just metadata that describes different pieces of it.
00:29:30.319 --> 00:29:40.559
If you write it out as pros, even if it's the same pros over and over again, the LLMs tend to work with that version of that content better.
00:29:40.799 --> 00:29:57.440
So transitioning the structured metadata into clear and simple statements, even if they're repetitive for the LLM, tends to produce a better result today with the LLMs we have today, because of how they've been trained and you know the biases and like the internal weightings that they have.
00:29:57.519 --> 00:30:00.799
Like you could train one that works better with the actual structure.
00:30:00.880 --> 00:30:02.799
That's just not the ones that we have today.
00:30:03.039 --> 00:30:06.160
So metadata touches a ton of different pieces of it.
00:30:06.240 --> 00:30:14.160
And I haven't even gone into like how metadata can then be used to build graphs and to create like more sophisticated like selection mechanisms.
00:30:14.400 --> 00:30:31.359
But the point is that once you get to any level of complexity, having the metadata on the content gives you so much more optionality in terms of how you build the systems around the LLM, and then also how you actually instruct the LLM when you're putting inputs into it.
00:30:31.680 --> 00:30:37.119
I think this was such a great overview because my goal here was to kind of step back.
00:30:37.440 --> 00:30:48.240
And for those amongst us who have very, I guess the as a technical writer, one would have that structured content, the concept conceptually.
00:30:48.720 --> 00:30:57.839
But if you don't, if you just have a Google Doc and you're starting off, then adding metadata to that, you've given us a why, a solid why.
00:30:58.000 --> 00:30:59.680
Why should you still consider it?
00:30:59.920 --> 00:31:04.400
And how that would kind of work under the hood that I keep going back to.
00:31:04.720 --> 00:31:12.240
And I would love for us to maybe even dive into the knowledge graphs and how they would relate to metadata.
00:31:19.200 --> 00:31:25.440
With more than 70 sessions and workshops featuring speakers from companies like Salesforce, TikTok, T-Mobile, and Amazon.
00:31:25.680 --> 00:31:33.119
LavaCon brings together content strategists, documentation leaders, and content professionals to share practical ideas you can apply right away.
00:31:33.279 --> 00:31:40.480
And yes, it's known as the Fun Conference 2 with networking events, therapy docs, comfort llamas, storytelling, and karaoke.
00:31:40.720 --> 00:31:44.799
Use discount code IDC26 to save$200 on registration.
00:31:45.440 --> 00:31:48.160
You did mention graphs here.
00:31:48.240 --> 00:31:52.640
So I think it's worth going into that a little bit, if not more.
00:31:53.119 --> 00:31:53.680
Yeah, sure.
00:31:53.920 --> 00:31:55.119
So I'm happy to go.
00:31:55.440 --> 00:31:58.799
Well, there are people who are are experts in knowledge graph.
00:31:58.960 --> 00:32:01.759
I wouldn't consider myself an expert in knowledge graphs.
00:32:01.920 --> 00:32:05.119
I do know I I would say a pretty good amount about them.
00:32:05.279 --> 00:32:11.599
So there's probably a limit to how far I can go, but I'm very happy to talk about them in relation to structured content metadata.
00:32:11.920 --> 00:32:15.759
So maybe uh start with just the definition of what a knowledge graph is.
00:32:16.079 --> 00:32:24.799
So so a knowledge graph is if you want to think about it in technical terms, it's a database that has typed nodes and typed connections between those nodes.
00:32:24.960 --> 00:32:33.759
So you could have people and then you could have like a nose or like this person knows this person or is married to.
00:32:33.920 --> 00:32:36.799
Obviously, that would be a one-to-one connection in most cases.
00:32:37.039 --> 00:32:39.359
But like, you know, so this is like the friend of a friend stuff.
00:32:39.440 --> 00:32:42.240
Like, so like ontologies, knowledge graphs, those kinds of things.
00:32:42.400 --> 00:32:52.880
When you put this information into graph databases, what it means is you can start asking multi-level questions of the database.
00:32:53.119 --> 00:33:04.799
So the thing that people really like about them is if you go back to like the people example, you could ask a question of a knowledge knowledge graph of like, give me all the companies for people that Patrick has been on podcasts with.
00:33:05.279 --> 00:33:34.400
And a good knowledge graph, like a good knowledge graph database would be able to go and say, Okay, well, Patrick is a podcaster, and then there would be a field that's like has been on, or something like that, and that would link out to a bunch of different podcasts, and then each of those podcasts would have guests and hosts, and each of those guests and hosts would be people, and all the people would have as employment or you know, works for something along those lines, right?
00:33:34.559 --> 00:33:41.759
And so you could have a query that would say like doop, doop, doop, doop, doop, and hop across all those things and select all of those companies.
00:33:42.000 --> 00:33:54.559
That's why people like knowledge graphs, is because it allows you to ask those kinds of questions of an information set, which are traditionally very difficult to ask in the context of like relational databases.
00:33:54.799 --> 00:34:00.160
So you can do some of that stuff in relational databases, but it's much less flexible, I guess is the best way to put it.
00:34:00.400 --> 00:34:09.440
So you're dealing with very large and you know well tagged or very semantic sets of content, like you're just gonna do better in a knowledge graph.
00:34:09.679 --> 00:34:20.320
So, how this collides with structured content and metadata is that it's not always that you're typically you're not actually keep maintaining your content in the knowledge graph, right?
00:34:20.400 --> 00:34:22.960
So you're maintaining your content in another location.
00:34:23.199 --> 00:34:30.159
So the way that you you correlate, you tie the content to the knowledge graph is through the metadata in many cases.
00:34:30.480 --> 00:34:37.440
So the metadata that becomes the selection mechanism that the the knowledge graph can use to pull that out.
00:34:37.679 --> 00:34:52.480
You can also use direct linking, but you oftentimes like the metadata is going to become like a major piece of like how those two things come together, or how the knowledge graph forms the nodes in the graph out of the content, like it contributes to those things.
00:34:52.960 --> 00:35:01.199
I'm thinking in a typical environment, you have documentation coming from product, from engineering, from support.
00:35:01.840 --> 00:35:12.239
How should the infrastructure establish authority and provenance and a source of truth within the context of that uh of structured content?
00:35:12.559 --> 00:35:15.440
Because that content may be structured differently.
00:35:16.239 --> 00:35:17.599
What's your take on that?
00:35:17.920 --> 00:35:20.320
Yeah, so I think this is a really timely question.
00:35:20.639 --> 00:35:36.880
So the the environment you've described, you know, very briefly, where like technical content comes from support, product engineering, um, the technical writers, like that is one version of companies where the the technical content is kind of like the result of all these contributors.
00:35:37.119 --> 00:35:39.440
That's how you get to those corpuses of content.
00:35:39.760 --> 00:35:50.639
The other structure of companies is where you have the technical writers is more of like a clearinghouse, like they're the ones that manage most of the content, and they probably write a large portion of it.
00:35:50.719 --> 00:36:11.280
And in those environments, you do tend to have knowledge bases that grow up around that, but there's a clear, very clear delineation between like the docs are gold plated, the knowledge bases can be a source, it can be leveraged, but it's not like the level of governance and quality that's expected is is very, very different.
00:36:12.079 --> 00:36:18.320
And I think it's very situational as to which which is going to work for any one individual company.
00:36:18.559 --> 00:36:28.320
You know, some companies can work really well in the more almost crowdsourced version of documentation, and then other companies really can't go that route.
00:36:28.400 --> 00:36:36.320
Like they have to have technical writers as the authorities on truth, more or less, like is the way to think about it.
00:36:36.480 --> 00:36:39.840
Um, and that's like a that's like um a center of excellence approach.
00:36:40.079 --> 00:36:46.880
I have a lot more experience in that approach because of our technology lends itself to that approach.
00:36:47.039 --> 00:36:53.599
Like if you have the highly like crowdsourced collaborative version of it, like that's just not where we fit.
00:36:53.760 --> 00:36:55.840
So I don't work in that world as much.
00:36:56.079 --> 00:37:06.480
When you're trying to establish authority and governance across multiple groups like that, so again, I guess there's kind of two situations there.
00:37:06.639 --> 00:37:13.039
One is they're all using the same thing, which in my experience there is almost always a wiki or Google Docs.
00:37:13.199 --> 00:37:15.679
Like I haven't seen very many variations on that.
00:37:15.760 --> 00:37:24.880
Um, and that's just because when you have a large group of people who are not writers writing the content, you have to give them a tool that's not a writer's tool.
00:37:25.119 --> 00:37:35.679
It's like it's super uncommon that you'll see a circumstance where you have engineering product support and technical, and like maybe you know, a couple of technical writers working in technical writing tools.
00:37:35.760 --> 00:37:42.400
Like I think I've maybe seen that once in my entire career doing this for like I'm not even gonna say how many years, but more than a decade.
00:37:42.639 --> 00:37:46.559
It's much more common that you'll see you hit you go to the lowest common denominator.
00:37:46.719 --> 00:37:48.559
Maybe that's that's not exactly what I mean.
00:37:48.639 --> 00:37:52.960
You go to the most available system, which is almost always a wiki.
00:37:53.199 --> 00:37:57.199
Wikis are not built for governance, like there's just no way around that.
00:37:57.280 --> 00:38:12.320
Like wikis are built for availability, and governance is an activity that imposes structure, it needfully imposes roadblocks and workflow and things like that, because that's where quality comes from.
00:38:12.480 --> 00:38:19.840
It comes from people doing the same things, knowing the patterns, checking themselves in predictable ways, that kind of stuff.
00:38:19.920 --> 00:38:21.280
Like that's governance, you know.
00:38:21.360 --> 00:38:24.320
Like you look at any like Six Sigma or Six Sigma, right?
00:38:24.400 --> 00:38:26.880
Like it's it's a set it's a set of patterns and practices.
00:38:27.119 --> 00:38:34.880
You can't apply those to a group of people that are not professionals in patterns and practices around that particular environment.
00:38:35.039 --> 00:38:37.440
So it becomes very challenging, is what it comes down to.
00:38:37.599 --> 00:38:42.000
You can't really have like broad inputs and high governance.
00:38:42.320 --> 00:38:52.800
And if you don't have high governance in your content, the work of like reverse engineering authority and like source of truth becomes much more complicated.
00:38:53.039 --> 00:38:55.440
It's not impossible in all cases.
00:38:55.679 --> 00:38:57.360
So I can give you an example of this.
00:38:57.519 --> 00:39:01.760
So like we have technical writers here, obviously, like that would be crazy if we didn't.
00:39:02.000 --> 00:39:10.639
But like our product team doesn't do the product specifications in Heretto, which is you know, Heretto is a documentation system.
00:39:10.719 --> 00:39:23.599
Like we do our product specifications in Confluence, and we do that because like we like product specifications are the sorry, these are the the design specs, not the actual, like these are these are not the final specs.
00:39:23.760 --> 00:39:25.760
We don't do the design specs in Heretto.
00:39:25.840 --> 00:39:28.800
So the design specs, because like they are working spaces, right?
00:39:28.880 --> 00:39:39.280
Like you've got a bunch of people in there and like they're throwing stuff at the wall, they're pulling things around, like they're just they're kind of screwing around in the content to figure out what's going to work while they're going through the process, the design process.
00:39:39.440 --> 00:39:48.960
It's a very creative, free-flowing process of like getting things together, and that's just not the right place to apply like high structure and high governance.
00:39:49.039 --> 00:39:59.199
Like, you need to give people the minimum amount of those things to give them the most creative freedom to get to the end design results they're looking for.
00:39:59.440 --> 00:40:07.519
Now, at the same time, the outcome of these initial design documents that they do, they tend to be very high quality.
00:40:07.760 --> 00:40:09.519
So they're totally inconsistent.
00:40:09.599 --> 00:40:21.840
Like there's no like like they're not, you know, they're not in the same format of the same structure, but like they have really, really good information in them because they are for the initial design of something, like they kind of are the single source of truth.
00:40:22.079 --> 00:40:30.079
So in our world, what we do there, this is before the thing is built, before the documentation is written, all those kinds of things, is you just lock it down.
00:40:30.159 --> 00:40:36.079
Like once it's done, you just reduce the number of people that can change it, and the people that can change it are much more mindful.
00:40:36.239 --> 00:40:44.960
Now, we also have the benefit of the people who are working on this, are people who are who have all of the benefits of knowing structured content and documentation because that's a tool that we work on.
00:40:45.119 --> 00:41:00.000
So, like inherently, they're gonna be better about some of these practices, and you do see that bleed into you know building just frankly, probably more information architecture, you know, kind of biosmosis in some of these documents.
00:41:00.079 --> 00:41:03.199
So, like we see a lot of content come from Confluence.
00:41:03.280 --> 00:41:15.039
It's very common that people move from Confluence to Heretto, and it tends it comes in a wide variety of of qualities and like like positions from the structure and like a governance perspective.
00:41:15.280 --> 00:41:22.079
So, but we use those as inputs to to different LLM systems that we have, those those Confluence pages.
00:41:22.320 --> 00:41:38.800
So before a feature is built and before the can the content is migrated into Heretto and it's put on our internal like our internal point of reference is a thing we have like an internal name for it, that is all designed and built in Heretto, and that is considered the highest order of truth in the organization.
00:41:38.960 --> 00:41:40.480
Like if it's there, it's true.
00:41:40.719 --> 00:41:49.920
But before, you know, in the design process, if people have questions about things, like they can ask an AI agent and it will source from those confluence pages.
00:41:50.079 --> 00:41:59.760
And it works quite well because it's a very small set of content, it's maintained by a very small team, it has a very known position in the cycle of information.
00:41:59.920 --> 00:42:04.159
Like it was designed at this point in time by these people for this purpose.
00:42:04.320 --> 00:42:11.519
So like there is a set of metadata that's assumed about that content simply by the definition of what it is.
00:42:11.599 --> 00:42:16.400
Like there's all these attributes that come with it that you can build into a system to get good results from.
00:42:16.960 --> 00:42:22.159
I was also thinking about loan writers and systems where they've not evolved.
00:42:22.400 --> 00:42:26.079
You gave them great examples of how to think about it.
00:42:26.159 --> 00:42:29.920
So if you don't have those systems in place, you can start small.
00:42:30.719 --> 00:42:32.639
Where can I start if I don't have it?
00:42:32.800 --> 00:42:39.199
How do I think about this where you don't have established processes or uh governance frameworks?
00:42:39.440 --> 00:42:49.519
How do you establish that so that you can get to the source of truth and locking it down and having the right people, get the right gatekeepers, so that eventually your content is gold cleader?
00:42:49.760 --> 00:42:53.360
Um Yeah, I can actually give one other piece of AI-related advice to that.
00:42:53.519 --> 00:43:03.280
So if you're a loan writer, like so if I was gonna go be a loan writer in an organization today, almost no chance that you have access to like true structure content.
00:43:03.360 --> 00:43:06.000
Like that's just not common for loan writers.
00:43:06.159 --> 00:43:11.840
So I think the most common circumstance for those individuals today is going to be that they're writing in Markdown.
00:43:12.079 --> 00:43:15.039
There's a bunch of other situations, but like that's probably the most common.
00:43:15.119 --> 00:43:16.400
So I'll focus on that one.
00:43:16.639 --> 00:43:39.039
One of the things that I would do after I wrote a very basic style guide, I'm not even gonna call it an information architecture at that point in time, but a set of rules and patterns around how it is that we created content, is I would feed that into an agent and I would say, you're now going to be my editor, and you're gonna hold me to these rules.
00:43:39.280 --> 00:43:50.639
And so then every time, so again, we're in Markdown, we're probably checking this content into GitHub, which at a you know, at if I'm maintaining a thousand topics or less, that's probably gonna work pretty well.
00:43:50.880 --> 00:43:56.719
Every time uh it was checked in, it just had the agent go and look look it over and have it give me feedback.
00:43:56.880 --> 00:43:59.519
Oh, you didn't put enough metadata on the top of this topic.
00:44:00.079 --> 00:44:01.199
Oh, this topic is too long.
00:44:01.360 --> 00:44:04.079
Oh, it it this topic contains too many claims.
00:44:04.320 --> 00:44:11.440
Like you should break this up into more more markdown files because you know you're mixing concepts and and tasks here.
00:44:11.599 --> 00:44:17.280
Like this is clearly, you know, this section is clearly a task, and this section can contains a bunch of claims.
00:44:17.519 --> 00:44:20.000
Like there's uh claims facts, sorry.
00:44:20.159 --> 00:44:31.679
So there's a bunch of things that you could do with a very simple configuration of Claude, or I've actually not done this in uh in Gemini or OpenAI, so like I'm just gonna say Claude, but I'm assuming you can do this in other LLMs.
00:44:31.840 --> 00:44:32.320
Yeah, yeah.
00:44:32.400 --> 00:44:48.719
So you could set all those things up and it could basically just help you enforce a number of rules that are going to get you a lot further down the road than you would be if you were you know just writing content as text and markdown or in YAML files or things like that.
00:44:48.880 --> 00:45:02.159
And I think you would find that if it did scale up and you hired more people and you did need to transition to a true structured format, you know, for personalization or reuse, those types of things, that you'd still have a lot of work to do, but you'd be in a better place.
00:45:02.559 --> 00:45:07.679
Yeah, I think that's a great suggestion to kind of lean into those systems available to you.
00:45:08.000 --> 00:45:11.039
You were designing a content architecture from scratch today.
00:45:11.199 --> 00:45:18.159
Let's say you've been hired, there are no systems, and you are going to be designing content architecture from scratch.
00:45:18.320 --> 00:45:27.599
Uh specifically for AI and agentic systems, what would you build from the source content all the way through retrieval context, uh governance, and execution?
00:45:27.920 --> 00:45:28.880
Oh my god.
00:45:30.800 --> 00:45:37.360
But you you do you you've been you've done great so far with all the I mean geez.
00:45:38.000 --> 00:45:43.280
Well, so it would depend a ton on there's so many variables that go into that.
00:45:43.440 --> 00:45:45.760
So I'll give you one version of it.
00:45:46.000 --> 00:46:03.599
Okay, so if I'm coming into an organization that has, let's say 10 proper technical writers maintaining a hundred thousand pages of content, or well, let's just say fifty thousand pages of content.
00:46:03.760 --> 00:46:06.880
It's that's a little more on the on the medians there.
00:46:07.280 --> 00:46:16.239
And that content is you know, has any level of complexity, which it almost always does at that size, like you can almost never get around that.
00:46:16.480 --> 00:46:21.360
The first thing I'm going to do, if they're not already, if so, like you said, the nothing exists.
00:46:21.440 --> 00:46:27.039
The first thing I'm gonna do is I'm gonna introduce structured content and a proper CCMS.
00:46:27.119 --> 00:46:28.559
Like that would be step one.
00:46:28.719 --> 00:46:32.400
And then from there, I'm going to look at the delivery requirements.
00:46:32.719 --> 00:46:40.159
So one of the things that I think people undervalue is the importance of having a really high-quality website today.
00:46:40.400 --> 00:46:43.679
Like that is where most of your AI answers are coming from.
00:46:43.840 --> 00:46:47.360
Like before you implement any AI, you need to feed the public AI.
00:46:47.519 --> 00:46:51.039
And to do that, you need a website that has all the different aspects.
00:46:51.199 --> 00:47:12.880
Many of them are just SEO things, but all the different aspects that are going to give you the proper authority with Google, OpenAI, Anthropic, and the other major models, so that when people are asking questions of those systems, they're properly retrieving information from your set of information for your customers.
00:47:13.119 --> 00:47:17.679
Like that's gonna be 90% of your customers' AI interaction.
00:47:17.920 --> 00:47:20.880
From there, it's a very it's gonna get very situational.
00:47:21.039 --> 00:47:36.000
If I've got a field service team, I'm gonna look at how I'm gonna deploy an agent to that field service team that covers the particular issues that they're having a hard time solving that are also their information issues.
00:47:36.239 --> 00:47:41.360
I would start deploying agents and connections to our support channel.
00:47:41.440 --> 00:47:43.360
So like there was a feedback loop there.
00:47:43.519 --> 00:47:47.199
You know, those things are maybe not simple, but fairly straightforward, right?
00:47:47.280 --> 00:47:49.840
Like that's kind of like, you know, that's known work at this point in time.
00:47:49.920 --> 00:47:53.280
I'm not gonna call it a solved problem, but it's known work at this point in time.
00:47:53.519 --> 00:48:04.000
I think if you you kind of started with that path in that environment, you're going to get yourself so far down the road that you're probably gonna be ahead of most people already.
00:48:04.159 --> 00:48:06.000
That I think that that is a blueprint.
00:48:06.239 --> 00:48:20.000
I guess the other thing I should say there too is when I said implement a CCMS, I meant do the information architecture work on the content, come up with an information model, get the content into really good shape, and then put it into a CCMS.
00:48:20.079 --> 00:48:22.079
So that's like the process of implementing structure.
00:48:22.239 --> 00:48:28.639
One of the biggest mistakes that people make is that they think they're gonna buy a tool and that's gonna solve their problems, and that's not the kit, not the case.
00:48:28.800 --> 00:48:37.840
Like the work to take to go from we have a ton of content that's unstructured to good structured content that's actually going to solve problems.
00:48:38.000 --> 00:48:43.760
That's an information modeling and information architecture and conversion process, and that is work.
00:48:44.000 --> 00:48:48.079
It has a huge payoff, it's the foundation for everything else.
00:48:48.320 --> 00:48:52.480
But if you don't do that work, you don't get the benefit out of structured content.
00:48:52.719 --> 00:48:54.320
So, like that would be the first step.
00:48:54.719 --> 00:48:57.119
First of all, my apologies, Patrick.
00:48:57.199 --> 00:49:00.800
I did not mean to quote unquote assault you with that question.
00:49:01.199 --> 00:49:02.159
No, no, no, no.
00:49:02.320 --> 00:49:03.280
It's a great question.
00:49:03.360 --> 00:49:05.280
It's just it's such a big question.
00:49:05.599 --> 00:49:12.559
So I was in such an on a team where the mindset was that the tool would solve all the problems.
00:49:12.800 --> 00:49:21.679
And thinking about information architecture and mod well, let's say that we had to revisit how car content was modeled and architected.
00:49:22.159 --> 00:49:23.840
The tool did not solve the problem.
00:49:24.239 --> 00:49:24.960
You think about it.
00:49:25.039 --> 00:49:32.719
If you were gonna go build a house and you showed up at the build site with a bunch of tools but no blueprints, yeah, what would you get?
00:49:32.880 --> 00:49:33.840
What would you end up with?
00:49:34.320 --> 00:49:35.280
Thank you so much for watching.
00:49:35.440 --> 00:49:38.480
This was a great conversation, and I hope you had fun as well.
00:49:38.639 --> 00:49:39.199
I did.
00:49:39.440 --> 00:49:40.320
Thanks again.
00:49:40.559 --> 00:49:41.440
Thank you.
00:49:42.320 --> 00:49:45.039
Thank you for listening to Insight Techcom.
00:49:45.280 --> 00:49:49.280
If there's a guest you would love to hear on the show, please let me know.
00:49:49.440 --> 00:49:54.559
And don't forget to follow and catch every episode on your favorite podcast app.
00:49:54.719 --> 00:49:55.840
See you next time.
00:00:02.240 --> 00:00:07.599
Hello friends and welcome to season eight of Inside Techcom with Zahra Mutabana.
00:00:07.919 --> 00:00:16.559
This season features a collection of conversations that explore the ideas, challenges, and opportunities shaping technical communication today.
00:00:16.719 --> 00:00:22.320
From AI and content strategy to leadership, product thinking, and the future of our profession.
00:00:22.559 --> 00:00:26.320
Each episode offers a fresh perspective from people doing the work.
00:00:26.480 --> 00:00:27.760
Let's get started.
00:00:31.839 --> 00:00:36.479
Season eight is proudly sponsored by LavaCon.
00:00:36.560 --> 00:00:43.200
Use discount code ITC26 to save$200 on registration for this year's conference.
00:00:43.359 --> 00:00:46.320
I'll share more about LavaCon later in this episode.
00:00:46.799 --> 00:00:47.840
Hello listeners.
00:00:47.920 --> 00:00:50.479
Welcome to another episode of Inside TechCon.
00:00:50.960 --> 00:00:54.240
Today we'll be talking with Patrick Gozek from Herero.
00:00:54.960 --> 00:01:03.600
And in the past, we've talked about structured content and its impact on how AI ingests that framework.
00:01:04.000 --> 00:01:05.680
Today we're going to do a deep dive.
00:01:06.159 --> 00:01:14.799
Patrick is going to be talking to us and educating us on how structured content contributes useful to the context under the hood and how it works.
00:01:15.120 --> 00:01:16.879
With that, Patrick, welcome to my show.
00:01:17.280 --> 00:01:18.480
Great to be here, Sarah.
00:01:18.560 --> 00:01:19.200
It's very nice.
00:01:19.599 --> 00:01:23.599
Patrick, first tell us a little about yourself, what you do, and yourself.
00:01:24.000 --> 00:01:24.799
Sure, happy to.
00:01:24.959 --> 00:01:26.480
So I'm Patrick Bozick.
00:01:26.799 --> 00:01:29.599
I am one of the founders of Heretto.
00:01:29.680 --> 00:01:30.799
I'm also the CEO.
00:01:31.040 --> 00:01:34.319
So I've been in the structured content industry for quite a while.
00:01:34.560 --> 00:01:39.760
As the uh some I guess I could be considered the face of the company sometimes.
00:01:40.000 --> 00:01:43.920
So in that role, I also do quite a bit of media things.
00:01:44.000 --> 00:01:46.319
So I have a talk show I do with Scott Abel.
00:01:46.640 --> 00:01:48.400
So that's called Coffee and Content.
00:01:48.480 --> 00:01:54.640
We've been on a little bit of a hiatus due to a technical issue, but we typically do coffee and contents every other week.
00:01:54.799 --> 00:01:58.640
So those are a lot of fun, cover a lot of different topics from the content world.
00:01:58.879 --> 00:02:02.480
We also have a podcast called Components, a content components.
00:02:02.560 --> 00:02:06.799
If you look it up on Spotify, that's got four or five seasons.
00:02:06.959 --> 00:02:10.479
It's all about documentation, structured content, those kinds of things.
00:02:10.560 --> 00:02:15.280
So if you're wanting nerd out about structured content, that's a great place to go.
00:02:15.599 --> 00:02:15.840
Yeah.
00:02:15.919 --> 00:02:18.080
And then I'm a developer by background.
00:02:18.159 --> 00:02:22.159
So I was one of the developers and architects of the first version of our software.
00:02:22.240 --> 00:02:26.639
So I'm I'm fairly technical, but I don't cause problems for our development team anymore.
00:02:26.719 --> 00:02:31.039
Nowadays, I focus on what's going to make customers happy and those kinds of things.
00:02:31.360 --> 00:02:36.000
What spawned this idea to look into structured content as a developer?
00:02:36.159 --> 00:02:37.039
I'm curious.
00:02:37.360 --> 00:02:37.919
Oh boy.
00:02:38.080 --> 00:02:41.759
So that's actually going back a lot further than I'd like to admit.
00:02:42.000 --> 00:03:04.800
But my co-founder and I, we got into structured content, not DITA, which is where we are today, but XML content with an actually with a custom schema when we were in college, actually, and we just saw that it was a superior way to store and manage content and that it was going to make some of the publishing activities we were working on easier and more capable.
00:03:05.039 --> 00:03:11.280
At that point in time, the technologies surrounding digital publishing were just far, far less well developed than they are today.
00:03:11.680 --> 00:03:19.280
So structured content provided it actually provided something quite different at that moment in time, now that I'm reflecting on it, than it does today.
00:03:19.520 --> 00:03:25.919
At that moment in time, it it basically gave you the ability to put a like a little bit of a database and a file.
00:03:26.159 --> 00:03:40.479
The amount of things you had to manage and the flexibility you got when you were using structured content at that point in time, if you're looking to apply that to like a very structured digital publishing environment, there was a lot of advantages there.
00:03:40.560 --> 00:03:42.719
Um so that was why we picked it up originally.
00:03:42.879 --> 00:04:02.400
And then over the years, we eventually drifted into, or I guess found our way, is maybe a better way of putting it, into technical content where you still get a lot of those benefits that we we were originally looking at with DIDA and in the digital publishing around uh technical content and documentation, but there's a ton more today.
00:04:02.560 --> 00:04:04.800
So there's there's so much more that you get from it.
00:04:05.199 --> 00:04:08.560
Yeah, we could probably have a whole episode on structured content.
00:04:08.639 --> 00:04:13.199
And I have, as I mentioned, had some episodes covering what that is.
00:04:13.360 --> 00:04:16.560
And it seems to me like the context can be different.
00:04:16.959 --> 00:04:28.560
Of course, data comes close to mind, but for those of us that don't use data, structured content still follows a certain framework, a certain guidelines, if you will.
00:04:28.800 --> 00:04:41.839
But for this conversation, I want to ground what that definition means to you, and then we can, I guess, start talking about how we can deep dive into how that lends to AI.
00:04:42.160 --> 00:04:49.199
Yeah, I should think that is really important to start with the definition of structured content because everybody does define it a little bit differently.
00:04:49.439 --> 00:04:53.360
And I think the individual definition matters.
00:04:53.759 --> 00:04:55.439
So I'll give you my definition.
00:04:55.519 --> 00:04:57.600
I'll also tell you why I think mine is right.
00:04:58.480 --> 00:05:01.519
But that doesn't mean that other people's don't uh can't be right too.
00:05:01.839 --> 00:05:22.639
So technically speaking, if you put all of your content into Excel or Google Sheets and you put it into, you know, let's say every topic was a column, or maybe better a row, and every portion of that topic or that piece of information was divided up into different columns.
00:05:23.040 --> 00:05:25.839
So you have the pieces broken up into the different cells.
00:05:26.079 --> 00:05:31.680
That would be structured content because you would have the content broken up into a structure.
00:05:31.839 --> 00:05:35.920
Like in the most basic definition, you have a structure for your content.
00:05:36.079 --> 00:05:41.920
So you're you're labeling the different pieces of it in a methodology, and that's structured content.
00:05:42.079 --> 00:05:43.839
Now, obviously, that's a terrible choice.
00:05:44.000 --> 00:05:48.399
Like that wouldn't be a thing that you'd want to do in almost any production environment.
00:05:48.480 --> 00:05:56.879
Although I will say that we have had customers migrate from that exact environment just like that to our tool, which is DIDA.
00:05:57.040 --> 00:06:10.319
But when you think about like how this comes together, the reason I start there is because there are a lot of systems that are they're effectively that, because if you think of what a form is, so like a digital form, it is just actually a table.
00:06:10.560 --> 00:06:12.959
You have your question one, question two.
00:06:13.199 --> 00:06:15.600
So your question one might be what is the heading for this file?
00:06:15.839 --> 00:06:19.920
Your question two might be who authored this file, question three might be the body.
00:06:20.160 --> 00:06:24.879
And people will call that structure content because you know you can put metadata in one place.
00:06:24.959 --> 00:06:29.680
And and in certain circumstances, that's what headless CMSs are, for instance.
00:06:29.759 --> 00:06:32.720
So, like things like contemptful, those kinds, that's that's what they are.
00:06:32.879 --> 00:06:35.519
In certain circumstances, that actually works quite well.
00:06:35.680 --> 00:06:38.319
Um, that's a really effective form of structured content.
00:06:38.480 --> 00:06:47.199
It's backed by a database, uh, it has the ability to create links between those pieces of information, uh, it has the metadata properly separated out.
00:06:47.279 --> 00:07:00.720
Like you have like there are certain things, but it in the background, it is functionally a table that is fundamentally different than what we work with, which is XML, because XML is a tree, which means that you can have structure below those big boxes.
00:07:00.959 --> 00:07:05.120
So when you go into a topic, there's going to be structure at multiple levels.
00:07:05.360 --> 00:07:10.639
So you'll have a piece, like let's say we'll use a procedure or a task topic, for instance.
00:07:10.879 --> 00:07:19.279
When you start creating that, you'll have a title, and then you'll have metadata around it, and then you'll have the prerequisites for running this procedure.
00:07:19.439 --> 00:07:24.319
And then the individual procedure, instead of being an ordered list, it would be steps, right?
00:07:24.879 --> 00:07:28.560
And then there's a step inside of it, there's structure inside of the steps.
00:07:28.800 --> 00:07:39.040
And so by having this multiple layers of structure and semantics, it means that you're able to manipulate that in a more granular way beyond just formatting.
00:07:39.199 --> 00:07:43.439
It's not just HTML and boxes, effectively, but they're both structure.
00:07:43.519 --> 00:07:48.160
Like there is a like both things are valid in terms of the structure content label.
00:07:48.399 --> 00:07:54.959
So, what you really need to ask yourself when you're thinking about, okay, well, we're moving to structure, it's beyond just the term structure.
00:07:55.040 --> 00:08:00.399
Like you should really be thinking about what is the form of structure that's going to serve our mission?
00:08:00.560 --> 00:08:04.079
What do we care about in terms of interoperability?
00:08:04.399 --> 00:08:07.439
What forms of linking are really important to us?
00:08:07.680 --> 00:08:10.480
What forms of output are going to be really important to us?
00:08:10.639 --> 00:08:18.959
Because those types of questions will guide you towards the type of structure that's going to be best for your particular implementation and your particular needs.
00:08:19.360 --> 00:08:24.800
I have to say that I thought my definition was going to be very different from yours or my understanding.
00:08:25.120 --> 00:08:28.480
But at some level, I think we are on the same page.
00:08:28.800 --> 00:08:29.040
Yeah.
00:08:29.279 --> 00:08:30.879
I mean, I think my definition is pretty broad.
00:08:30.959 --> 00:08:32.080
Like I said, I think it's right too.
00:08:32.399 --> 00:08:34.000
Yes, it's very inclusive.
00:08:34.320 --> 00:08:39.200
So that allows me to be kind of okay, that definition sounds close to what I have in my mind.
00:08:40.000 --> 00:08:40.639
Fantastic.
00:08:41.120 --> 00:08:41.440
Yes.
00:08:41.759 --> 00:08:49.519
I have not worked with Ditta, and you mentioned Ditta, and Dita comes up a lot when we talk about structured content.
00:08:49.840 --> 00:08:58.080
We've touched upon this before, but I think for the benefit of the audience, it would be nice to quickly do a shout out to what Dita is.
00:08:58.399 --> 00:08:58.960
Yeah, for sure.
00:08:59.120 --> 00:09:03.200
So Dita, in the most basic definition, it's it's a form of XML.
00:09:03.360 --> 00:09:08.000
So in that way, you can actually think of it as like a cousin to HTML in some ways.
00:09:08.080 --> 00:09:11.279
So it actually shares a lot of the same elements that HTML has.
00:09:11.519 --> 00:09:20.960
The thing to know about DITA is that it has built in many of the patterns that you need to scale up technical content.
00:09:21.039 --> 00:09:32.240
So there's a ton of stuff that we've pretty much figured out over the last call it 15 or 20 years, are really critical to larger or more complex sets of technical content.
00:09:32.559 --> 00:09:39.840
And the DITA committee, so the people who work on the open source standard, they put all of those things into the standard.
00:09:39.919 --> 00:09:41.600
So you don't have to recreate them.
00:09:41.759 --> 00:09:48.879
And then perhaps just as importantly, beyond not having to recreate them, you don't have any vendor opinion or lock-in.
00:09:49.279 --> 00:10:05.120
So one of the challenges that the that DITA was trying to solve, and I think has solved pretty effectively, is that when you have vendor-specific formats, it means that okay, well, we've reused content, but we've reused content in this particular system.
00:10:05.360 --> 00:10:14.639
And once you get to a certain level of reuse, if you didn't have a portable technology underneath it, your ability to migrate someplace else is I mean, it's it's almost impossible.
00:10:14.960 --> 00:10:18.320
It just becomes the lock in there, it's just so high.
00:10:18.639 --> 00:10:29.440
So what Ditter really solves is it solves having good patterns for the core things that you really need to do with technical content, and then it also solves an interoperability problem.
00:10:29.600 --> 00:10:34.480
And you know, there's some of the criticisms of Ditto are that it comes with you know like a high learning curve.
00:10:34.559 --> 00:10:36.480
Yeah, I don't know if I actually agree with that.
00:10:36.559 --> 00:10:38.879
Like I've seen a lot of people learn it pretty quickly.
00:10:39.120 --> 00:10:42.080
There's people feel that it's too heavy for today's uses.
00:10:42.159 --> 00:10:43.360
That's another criticism.
00:10:43.600 --> 00:10:45.679
Those people tend to come from like the markdown world.
00:10:45.840 --> 00:10:51.039
So, like in comparison to Markdown, you know, markdown is a text format, DITA is an is an XML format.
00:10:51.279 --> 00:10:52.799
I think that's just a use case thing.
00:10:52.879 --> 00:10:59.279
So like DITA is not a great, is not like you never write the README files for your vibe coded project in DITA.
00:10:59.360 --> 00:11:00.159
I don't do that.
00:11:00.320 --> 00:11:05.360
Like when I have small apps and things like that that I build, I don't write any of the documentation in DITA.
00:11:05.440 --> 00:11:06.399
It doesn't make any sense.
00:11:06.639 --> 00:11:13.840
Did it is a format that you use when you arrive at a sufficient level of complexity and a sufficient scale.
00:11:14.080 --> 00:11:24.720
So if you want like really rough and tough numbers, what we've found over the years is that if you're below a thousand pages of content, DITA probably doesn't make sense.
00:11:24.879 --> 00:11:31.919
Um, there are some exceptions, but even at a thousand pages of content, like it can be questionable there depending on your needs.
00:11:32.159 --> 00:11:40.000
And so where it really starts to make sense is when you have multiple variations of products or variations of content, audiences.
00:11:40.320 --> 00:11:49.919
So you have anything that's like personalization, and there's a bunch of different terms that people use for the content changes when somebody looks at it for a different person looks at it for a different reason.
00:11:50.159 --> 00:11:53.519
Localization can be a driver, although I think less so today.
00:11:53.679 --> 00:11:56.159
And then just sheer scale with like reuse.
00:11:56.240 --> 00:12:02.240
So if you have a lot of content that you is used in multiple places, that's when you know data starts to really shine.
00:12:11.279 --> 00:12:12.639
In a simplified way.
00:12:13.039 --> 00:12:31.519
Since we are going to do a deep dive into how structured content lends itself to AI, does data lend itself better, or just this the generic definition of the structured content that we ground it, that we are trying to ground this conversation in also lends itself to better AI interactions?
00:12:31.840 --> 00:12:33.279
That is such a good question.
00:12:33.600 --> 00:12:36.960
So the answer is yes, it depends in here.
00:12:37.120 --> 00:12:42.080
And I feel like I'm among the right people to give like engineering style, it depends answers.
00:12:42.240 --> 00:12:46.720
One of the jokes in our leadership team is that when you ask the business people a question, they give you an answer.
00:12:46.960 --> 00:12:49.759
When you ask the technical people a question, they say it depends.
00:12:50.000 --> 00:12:53.600
And so I think that I'm I'm filling that role in this particular moment.
00:12:53.840 --> 00:12:57.360
So, you know, this is kind of like what we we wanted to cover today.
00:12:57.440 --> 00:13:00.320
So it is a it's a very, very long answer.
00:13:00.720 --> 00:13:10.879
At the very core of it, if like he wanted like the too long didn't listen version of it, structured content is going to give you better structured contents with the broad definition.
00:13:10.960 --> 00:13:20.559
So like anything that is broken up into labeled pieces that are then stored consistently, like that definition of structured content.
00:13:20.639 --> 00:13:29.200
So did inclusive of data, inclusive of uh headless CMSs, inclusive of anything that has a well-defined schema applied to content.
00:13:29.360 --> 00:13:32.240
So, why would that be better for AI?
00:13:32.320 --> 00:13:37.759
And it really just comes down to chunking, like that's the fundamental difference between those things.
00:13:38.080 --> 00:13:47.039
So the problem with that statement is that when you get to certain forms of structured content, and I'll like I'll give you like an example from a headless CMS.
00:13:47.200 --> 00:13:50.799
The headless CMS is every headless CMS is gonna have a different schema.
00:13:50.960 --> 00:13:56.240
Like part of the the appeal of a headless CMS is that you can create your own schema.
00:13:56.399 --> 00:14:06.240
So you go in and you get a blanks blank page, and all you have to have is a title, and then you can add a body and an afterthoughts, and you could add you know links to other things.
00:14:06.399 --> 00:14:13.600
And like the standard thing is like, you know, how would you lay out a one of the standard examples is how would you lay that out for like a music venue?
00:14:13.759 --> 00:14:18.879
And you'd have one thing that was a band, and then you'd have an artist, and then you'd have like those kinds of things, right?
00:14:18.960 --> 00:14:20.879
And you'd you build those schemas.
00:14:21.120 --> 00:14:31.679
Well, in a really well-designed schema, it's gonna function really well as giving you good sized pieces of content for the chunking.
00:14:31.919 --> 00:14:40.879
In a poorly designed schema, what's gonna happen is you're gonna get people basically putting in a title and a body, and like that's really not any different than a Google Doc, right?
00:14:41.039 --> 00:14:46.159
So at that point in time, people are just gonna throw huge quantities of content into that body field.
00:14:46.320 --> 00:14:55.200
So even though technically you have two fields, the structure is two boxes, like it's one box, it's it's two boxes with unstructured content in one of them.
00:14:55.360 --> 00:14:57.840
And so then you lose the benefits of chunking.
00:14:57.919 --> 00:15:19.679
So now there's a bunch of other things, but if you like if you want the biggest bang for your buck, so like the I don't know, the 10% that gives you 60% outcome, it's having really well clean defined borders around your content, which means smaller pieces, more intentional pieces, pieces of content that contain ideally one piece of information.
00:15:19.919 --> 00:15:23.440
So in DITA, like a topic is supposed to be about one thing.
00:15:23.679 --> 00:15:28.639
It like if it's a concept topic, it's supposed to teach you one thing or explain one thing.
00:15:28.799 --> 00:15:33.600
If it's a task, it's supposed to give you a procedure that does one thing.
00:15:33.759 --> 00:15:39.919
Um, so it takes you from from the beginning to end of one thing you have to complete, and there's other versions of that as well.
00:15:40.159 --> 00:16:05.360
So in the AI world, when you go to produce results with an AI system, being able to provide it very clean chunking, so very clean set like pieces of content for the classification on when you're putting it into a rag system, and then also for retrieval, there's a bunch of other things, but that's the thing that gets you the biggest bang for your buck.
00:16:05.759 --> 00:16:12.480
Modern language models are pretty good at extracting meaning from unstructured content.
00:16:13.440 --> 00:16:16.159
Does that reduce the need for structured content?
00:16:16.559 --> 00:16:21.759
Because we're trying to make the case that you need to have chunking with structured content.
00:16:22.320 --> 00:16:33.039
But I'm kind of trying to flip that question and see if the LLMs have gotten so bad so good, or is there still an advantage to use structured content as we have established?
00:16:33.360 --> 00:16:35.279
Yeah, so I think that is also a great question.
00:16:35.440 --> 00:16:41.759
It's one of the places that there's I don't want to say misunderstanding because I think that's the wrong way to think about it.
00:16:42.000 --> 00:16:45.679
But I think that there's so maybe this is a better way to put it.
00:16:46.000 --> 00:16:52.720
You're right that the the modern LLMs have gotten very good at extracting information when the question is basic.
00:16:52.960 --> 00:17:03.759
And the reality is that most of the time when people are interacting with these LLMs, their their modalities of interaction fall into a couple of categories.
00:17:04.000 --> 00:17:06.960
One is coding, and coding is a verifiable tree.
00:17:07.039 --> 00:17:19.440
So like part of the reason coding has gotten so good is not because the LLMs have gotten so good, it's because the harnesses have gotten so good at managing something that is verifiable, like that.
00:17:19.599 --> 00:17:20.960
It's logically verifiable.
00:17:21.200 --> 00:17:26.720
The other modality that we use, we use these in is effectively a replacement for search, right?
00:17:26.799 --> 00:17:29.200
So you go to Google and like you look at the AI result.
00:17:29.759 --> 00:17:35.839
Again, a lot of the advancement there is actually in the deterministic harness under the hood.
00:17:36.000 --> 00:17:52.160
Like the amount of engineering that Google and Anthropic and Chat or and OpenAI have done so that what is being delivered to the AI for consideration is high quality, is probably equivalent to the amount of like engineering they've done in the AI model.
00:17:52.240 --> 00:17:55.200
Well, maybe not, I don't know, but like it's a lot is the point.
00:17:55.440 --> 00:18:05.519
And then when you're but when you're thinking about interacting with these systems in a more general sense, typically people are putting very small quantities of information into them.
00:18:05.759 --> 00:18:15.200
So like if I go to notebook LLM and I throw in a couple of PDFs, it does work quite well because the quantity of information is very small.
00:18:15.359 --> 00:18:21.200
Now, in the background, it is still chopping that PDF up and it is still indexing it.
00:18:21.359 --> 00:18:26.160
Like it is not just feeding that PDF full stock to the to the LLM.
00:18:26.720 --> 00:18:46.960
But what's happening when you ask that the question, or you're working with the system to get information out of that, is that because it's a small number of chunks, and because the system is then reducing down what it's sending to the LLM to a smaller set, it's not that big of a deal.
00:18:47.200 --> 00:18:49.200
The problem is when you scale up.
00:18:49.519 --> 00:18:56.240
So there's a reason you can only put, I don't know what it is, it's not a huge quantity of content into notebook LLM.
00:18:56.480 --> 00:19:10.880
When you start talking about thousands of pages or tens of thousands of pages or more, at that point in time, especially if you're asking more human questions as opposed to like search queries, that's when it absolutely still breaks down.
00:19:11.119 --> 00:19:13.039
And like we have research that shows this.
00:19:13.119 --> 00:19:16.480
So like this isn't this isn't just me hypothesizing these things.
00:19:16.559 --> 00:19:18.799
Like this is this is stuff that we can show.
00:19:19.039 --> 00:19:36.319
And in those worlds where having really strong rag retrieval, which is built on the foundation of having good tokenization of the content, you start to see structured content outperform unstructured content by about 50%.
00:19:36.960 --> 00:19:38.480
Now it's use case dependent.
00:19:38.559 --> 00:19:44.160
Like there are like simpler questions will do better in both cases, more complex questions will do better.
00:19:44.319 --> 00:19:46.559
So there's there's a lot of it depends in there.
00:19:46.720 --> 00:19:56.079
But for things that really matter, especially in like regulated industries, or I mean, I guess everybody should care about their customers getting the right answer, but you know, like some people care more than others, I suppose.
00:19:56.240 --> 00:20:02.640
It still is a substantial difference in overall quality of retrieval and accuracy.
00:20:03.039 --> 00:20:11.519
So I think this is a great, I guess, segue into the RAG, uh, the retrieve and generate model that we start looking into.
00:20:11.920 --> 00:20:20.160
Under the hood, retrieval is evolving from a fairly linear model towards more agents that can decide what information they need to go and find.
00:20:20.319 --> 00:20:25.440
Where does structured content fit into this newer model of agentic retrieval and context engineering?
00:20:25.680 --> 00:20:29.599
Yeah, I so I think that's an interesting area of evolution right now.
00:20:29.759 --> 00:20:35.839
And I guess the first thing I want to call out here is that the models are themselves getting better.
00:20:36.000 --> 00:20:37.440
Like there's no doubt about that.
00:20:37.599 --> 00:20:46.400
But I actually think the advancement in rag systems is more in the harness sophistication than it actually is in the models right now.
00:20:46.640 --> 00:20:56.559
So when you think about like how does an AI work with a a rag like content operations or rag architecture such that it's going to produce a better outcome.
00:20:56.720 --> 00:21:00.480
Well, actually, should we stop and like give a definition of what of like what rag is before?
00:21:00.960 --> 00:21:01.519
Yes, please.
00:21:01.599 --> 00:21:02.079
Absolutely.
00:21:02.400 --> 00:21:02.799
Go for it.
00:21:03.039 --> 00:21:03.519
Yeah, okay.
00:21:03.680 --> 00:21:11.680
So rag is retrieval augmented generation, which is technical people trying to confuse the world with with fancy words.
00:21:11.839 --> 00:21:15.920
All it means is that you're running a deterministic on the well, you're running a search.
00:21:16.079 --> 00:21:18.720
I was about to say deterministic search, but that's not necessarily true.
00:21:18.960 --> 00:21:27.039
You're running a search, the output of that search is then being put into the context for an AI agent to generate a result.
00:21:27.119 --> 00:21:27.839
That's all it means.
00:21:27.920 --> 00:21:31.279
So it's it's just search and then generate, is what it comes down to.
00:21:31.519 --> 00:21:36.160
Now, most rag systems are built on vector databases.
00:21:36.400 --> 00:21:53.359
And why it matters to call that out is because fundamentally, vector databases, what they are is they're just they're calculations around a piece of information as to like kind of where it sits in space, and they what they do is they retrieve things that are close together.
00:21:53.519 --> 00:21:56.720
Now, those calculations are done by LLMs typically.
00:21:57.039 --> 00:22:04.720
When you look at like some of the research that we've done, like we've Tried a bunch of different vector databases and a bunch of different ways of calculating the embeddings.
00:22:04.880 --> 00:22:08.880
So the embeddings are like the position, I guess, if you want to think about it that way.
00:22:09.119 --> 00:22:16.400
And the way that we we do that is that you can calculate the embeddings with a bunch of different with any of the AI models that are out there.
00:22:16.640 --> 00:22:24.799
So what we've found is that in our testing, open AI tends to be the one that's most cost effective and easiest to do.
00:22:24.880 --> 00:22:26.559
And so that's the one that we've used.
00:22:26.720 --> 00:22:32.480
We've tried a bunch, but that's the one we've kind of settled on as like our baseline, which is funny because we use anthropic everywhere else.
00:22:32.640 --> 00:22:34.240
But like that's how that comes together.
00:22:34.400 --> 00:22:46.160
So then you have the content in the vector database, the way you retrieve it is through these vector searches, and then when you retrieve sets of content, that's going to be put into the LLM.
00:22:46.400 --> 00:22:53.920
So the major difference here between like the first versions of these and the versions that we deal with today, the models are better, yes.
00:22:54.079 --> 00:23:10.960
But now you may see being a in a place where you send the question to the LLM, the LLM modifies the question or it breaks it down into some way, or it clarifies the question, changes the question, and then it passes the query to the vector database.
00:23:11.119 --> 00:23:14.640
The vector database returns results, and that loop can continue.
00:23:14.880 --> 00:23:17.119
So reasoning is largely just a loop, right?
00:23:17.200 --> 00:23:19.279
Like they basically feed the results back in.
00:23:19.359 --> 00:23:26.160
There could be some deterministic logic that does some parsing or extraction between the LLM and the database.
00:23:26.319 --> 00:23:32.880
Like there's other ways that they, you know, they can basically upgrade the overall loop, but it really fundamentally is a loop.
00:23:33.039 --> 00:23:39.119
So I think the misconception that a lot of people have is they think all of this is happening inside of the LLM.
00:23:39.279 --> 00:23:45.519
Like the LLM is making a decision about what information it's retrieving, but like the information is in the LLM.
00:23:45.599 --> 00:23:49.359
The information that you're when you're talking about RAG is not in the LLM.
00:23:49.440 --> 00:23:55.440
It's in a separate piece of technology that the LLM is making, is having an interaction with?
00:23:55.920 --> 00:24:04.880
When I started working with structured content, metadata became a centerpiece to how a cornerstone to how search was done.
00:24:04.960 --> 00:24:10.079
So the the richer the metadata, the better the output was.
00:24:10.720 --> 00:24:12.960
So organizing information was super critical.
00:24:13.200 --> 00:24:24.559
What changes when a machine is using that metadata to decide what information belongs in an AI models context with all the technology breakdown that you've given us?
00:24:24.960 --> 00:24:25.279
Yeah.
00:24:26.240 --> 00:24:29.519
So this could be a whole series of podcasts in and of itself.
00:24:30.079 --> 00:24:30.319
Of course.
00:24:30.799 --> 00:24:31.440
What changes?
00:24:31.680 --> 00:24:33.200
Well, oh boy.
00:24:33.519 --> 00:24:41.599
The first thing I think that's important to think about here is that metadata gives you a lot of options in how you process something.
00:24:42.079 --> 00:24:58.640
So one of the key aspects of having really high quality metadata on your content is it's hooks and like levers that you can pull in deterministic processes that decide how content ends up and how it shows up.
00:24:58.960 --> 00:24:59.519
Right.
00:24:59.759 --> 00:25:13.599
So when you're thinking about what you're gonna end up providing to an LLM, you almost always have some type of a pipeline that's going to be coming into whatever your rag system is or whatever, wherever the LLM is actually going to be touching that content.
00:25:13.839 --> 00:25:22.960
And metadata means that you can be more specific about what and how those things go into the uh the LLM or the AI-based system.
00:25:23.119 --> 00:25:27.359
So it gives you a lot more control with traditional code.
00:25:27.599 --> 00:25:32.160
And I'm emphasizing that because it really shouldn't be undervalued.
00:25:32.319 --> 00:25:34.559
That is still enormously powerful.
00:25:34.799 --> 00:25:50.400
The more specific you can be in a deterministic way in the systems that you build, the more confidence you can have and the more sustainability those systems are going to have, because you can write unit tests, right?
00:25:50.480 --> 00:25:53.519
Like things in deterministic systems have deterministic outcomes.
00:25:53.680 --> 00:25:56.480
Like that's fundamentally how they're different than AI systems.
00:25:56.720 --> 00:25:59.839
So metadata still has a ton of value in that way.
00:26:00.079 --> 00:26:03.759
This is kind of a the other side of that coin, but I think also valuable.
00:26:04.000 --> 00:26:15.920
There are circumstances where traditional search, if metadata is effectively applied, will perform equally well to vector search, and it's a lot less expensive.
00:26:16.160 --> 00:26:20.240
So there's certain applications where the old stuff works just as well.
00:26:20.400 --> 00:26:26.400
And there's actually cases where it works better, but then you know, we don't have to go, we don't have to dig too deeply into those things.
00:26:26.559 --> 00:26:30.079
But the point is that metadata also gives you options.
00:26:30.240 --> 00:26:46.720
So if you're in a place where you're completely reliant on AI systems and you don't have really strong metadata around how you're trying to build a retrieval pipeline, um, then you're totally reliant on the inference of context from the AI into a vector database.
00:26:46.960 --> 00:26:58.000
Whereas if you had that metadata, you might be in a place where you could have a deterministic system or a traditional search that was just as just as good or better, um, and also gave you some other options.
00:26:58.160 --> 00:27:04.240
So metadata can be really valuable in terms of comparison of systems, not just an optionality for your system.
00:27:04.319 --> 00:27:07.359
So you have optionality, you have comparison, like you have those types of things.
00:27:07.519 --> 00:27:19.839
And then when you think about like how does it actually benefit the LLMs themselves, this is where you get into like a very interesting discussion because there are aspects of it that can be really beneficial.
00:27:20.079 --> 00:27:33.119
Typically, what it means is that you're transitioning a structured metadata into, I would say, like statements for the LLM, such that it is still context, right?
00:27:33.200 --> 00:27:39.279
But like LLMs receive context differently than like humans and traditional machines to some extent.
00:27:39.440 --> 00:27:42.960
But if you have everything tagged for, so I'll give you an example.
00:27:43.119 --> 00:27:45.200
Let's say you have two products.
00:27:45.359 --> 00:27:48.000
You have a we'll use washing machines, right?
00:27:48.079 --> 00:27:55.440
Let's say you have a 1000 version and a 1001 version, and those are very, very similar systems, right?
00:27:55.519 --> 00:27:57.359
They're the same line of washing machine.
00:27:57.519 --> 00:27:58.880
There's a few differences.
00:27:59.440 --> 00:28:07.359
Well, you've got a hundred topics for each of those systems, and let's say 95 of them are the same.
00:28:07.519 --> 00:28:12.799
So they all carry the 1000 and the 1001 tag, and then you have five that diverge.
00:28:12.880 --> 00:28:19.279
So you have 105 topics in total, and you know, five with a thousand, five with a thousand and one.
00:28:19.599 --> 00:28:26.880
If you're just stuffing all of that into an LLM, and remember, like this is only 105 topics, so you can probably get away with that.
00:28:27.119 --> 00:28:33.599
The way you would use metadata there, and notice I'm specifically avoiding rag and search in this particular example.
00:28:33.680 --> 00:28:57.200
Like I'm just talking about how you put the whole 105 in, is instead of having a tag on that, you would probably be better off actually having a statement at the top of it where before you put it in, you'd use a deterministic system to take the tag from you know product and then it had the tag name to this is the 1001 version of this washing machine.
00:28:57.440 --> 00:29:00.000
This is the 1000 version of this washing machine.
00:29:00.079 --> 00:29:01.839
You just put that at the top of every topic.
00:29:02.079 --> 00:29:08.160
And an LLM is going to interpret that better than having like the actual tag on there.
00:29:08.319 --> 00:29:10.240
And so I've done a bunch of testing around this.
00:29:10.319 --> 00:29:12.640
Like, this also is the same with like glossary terms.
00:29:12.720 --> 00:29:21.599
So if you have like structured glossaries, um, different LLMs will receive the pieces of that glossary, which they're tags, right?
00:29:21.680 --> 00:29:22.480
Like it's metadata.
00:29:22.559 --> 00:29:26.559
Like you're like, you know, this is an abbreviation, this is another way of saying it.
00:29:26.640 --> 00:29:30.160
Like, there's like right, it that's just metadata that describes different pieces of it.
00:29:30.319 --> 00:29:40.559
If you write it out as pros, even if it's the same pros over and over again, the LLMs tend to work with that version of that content better.
00:29:40.799 --> 00:29:57.440
So transitioning the structured metadata into clear and simple statements, even if they're repetitive for the LLM, tends to produce a better result today with the LLMs we have today, because of how they've been trained and you know the biases and like the internal weightings that they have.
00:29:57.519 --> 00:30:00.799
Like you could train one that works better with the actual structure.
00:30:00.880 --> 00:30:02.799
That's just not the ones that we have today.
00:30:03.039 --> 00:30:06.160
So metadata touches a ton of different pieces of it.
00:30:06.240 --> 00:30:14.160
And I haven't even gone into like how metadata can then be used to build graphs and to create like more sophisticated like selection mechanisms.
00:30:14.400 --> 00:30:31.359
But the point is that once you get to any level of complexity, having the metadata on the content gives you so much more optionality in terms of how you build the systems around the LLM, and then also how you actually instruct the LLM when you're putting inputs into it.
00:30:31.680 --> 00:30:37.119
I think this was such a great overview because my goal here was to kind of step back.
00:30:37.440 --> 00:30:48.240
And for those amongst us who have very, I guess the as a technical writer, one would have that structured content, the concept conceptually.
00:30:48.720 --> 00:30:57.839
But if you don't, if you just have a Google Doc and you're starting off, then adding metadata to that, you've given us a why, a solid why.
00:30:58.000 --> 00:30:59.680
Why should you still consider it?
00:30:59.920 --> 00:31:04.400
And how that would kind of work under the hood that I keep going back to.
00:31:04.720 --> 00:31:12.240
And I would love for us to maybe even dive into the knowledge graphs and how they would relate to metadata.
00:31:19.200 --> 00:31:25.440
With more than 70 sessions and workshops featuring speakers from companies like Salesforce, TikTok, T-Mobile, and Amazon.
00:31:25.680 --> 00:31:33.119
LavaCon brings together content strategists, documentation leaders, and content professionals to share practical ideas you can apply right away.
00:31:33.279 --> 00:31:40.480
And yes, it's known as the Fun Conference 2 with networking events, therapy docs, comfort llamas, storytelling, and karaoke.
00:31:40.720 --> 00:31:44.799
Use discount code IDC26 to save$200 on registration.
00:31:45.440 --> 00:31:48.160
You did mention graphs here.
00:31:48.240 --> 00:31:52.640
So I think it's worth going into that a little bit, if not more.
00:31:53.119 --> 00:31:53.680
Yeah, sure.
00:31:53.920 --> 00:31:55.119
So I'm happy to go.
00:31:55.440 --> 00:31:58.799
Well, there are people who are are experts in knowledge graph.
00:31:58.960 --> 00:32:01.759
I wouldn't consider myself an expert in knowledge graphs.
00:32:01.920 --> 00:32:05.119
I do know I I would say a pretty good amount about them.
00:32:05.279 --> 00:32:11.599
So there's probably a limit to how far I can go, but I'm very happy to talk about them in relation to structured content metadata.
00:32:11.920 --> 00:32:15.759
So maybe uh start with just the definition of what a knowledge graph is.
00:32:16.079 --> 00:32:24.799
So so a knowledge graph is if you want to think about it in technical terms, it's a database that has typed nodes and typed connections between those nodes.
00:32:24.960 --> 00:32:33.759
So you could have people and then you could have like a nose or like this person knows this person or is married to.
00:32:33.920 --> 00:32:36.799
Obviously, that would be a one-to-one connection in most cases.
00:32:37.039 --> 00:32:39.359
But like, you know, so this is like the friend of a friend stuff.
00:32:39.440 --> 00:32:42.240
Like, so like ontologies, knowledge graphs, those kinds of things.
00:32:42.400 --> 00:32:52.880
When you put this information into graph databases, what it means is you can start asking multi-level questions of the database.
00:32:53.119 --> 00:33:04.799
So the thing that people really like about them is if you go back to like the people example, you could ask a question of a knowledge knowledge graph of like, give me all the companies for people that Patrick has been on podcasts with.
00:33:05.279 --> 00:33:34.400
And a good knowledge graph, like a good knowledge graph database would be able to go and say, Okay, well, Patrick is a podcaster, and then there would be a field that's like has been on, or something like that, and that would link out to a bunch of different podcasts, and then each of those podcasts would have guests and hosts, and each of those guests and hosts would be people, and all the people would have as employment or you know, works for something along those lines, right?
00:33:34.559 --> 00:33:41.759
And so you could have a query that would say like doop, doop, doop, doop, doop, and hop across all those things and select all of those companies.
00:33:42.000 --> 00:33:54.559
That's why people like knowledge graphs, is because it allows you to ask those kinds of questions of an information set, which are traditionally very difficult to ask in the context of like relational databases.
00:33:54.799 --> 00:34:00.160
So you can do some of that stuff in relational databases, but it's much less flexible, I guess is the best way to put it.
00:34:00.400 --> 00:34:09.440
So you're dealing with very large and you know well tagged or very semantic sets of content, like you're just gonna do better in a knowledge graph.
00:34:09.679 --> 00:34:20.320
So, how this collides with structured content and metadata is that it's not always that you're typically you're not actually keep maintaining your content in the knowledge graph, right?
00:34:20.400 --> 00:34:22.960
So you're maintaining your content in another location.
00:34:23.199 --> 00:34:30.159
So the way that you you correlate, you tie the content to the knowledge graph is through the metadata in many cases.
00:34:30.480 --> 00:34:37.440
So the metadata that becomes the selection mechanism that the the knowledge graph can use to pull that out.
00:34:37.679 --> 00:34:52.480
You can also use direct linking, but you oftentimes like the metadata is going to become like a major piece of like how those two things come together, or how the knowledge graph forms the nodes in the graph out of the content, like it contributes to those things.
00:34:52.960 --> 00:35:01.199
I'm thinking in a typical environment, you have documentation coming from product, from engineering, from support.
00:35:01.840 --> 00:35:12.239
How should the infrastructure establish authority and provenance and a source of truth within the context of that uh of structured content?
00:35:12.559 --> 00:35:15.440
Because that content may be structured differently.
00:35:16.239 --> 00:35:17.599
What's your take on that?
00:35:17.920 --> 00:35:20.320
Yeah, so I think this is a really timely question.
00:35:20.639 --> 00:35:36.880
So the the environment you've described, you know, very briefly, where like technical content comes from support, product engineering, um, the technical writers, like that is one version of companies where the the technical content is kind of like the result of all these contributors.
00:35:37.119 --> 00:35:39.440
That's how you get to those corpuses of content.
00:35:39.760 --> 00:35:50.639
The other structure of companies is where you have the technical writers is more of like a clearinghouse, like they're the ones that manage most of the content, and they probably write a large portion of it.
00:35:50.719 --> 00:36:11.280
And in those environments, you do tend to have knowledge bases that grow up around that, but there's a clear, very clear delineation between like the docs are gold plated, the knowledge bases can be a source, it can be leveraged, but it's not like the level of governance and quality that's expected is is very, very different.
00:36:12.079 --> 00:36:18.320
And I think it's very situational as to which which is going to work for any one individual company.
00:36:18.559 --> 00:36:28.320
You know, some companies can work really well in the more almost crowdsourced version of documentation, and then other companies really can't go that route.
00:36:28.400 --> 00:36:36.320
Like they have to have technical writers as the authorities on truth, more or less, like is the way to think about it.
00:36:36.480 --> 00:36:39.840
Um, and that's like a that's like um a center of excellence approach.
00:36:40.079 --> 00:36:46.880
I have a lot more experience in that approach because of our technology lends itself to that approach.
00:36:47.039 --> 00:36:53.599
Like if you have the highly like crowdsourced collaborative version of it, like that's just not where we fit.
00:36:53.760 --> 00:36:55.840
So I don't work in that world as much.
00:36:56.079 --> 00:37:06.480
When you're trying to establish authority and governance across multiple groups like that, so again, I guess there's kind of two situations there.
00:37:06.639 --> 00:37:13.039
One is they're all using the same thing, which in my experience there is almost always a wiki or Google Docs.
00:37:13.199 --> 00:37:15.679
Like I haven't seen very many variations on that.
00:37:15.760 --> 00:37:24.880
Um, and that's just because when you have a large group of people who are not writers writing the content, you have to give them a tool that's not a writer's tool.
00:37:25.119 --> 00:37:35.679
It's like it's super uncommon that you'll see a circumstance where you have engineering product support and technical, and like maybe you know, a couple of technical writers working in technical writing tools.
00:37:35.760 --> 00:37:42.400
Like I think I've maybe seen that once in my entire career doing this for like I'm not even gonna say how many years, but more than a decade.
00:37:42.639 --> 00:37:46.559
It's much more common that you'll see you hit you go to the lowest common denominator.
00:37:46.719 --> 00:37:48.559
Maybe that's that's not exactly what I mean.
00:37:48.639 --> 00:37:52.960
You go to the most available system, which is almost always a wiki.
00:37:53.199 --> 00:37:57.199
Wikis are not built for governance, like there's just no way around that.
00:37:57.280 --> 00:38:12.320
Like wikis are built for availability, and governance is an activity that imposes structure, it needfully imposes roadblocks and workflow and things like that, because that's where quality comes from.
00:38:12.480 --> 00:38:19.840
It comes from people doing the same things, knowing the patterns, checking themselves in predictable ways, that kind of stuff.
00:38:19.920 --> 00:38:21.280
Like that's governance, you know.
00:38:21.360 --> 00:38:24.320
Like you look at any like Six Sigma or Six Sigma, right?
00:38:24.400 --> 00:38:26.880
Like it's it's a set it's a set of patterns and practices.
00:38:27.119 --> 00:38:34.880
You can't apply those to a group of people that are not professionals in patterns and practices around that particular environment.
00:38:35.039 --> 00:38:37.440
So it becomes very challenging, is what it comes down to.
00:38:37.599 --> 00:38:42.000
You can't really have like broad inputs and high governance.
00:38:42.320 --> 00:38:52.800
And if you don't have high governance in your content, the work of like reverse engineering authority and like source of truth becomes much more complicated.
00:38:53.039 --> 00:38:55.440
It's not impossible in all cases.
00:38:55.679 --> 00:38:57.360
So I can give you an example of this.
00:38:57.519 --> 00:39:01.760
So like we have technical writers here, obviously, like that would be crazy if we didn't.
00:39:02.000 --> 00:39:10.639
But like our product team doesn't do the product specifications in Heretto, which is you know, Heretto is a documentation system.
00:39:10.719 --> 00:39:23.599
Like we do our product specifications in Confluence, and we do that because like we like product specifications are the sorry, these are the the design specs, not the actual, like these are these are not the final specs.
00:39:23.760 --> 00:39:25.760
We don't do the design specs in Heretto.
00:39:25.840 --> 00:39:28.800
So the design specs, because like they are working spaces, right?
00:39:28.880 --> 00:39:39.280
Like you've got a bunch of people in there and like they're throwing stuff at the wall, they're pulling things around, like they're just they're kind of screwing around in the content to figure out what's going to work while they're going through the process, the design process.
00:39:39.440 --> 00:39:48.960
It's a very creative, free-flowing process of like getting things together, and that's just not the right place to apply like high structure and high governance.
00:39:49.039 --> 00:39:59.199
Like, you need to give people the minimum amount of those things to give them the most creative freedom to get to the end design results they're looking for.
00:39:59.440 --> 00:40:07.519
Now, at the same time, the outcome of these initial design documents that they do, they tend to be very high quality.
00:40:07.760 --> 00:40:09.519
So they're totally inconsistent.
00:40:09.599 --> 00:40:21.840
Like there's no like like they're not, you know, they're not in the same format of the same structure, but like they have really, really good information in them because they are for the initial design of something, like they kind of are the single source of truth.
00:40:22.079 --> 00:40:30.079
So in our world, what we do there, this is before the thing is built, before the documentation is written, all those kinds of things, is you just lock it down.
00:40:30.159 --> 00:40:36.079
Like once it's done, you just reduce the number of people that can change it, and the people that can change it are much more mindful.
00:40:36.239 --> 00:40:44.960
Now, we also have the benefit of the people who are working on this, are people who are who have all of the benefits of knowing structured content and documentation because that's a tool that we work on.
00:40:45.119 --> 00:41:00.000
So, like inherently, they're gonna be better about some of these practices, and you do see that bleed into you know building just frankly, probably more information architecture, you know, kind of biosmosis in some of these documents.
00:41:00.079 --> 00:41:03.199
So, like we see a lot of content come from Confluence.
00:41:03.280 --> 00:41:15.039
It's very common that people move from Confluence to Heretto, and it tends it comes in a wide variety of of qualities and like like positions from the structure and like a governance perspective.
00:41:15.280 --> 00:41:22.079
So, but we use those as inputs to to different LLM systems that we have, those those Confluence pages.
00:41:22.320 --> 00:41:38.800
So before a feature is built and before the can the content is migrated into Heretto and it's put on our internal like our internal point of reference is a thing we have like an internal name for it, that is all designed and built in Heretto, and that is considered the highest order of truth in the organization.
00:41:38.960 --> 00:41:40.480
Like if it's there, it's true.
00:41:40.719 --> 00:41:49.920
But before, you know, in the design process, if people have questions about things, like they can ask an AI agent and it will source from those confluence pages.
00:41:50.079 --> 00:41:59.760
And it works quite well because it's a very small set of content, it's maintained by a very small team, it has a very known position in the cycle of information.
00:41:59.920 --> 00:42:04.159
Like it was designed at this point in time by these people for this purpose.
00:42:04.320 --> 00:42:11.519
So like there is a set of metadata that's assumed about that content simply by the definition of what it is.
00:42:11.599 --> 00:42:16.400
Like there's all these attributes that come with it that you can build into a system to get good results from.
00:42:16.960 --> 00:42:22.159
I was also thinking about loan writers and systems where they've not evolved.
00:42:22.400 --> 00:42:26.079
You gave them great examples of how to think about it.
00:42:26.159 --> 00:42:29.920
So if you don't have those systems in place, you can start small.
00:42:30.719 --> 00:42:32.639
Where can I start if I don't have it?
00:42:32.800 --> 00:42:39.199
How do I think about this where you don't have established processes or uh governance frameworks?
00:42:39.440 --> 00:42:49.519
How do you establish that so that you can get to the source of truth and locking it down and having the right people, get the right gatekeepers, so that eventually your content is gold cleader?
00:42:49.760 --> 00:42:53.360
Um Yeah, I can actually give one other piece of AI-related advice to that.
00:42:53.519 --> 00:43:03.280
So if you're a loan writer, like so if I was gonna go be a loan writer in an organization today, almost no chance that you have access to like true structure content.
00:43:03.360 --> 00:43:06.000
Like that's just not common for loan writers.
00:43:06.159 --> 00:43:11.840
So I think the most common circumstance for those individuals today is going to be that they're writing in Markdown.
00:43:12.079 --> 00:43:15.039
There's a bunch of other situations, but like that's probably the most common.
00:43:15.119 --> 00:43:16.400
So I'll focus on that one.
00:43:16.639 --> 00:43:39.039
One of the things that I would do after I wrote a very basic style guide, I'm not even gonna call it an information architecture at that point in time, but a set of rules and patterns around how it is that we created content, is I would feed that into an agent and I would say, you're now going to be my editor, and you're gonna hold me to these rules.
00:43:39.280 --> 00:43:50.639
And so then every time, so again, we're in Markdown, we're probably checking this content into GitHub, which at a you know, at if I'm maintaining a thousand topics or less, that's probably gonna work pretty well.
00:43:50.880 --> 00:43:56.719
Every time uh it was checked in, it just had the agent go and look look it over and have it give me feedback.
00:43:56.880 --> 00:43:59.519
Oh, you didn't put enough metadata on the top of this topic.
00:44:00.079 --> 00:44:01.199
Oh, this topic is too long.
00:44:01.360 --> 00:44:04.079
Oh, it it this topic contains too many claims.
00:44:04.320 --> 00:44:11.440
Like you should break this up into more more markdown files because you know you're mixing concepts and and tasks here.
00:44:11.599 --> 00:44:17.280
Like this is clearly, you know, this section is clearly a task, and this section can contains a bunch of claims.
00:44:17.519 --> 00:44:20.000
Like there's uh claims facts, sorry.
00:44:20.159 --> 00:44:31.679
So there's a bunch of things that you could do with a very simple configuration of Claude, or I've actually not done this in uh in Gemini or OpenAI, so like I'm just gonna say Claude, but I'm assuming you can do this in other LLMs.
00:44:31.840 --> 00:44:32.320
Yeah, yeah.
00:44:32.400 --> 00:44:48.719
So you could set all those things up and it could basically just help you enforce a number of rules that are going to get you a lot further down the road than you would be if you were you know just writing content as text and markdown or in YAML files or things like that.
00:44:48.880 --> 00:45:02.159
And I think you would find that if it did scale up and you hired more people and you did need to transition to a true structured format, you know, for personalization or reuse, those types of things, that you'd still have a lot of work to do, but you'd be in a better place.
00:45:02.559 --> 00:45:07.679
Yeah, I think that's a great suggestion to kind of lean into those systems available to you.
00:45:08.000 --> 00:45:11.039
You were designing a content architecture from scratch today.
00:45:11.199 --> 00:45:18.159
Let's say you've been hired, there are no systems, and you are going to be designing content architecture from scratch.
00:45:18.320 --> 00:45:27.599
Uh specifically for AI and agentic systems, what would you build from the source content all the way through retrieval context, uh governance, and execution?
00:45:27.920 --> 00:45:28.880
Oh my god.
00:45:30.800 --> 00:45:37.360
But you you do you you've been you've done great so far with all the I mean geez.
00:45:38.000 --> 00:45:43.280
Well, so it would depend a ton on there's so many variables that go into that.
00:45:43.440 --> 00:45:45.760
So I'll give you one version of it.
00:45:46.000 --> 00:46:03.599
Okay, so if I'm coming into an organization that has, let's say 10 proper technical writers maintaining a hundred thousand pages of content, or well, let's just say fifty thousand pages of content.
00:46:03.760 --> 00:46:06.880
It's that's a little more on the on the medians there.
00:46:07.280 --> 00:46:16.239
And that content is you know, has any level of complexity, which it almost always does at that size, like you can almost never get around that.
00:46:16.480 --> 00:46:21.360
The first thing I'm going to do, if they're not already, if so, like you said, the nothing exists.
00:46:21.440 --> 00:46:27.039
The first thing I'm gonna do is I'm gonna introduce structured content and a proper CCMS.
00:46:27.119 --> 00:46:28.559
Like that would be step one.
00:46:28.719 --> 00:46:32.400
And then from there, I'm going to look at the delivery requirements.
00:46:32.719 --> 00:46:40.159
So one of the things that I think people undervalue is the importance of having a really high-quality website today.
00:46:40.400 --> 00:46:43.679
Like that is where most of your AI answers are coming from.
00:46:43.840 --> 00:46:47.360
Like before you implement any AI, you need to feed the public AI.
00:46:47.519 --> 00:46:51.039
And to do that, you need a website that has all the different aspects.
00:46:51.199 --> 00:47:12.880
Many of them are just SEO things, but all the different aspects that are going to give you the proper authority with Google, OpenAI, Anthropic, and the other major models, so that when people are asking questions of those systems, they're properly retrieving information from your set of information for your customers.
00:47:13.119 --> 00:47:17.679
Like that's gonna be 90% of your customers' AI interaction.
00:47:17.920 --> 00:47:20.880
From there, it's a very it's gonna get very situational.
00:47:21.039 --> 00:47:36.000
If I've got a field service team, I'm gonna look at how I'm gonna deploy an agent to that field service team that covers the particular issues that they're having a hard time solving that are also their information issues.
00:47:36.239 --> 00:47:41.360
I would start deploying agents and connections to our support channel.
00:47:41.440 --> 00:47:43.360
So like there was a feedback loop there.
00:47:43.519 --> 00:47:47.199
You know, those things are maybe not simple, but fairly straightforward, right?
00:47:47.280 --> 00:47:49.840
Like that's kind of like, you know, that's known work at this point in time.
00:47:49.920 --> 00:47:53.280
I'm not gonna call it a solved problem, but it's known work at this point in time.
00:47:53.519 --> 00:48:04.000
I think if you you kind of started with that path in that environment, you're going to get yourself so far down the road that you're probably gonna be ahead of most people already.
00:48:04.159 --> 00:48:06.000
That I think that that is a blueprint.
00:48:06.239 --> 00:48:20.000
I guess the other thing I should say there too is when I said implement a CCMS, I meant do the information architecture work on the content, come up with an information model, get the content into really good shape, and then put it into a CCMS.
00:48:20.079 --> 00:48:22.079
So that's like the process of implementing structure.
00:48:22.239 --> 00:48:28.639
One of the biggest mistakes that people make is that they think they're gonna buy a tool and that's gonna solve their problems, and that's not the kit, not the case.
00:48:28.800 --> 00:48:37.840
Like the work to take to go from we have a ton of content that's unstructured to good structured content that's actually going to solve problems.
00:48:38.000 --> 00:48:43.760
That's an information modeling and information architecture and conversion process, and that is work.
00:48:44.000 --> 00:48:48.079
It has a huge payoff, it's the foundation for everything else.
00:48:48.320 --> 00:48:52.480
But if you don't do that work, you don't get the benefit out of structured content.
00:48:52.719 --> 00:48:54.320
So, like that would be the first step.
00:48:54.719 --> 00:48:57.119
First of all, my apologies, Patrick.
00:48:57.199 --> 00:49:00.800
I did not mean to quote unquote assault you with that question.
00:49:01.199 --> 00:49:02.159
No, no, no, no.
00:49:02.320 --> 00:49:03.280
It's a great question.
00:49:03.360 --> 00:49:05.280
It's just it's such a big question.
00:49:05.599 --> 00:49:12.559
So I was in such an on a team where the mindset was that the tool would solve all the problems.
00:49:12.800 --> 00:49:21.679
And thinking about information architecture and mod well, let's say that we had to revisit how car content was modeled and architected.
00:49:22.159 --> 00:49:23.840
The tool did not solve the problem.
00:49:24.239 --> 00:49:24.960
You think about it.
00:49:25.039 --> 00:49:32.719
If you were gonna go build a house and you showed up at the build site with a bunch of tools but no blueprints, yeah, what would you get?
00:49:32.880 --> 00:49:33.840
What would you end up with?
00:49:34.320 --> 00:49:35.280
Thank you so much for watching.
00:49:35.440 --> 00:49:38.480
This was a great conversation, and I hope you had fun as well.
00:49:38.639 --> 00:49:39.199
I did.
00:49:39.440 --> 00:49:40.320
Thanks again.
00:49:40.559 --> 00:49:41.440
Thank you.
00:49:42.320 --> 00:49:45.039
Thank you for listening to Insight Techcom.
00:49:45.280 --> 00:49:49.280
If there's a guest you would love to hear on the show, please let me know.
00:49:49.440 --> 00:49:54.559
And don't forget to follow and catch every episode on your favorite podcast app.
00:49:54.719 --> 00:49:55.840
See you next time.
