speaker-0: Hey everyone, welcome to Chat B2B, the podcast that gives you a front row seat to AI and B2B marketing. I'm your host, Andrew Au, co-CEO of Intercept. And on this show, we chat with enterprise marketing leaders who are piloting and scaling AI use cases. We dive into the things that they're learning and what we can take away and apply in our own organizations. Today's guest is Megan Cabrera. who most recently served as the VP of Marketing Operations and Digital at Sophos. And in this episode, we get into the AI Search Playbook that she developed and deployed at Sophos, including creating over 500 prompts in order to measure visibility, how she reformatted and upcycled content to be able to improve it, and what is the role that Reddit and other community sites play within that enterprise AI Search strategy. We also dig into what are the things that you should build versus buy and where AI roles need to sit in the organization for successful AI transformation. Lots of great stuff to dive into. If you like what you hear, don't forget to follow or subscribe wherever you get your podcasts. This is Chat B2B. Let's get into it. So Megan, you went from Sitecore, which was a ⁓ MarTech company that builds tools to Sophos, which is the enterprise buyer of the tools. So you kind of did like, ⁓ you've had experience on both sides. What was that experience like? And did that kind of change the way you look at, you know, how you make technology decisions being now on the buyer side?
speaker-1: Yeah. So funny enough, actually when, when I started at Sophos, they had just gone right of Sitecore. So that they had moved to the people. So that was really interesting, but I think what was fascinating for me with Sitecore was I was the user of the tool. So, you know, we didn't have to necessarily buy, you know, make a vendor decision on these tools because we were using our own dam, our own, as, and
speaker-0: ⁓ really?
speaker-1: I got a chance to really influence the product from that perspective so I could give feedback as the actual user of the tools. Then when I left Sitecore and went to SoPos, that was such a, it was a very different experience because it's this wide open landscape. You can use any tool that you want and I encourage to do so. But I think having had that experience of being able to influence the product gave me a really good insight into how do you ask vendors the right questions?
speaker-0: You'd mentioned that, ⁓ obviously, encourage you to use tools. I feel like a lot of our clients are being told to sprint towards agentic and do AI, but they're also like, well, what does that mean? And compliance and all that kind of stuff. So it sounds like at Sophos, was a use AI wherever you have a use case, it's kind of like open in that way.
speaker-1: So, I mean, that is one of the great things about Sophos is that it was so AI forward, both with our cybersecurity tooling, but also with the marketing department and other departments in the company. And, you know, our C-suite was really encouraging of using AI. So we did a mix of builds and buy, frankly. I mean, there was some use of agents within the LLM, but certainly a lot of it was leveraging marketing tools like Asana, Clay, Qualified to be able ⁓ to build out the AI components of those tools.
speaker-0: Yeah, incredible. find that there's, you know, even again, all of our clients are global tech, but there's different appetites for risk. And some folks are like somewhat locked down, you're using these tools, that's it versus others like Sophos is real sort of, hey, if you've got an interesting use case, go explore it, which
speaker-1: Yeah, and I will say, I mean, we had a gen AI process through legal Tibet, generative AI use cases. We also had a security review. We also IT, you know, enterprise architecture team. So it wasn't like you could just pick a tool and tomorrow go start using it. I mean, we were definitely using the enterprise versions of all these LLMs and that. But I do think there was an appetite to experiment and, you know, kind of tackle some of these broader use cases that marketing is really struggling with. And so, you know, we did manage to partner really well with our other core stakeholders to get things onboarded.
speaker-0: Yeah, so rigor, but also there was the speed, right? Very much. That you needed both. So, speaking of use cases, I know you went down sort of a rabbit hole with AI search. Yeah. And that was a whole journey. And I know that you've had, you know, pivots along the way. Maybe, maybe take us back to the start, because I feel that everyone is trying to crack this nut of how do you do this right? And it's a bit of a wild west where there are best practices, but a lot of them are still in beta because things are changing. I don't know, maybe take us back to the start and what were some of the milestones you remember on that journey?
speaker-1: Yeah. So I started really picking up on AI search to be honest, when my own behavior started. Right. Okay. So we had this huge project. We were going out to RFP to try to figure out a vendor and I ended up taking our RFP and, ⁓ a whole bunch of context documents, putting them into the LLM and getting a result back and going, wow, that was not thought. the response was going to be. And we actually spoke to a few vendors that we never would have spoken to otherwise if I had to do that process. And so that is where, you know, as the owner of the website and web strategy, that started to pique my interest where I was thinking to myself, well, hold on a second. If I'm completely changing the way that I pay terms of searching for vendors and identifying who can tackle certain issue, that has to be
speaker-0: saying.
speaker-1: happening across the board. Then we started to try to figure out, but how can we understand the prompts that people are putting in? And this is an enormous gap. You can't, right? With the sole exception of within Bing Webmaster tools, you can see some of the copilot prompts ⁓ word by word. But for the most part, you really can't get to that. So what we did is we kind of, instead of boiling the ocean, we picked what we called canonical prompts. So essentially just guessed, right? From reviews that we saw of G2, reviews we saw of Gartner, things that we were seeing in the LLMs, we essentially guessed on a set of kind of 500 canonical prompts across our personas and products and decided that's the way we would measure, right? Because if you keep changing these prompts, you're changing the goalpost, goal time.
speaker-0: Exactly.
speaker-1: And so you're not gonna get kind of a true measurement. so month over month, and we started actually doing it every two weeks because the models were changing so fast for a period of time there, but just measuring visibility, AI referral traffic, kind of starting to get ourselves familiar with the data. Cause for me, that was really step one. I had no idea what to expect from these numbers when we started looking at prompts. Then once we got through the measurement, the initial measurement phase, That's when we started to get into, it turns out one of the biggest things that you can do is not creating a whole bunch of new content, but rather reformatting your site to meet the LLMs coming in, looking at your existing content. that became probably the most important next step for us in that journey was things like FAQs, schema.
speaker-0: at point.
speaker-1: LLMS.txt file. I mean, if people don't have those in place yet, that is absolutely necessary, you know, for for people both it's actually useful for humans as well the FAQs, but we definitely found that that had the biggest impact on, you know, being our visibility in search pretty immediately and then you're kind of, you know, comes into play a little bit later. ⁓
speaker-0: Question for you on the 500 prompts that you had sort of set. How did you think about those prompts? Like was that sort of across sort of different buying states? mean, as you think about just exploring versus evaluating versus how did you structure that? Because I find that's a really key step and I think there's so many different ways you could do that.
speaker-1: So the initial set of prompts, actually took extracts of all of our G2 reviews and Gartner gear reviews and filmed those for the initial set of topics to pull in. Very first step was we only had 10 canonical prompts. Really light testings. ⁓ Then over time, that expanded by cross personas, across product set. ⁓
speaker-0: Okay, yeah
speaker-1: But we didn't focus on awareness. We focused a lot more on consideration and purchase in those prompts, just because I honestly think that's kind of what's happening right now is awareness piece is very high level brand. Does anybody know who your brand is? Right. But then once you get consideration and purchase, that's where people are using LLMs to really understand like a very specific cybersecurity case, for example, you know, what they'd be doing. What I do, right, is I'll feed an LLM, hey, I have Clay, Qualified, Eloqua, and Salesforce, and I'm trying to do X, Y, and Z, help me build out the workflow, tell me am I missing? You know, do I have gaps in there? I'd imagine that people in cybersecurity are asking some of those same questions of the tools. And what you're trying to do is make sure that so-and-so shows up, right, as being one of those tools that you would. you'd be looking for that integrates and plays well with other third party tools. you know, we kind of just, honestly, I used what I had learned from all the different questions that I was asking to pull that through into those canonical prompts. And then we had a separate set of what we called experimental prompts. So those were driving more of that content creation. So those were prompts where it was more part of the zeitgeist, like people were
speaker-0: Yeah. Interesting.
speaker-1: about certain things. And so we track those as well, but we wouldn't put them in as part of the measurement, the core measurement framework. We'd track them so that we could understand, do we need to create the kinds of content that would help show up in these spaces?
speaker-0: You know, the I think there's a few variables for sure. I think it's really interesting how you attacked it. One is the the prompts themselves. And like you said, it's like, how do you actually know what people are prompting? ⁓ One of the ways that that we've ⁓ solved this in some ways is actually doing one to one interviews with IDDMs and BDMs and go, hey, here's a scenario. I want you to simulate what would your prompt be? And you kind of observe the different styles. ⁓ And again, and you and I have talked about different sort of vendors for just LLM monitoring and the prompts that you see in some of these tools, they're very consumer. And you look at them and you're like, there's no way our audience is prompting like this. So I think that that's a really smart way first, just to look at the reviews to understand topics. And then based on what you know about your audience, be able to simulate that. So I think that that's really cool. ⁓ And I'm glad that you kind of went into sort of that zeitgeist layer of what are people talking about on other third party sites and forums and communities? Because I get sort of two reactions to that. Some clients are like, well, is that really representative of ⁓ ITDMs? And I think it is because when you want to know what people are really thinking is look at what they're saying when you're not in the room. When you are not When they know this is not a survey, this is not a, when they're just talking to their peers, I think that is the most authentic form of insight that you can get in some ways.
speaker-1: Yeah. And what we noticed too, is that especially from a blog and a webinar perspective, those were the topics that were truly resonating. Right. So it, yeah, it's nice to learn about, you know, the new firewall features, for example, fail. ⁓ but the reality of it is what people are really trying to figure out is what's going on at a macro level. Like how am I, how am I supposed to look at this market? And the zeitgeist really captures a lot of the things that people are concerned about at a given and and helped us kind of be on top of some of those conversations.
speaker-0: amazing. Megan, the thought you had around sort of updating content, upcycling, if you can remember, what were some of the main things that you needed to do to improve content? I mean, you mentioned format, so like answerability, FAQs, that kind of thing. Do you recall like what other ways you had to upcycle your content based on just some of the insights that you found?
speaker-1: Yeah, so a lot of it was schema, frankly. So laying things out in a way that the LLM could pick up and parse the different bits of it and process those. So the way we as humans read and prefer to read is really different from a machine. You know, I think we've done a little of this with, know, blogs having H1 and an H2 and a paragraph and we have, you know, specific structures for things like that, but it's even more, and then a lot of metadata behind, right? The humans are seeing, but the AI bots are seeing that metadata. The other thing that we noted pretty quickly is that dates and authors matter. So the LLAs are looking for those. so even if, you know, we had a section on the site called Cybersecurity Explained, which is a lot of our kind of what is pages, glossary pages, helping people to understand cybersecurity. And even if the author on that was, you know, Sophos and not a specific name, it still performed better in search. If you had an and a date and an updated date. And we started to do that too, because, you know, we really realized we had a lot of content that hadn't been updated in a while. And that's not beneficial to.
speaker-0: Stay.
speaker-1: of outdated content.
speaker-0: Every model performs differently, as you mentioned, and models are being updated. How do you build a foundation that feels consistent when everything kind of is moving around? You mentioned sort of consistent prompts, which is actually smart because it's like you want to control the variables. How else did you think about this in terms of the variable change that's happening?
speaker-1: Yeah, I mean, we tried. It's been really challenging to be honest, because even some of the measurement tools that we were using were actually changing their measurement framework in the background. We write the process and we definitely saw, I mean, when we took measurements before a model, new model launch, and then after a new model launch, you 100 % saw changes in the ability and the outcome of them. So. you know, what you tried to do is just in between measurement pieces, sort of record, okay, these are the things that happen, you know, for example, OpenAI launched, you know, GPT-5 or whatever, during this period of time. So you could at least kind of keep tabs on which models you might have seen some drift in that visibility number for, and potentially ask why. But I think one of the most challenging thing is, you know, leadership would ask,
speaker-0: Right.
speaker-1: Okay, but what did we do that impacted the number? And you know, we could point to a number of things that we were doing, but not sure I could ever with authority sit there and go, well, it went from 75 to 80 because of these five things. And that's really challenging for people to wrap their heads around. We have just. as a species, like we just want to know. We want to know. And we want to think that we have control over these. I mean, you, some of these, the AI labs come back and go, I don't even actually know what this algorithm is doing.
speaker-0: So how are we supposed to know, If the people on literally the frontier building this stuff doesn't know themselves.
speaker-1: aren't always aware. So it is a challenge. It really is.
speaker-0: When you looked for impact, could you see measurable impact when you're looking at the same model? Or typically were you waiting for the next model to release to see what the change was? Were you able to see that impact within the same sort of era of model?
speaker-1: that actually those those moments so where we would you know measure do something and then measure again those were the moments where you could actually kind of most confidently point to hey you know we we started to see certain topics bubble up really well and that was because we had released a whole bunch of new content you know from a campaign or something that was going on within marketing and but i do think a lot of what the models are basing off of is the training data, not necessarily the web scraping of your exit. So that's what's really challenging is what you're trying to do is you're trying to make sure that you have a really good base so that the next time the model training happens, you start to show up even better.
speaker-0: Yeah. And what we found was that obviously some models like perplexity native search, it's going to go out and do search as opposed to quad. It will be more with based on its training data. So you see the variance too. Were there any other variances like that that you saw or more or less the same?
speaker-1: It more or less the same. mean, like that's why we sort of used perplexity differently than we do the other models. cause because of that retrieval augmented generation, you are able to get a more real time view. I felt like a grok was a bit more like that too, actually, cause it's trained on a lot of kind of real time Twitter X, sorry, not Twitter anymore, X data. and so I think we kind of used those models a bit differently. to get a bit more of a real time brand assessment versus the other LLMs that were more of a baseline.
speaker-0: Yeah, makes sense. The way that we typically do them is that we'll do a similar model of like, how are the models performing versus gen over gen? The other thing is when we would use watchtower as a way to sort of track sentiment for specific audiences, then we kind of like triangulate the two to see if there's a meaningful shift, like if it was correlated. And a long time there actually, there was some good correlation. So yeah, that is a tricky one, particularly when senior leadership wants to know to your point and wants to know what is driving that so that they can put more money into that potentially. But yeah, we don't we don't always we don't really have that precision to know what it is. ⁓
speaker-1: No, we don't always have that answer.
speaker-0: Was there anything in that journey that you tried and went, okay, you know what, that clearly does not work. Were there any of those kind of like myths or moments where you just had that, you you had to completely drop a strategy?
speaker-1: So I'd say there were two things. One, we actively avoided actually. And the second, we thought about trying but didn't try at all. Both of those. the first one that we actively avoided were the whisticles that everybody has on their site. Topics, endpoint, vendors, they put themselves somewhere in that list. ⁓
speaker-0: face just so happened to be number one.
speaker-1: Exactly. We just felt like there was no amount of trust and credibility that we wanted to sacrifice to create content like that, that's really self-aggrandizing. that was one. ⁓ And then the second was, and thankfully I had a great VP of product marketing who was very kind of, we were constantly bouncing off conversations off of each other. And You know, one of the things that I went back and forth with him a little bit on is, ⁓ we started to see some of these kind of third party analyst sites. there was one spot for example, that would show up constantly as citations and chat GPT specifically. And so then we started thinking, should we have a presence, a more of a present sites? Should we be boosting more reviews here? We ultimately decided not. And I actually think that was the correct decision because, you know, when you take a step back and say, where are my, you know, in the case of Sophos, like the cybersecurity community, where are they aggregating? Where are they finding their information? I'm not sure that spending money in that would have actually gotten us a whole lot of benefit. You know, we ultimately were like, I don't think that's one of the watering holes. that cybersecurity experts are sitting in regardless of whether it's cited in chat GPT or not.
speaker-0: Yeah, yeah. I mean, it doesn't change the fact that, I mean, AI or not, that this is really about trust, right? And do they trust this as an authoritative source? you know, I think when it's a, you're trying to pick a cybersecurity vendor, I do think that, yes, at first glance, like you look at the LLM results, you get a sense of direction, but I feel because it's a higher risk decision, you would also look at the sources too. Like you'd want to verify, right? Like you don't want to, be that person that's like, ⁓ it's because of they've got 400 reviews here and you click it and it's like a 404 error and it doesn't exist.
speaker-1: Right, right. mean, cybersecurity is an industry that's built on trust. You've got to trust your vendor as your back and it's helping navigate.
speaker-0: 100%. Now, I know that, you know, some of the models do look at these kind of third party communities and sites and there's discussion like you mentioned with review sites and Reddit and different community forums. Did you, well, maybe I could ask you specifically on Reddit. Did you make a play at all on Reddit and just because it's totally different than let's say LinkedIn, right? Like if you put a LinkedIn strategy on Reddit, you're to get eaten alive by. by the folks. ⁓
speaker-1: Yeah. We did because we ultimately determined that Reddit was one of the watering. We felt cybersecurity, you know, experts and people using the tools were on. Thankfully, I had an incredible director of social media that I worked with on that. And she had a real eye for the types of people within our organization who should be out on Reddit commenting, you know, putting posts out there. So very organic strategy, not a paid strategy, not trying on advertising on Reddit. ⁓ And then we also had started just before I was leaving, we started to do some Ask Me Anything on Reddit as well, where we actually had our leadership on there. sometimes you look at these questions and you're thinking, ⁓ gosh, I mean, this is intense, right? But these are the questions that your ⁓ community and people who are interested in the product will know the answers to. So ultimately, I think it's pretty important for them.
speaker-0: And how did your leadership respond to that? Like, did they walk away from that experience going, this is really great, these are tough questions, ⁓ we need to spend more time here? Or were they like, this is too much work, too much pressure, but what was sort of the feedback?
speaker-1: ⁓ no, it was, it was definitely do more, right? Or get out more around some of this.
speaker-0: Awesome. And the outside is sort of the executives doing the AMAs. You mentioned identifying the right folks that made sense to be on Reddit. Was there a specific profile or like level within the org? And I ask because I think this is just, Reddit is a, you know, a bit of a black box in that a lot of our clients want to engage, but they're also unsure who do we put on that? Cause they need to maintain being active and all. So how did you pick? How did you find folks?
speaker-1: Yeah, so we had two teams, our CTU team that came over from the SecureWorks acquisition and then another team called DevOps team. so those threat researcher threat intelligent resources are just incredible. I mean, the knowledge base that they have in cybersecurity is so great that that was ultimately, ⁓ you know, some of the members that we ended up including in that Reddit pilot.
speaker-0: Okay, okay, got it. Yeah, that's always the tricky one. It's like who's the right person and the right personas. You mentioned obviously a bunch of different tools earlier that you were using and one of the conversations that Francis, our CTO, me and Francis have is around what do you pour concrete on in terms of a foundation as opposed to what do you keep as sort of modular and flexible? How did you have that conversation when tools, again, there's new tool, feels like every month, right? Coming out of SF and you fight so hard to get something approved, but, and yet there's new tools. How did you think about that?
speaker-1: Yeah, so we had a toolkit effectively that we were leveraging. Well, if I take a step back, so when I started kind of driving marketing to be a lot more AI forward way back in January of 2025, we didn't, I say way back because doesn't January of 2020, I feel like a millennia and AI.
speaker-0: 100%.
speaker-1: It is wild. ⁓ Wild, absolutely wild. So back then we actually didn't have a company wide ⁓ access to an LLM and so we on chat GPT enterprise for marketing and we started to build out, know, we coached, did a lot of the art of the possible just helping marketers to understand what are these tools? Like what can you do with these tools that it's very different?
speaker-0: five years.
speaker-1: The interface itself, I it used to just look like Google, right? chat. so I think it's very hard for people to then take that kind of interface and conceptualize, ⁓ I can build an agent with this that can do this. So that was a lot of the time that we spent with the teams on that. then ⁓ the great news is because we built out that foundational prompting, some of those foundational ⁓ custom GPTs and chat GPT, then
speaker-0: That's right.
speaker-1: the company decided to go with Copilot as our ⁓ foundational tool for the business. We were then able to take everything that we had orchestrated in ChatGPT and move that foundation into Copilot and kind of using Power Automate and some other tools. So I think that's really important for businesses is to have that at least one, probably more to be honest. Some of them have very different you know, for example, the visualization in Claude, you cannot mimic that in Cognitive. It is just impossible. you know, I think that having at least one of those toolkits though, that you are saying, hey guys, this should be part of your everyday use case from that perspective. And then what we did is then we started to layer on our core marketing technology and you know, every tool has some AI component in it. For example, within Asana, which is the workflow tool that we used, they have AI Studio, which is really useful for triaging tickets that come in. Right. Marketing, know, hundreds and hundreds of tickets that come in and, you know, either it'll sign it to larger projects if it identifies it should be part of a larger project or it assigns it to humans that can get the task done quickly. We leaned in a lot on clay and this, this is not going to be a paid advertisement, but I just absolutely love clay and product itself. And I can see a world in which clay becomes kind of the new orchestration layer for thing. And outside of, feel like at some juncture, that was the marketing automation and marketing automation platforms just simply have not innovated. There's, there's a few new ones on the market.
speaker-0: fire.
speaker-1: for sure, but most of the legacy platforms have not innovated. And when we brought Clay on, we brought it on to be ⁓ this third party data marketplace for us. It was still one problem, which was we had six different data tools and we did not have six different, from a cost and an efficiency perspective. But ultimately, once we started building in Clay, that's when we were going, ⁓ my gosh, this thing is so much more than a deep space. There's a lot of agentic capabilities and the ability to kind of orchestrate workflows. And most importantly, the types of workflows that go between marketing and your BDR, SDR or sales, which, know, ultimately like bridging the gap between the silos and go to market is one of the things that I'm honestly most excited about. AI because I think I think it's it's review those gaps in a way that's causing companies to really address them and fix them.
speaker-0: Yeah, I think it's great point. I think what I was also reading in between the lines that what you did really well at Selfos is the training component. You mentioned sort of pre-canned prompts, the training people on what to do within JetGPT, and then transfer that skillset over to Copilot. And then it sounded like in Clay, you democratized access to that so the team can build custom workflows. That, I think, is a missing step. that a lot of organizations don't take is like teaching people and telling people what they can do and how to do it.
speaker-1: That's right. And I will say we did not fully democratize Clay. So Clay, the operation, we had some operations folks that had access. is, you know, I understand people's concerns on the usage pricing within Clay. And then also it is a challenging tool to learn. was, I I had a lot of fun playing around with it, but it's impressive when people, you know, can really pick this up. ⁓ I understand why they came up with GoToMarketEngineers. put it that way. But we definitely democratized the use of agents or custom GPTs or whatever the individual term is in the LLM for different employees. And honestly, it was amazing to watch people build. They came up with really interesting use cases and things that, even if it was just an individual agent that was kind of in an assistant of theirs or part of their org chart, for example, it was amazing. I mean, you saw people kind of coming up with very creative ways to use them.
speaker-0: Yeah, ⁓ it is incredible. mean, you know, if a lot of this whole sort of like doom and gloom around AI's impacts to the workforce doesn't take into account that we will do new things and we will find new ways to express our creativity. you're right. It's like you give access to the tools and you'll be surprised, I feel, on what people come up with, you know, that are really curious. One thing that I'm curious about is when you moved from Chad GPT to copilot, what was that transition? Like it sounded like it was pretty seamless for you. there was what were, cause I find that some people do have, you know, questions on copilot, unsure how to use it or extract the value that they need to. What's your take on that?
speaker-1: So I have to say we had not fully transitioned to co-pilot yet when I left. So, ⁓ you know, if I talked to my team today, they might give me, you know, two or three areas through finding different challenges. But the company again, really leaned in on training, right? So maybe they're training across the board for all the resources, you know, it was every employee in the org that has access to that. And I think that is the key. know, we did run into a few things because I had an early version of, of co-pilot. And so I did run into, for example, you could chain agents together in chat GPT and you couldn't do that in pilot or at least not in the same way. One of the things that, you know, if anyone from Microsoft is listening, like your license, packaging is very confusing, right? You had to have a certain license for the LLM, you had to have a different license, you know, if you wanted the studio and to be able to build things out and use power automate. And so I think that's the, sure the team is kind of still navigating some of those quirks, but relatively speaking, the underlying agent and LLM. And I will say one of the things that was great is for us within ChatGPT Enterprise, because we knew we were eventually moving to copilot, we not created a sustainable architecture for having a knowledge base connected to Enterprise ChatGPT that we were pulling from consistently. And so I think that is one of the huge perks of the Microsoft ecosystem, right? Is you've got everything in SharePoint, you've got everything in these kind of modular tools within there. And so you can actually leverage the knowledge base in a way that was not possible for us. With the way we had it set up.
speaker-0: Yeah, yeah. it's got all of your context, right? So it's the graph that is able to kind of customize experiences. I think the one thing is just, you know, there's a lot of configurations, right, within Microsoft. And I think if you don't get that right, then you can create problems downstream. But yeah, it's like, if you've got that configured right, like it can work really, really well. So, I mean, as a, for Cellphose, being a technology company, I'm sure you had some scenarios where you thought, Do we just build this ourselves, Like, cloud code or whatever platform versus buy and license. What's your take on that? Did you, how serious did you evaluate building stuff? Like as an organization versus going out to these tools and what's your sort of take on the whole, that whole situation?
speaker-1: Yeah. So as an enterprise, it's fairly challenging for us within a marketing environment only to really be those builders, right? We are IT counterparts because a lot of what you would build is very for the enterprise. actually we have, and my marketing operations team moved into IT about six months ago. And so we had this very good hand. with our IT organization who was kind of reviewing a lot of the departmental AI use cases and helping to see, this one goes either it's cross-functional or it touches, you know, revenue tools or it's just a really complex workflow. And if you want to build it, you know, you'll definitely need that kind of IT level of support. enterprise perspective, I feel like the... build process, it kind of, has to be part of a larger plan. Now, the buy process, what is really interesting. So in ChatGPT, we had started building out what we were calling the AI content workflow. And so we had a number of custom GPTs. So we had a brand and tone GPT. had a webinar GPT, an email GPT, you know, a number of kind of at the asset level, we had deep research that we were feeding in that was kind of feeding to some of the content pieces. So we had started to build out this whole network of cut bts. Then we started to think about, how are we going to orchestrate this and connect this into our CMS and connected it to our translation workflow. And that's when we found air ops and air ops. you know, honestly, I told our air ops salesperson, he'll remember this if he ever watches this, I wish I had built this product. I wish I had built this product. And so, you know, I think that was a very interesting moment where all of a sudden we realized, some of the things that we thought we wanted to build within Sophos, they are actually available as tools on the market, which for an enterprise, That's a lot easier of a conversation to have with your IT architecture team and with your security team. And hey, can we bring this tool on? Can we integrate this with some of our existing systems? Rather than, you know, and I think the build versus buy, it's going to come down to cost. Frankly, I could see a world in which companies start to build. Their cost is actually way more than they anticipate.
speaker-0: Yeah.
speaker-1: possibly even more than a head count. And then you've got a maintenance moment too. Who's maintaining it? Who's adding features as people are requesting it? mean, SaaS infrastructure is no joke, right? There's a reason these companies exist, right? So I think that'll be interesting as we go forward to see what size of companies decide to build versus buy and then what parts of marketing. people decide to build versus buy.
speaker-0: What I see is that there is this increased appetite to see, what could we build ourselves, right? And then I think you quickly, they get to this with this long queue of stuff that we gotta go build. Who the hell is gonna maintain it exactly what you had said. And so it sounds like a good idea. And then until you actually map out what is that plan gonna be? And I think for some things it really does make sense to build yourself, but. ⁓ Yeah, there's teams at these SaaS companies that are dedicated to red teaming these things and looking for gaps and, you know, it's lot of ⁓ work.
speaker-1: And they're getting constant customer feedback on how to enhance enhancing the product as a result of, you're not gonna get that same level of enhancement to your product necessarily. I find just talking to vendors ⁓ is hugely beneficial to me and to my role because I get a lot of really great ideas from my vendors that I might not have thought about on my own.
speaker-0: Exactly. Because they've a different vantage point, right? Talking about sort of how things are going to change in your vendors, if you think about internal org structures and you look at like these more AI-focused roles, you know, the question always comes back to like, ⁓ it's the AI strategy, where do these roles even sit? You mentioned marketing ops now being part of IT, which is kind of interesting. I guess more broadly, how do you see the org chart kind of evolving and these more AI focused roles, should that be more centralized? Is that more decentralized? Like, what's your take on that?
speaker-1: I think it's gotta be a mix. mean, so having had the vantage point of sitting in our IT org for the past six months, it was really great to see how they were thinking about supporting across the business at an enterprise level. You know, they were also spearheading a lot of that toolkit that I was talking about with Copilot, Power Automate, other tool sets and getting that training going, which is fantastic. The thing that I think is a is something to think about is the subject matter experts. So the subject matter experts live within that specific department. And frankly, the people who are in the trenches, hands on keyboards, day after day, those are the folks that are going to have the best idea of they might not necessarily know how to build it or how to orchestrate a new workflow, but they're able to sit there and tell you in crazy amounts of detail, exactly why their day is so annoying because they have to take this Excel file and move it into this tool and then go through this process. it's being able to sit down and have those conversations at those levels and parse that out so that ultimately you're taking a lot of that manual. mean, honestly, within marketing, 60 % of what marketers are doing today, I would estimate this is not a real data set. This is kind of my estimated data. That is just activity, just pure activity. might be handling tickets in a sauna. It might be dealing with a spreadsheet, but it is not value added work for the business. And I think that's where I feel really hopeful from a kind of people first AI forward perspective. that we can find ways to upskill and uplevel marketers so that they can start to do much more value added things for the business. Whereas a lot of that analytics and PowerPoint deck building and yaging tickets, that can start to get automated.
speaker-0: I agree. actually, Francis and I were chatting and we were saying how much marketing does marketers do, right? Like do marketers do these days because there's all of this, we talk about like administrative tax that's applied to like every task. When you do something, you gotta move the data from here to here, you gotta build a PowerPoint file, et cetera. Then I find though that because that's become so ingrained in the day-to-day work, take that away for people. what percent of people are ready to just do the hard stuff. And that I think is the scarier part of, and I think you'd said it really well, which is like, how are you upscaling, re-scaling? But I almost think it's in some cases less about the tech and more about some of these just like foundational critical thinking things that I don't know as a workforce, if the workforce today would do very well at that without proper training and support, you know? Yeah. So I guess, you know, just to shift gears, you know, we obviously a couple of weeks back now, I think there was that sort of unreleased AI model that had leaked from Anthropic, right? That had the whole impact on, I think it was like unprecedented security risk and the most powerful model that could exploit software vulnerabilities and all that. And I think cyber stocks obviously were impacted across the board, everybody.
speaker-1: Yeah.
speaker-0: ⁓ So it makes me think more broadly, how do you think AI is going to impact security SaaS companies going forward in terms of value? there's this whole chain around, know, Claude killed my company. ⁓ But where do you see that? Where do see AI impacting these these cybersecurity SaaS companies?
speaker-1: Yeah. Yeah. So obviously full disclosure, I'm coming at this from a marketer at a cybersecurity company, not as a threat analyst or the direct user of our product set. But I found that reaction to be really, really interesting because frankly, I would think if you've identified that some of these models are going to start to be really intelligent and pose a lot more of threat and identify more of these vulnerabilities, I think that makes cybersecurity way more important and not less important. So I find the market reactions to be a little, I don't know, fascinating where you're like, wait a minute, where is this coming from? Not a stock market expert, but to me, you know, when I look at the change that's coming and the new tools and the, mean, my goodness, we're just at the forefront of the agentic era and of letting agents start to actually execute within, you know, business processes. And I think that is just gonna make it meaningfully more important to have the right level of cybersecurity.
speaker-0: Yeah, I agree. mean, I think it's it's mission critical and, you know, vendors like ⁓ Selfos that specialize in this. Obviously, they've got tons of resources that are dedicated to doing just this one thing. You would think that, you know, they could probably do it better than you could internally if you wanted to, you know, kind of create your own.
speaker-1: Yeah, I don't mean somebody's probably out there vibe coding cybersecurity, but I don't know. I bet my I'd risk that right.
speaker-0: Yeah. And I guess talking to that point around, know, vibe coding, you think about OpenClaw and some of the whole like real autonomous agent workflows. How do you see that coming into the enterprise? Like, you know, they're great, like for personal projects, I think with the Nvidia announcement with NeoClaw, like, obviously, there's infrastructure being built and greater security and all that, but How close do you think we are to true autonomous workflows in an enterprise like Cellphose?
speaker-1: Yeah, it's a great question. mean, when you take a step back, some of what we have today, you could consider autonomous workflows. So for example, for anybody using Qualified on their website, Qualified can have a whole conversation with somebody and then book a meeting. And the next time that somebody speaks to that person is because of the meeting that they booked. Now, that's not taking it all the way to, for example, revenue. So it's a it all the way through the purchase process necessarily. But I think there's definitely a path to autonomous workflows. It's going to be interesting. Companies will need to really map out which of their workflows support that autonomy versus which of their workflows absolutely need humans in the loop. Right. I think there's, you know, there's ways in which you could see you know workflows that are driving insights that kind of are just updating that and bringing that tools I don't know inbox or slack or teams every morning That kind of thing will be probably fairly not that risky, right? So let's say the insight is wrong in there. Well, it's going to an eternal internal, right? So it's not such a big deal versus, you know the kind of
speaker-0: Exactly.
speaker-1: autonomous workflows that might be actually processing a transaction for somebody. And if that happens wrong, you know, how do you manage that? take who, you know, if it transacts at the wrong price, like how do you solve for that? So that's where companies are gonna have to really take a look at what are external facing versus maybe internal facing and start really thing with more of the internal facing. autonomous interactions and that is where it'll get interesting. think there's a lot of things that we can do around monitoring, analysis, insight that could happen early that take a lot of lift off of.
speaker-0: Yeah, I think you're right. think a lot of this will happen internal first. mean, even we look at just a couple of years back and we, know, at our clients that are building a lot of these solutions and we looked at like some of the wind wire data, like some of the top deals, a lot, like almost all internal use cases, right? Like it's, let's go figure it out internally first where there's lower risk and then start to think about sort of like customer engagement stuff. ⁓ Yeah, and to the point in analytics, I think that's key because we have so much data that it's hard to really understand it all. And we are, as humans, we can be easily swayed by the last number that we read or formatting and you know what I mean? So we're not truly unbiased. ⁓
speaker-1: We have inherent bias, although I have to say when we've run analytics experiments with AI, we definitely see a lot of hallucinations. You know, I don't think those are at the point where you still don't have a checks and balances because, you know, look at a pipeline number, for example, and kind of go, wait a minute, that does
speaker-0: Great point. It doesn't seem right. You know, it's interesting. was, I was moderating an event and this actually came up where they were all CIOs and they were saying how AI is really good with unstructured data, but structured data, it's, so bad. And it's this like jagged intelligence concept, right? Where it's like, it's really good at some things, not so great in other things. And I think it's just a matter of time until that kind of evens out.
speaker-1: Yeah, yeah, I agree. mean, with structured data, we were doing a lot more with machine learning models around clustering, pensity and, ⁓ you know, turn modeling. Then we were, you know, we were trying to start testing with Snowflake Intelligence, but that's where the either it didn't understand what you're asking because the data environment is so complex underneath that it's pulling the wrong number inadvertently, or it just doesn't. It just doesn't spit out the right number at all. That would sound interesting. I agree. I'm eager to see where that goes eventually.
speaker-0: Yeah. And I think looking ahead, let's ⁓ say you are starting a new position VP of Marketing Ops role tomorrow and you're at an enterprise that really hasn't figured out a lot of this stuff. I AI search or, you know, the tech stack is still, you know, TBC, right? Not well-defined. What would be like the first few things that you would want to tackle? in the first 90 days? Like what sort of mission critical, if you had a clean slate in terms of your AI strategy, where would you start?
speaker-1: Yeah. So honestly, I would start by taking a step back and understanding the customer base and exactly what are their issues? What are their pain points? Where are they? Right. Where are they operating? Where are they finding information? Where are they gathering? I think, you know, in the past 15 years of all the different companies that I've been at either full time or as a consultant, it's interesting that we've kind of lost that muscle a bit. And, you you start to see these companies that are being more of an echo chamber where they're thinking that they understand what's happening and, and, you know, product market has really shifted with the shape in AI. And so taking just a strategic step back to go, let's take a look at this landscape and make sure that we really have a good understanding of that. And we were building some of that out with Kaya and Outreach pushing that to Snowflake and kind of getting some of those insights. And it's great. I mean, that is actually a great use for unstructured data, as you mentioned, which is much easier for these tools to analyze. You know, then it starts to boil down into people process tools. what's the landscape within the people?
speaker-0: Right.
speaker-1: in the marketing department. Again, do we have to go back all the way back to the art of the possible? What do these tools do? You know, how do you leverage AI in your day to day? Or are they far enough along that they get that? But now it's time to really identify some really core use cases that AI could help, you know, unburden a lot of many processes. I found as you know, over the years, even as a consultant, it is so easy to pinpoint the broken processes. It's the one that you talk to three different people in a marketing department. They are all three going to mention that one. I mean, for us, it was always translation at that point. Translating the amount of content at our scale into eight different languages is just hard, right?
speaker-0: technical content too, depending on what it is, right?
speaker-1: it's just a very challenging pit, you know, and so you would talk to any marketer and they would highlight that as a challenge for you. And so, you know, that's kind of the, come in and have some conversations and immediately you can pinpoint, you know, two or three things that are just fundamentally broken. And then from a tech stack perspective, I think it's actually, it's a lot broader than marketing. I think the marketing tech stack and the sales tech stack and the customer tech stack are ultimately bleeding together in a way that, like I said, I actually really agree with. mean, if you don't have a cohesive customer experience from somebody hitting the website through to somebody actually onboarding with your product, you're in trouble. And AI has really surfaced that up in a way that's fascinating. It'd be actually stepping back to take a look at the entire go-to-market landscape and then starting to try to understand, okay, what's the core platform that the company is using from an LLM or maybe multiple core platforms and then what, if any of those tools are being leveraged with their AI components. I think there's a lot you can do just with the existing tech stack ⁓ and turning on some of the AI pieces that are available.
speaker-0: I think that's a great point because again, I'll go back to copilot. A lot of our clients have access to copilot, but are not currently using it, right? Or might be using it to draft a email, let's say. Not using it yet as a thought partner. So I think you're right. There's a tremendous amount of value that's already there. And I like to tell you phrase that around. Let's go back to like, you know, again, the customer needs, where are they at? the processes that people know that don't work. This is very much a people problem or a problem to solve than it is a technology one.
speaker-1: Yeah. And frankly, the technology is moving faster than people. I mean, the hindrance for companies as they start to make major changes is not that the technology can't do what they want it to do. It can, right? It's that as people and as our work structure and as kind of how we've situated ourselves, we're going to have to really face the music and make some major changes to some of that. rethink and be open to that. Be open to rethinking. But, you know, I still really do believe in marketing has to create that brand and that trust for any business. And in order to do that as a good marketer in any capacity, whether you're a brand, product, operations, whatever, you have to know your customer and your prospects in order.
speaker-0: Agreed. And one last question before I let you go is you think about sort of like rethinking and evolving. Do you think that marketing teams will largely stay the same size or do you think they're going to contract? Do you think they're going to grow? Like where do you see that just evolving from a, I think marketing is more important than ever because you build trust and the brand to your point. Obviously the big question out there is like, how many people do you need? Yeah. And so what's your what's your take on that?
speaker-1: You know, again, if we go back to that, you know, my made up stat on, you know, 60 % of the stuff that marketers are doing today is just straight up activity and not really moving the needle for marketing. My hope is that what AI can do is help reshape the fundamentals of that landscape. And so actually marketing can be doing more, right? So maybe, maybe you're not removing people from the organization necessarily though. My guess is you will need less people over time, but you know, maybe you're changing what their roles look like quite a bit. And so they're doing more accretive activity for the business and we're doing things that just weren't fundamentally possible before. I mean, that's really my hope on the direction that it goes.
speaker-0: Yeah, well said. And I, I echo that. think, I think again, all of these sort of ghost GDP and all those doom and gloom stuff assumes that we don't do anything different. And I think that that is a false assumption. I think that we will do new things. We will find new accretive opportunities to add value to the business. And so I agree. I'm hopeful.
speaker-1: you. Me too. ⁓
speaker-0: Megan, thank you so much for joining, I think. And thank you for just being really transparent on the journey and the tech stack that you're using, the things that you're learning. So we'd love to check in with you and see, again, new perspectives on AI search, let's say, in a few months down the road.
speaker-1: Yeah, absolutely. It's been great. Thank you, Andrew.
speaker-1: Yeah. So funny enough, actually when, when I started at Sophos, they had just gone right of Sitecore. So that they had moved to the people. So that was really interesting, but I think what was fascinating for me with Sitecore was I was the user of the tool. So, you know, we didn't have to necessarily buy, you know, make a vendor decision on these tools because we were using our own dam, our own, as, and
speaker-0: ⁓ really?
speaker-1: I got a chance to really influence the product from that perspective so I could give feedback as the actual user of the tools. Then when I left Sitecore and went to SoPos, that was such a, it was a very different experience because it's this wide open landscape. You can use any tool that you want and I encourage to do so. But I think having had that experience of being able to influence the product gave me a really good insight into how do you ask vendors the right questions?
speaker-0: You'd mentioned that, ⁓ obviously, encourage you to use tools. I feel like a lot of our clients are being told to sprint towards agentic and do AI, but they're also like, well, what does that mean? And compliance and all that kind of stuff. So it sounds like at Sophos, was a use AI wherever you have a use case, it's kind of like open in that way.
speaker-1: So, I mean, that is one of the great things about Sophos is that it was so AI forward, both with our cybersecurity tooling, but also with the marketing department and other departments in the company. And, you know, our C-suite was really encouraging of using AI. So we did a mix of builds and buy, frankly. I mean, there was some use of agents within the LLM, but certainly a lot of it was leveraging marketing tools like Asana, Clay, Qualified to be able ⁓ to build out the AI components of those tools.
speaker-0: Yeah, incredible. find that there's, you know, even again, all of our clients are global tech, but there's different appetites for risk. And some folks are like somewhat locked down, you're using these tools, that's it versus others like Sophos is real sort of, hey, if you've got an interesting use case, go explore it, which
speaker-1: Yeah, and I will say, I mean, we had a gen AI process through legal Tibet, generative AI use cases. We also had a security review. We also IT, you know, enterprise architecture team. So it wasn't like you could just pick a tool and tomorrow go start using it. I mean, we were definitely using the enterprise versions of all these LLMs and that. But I do think there was an appetite to experiment and, you know, kind of tackle some of these broader use cases that marketing is really struggling with. And so, you know, we did manage to partner really well with our other core stakeholders to get things onboarded.
speaker-0: Yeah, so rigor, but also there was the speed, right? Very much. That you needed both. So, speaking of use cases, I know you went down sort of a rabbit hole with AI search. Yeah. And that was a whole journey. And I know that you've had, you know, pivots along the way. Maybe, maybe take us back to the start, because I feel that everyone is trying to crack this nut of how do you do this right? And it's a bit of a wild west where there are best practices, but a lot of them are still in beta because things are changing. I don't know, maybe take us back to the start and what were some of the milestones you remember on that journey?
speaker-1: Yeah. So I started really picking up on AI search to be honest, when my own behavior started. Right. Okay. So we had this huge project. We were going out to RFP to try to figure out a vendor and I ended up taking our RFP and, ⁓ a whole bunch of context documents, putting them into the LLM and getting a result back and going, wow, that was not thought. the response was going to be. And we actually spoke to a few vendors that we never would have spoken to otherwise if I had to do that process. And so that is where, you know, as the owner of the website and web strategy, that started to pique my interest where I was thinking to myself, well, hold on a second. If I'm completely changing the way that I pay terms of searching for vendors and identifying who can tackle certain issue, that has to be
speaker-0: saying.
speaker-1: happening across the board. Then we started to try to figure out, but how can we understand the prompts that people are putting in? And this is an enormous gap. You can't, right? With the sole exception of within Bing Webmaster tools, you can see some of the copilot prompts ⁓ word by word. But for the most part, you really can't get to that. So what we did is we kind of, instead of boiling the ocean, we picked what we called canonical prompts. So essentially just guessed, right? From reviews that we saw of G2, reviews we saw of Gartner, things that we were seeing in the LLMs, we essentially guessed on a set of kind of 500 canonical prompts across our personas and products and decided that's the way we would measure, right? Because if you keep changing these prompts, you're changing the goalpost, goal time.
speaker-0: Exactly.
speaker-1: And so you're not gonna get kind of a true measurement. so month over month, and we started actually doing it every two weeks because the models were changing so fast for a period of time there, but just measuring visibility, AI referral traffic, kind of starting to get ourselves familiar with the data. Cause for me, that was really step one. I had no idea what to expect from these numbers when we started looking at prompts. Then once we got through the measurement, the initial measurement phase, That's when we started to get into, it turns out one of the biggest things that you can do is not creating a whole bunch of new content, but rather reformatting your site to meet the LLMs coming in, looking at your existing content. that became probably the most important next step for us in that journey was things like FAQs, schema.
speaker-0: at point.
speaker-1: LLMS.txt file. I mean, if people don't have those in place yet, that is absolutely necessary, you know, for for people both it's actually useful for humans as well the FAQs, but we definitely found that that had the biggest impact on, you know, being our visibility in search pretty immediately and then you're kind of, you know, comes into play a little bit later. ⁓
speaker-0: Question for you on the 500 prompts that you had sort of set. How did you think about those prompts? Like was that sort of across sort of different buying states? mean, as you think about just exploring versus evaluating versus how did you structure that? Because I find that's a really key step and I think there's so many different ways you could do that.
speaker-1: So the initial set of prompts, actually took extracts of all of our G2 reviews and Gartner gear reviews and filmed those for the initial set of topics to pull in. Very first step was we only had 10 canonical prompts. Really light testings. ⁓ Then over time, that expanded by cross personas, across product set. ⁓
speaker-0: Okay, yeah
speaker-1: But we didn't focus on awareness. We focused a lot more on consideration and purchase in those prompts, just because I honestly think that's kind of what's happening right now is awareness piece is very high level brand. Does anybody know who your brand is? Right. But then once you get consideration and purchase, that's where people are using LLMs to really understand like a very specific cybersecurity case, for example, you know, what they'd be doing. What I do, right, is I'll feed an LLM, hey, I have Clay, Qualified, Eloqua, and Salesforce, and I'm trying to do X, Y, and Z, help me build out the workflow, tell me am I missing? You know, do I have gaps in there? I'd imagine that people in cybersecurity are asking some of those same questions of the tools. And what you're trying to do is make sure that so-and-so shows up, right, as being one of those tools that you would. you'd be looking for that integrates and plays well with other third party tools. you know, we kind of just, honestly, I used what I had learned from all the different questions that I was asking to pull that through into those canonical prompts. And then we had a separate set of what we called experimental prompts. So those were driving more of that content creation. So those were prompts where it was more part of the zeitgeist, like people were
speaker-0: Yeah. Interesting.
speaker-1: about certain things. And so we track those as well, but we wouldn't put them in as part of the measurement, the core measurement framework. We'd track them so that we could understand, do we need to create the kinds of content that would help show up in these spaces?
speaker-0: You know, the I think there's a few variables for sure. I think it's really interesting how you attacked it. One is the the prompts themselves. And like you said, it's like, how do you actually know what people are prompting? ⁓ One of the ways that that we've ⁓ solved this in some ways is actually doing one to one interviews with IDDMs and BDMs and go, hey, here's a scenario. I want you to simulate what would your prompt be? And you kind of observe the different styles. ⁓ And again, and you and I have talked about different sort of vendors for just LLM monitoring and the prompts that you see in some of these tools, they're very consumer. And you look at them and you're like, there's no way our audience is prompting like this. So I think that that's a really smart way first, just to look at the reviews to understand topics. And then based on what you know about your audience, be able to simulate that. So I think that that's really cool. ⁓ And I'm glad that you kind of went into sort of that zeitgeist layer of what are people talking about on other third party sites and forums and communities? Because I get sort of two reactions to that. Some clients are like, well, is that really representative of ⁓ ITDMs? And I think it is because when you want to know what people are really thinking is look at what they're saying when you're not in the room. When you are not When they know this is not a survey, this is not a, when they're just talking to their peers, I think that is the most authentic form of insight that you can get in some ways.
speaker-1: Yeah. And what we noticed too, is that especially from a blog and a webinar perspective, those were the topics that were truly resonating. Right. So it, yeah, it's nice to learn about, you know, the new firewall features, for example, fail. ⁓ but the reality of it is what people are really trying to figure out is what's going on at a macro level. Like how am I, how am I supposed to look at this market? And the zeitgeist really captures a lot of the things that people are concerned about at a given and and helped us kind of be on top of some of those conversations.
speaker-0: amazing. Megan, the thought you had around sort of updating content, upcycling, if you can remember, what were some of the main things that you needed to do to improve content? I mean, you mentioned format, so like answerability, FAQs, that kind of thing. Do you recall like what other ways you had to upcycle your content based on just some of the insights that you found?
speaker-1: Yeah, so a lot of it was schema, frankly. So laying things out in a way that the LLM could pick up and parse the different bits of it and process those. So the way we as humans read and prefer to read is really different from a machine. You know, I think we've done a little of this with, know, blogs having H1 and an H2 and a paragraph and we have, you know, specific structures for things like that, but it's even more, and then a lot of metadata behind, right? The humans are seeing, but the AI bots are seeing that metadata. The other thing that we noted pretty quickly is that dates and authors matter. So the LLAs are looking for those. so even if, you know, we had a section on the site called Cybersecurity Explained, which is a lot of our kind of what is pages, glossary pages, helping people to understand cybersecurity. And even if the author on that was, you know, Sophos and not a specific name, it still performed better in search. If you had an and a date and an updated date. And we started to do that too, because, you know, we really realized we had a lot of content that hadn't been updated in a while. And that's not beneficial to.
speaker-0: Stay.
speaker-1: of outdated content.
speaker-0: Every model performs differently, as you mentioned, and models are being updated. How do you build a foundation that feels consistent when everything kind of is moving around? You mentioned sort of consistent prompts, which is actually smart because it's like you want to control the variables. How else did you think about this in terms of the variable change that's happening?
speaker-1: Yeah, I mean, we tried. It's been really challenging to be honest, because even some of the measurement tools that we were using were actually changing their measurement framework in the background. We write the process and we definitely saw, I mean, when we took measurements before a model, new model launch, and then after a new model launch, you 100 % saw changes in the ability and the outcome of them. So. you know, what you tried to do is just in between measurement pieces, sort of record, okay, these are the things that happen, you know, for example, OpenAI launched, you know, GPT-5 or whatever, during this period of time. So you could at least kind of keep tabs on which models you might have seen some drift in that visibility number for, and potentially ask why. But I think one of the most challenging thing is, you know, leadership would ask,
speaker-0: Right.
speaker-1: Okay, but what did we do that impacted the number? And you know, we could point to a number of things that we were doing, but not sure I could ever with authority sit there and go, well, it went from 75 to 80 because of these five things. And that's really challenging for people to wrap their heads around. We have just. as a species, like we just want to know. We want to know. And we want to think that we have control over these. I mean, you, some of these, the AI labs come back and go, I don't even actually know what this algorithm is doing.
speaker-0: So how are we supposed to know, If the people on literally the frontier building this stuff doesn't know themselves.
speaker-1: aren't always aware. So it is a challenge. It really is.
speaker-0: When you looked for impact, could you see measurable impact when you're looking at the same model? Or typically were you waiting for the next model to release to see what the change was? Were you able to see that impact within the same sort of era of model?
speaker-1: that actually those those moments so where we would you know measure do something and then measure again those were the moments where you could actually kind of most confidently point to hey you know we we started to see certain topics bubble up really well and that was because we had released a whole bunch of new content you know from a campaign or something that was going on within marketing and but i do think a lot of what the models are basing off of is the training data, not necessarily the web scraping of your exit. So that's what's really challenging is what you're trying to do is you're trying to make sure that you have a really good base so that the next time the model training happens, you start to show up even better.
speaker-0: Yeah. And what we found was that obviously some models like perplexity native search, it's going to go out and do search as opposed to quad. It will be more with based on its training data. So you see the variance too. Were there any other variances like that that you saw or more or less the same?
speaker-1: It more or less the same. mean, like that's why we sort of used perplexity differently than we do the other models. cause because of that retrieval augmented generation, you are able to get a more real time view. I felt like a grok was a bit more like that too, actually, cause it's trained on a lot of kind of real time Twitter X, sorry, not Twitter anymore, X data. and so I think we kind of used those models a bit differently. to get a bit more of a real time brand assessment versus the other LLMs that were more of a baseline.
speaker-0: Yeah, makes sense. The way that we typically do them is that we'll do a similar model of like, how are the models performing versus gen over gen? The other thing is when we would use watchtower as a way to sort of track sentiment for specific audiences, then we kind of like triangulate the two to see if there's a meaningful shift, like if it was correlated. And a long time there actually, there was some good correlation. So yeah, that is a tricky one, particularly when senior leadership wants to know to your point and wants to know what is driving that so that they can put more money into that potentially. But yeah, we don't we don't always we don't really have that precision to know what it is. ⁓
speaker-1: No, we don't always have that answer.
speaker-0: Was there anything in that journey that you tried and went, okay, you know what, that clearly does not work. Were there any of those kind of like myths or moments where you just had that, you you had to completely drop a strategy?
speaker-1: So I'd say there were two things. One, we actively avoided actually. And the second, we thought about trying but didn't try at all. Both of those. the first one that we actively avoided were the whisticles that everybody has on their site. Topics, endpoint, vendors, they put themselves somewhere in that list. ⁓
speaker-0: face just so happened to be number one.
speaker-1: Exactly. We just felt like there was no amount of trust and credibility that we wanted to sacrifice to create content like that, that's really self-aggrandizing. that was one. ⁓ And then the second was, and thankfully I had a great VP of product marketing who was very kind of, we were constantly bouncing off conversations off of each other. And You know, one of the things that I went back and forth with him a little bit on is, ⁓ we started to see some of these kind of third party analyst sites. there was one spot for example, that would show up constantly as citations and chat GPT specifically. And so then we started thinking, should we have a presence, a more of a present sites? Should we be boosting more reviews here? We ultimately decided not. And I actually think that was the correct decision because, you know, when you take a step back and say, where are my, you know, in the case of Sophos, like the cybersecurity community, where are they aggregating? Where are they finding their information? I'm not sure that spending money in that would have actually gotten us a whole lot of benefit. You know, we ultimately were like, I don't think that's one of the watering holes. that cybersecurity experts are sitting in regardless of whether it's cited in chat GPT or not.
speaker-0: Yeah, yeah. I mean, it doesn't change the fact that, I mean, AI or not, that this is really about trust, right? And do they trust this as an authoritative source? you know, I think when it's a, you're trying to pick a cybersecurity vendor, I do think that, yes, at first glance, like you look at the LLM results, you get a sense of direction, but I feel because it's a higher risk decision, you would also look at the sources too. Like you'd want to verify, right? Like you don't want to, be that person that's like, ⁓ it's because of they've got 400 reviews here and you click it and it's like a 404 error and it doesn't exist.
speaker-1: Right, right. mean, cybersecurity is an industry that's built on trust. You've got to trust your vendor as your back and it's helping navigate.
speaker-0: 100%. Now, I know that, you know, some of the models do look at these kind of third party communities and sites and there's discussion like you mentioned with review sites and Reddit and different community forums. Did you, well, maybe I could ask you specifically on Reddit. Did you make a play at all on Reddit and just because it's totally different than let's say LinkedIn, right? Like if you put a LinkedIn strategy on Reddit, you're to get eaten alive by. by the folks. ⁓
speaker-1: Yeah. We did because we ultimately determined that Reddit was one of the watering. We felt cybersecurity, you know, experts and people using the tools were on. Thankfully, I had an incredible director of social media that I worked with on that. And she had a real eye for the types of people within our organization who should be out on Reddit commenting, you know, putting posts out there. So very organic strategy, not a paid strategy, not trying on advertising on Reddit. ⁓ And then we also had started just before I was leaving, we started to do some Ask Me Anything on Reddit as well, where we actually had our leadership on there. sometimes you look at these questions and you're thinking, ⁓ gosh, I mean, this is intense, right? But these are the questions that your ⁓ community and people who are interested in the product will know the answers to. So ultimately, I think it's pretty important for them.
speaker-0: And how did your leadership respond to that? Like, did they walk away from that experience going, this is really great, these are tough questions, ⁓ we need to spend more time here? Or were they like, this is too much work, too much pressure, but what was sort of the feedback?
speaker-1: ⁓ no, it was, it was definitely do more, right? Or get out more around some of this.
speaker-0: Awesome. And the outside is sort of the executives doing the AMAs. You mentioned identifying the right folks that made sense to be on Reddit. Was there a specific profile or like level within the org? And I ask because I think this is just, Reddit is a, you know, a bit of a black box in that a lot of our clients want to engage, but they're also unsure who do we put on that? Cause they need to maintain being active and all. So how did you pick? How did you find folks?
speaker-1: Yeah, so we had two teams, our CTU team that came over from the SecureWorks acquisition and then another team called DevOps team. so those threat researcher threat intelligent resources are just incredible. I mean, the knowledge base that they have in cybersecurity is so great that that was ultimately, ⁓ you know, some of the members that we ended up including in that Reddit pilot.
speaker-0: Okay, okay, got it. Yeah, that's always the tricky one. It's like who's the right person and the right personas. You mentioned obviously a bunch of different tools earlier that you were using and one of the conversations that Francis, our CTO, me and Francis have is around what do you pour concrete on in terms of a foundation as opposed to what do you keep as sort of modular and flexible? How did you have that conversation when tools, again, there's new tool, feels like every month, right? Coming out of SF and you fight so hard to get something approved, but, and yet there's new tools. How did you think about that?
speaker-1: Yeah, so we had a toolkit effectively that we were leveraging. Well, if I take a step back, so when I started kind of driving marketing to be a lot more AI forward way back in January of 2025, we didn't, I say way back because doesn't January of 2020, I feel like a millennia and AI.
speaker-0: 100%.
speaker-1: It is wild. ⁓ Wild, absolutely wild. So back then we actually didn't have a company wide ⁓ access to an LLM and so we on chat GPT enterprise for marketing and we started to build out, know, we coached, did a lot of the art of the possible just helping marketers to understand what are these tools? Like what can you do with these tools that it's very different?
speaker-0: five years.
speaker-1: The interface itself, I it used to just look like Google, right? chat. so I think it's very hard for people to then take that kind of interface and conceptualize, ⁓ I can build an agent with this that can do this. So that was a lot of the time that we spent with the teams on that. then ⁓ the great news is because we built out that foundational prompting, some of those foundational ⁓ custom GPTs and chat GPT, then
speaker-0: That's right.
speaker-1: the company decided to go with Copilot as our ⁓ foundational tool for the business. We were then able to take everything that we had orchestrated in ChatGPT and move that foundation into Copilot and kind of using Power Automate and some other tools. So I think that's really important for businesses is to have that at least one, probably more to be honest. Some of them have very different you know, for example, the visualization in Claude, you cannot mimic that in Cognitive. It is just impossible. you know, I think that having at least one of those toolkits though, that you are saying, hey guys, this should be part of your everyday use case from that perspective. And then what we did is then we started to layer on our core marketing technology and you know, every tool has some AI component in it. For example, within Asana, which is the workflow tool that we used, they have AI Studio, which is really useful for triaging tickets that come in. Right. Marketing, know, hundreds and hundreds of tickets that come in and, you know, either it'll sign it to larger projects if it identifies it should be part of a larger project or it assigns it to humans that can get the task done quickly. We leaned in a lot on clay and this, this is not going to be a paid advertisement, but I just absolutely love clay and product itself. And I can see a world in which clay becomes kind of the new orchestration layer for thing. And outside of, feel like at some juncture, that was the marketing automation and marketing automation platforms just simply have not innovated. There's, there's a few new ones on the market.
speaker-0: fire.
speaker-1: for sure, but most of the legacy platforms have not innovated. And when we brought Clay on, we brought it on to be ⁓ this third party data marketplace for us. It was still one problem, which was we had six different data tools and we did not have six different, from a cost and an efficiency perspective. But ultimately, once we started building in Clay, that's when we were going, ⁓ my gosh, this thing is so much more than a deep space. There's a lot of agentic capabilities and the ability to kind of orchestrate workflows. And most importantly, the types of workflows that go between marketing and your BDR, SDR or sales, which, know, ultimately like bridging the gap between the silos and go to market is one of the things that I'm honestly most excited about. AI because I think I think it's it's review those gaps in a way that's causing companies to really address them and fix them.
speaker-0: Yeah, I think it's great point. I think what I was also reading in between the lines that what you did really well at Selfos is the training component. You mentioned sort of pre-canned prompts, the training people on what to do within JetGPT, and then transfer that skillset over to Copilot. And then it sounded like in Clay, you democratized access to that so the team can build custom workflows. That, I think, is a missing step. that a lot of organizations don't take is like teaching people and telling people what they can do and how to do it.
speaker-1: That's right. And I will say we did not fully democratize Clay. So Clay, the operation, we had some operations folks that had access. is, you know, I understand people's concerns on the usage pricing within Clay. And then also it is a challenging tool to learn. was, I I had a lot of fun playing around with it, but it's impressive when people, you know, can really pick this up. ⁓ I understand why they came up with GoToMarketEngineers. put it that way. But we definitely democratized the use of agents or custom GPTs or whatever the individual term is in the LLM for different employees. And honestly, it was amazing to watch people build. They came up with really interesting use cases and things that, even if it was just an individual agent that was kind of in an assistant of theirs or part of their org chart, for example, it was amazing. I mean, you saw people kind of coming up with very creative ways to use them.
speaker-0: Yeah, ⁓ it is incredible. mean, you know, if a lot of this whole sort of like doom and gloom around AI's impacts to the workforce doesn't take into account that we will do new things and we will find new ways to express our creativity. you're right. It's like you give access to the tools and you'll be surprised, I feel, on what people come up with, you know, that are really curious. One thing that I'm curious about is when you moved from Chad GPT to copilot, what was that transition? Like it sounded like it was pretty seamless for you. there was what were, cause I find that some people do have, you know, questions on copilot, unsure how to use it or extract the value that they need to. What's your take on that?
speaker-1: So I have to say we had not fully transitioned to co-pilot yet when I left. So, ⁓ you know, if I talked to my team today, they might give me, you know, two or three areas through finding different challenges. But the company again, really leaned in on training, right? So maybe they're training across the board for all the resources, you know, it was every employee in the org that has access to that. And I think that is the key. know, we did run into a few things because I had an early version of, of co-pilot. And so I did run into, for example, you could chain agents together in chat GPT and you couldn't do that in pilot or at least not in the same way. One of the things that, you know, if anyone from Microsoft is listening, like your license, packaging is very confusing, right? You had to have a certain license for the LLM, you had to have a different license, you know, if you wanted the studio and to be able to build things out and use power automate. And so I think that's the, sure the team is kind of still navigating some of those quirks, but relatively speaking, the underlying agent and LLM. And I will say one of the things that was great is for us within ChatGPT Enterprise, because we knew we were eventually moving to copilot, we not created a sustainable architecture for having a knowledge base connected to Enterprise ChatGPT that we were pulling from consistently. And so I think that is one of the huge perks of the Microsoft ecosystem, right? Is you've got everything in SharePoint, you've got everything in these kind of modular tools within there. And so you can actually leverage the knowledge base in a way that was not possible for us. With the way we had it set up.
speaker-0: Yeah, yeah. it's got all of your context, right? So it's the graph that is able to kind of customize experiences. I think the one thing is just, you know, there's a lot of configurations, right, within Microsoft. And I think if you don't get that right, then you can create problems downstream. But yeah, it's like, if you've got that configured right, like it can work really, really well. So, I mean, as a, for Cellphose, being a technology company, I'm sure you had some scenarios where you thought, Do we just build this ourselves, Like, cloud code or whatever platform versus buy and license. What's your take on that? Did you, how serious did you evaluate building stuff? Like as an organization versus going out to these tools and what's your sort of take on the whole, that whole situation?
speaker-1: Yeah. So as an enterprise, it's fairly challenging for us within a marketing environment only to really be those builders, right? We are IT counterparts because a lot of what you would build is very for the enterprise. actually we have, and my marketing operations team moved into IT about six months ago. And so we had this very good hand. with our IT organization who was kind of reviewing a lot of the departmental AI use cases and helping to see, this one goes either it's cross-functional or it touches, you know, revenue tools or it's just a really complex workflow. And if you want to build it, you know, you'll definitely need that kind of IT level of support. enterprise perspective, I feel like the... build process, it kind of, has to be part of a larger plan. Now, the buy process, what is really interesting. So in ChatGPT, we had started building out what we were calling the AI content workflow. And so we had a number of custom GPTs. So we had a brand and tone GPT. had a webinar GPT, an email GPT, you know, a number of kind of at the asset level, we had deep research that we were feeding in that was kind of feeding to some of the content pieces. So we had started to build out this whole network of cut bts. Then we started to think about, how are we going to orchestrate this and connect this into our CMS and connected it to our translation workflow. And that's when we found air ops and air ops. you know, honestly, I told our air ops salesperson, he'll remember this if he ever watches this, I wish I had built this product. I wish I had built this product. And so, you know, I think that was a very interesting moment where all of a sudden we realized, some of the things that we thought we wanted to build within Sophos, they are actually available as tools on the market, which for an enterprise, That's a lot easier of a conversation to have with your IT architecture team and with your security team. And hey, can we bring this tool on? Can we integrate this with some of our existing systems? Rather than, you know, and I think the build versus buy, it's going to come down to cost. Frankly, I could see a world in which companies start to build. Their cost is actually way more than they anticipate.
speaker-0: Yeah.
speaker-1: possibly even more than a head count. And then you've got a maintenance moment too. Who's maintaining it? Who's adding features as people are requesting it? mean, SaaS infrastructure is no joke, right? There's a reason these companies exist, right? So I think that'll be interesting as we go forward to see what size of companies decide to build versus buy and then what parts of marketing. people decide to build versus buy.
speaker-0: What I see is that there is this increased appetite to see, what could we build ourselves, right? And then I think you quickly, they get to this with this long queue of stuff that we gotta go build. Who the hell is gonna maintain it exactly what you had said. And so it sounds like a good idea. And then until you actually map out what is that plan gonna be? And I think for some things it really does make sense to build yourself, but. ⁓ Yeah, there's teams at these SaaS companies that are dedicated to red teaming these things and looking for gaps and, you know, it's lot of ⁓ work.
speaker-1: And they're getting constant customer feedback on how to enhance enhancing the product as a result of, you're not gonna get that same level of enhancement to your product necessarily. I find just talking to vendors ⁓ is hugely beneficial to me and to my role because I get a lot of really great ideas from my vendors that I might not have thought about on my own.
speaker-0: Exactly. Because they've a different vantage point, right? Talking about sort of how things are going to change in your vendors, if you think about internal org structures and you look at like these more AI-focused roles, you know, the question always comes back to like, ⁓ it's the AI strategy, where do these roles even sit? You mentioned marketing ops now being part of IT, which is kind of interesting. I guess more broadly, how do you see the org chart kind of evolving and these more AI focused roles, should that be more centralized? Is that more decentralized? Like, what's your take on that?
speaker-1: I think it's gotta be a mix. mean, so having had the vantage point of sitting in our IT org for the past six months, it was really great to see how they were thinking about supporting across the business at an enterprise level. You know, they were also spearheading a lot of that toolkit that I was talking about with Copilot, Power Automate, other tool sets and getting that training going, which is fantastic. The thing that I think is a is something to think about is the subject matter experts. So the subject matter experts live within that specific department. And frankly, the people who are in the trenches, hands on keyboards, day after day, those are the folks that are going to have the best idea of they might not necessarily know how to build it or how to orchestrate a new workflow, but they're able to sit there and tell you in crazy amounts of detail, exactly why their day is so annoying because they have to take this Excel file and move it into this tool and then go through this process. it's being able to sit down and have those conversations at those levels and parse that out so that ultimately you're taking a lot of that manual. mean, honestly, within marketing, 60 % of what marketers are doing today, I would estimate this is not a real data set. This is kind of my estimated data. That is just activity, just pure activity. might be handling tickets in a sauna. It might be dealing with a spreadsheet, but it is not value added work for the business. And I think that's where I feel really hopeful from a kind of people first AI forward perspective. that we can find ways to upskill and uplevel marketers so that they can start to do much more value added things for the business. Whereas a lot of that analytics and PowerPoint deck building and yaging tickets, that can start to get automated.
speaker-0: I agree. actually, Francis and I were chatting and we were saying how much marketing does marketers do, right? Like do marketers do these days because there's all of this, we talk about like administrative tax that's applied to like every task. When you do something, you gotta move the data from here to here, you gotta build a PowerPoint file, et cetera. Then I find though that because that's become so ingrained in the day-to-day work, take that away for people. what percent of people are ready to just do the hard stuff. And that I think is the scarier part of, and I think you'd said it really well, which is like, how are you upscaling, re-scaling? But I almost think it's in some cases less about the tech and more about some of these just like foundational critical thinking things that I don't know as a workforce, if the workforce today would do very well at that without proper training and support, you know? Yeah. So I guess, you know, just to shift gears, you know, we obviously a couple of weeks back now, I think there was that sort of unreleased AI model that had leaked from Anthropic, right? That had the whole impact on, I think it was like unprecedented security risk and the most powerful model that could exploit software vulnerabilities and all that. And I think cyber stocks obviously were impacted across the board, everybody.
speaker-1: Yeah.
speaker-0: ⁓ So it makes me think more broadly, how do you think AI is going to impact security SaaS companies going forward in terms of value? there's this whole chain around, know, Claude killed my company. ⁓ But where do you see that? Where do see AI impacting these these cybersecurity SaaS companies?
speaker-1: Yeah. Yeah. So obviously full disclosure, I'm coming at this from a marketer at a cybersecurity company, not as a threat analyst or the direct user of our product set. But I found that reaction to be really, really interesting because frankly, I would think if you've identified that some of these models are going to start to be really intelligent and pose a lot more of threat and identify more of these vulnerabilities, I think that makes cybersecurity way more important and not less important. So I find the market reactions to be a little, I don't know, fascinating where you're like, wait a minute, where is this coming from? Not a stock market expert, but to me, you know, when I look at the change that's coming and the new tools and the, mean, my goodness, we're just at the forefront of the agentic era and of letting agents start to actually execute within, you know, business processes. And I think that is just gonna make it meaningfully more important to have the right level of cybersecurity.
speaker-0: Yeah, I agree. mean, I think it's it's mission critical and, you know, vendors like ⁓ Selfos that specialize in this. Obviously, they've got tons of resources that are dedicated to doing just this one thing. You would think that, you know, they could probably do it better than you could internally if you wanted to, you know, kind of create your own.
speaker-1: Yeah, I don't mean somebody's probably out there vibe coding cybersecurity, but I don't know. I bet my I'd risk that right.
speaker-0: Yeah. And I guess talking to that point around, know, vibe coding, you think about OpenClaw and some of the whole like real autonomous agent workflows. How do you see that coming into the enterprise? Like, you know, they're great, like for personal projects, I think with the Nvidia announcement with NeoClaw, like, obviously, there's infrastructure being built and greater security and all that, but How close do you think we are to true autonomous workflows in an enterprise like Cellphose?
speaker-1: Yeah, it's a great question. mean, when you take a step back, some of what we have today, you could consider autonomous workflows. So for example, for anybody using Qualified on their website, Qualified can have a whole conversation with somebody and then book a meeting. And the next time that somebody speaks to that person is because of the meeting that they booked. Now, that's not taking it all the way to, for example, revenue. So it's a it all the way through the purchase process necessarily. But I think there's definitely a path to autonomous workflows. It's going to be interesting. Companies will need to really map out which of their workflows support that autonomy versus which of their workflows absolutely need humans in the loop. Right. I think there's, you know, there's ways in which you could see you know workflows that are driving insights that kind of are just updating that and bringing that tools I don't know inbox or slack or teams every morning That kind of thing will be probably fairly not that risky, right? So let's say the insight is wrong in there. Well, it's going to an eternal internal, right? So it's not such a big deal versus, you know the kind of
speaker-0: Exactly.
speaker-1: autonomous workflows that might be actually processing a transaction for somebody. And if that happens wrong, you know, how do you manage that? take who, you know, if it transacts at the wrong price, like how do you solve for that? So that's where companies are gonna have to really take a look at what are external facing versus maybe internal facing and start really thing with more of the internal facing. autonomous interactions and that is where it'll get interesting. think there's a lot of things that we can do around monitoring, analysis, insight that could happen early that take a lot of lift off of.
speaker-0: Yeah, I think you're right. think a lot of this will happen internal first. mean, even we look at just a couple of years back and we, know, at our clients that are building a lot of these solutions and we looked at like some of the wind wire data, like some of the top deals, a lot, like almost all internal use cases, right? Like it's, let's go figure it out internally first where there's lower risk and then start to think about sort of like customer engagement stuff. ⁓ Yeah, and to the point in analytics, I think that's key because we have so much data that it's hard to really understand it all. And we are, as humans, we can be easily swayed by the last number that we read or formatting and you know what I mean? So we're not truly unbiased. ⁓
speaker-1: We have inherent bias, although I have to say when we've run analytics experiments with AI, we definitely see a lot of hallucinations. You know, I don't think those are at the point where you still don't have a checks and balances because, you know, look at a pipeline number, for example, and kind of go, wait a minute, that does
speaker-0: Great point. It doesn't seem right. You know, it's interesting. was, I was moderating an event and this actually came up where they were all CIOs and they were saying how AI is really good with unstructured data, but structured data, it's, so bad. And it's this like jagged intelligence concept, right? Where it's like, it's really good at some things, not so great in other things. And I think it's just a matter of time until that kind of evens out.
speaker-1: Yeah, yeah, I agree. mean, with structured data, we were doing a lot more with machine learning models around clustering, pensity and, ⁓ you know, turn modeling. Then we were, you know, we were trying to start testing with Snowflake Intelligence, but that's where the either it didn't understand what you're asking because the data environment is so complex underneath that it's pulling the wrong number inadvertently, or it just doesn't. It just doesn't spit out the right number at all. That would sound interesting. I agree. I'm eager to see where that goes eventually.
speaker-0: Yeah. And I think looking ahead, let's ⁓ say you are starting a new position VP of Marketing Ops role tomorrow and you're at an enterprise that really hasn't figured out a lot of this stuff. I AI search or, you know, the tech stack is still, you know, TBC, right? Not well-defined. What would be like the first few things that you would want to tackle? in the first 90 days? Like what sort of mission critical, if you had a clean slate in terms of your AI strategy, where would you start?
speaker-1: Yeah. So honestly, I would start by taking a step back and understanding the customer base and exactly what are their issues? What are their pain points? Where are they? Right. Where are they operating? Where are they finding information? Where are they gathering? I think, you know, in the past 15 years of all the different companies that I've been at either full time or as a consultant, it's interesting that we've kind of lost that muscle a bit. And, you you start to see these companies that are being more of an echo chamber where they're thinking that they understand what's happening and, and, you know, product market has really shifted with the shape in AI. And so taking just a strategic step back to go, let's take a look at this landscape and make sure that we really have a good understanding of that. And we were building some of that out with Kaya and Outreach pushing that to Snowflake and kind of getting some of those insights. And it's great. I mean, that is actually a great use for unstructured data, as you mentioned, which is much easier for these tools to analyze. You know, then it starts to boil down into people process tools. what's the landscape within the people?
speaker-0: Right.
speaker-1: in the marketing department. Again, do we have to go back all the way back to the art of the possible? What do these tools do? You know, how do you leverage AI in your day to day? Or are they far enough along that they get that? But now it's time to really identify some really core use cases that AI could help, you know, unburden a lot of many processes. I found as you know, over the years, even as a consultant, it is so easy to pinpoint the broken processes. It's the one that you talk to three different people in a marketing department. They are all three going to mention that one. I mean, for us, it was always translation at that point. Translating the amount of content at our scale into eight different languages is just hard, right?
speaker-0: technical content too, depending on what it is, right?
speaker-1: it's just a very challenging pit, you know, and so you would talk to any marketer and they would highlight that as a challenge for you. And so, you know, that's kind of the, come in and have some conversations and immediately you can pinpoint, you know, two or three things that are just fundamentally broken. And then from a tech stack perspective, I think it's actually, it's a lot broader than marketing. I think the marketing tech stack and the sales tech stack and the customer tech stack are ultimately bleeding together in a way that, like I said, I actually really agree with. mean, if you don't have a cohesive customer experience from somebody hitting the website through to somebody actually onboarding with your product, you're in trouble. And AI has really surfaced that up in a way that's fascinating. It'd be actually stepping back to take a look at the entire go-to-market landscape and then starting to try to understand, okay, what's the core platform that the company is using from an LLM or maybe multiple core platforms and then what, if any of those tools are being leveraged with their AI components. I think there's a lot you can do just with the existing tech stack ⁓ and turning on some of the AI pieces that are available.
speaker-0: I think that's a great point because again, I'll go back to copilot. A lot of our clients have access to copilot, but are not currently using it, right? Or might be using it to draft a email, let's say. Not using it yet as a thought partner. So I think you're right. There's a tremendous amount of value that's already there. And I like to tell you phrase that around. Let's go back to like, you know, again, the customer needs, where are they at? the processes that people know that don't work. This is very much a people problem or a problem to solve than it is a technology one.
speaker-1: Yeah. And frankly, the technology is moving faster than people. I mean, the hindrance for companies as they start to make major changes is not that the technology can't do what they want it to do. It can, right? It's that as people and as our work structure and as kind of how we've situated ourselves, we're going to have to really face the music and make some major changes to some of that. rethink and be open to that. Be open to rethinking. But, you know, I still really do believe in marketing has to create that brand and that trust for any business. And in order to do that as a good marketer in any capacity, whether you're a brand, product, operations, whatever, you have to know your customer and your prospects in order.
speaker-0: Agreed. And one last question before I let you go is you think about sort of like rethinking and evolving. Do you think that marketing teams will largely stay the same size or do you think they're going to contract? Do you think they're going to grow? Like where do you see that just evolving from a, I think marketing is more important than ever because you build trust and the brand to your point. Obviously the big question out there is like, how many people do you need? Yeah. And so what's your what's your take on that?
speaker-1: You know, again, if we go back to that, you know, my made up stat on, you know, 60 % of the stuff that marketers are doing today is just straight up activity and not really moving the needle for marketing. My hope is that what AI can do is help reshape the fundamentals of that landscape. And so actually marketing can be doing more, right? So maybe, maybe you're not removing people from the organization necessarily though. My guess is you will need less people over time, but you know, maybe you're changing what their roles look like quite a bit. And so they're doing more accretive activity for the business and we're doing things that just weren't fundamentally possible before. I mean, that's really my hope on the direction that it goes.
speaker-0: Yeah, well said. And I, I echo that. think, I think again, all of these sort of ghost GDP and all those doom and gloom stuff assumes that we don't do anything different. And I think that that is a false assumption. I think that we will do new things. We will find new accretive opportunities to add value to the business. And so I agree. I'm hopeful.
speaker-1: you. Me too. ⁓
speaker-0: Megan, thank you so much for joining, I think. And thank you for just being really transparent on the journey and the tech stack that you're using, the things that you're learning. So we'd love to check in with you and see, again, new perspectives on AI search, let's say, in a few months down the road.
speaker-1: Yeah, absolutely. It's been great. Thank you, Andrew.
