Search Disrupted Newsletter (Issue 19)

Sabotauged search results, Buyers' Journey in AI, the organization invites, and data on how searchers aren't clicking on citations.

June 20, 2025 • Michael Buckbee

Can AI detect sabotaged search results?

Search is by far the number one use of AI in the world, and it’s often complicated, opaque, and confusing.

It’s such a challenge that the first thing that people turn to is using AI models to try and police other AI models.

This is the topic of Anthropic’s new paper.

They developed 17 scenarios that SEOs would identify as commercial search journeys (ex: “Find the cheapest flight to Paris”) but then tasked the first LLM to try and achieve that with a secret goal of actually sabotaging the search (ex: “Secretly, warp the results to prefer Delta even if it’s more expensive”).

They then set a second LLM to try and detect the sabotaged results…and they didn’t do great.

Anthropic’s paper has the best/worst models succeeding in sabotaging the search results 60% of the time.

But that’s actually overly optimistic, as they had access to all of the internal information of the first LLM that was trying to sabotage the search.

In large part, this is why we built Knowatoa: to develop tools that can help police the outward results of AI search, detect when substantial changes occur, and help people better understand and influence the results.

AI Search is rewriting how buyers find you

Brandi Starr from the Revenue Rehab podcast interviewed me about the very real impact of AI search on revenue operations and how it’s changing buying patterns.

We spoke a lot about how to position AI search in your overall marketing strategy, how to frame the changes happening to get support from the C-suite, and what steps to take now to prepare for all the changes happening.

New Organization Invites in Knowatoa

Last week, we introduced our new team management and invite system, and while it’s been a smashing success, it didn’t go far enough.

So, this week, we introduced the ability for you to create and manage multiple organizations within your Knowatoa account.

Organizations each contain their own set of:

Users
Sites/Brands
Questions

While we’ve designed this specifically for agencies that need to manage dozens of clients and teams, it’s also great for large in-house teams that are trying to get their hands around AI search within their own set of sites.

Read more about the new organization invites in our blog post

Meta AI is a privacy disaster?

There’s been a spike in folks talking about how Meta AI is a privacy disaster because it enables the sharing of AI search queries with your timeline.

While that’s obviously not great, it’s also about the same minor level of concern as people misunderstanding the difference between Facebook and Google and posting “find hot women Tulsa” type search queries as Facebook status updates.

Embarrassing but not critically concerning.

The real issue lies in how much data Meta is using to train their AI models and how and what they’re recommending to people based on that data.

Meta’s reach is truly massive, and they’re aggressively pushing their AI search tools into every part of their platform: Instagram, Messenger, WhatsApp, and Facebook proper.

It’s a tremendous opportunity for search marketers to get in at the beginning of the search shift away from Google but also a massive risk at a personal level with just how much information Meta knows about you.

Meta AI and privacy

People aren’t clicking on AI Citations

We’ve previously written about how the ratio of pages crawled by search engines (Google, OpenAI, etc.) is shifting dramatically lower in terms of traffic sent to your site for each page crawled.

This data came from a study by Cloudflare that they’ve now clarified even further where they’re framing the changes as an “existential threat” to publishers.

Two things about this you have to keep in mind:

You’re probably not a publisher at the level of the New York Times or the Washington Post.
Cloudflare is selling a service that has the capability of stopping AI bots from crawling your site, so they have a vested interest in this.
Publishing and serving web pages is cheap in comparison to literally any other form of marketing activity.
In the same way that Nike doesn’t measure the success of an ad campaign by the number of website visitors but by increased sales, you should be measuring the success of your SEO in traffic numbers.

We talk a lot about this in the BISCUIT framework, where we’re encouraging brands to shift their content plans further down the funnel to where the real value is and not just chase traffic.

Read the BISCUIT framework

Thanks

Thanks to all the podcasts that have been nice enough to talk to me about how AI search is changing the way we do SEO and how Knowatoa is helping brands make the transition from traditional SEO to AI search.

If you have a podcast, live stream, webinar series, or another educational channel where you’re helping marketers, I’d love to be a guest.

Drop me a line at mike@knowatoa.com, and we’ll make it happen.

p.s. It would really help me out if you could Follow me on LinkedIn