bonus rant #4: the new padded room of the internet

For nearly 20 years, I’ve basically been Chronically Online which, when coupled with a love of e-commerce-flavored-dopamine, means that I am in an advertising behavior demographic I like to imagine is called “Shit She Doesn’t Need But Is Convinced For At Least A Week That It’s Probably the Solution to Her Life She’s Been Looking For.”

Combine these things with the rapid progression of age and a life mostly spent sitting (working at a desk, sitting in bed watching TV, sitting on trains and planes and in cars, even sitting to exercise on a rowing machine), and the ads I’m getting are things like the Callixe Thera-Pillow, promised to be a 3-in-1 neck massager with heat, vibration, and traction therapy. Naturally, this product was probably the solution to my life (read: back pain) I’ve been looking for.

But reader, I am no fool. I am no novice. I verify! With reviews! So I search for the brand and the product and, after a few marketplace and brand links on the search engine results page (also called a SERP - this will be important later), I arrive at several review pages. I open 10 of them successively and dig into the first. This review is promising, from a chiropractic office. Then the second…which…is looking an awful lot like the first. The domain name was different, the reviewer was supposedly a travel blogger, but the visual structure of the site and its blog was identical to the first, with the exception of different colors. So I go to the third. Then the fourth. And the fifth. And link after link has identical-looking favicons, differing only in letter, but different domains, different headlines, supposedly different content, but all the exact same webpage style, structure, visuals. 

It dawned on me that this SERP was full of exclusively AI-generated content, likely initiated by the brand to pad their results with artificially generated reviews. NOT reviews by actual consumers who had tried the actual product, but by robots fed a list of product benefits expressly to churn out these results for people looking to find reviews of the product. 

SURELY THIS IS NOT THE INTERNET THAT I SIGNED UP FOR SO LONG AGO. 

While most things about technology anger me these days, this was next-level. If I can’t even find another human’s opinion of a product,¹ because ANY opinions at all are completely fucking buried by AI-generated results…what fucking good is this place?

So when I read an Instagram post, only weeks after this incident, sharing that Google announced it would move rapidly to favoring AI results in its SERPs, I was horrified. I’m imagining just…pages and pages and pages of AI-generated content, feeling my stomach drop to my butt, wondering how the fuck I’ll ever be able to search for reliable information again.

Well, that’s not exactly what Google announced. It’s also why you shouldn’t just listen to Instagram posts, and instead wait for people who know WTF they’re talking about (or who researched literally fucking endlessly like I did for this piece) to outline the facts. This is a long overview; after punting this piece last week to the next week, then from that week to this week, you can imagine I’ve got a lot to explain. Though I may be biased, I truly do believe that what I’m sharing here is important, and if you’re interested in not just why this is happening and where it’s all leading, but also why you SHOULD care (even if you don’t currently), I’ve broken it all down for you here. I also know that my readership is mixed—some folks understand some of the inner workings and variables at play within Google, and for some, it will be very new but VERY critical information. As such, I’ve made a little clickable index/table of contents for you. It’s long AND I think it’s all worth reading; I think the setup is particularly important. But click through and read them almost like separate lil blogs as you need/want:

Lil Index/ToC

So…what the fuck IS happening with Google and its new search announcement?

Many of you have likely already seen the changes to your Google Search (product name) experience. However, in my sleuthing, I found that the changes had NOT been rolled out to ALL my accounts, so it’s possible that you haven’t seen it yet and may soon.

In May 2026, Google made an announcement that they were going to change the way that users engaged with Google Search. In their promo video, it all looks…pretty slick. That’s what promo videos are designed to do. The idea would be that, while “AI Mode” had previously been a separate button you had to push to engage, and though Gemini (Google’s AI product name) “AI Overviews” have been in effect for the last year on most searches, they would be weaving AI deeper into the search experience. 

If you’ve done a Google search ON Google in the past year (not on another search engine, which is different from a browser, like I covered last week), you’ve likely seen the AI Overviews. Essentially, Gemini crawls Google’s own search engine results to your query and summarizes the salient details for you. Early on, the results were…not great (more on that later). The AI Overviews have improved over time, including link citations so that you can not only read the AI Overview, but also see where the fuck in its ass Google pulled that information from, so it’s not just a “trust me, bro” situation. 

In the new version of Google Search, those AI Overviews do still exist. They will also include, at the bottom of the AI Overview, a chatbox and prompt for you to type in if, say, you read the AI Overview, but you still have questions. This does not run another search, to be clear. Instead, it engages Gemini in an AI agent fashion, just like engaging with ChatGPT or Claude AI. You type in the question or prompt or search query, and the AI “agent” will respond to you conversationally. This part is new and used to only be engaged if you actively clicked on the “AI Mode” button. 

The other part that is new is that, while the AI Overviews will still summarize Google’s Favorite Parts of the Internet (again, more on this later), the link citations will heavily favor Google’s own AI. What does that mean? It doesn’t mean a SERP of AI-generated content links, like in my neck massager example. What it DOES mean is that the link citations will actually favor Google’s FavoritEST Parts of the Internet: The Parts They Own. Favored citation destinations will include YouTube, a Google-owned property, as well as…other Google searches.²

Now, you might be asking yourself, what exactly does that mean? And that would be a good question, because it’s fucking stupid. Essentially, when you type in a query like, say, “best back pillows”, and Gemini provides an AI Overview with link citations, those citations will favor…another Google search. As you can see in the image below, on the right margin are the citations for the query “best back pillows.” Casper is at the top, shilling their own, of course. But then the next two citations have the Google logo/favicon next to them. In this case, I can click on one of two links to products that aren’t owned by Google, but are likely sold on Google Shopping. 

But the important part here is that those links essentially function as another Google search query. So when you click the citation for more information, you’re not getting the answers so much as Google is answering your question with another question so they can run another search for you to (fingers crossed) find the answers you were already looking for there.

And we all know just how much we love it when people answer a question with a question, especially when they know the answer and just…don’t wanna tell you.

Another example from this piece in WIRED detailed a search for Oscars 2026 ceremony, in which the AI Overview provided citations that were just…links to other Google search results for individual movies like One Battle After Another. So instead of providing link citations with the answers to the questions like, say, the IMDB page for One Battle After Another or a Variety article on Oscar contender announcements, it’s sending you to a SERP as though you asked it another question. The point is that, while a citation should normally point you to the source material that answers your question, Google is instead pointing you to another search engine results page for a different question. Your next step is to go to more Google search results instead of just…getting you to the page that is gonna have the answer.

(Honestly, it kind of reminds me of high school Student Congress in North Dakota, where there was one kid I competed against constantly who would only cite his own dad or himself as sources in his speeches. You might be tempted to believe his dad was an expert in the thing the speech was about, and you would be forgiven for believing that, because that would make sense, but actually his dad was not an expert in anything we debated and he himself was just a very average 16-year-old boy exhibiting early and strong signs of confirmation bias.)

One thing you learn quickly if you work in User Experience fields is that people don’t generally like it when you add MORE steps to a process. Where you can, eliminating steps is considered best practice. So if you find yourself puzzled at why Google would add more steps and make it more complicated to find answers to your questions, if you are confused at how a behemoth of a company with more user data than God seems to be intentionally obfuscating a pretty basic, cut-and-dry process and adding more circularity, if you’re scratching your head at the flagrance of Google’s self-promotion after being found in court to be a monopoly that requires busting up…you’re not alone.

So why the fuck are they doing this then? What do they stand to gain?

Likely none of my readers will be shocked that the short answer here is: money. But let’s unpack it a little more. If you are already VERY familiar with the inner workings of digital advertising (like…you’re a paid ads professional, not that you’ve ever run an ad), you can skip to this section here. If you don’t know the ins and outs of how the digital advertising sausage is ACTUALLY made, I’m recommending you read this next part. I think it’s important for ALL people to understand exactly how their eyeballs are monetized.

Google originally had Search as its only real product. Then after expanding into advertising with Google AdWords (later Google Ads), they had their real money-maker. Google Search, as a product, is only really profitable because it exists as glorified digital billboard space. Google Ads can sell its own algorithm-recommended/ranked search results as advertising space AND it can push ads to websites that opt in to essentially loan their digital space out to Google. This is why you see ads on so many websites that aren’t Google–the advertisers likely are not paying the recipe blogger directly or the tech news publisher directly. They are likely paying Google, who “rents” this space on these websites and pays the websites just a lil bit of what the advertisers pay Google, specifically. This is called the Google Display Network.

For the advertising-uninitiated (what is it like to know peace), there are two ways that companies pay for ads: the CPC and CPM models. The CPC model stands for Cost Per Click–this means that the company only pays Google if someone actually clicks on their ad. CPM stands for Cost Per Mille, which effectively translates to Cost Per 1,000 Impressions—an advertiser pays for each “impression,” which is any time that the ad is seen. So if the ad is shown to three people, but one person sees it twice, that is four impressions. When it comes to advertising search products, Google Search ads (the ones on the actual SERPs) are CPC models, so advertisers only pay if the links are clicked. Google Display Network (GDN) are the image/video ads you see on other websites; GDN ads are CPM models, so advertisers pay for the ads to simply be seen, whether or not they are clicked. 

Advertisers benefit from CPC models more than CPM models, because a click is supposed to indicate at least a modicum of interest in the ad, product, or brand. A click could turn into a sale. And since only 4-6% of all impressions, on average, actually turn into clicks, it’s far more cost-efficient for an advertiser to pay for 4 clicks vs 100 impressions. Google, however, benefits from both models, but benefits from a CPM model more than the advertiser does. Why? Because they are charging based on the amount of times the ad is seen–not considered, not clicked, but seen. This is the equivalent of an advertiser paying for every time a billboard is passed on the interstate, whether or not the person passing it even noticed. It’s not a good model for the advertiser, because they’re paying for a lot of people who don’t give a shit. 

Google Search ad campaigns are CPC models. Google Display campaigns are CPM models. Now, I just told you that Google benefits from CPM model more than the advertiser, but because impressions generally are a lower price than clicks, Google actually makes more money off of Google Search ads on a CPC model, because clicks, though fewer and further between, are paid at a higher premium than impressions. This part is important.

To add to your education (and I promise this part is important as we look to the future), Google also launched, in the last few years, an ad product called Performance Max (PMAX) that promises to be your advertising rocket ship to results. PMAX users are likely (or at least, were likely when it first came out) to tell you that the hype was real - PMAX delivered better results than any of their other campaigns. The trade-off, however (and this is important), is that you have no idea what PMAX actually is, what it actually does, how it actually works, or what’s actually being delivered on your ads.³

Why do we give a fuck? Because while Google’s reporting appears to show better results and, indeed, many PMAX users report better ROI on PMAX ads, you have next-to-no understanding as an advertiser how it works. Which also means that how it works can change at any point in time, and you’ll also never know that in any verifiable way. 

With PMAX, the general understanding for an advertiser reading a report is that if they paid for a click, it was a Search ad. If they paid for an impression, it was a Display ad. But again, and most importantly: This is not something they can verify, because PMAX is the blackest of black-box ad delivery methods on the market. There is no way to know for sure what you are paying for. 

PMAX is the ultimate “trust me, bro” product, and Google has gotten away with it because, from what advertisers can see, it works. How do they know? They look at their ad reports, then they look at their sales data, and they can infer conclusions. They’ve also gotten away with it by helping to progressively “solve” the problem they themselves created, giving SLIGHTLY more insights to advertisers over the years…just enough to make them feel like they finally have SOME information so they stop noticing how little they ACTUALLY have. Google justifies a lack of transparency with an “I got you, bro” and you forfeit visibility and control for results. Every ad platform, especially with AI models running on them, tries to push advertisers this direction, promising that if you give up control, they’ll deliver for you. (As an advertiser with decades of experience, I can promise you that giving up control in this case is never the correct general strategic choice.)

Now let’s pull this all together. Google’s AI Overviews naturally stand to reduce ad inventory, or the space available for rent, but we’ll talk more about that later. By directing you to another of their search engine results pages in their AI Overviews, they are essentially forcing you to do another search. Another search means more opportunity to serve up Google Search ads, their top moneymaker. And that means more ad impressions AND more opportunities to click on ads, which is how they make their money.

But remember PMAX, that ol’ black box? And how you can’t really tell what you’re paying for? And you don’t really know how it works so you won’t really know if it changes? Something I didn’t mention earlier is that advertisers are, more and more often, being cornered into using PMAX as their ad delivery method instead of simply building their own Search or Display ads manually. Why? Because it makes Google more money. My tin foil hat hunch is that we will see a day very soon (if it’s not already in motion) where Google shifts to a CPM default model on Search ads that becomes a CPC model if the user clicks. More simply put, you’ll pay for an impression no matter what, unless someone clicks, and then you’ll pay for the click. 

And you (or advertisers) either won’t know or won’t be able to control it.

If this happens (and I have a pretty decent prediction track record for companies setting the stage to pull really heinous bullshit), it means that directing users to another SERP will make them more money no matter what, because they’ll collect on both the impression and the click. And they’ll also make more money sending people to YouTube, since they deliver ads there as well.

Oh, and YouTube ads are paid for every time someone views them past a specific time threshold. So…more money there, for sure.

How Google’s shaky relationship with the truth affects us all

Ultimately, all of this is how Google sells the “inevitability” narrative of AI to the world. First to advertisers, so they can make more money, then to the press and to the rest of us, to push us toward using their AI products (that they will eventually charge individual users for once they have formed & established both user behavior and reliance.)

Google has said that AI Mode in Search has been “wildly successful” already and that people use Google Search more frequently because of AI Mode. But…duh? If, when a user does a search, you direct them to…more searches…you’ve artificially manufactured an environment where the user “uses Google Search more frequently because of AI Mode.” That’s not user choice. That’s simply a known impact of limiting their options and directing them to another search.

Additionally, you should exercise a HEALTHY amount of skepticism when reading any major corporation’s absolutely GLOWING reports on AI use and adoption, but especially Google. They say that AI Mode currently boasts 1 BILLION monthly users. But the follow-up question should always be, “How did you get those numbers?” As in, what are you counting as a monthly user? How many of them are new, as in just trying it out once and never again, vs recurring? Are AI Overviews counted as “AI Mode”, so anyone who clicks to expand an AI Overview is now suddenly considered a part of this BILLION monthly users? 

Most reports put out by AI-favoring corporations that cite widespread use and adoption also conveniently leave out the way they force adoption. As in, if Microsoft Copilot is forcibly installed on all the computers at a new company because of their license with Microsoft…does that count as “adoption”, simply because it exists on a computer, without ever being used? They never share their methodology, and with good reason: The data analyst in me needs you to know that companies are always, always, always padding these numbers. They count things that could not, to any reasonable person, be counted. Why? Because it impresses investors, it makes the stock market optimistic on them so their stock price increases, and it tells the press a story they can sell. 

Like when the BBC says that 1 BILLION people “use AI chatbots regularly,” you’ll notice that they don’t cite the original source. They just say it. And that number sounds an awful lot like the one Google announced about their own AI Mode users, which should lead us to a tentative conclusion that the BBC is simply parroting Google’s own words, but shifting them slightly. And whether or not they want to own it, that slight shift is critical. It goes from “one billion people use GOOGLE’S AI Mode monthly” to “one billion people use AI chatbots monthly”--this slight tweak in language makes people bullish on AI and reinforces the “inevitability narrative” that the Accelerationists work so hard to push. 

By not interrogating the source material claims or the methodology at all, press like this contributes to a warped understanding of both our present and our future dances with world-altering technology. Messaging shapes behavior. But this is part of why I am so persistent on not just calling this out, but EXPLAINING THIS ECOSYSTEM AD NAUSEAM, because we NEED to examine that messaging (all of it) critically. To include the question: What, exactly, does it mean for something to be “wildly successful?”

When Google says that their AI is “wildly successful,” what is the metric by which this is concluded? Wildly successful for whom? Because it doesn’t feel like a success for the end-user, who ends up in Dante’s Seventh Circle of Search Circularity. Is “wild success” simply about the volume of information that is spit out (more search engine results) and not the quality of that information? (Signs point to yes.)

“Using Google to do more searches” is not the same as success or a positive outcome; even if it’s both of those things for Google, it certainly does not mean success for the rest of us. We shouldn’t let the broad strokes of that messaging be a blanket we even reluctantly wrap ourselves in. Regardless of the changes, just as Google’s algorithm has always allowed us to find answers to our questions faster, it has never been the equivalent of finding the best answers to our questions, or ensuring that we receive the highest quality information. You’re simply finding lots of answers…and because Google wants you to continue using it, those answers are likely the ones that confirm your existing biases, since those ones are proven to get you to interact more.

That…doesn’t feel like a success for anyone except the people making money off of us.

Okay, skip to the part where I should give a shit and why.

Gladly. I’ve split it out so you can self-select your journey and how it will impact you:

How it will affect business owners, small businesses, founders, marketers, and advertisers

Let’s start with Organic Search. I’ll give it to you straight: These changes mean Google is stealing clicks from you, clicks that you could get from users to your website.

How, exactly? First off, because the AI Overview interface that combines the new AI Agent chatbox effectively pushes all other links (like the ones to your business or your content) “below the fold.” If you’re not familiar with this term, it basically means people don’t like to scroll, so you keep the important shit up top where it can be seen quickly and easily. In the past, top-ranked links could be seen underneath AI Overviews as the screen loaded, signaling visually to the searcher that there were other websites they could go to. Like yours. Now, they’re pushed further down, which means that even if you were a top-ranked website on Google…that could basically mean fuck-all now. 

Pew Research shows that the vast majority of people who engage with AI Overviews simply rely on the Overview. Some will click the citations. But if they engage with the AI Overview, they’re not likely to keep scrolling. So if Gemini doesn’t include your business in its AI Overview…you don’t get seen. Previously, you might have been able to fix this and pivot to optimizing your website for AI. With these changes, however, it means that Google doesn’t care as much about including your shit in its AI Overviews because they really mostly care about including their shit in those AI Overviews, for all the aforementioned reasons.

But hey, let’s say best case scenario is that you’ve got a stellar AIO/GEO plan and your business is showing up in AI recommendations–nice job, babe. Except that, while AI isn’t technically a living, breathing thing, it is a thing that is constantly changing. So who is monitoring or how are you monitoring what it says about your business? What if a negative review gets pulled in, or worse, a whole slate of negative review spam, and now it’s not so much recommending your business as sharing incorrect, user-sourced information? What happens when (and it will happen) hijackers threaten to spam Reddit about your business until it shows up in AI Overviews when people search for your business or search for your industry, unless you pay them? 

While the latter might seem like scare tactic (it’s not, check out the way these same people are blackmailing schools right now, present day), the former is going to have the most impact for the most people most often. Engaging with AI Overviews means a 61% drop in people visiting your website, which means you don’t get the click, so you can’t retarget them on other platforms with ads and you may never get their information or make the sale. 

But then there’s advertising, too (if you don’t advertise, you can skip this if you want). Search Ads are more expensive than Display Ads in part because they’re on a CPC model, but also in part because the ad inventory is limited specifically to Google SERPs and not, like, ALL of the internet (the GDN). Less space = higher premium. Now, pushing everyone below the fold means that inventory just got reduced even more, so you can expect your ad costs to spike.

Again, my hunch is that PMAX is about to start charging for impressions in the near future so that Google gets its bag no matter what. And what are you gonna do? They’re the largest search engine/DSP in the Western hemisphere, at least…they basically know they can do what they want and still blame you if the results don’t work for you.

Let’s say I’m wrong and they don’t end up charging per impression on Search or in PMAX in this way. Have we talked about bot/AI agent traffic? The ones that can click on your ads and none of these DSPs even give a shit anymore because how are you gonna prove it? So you’re paying for clicks that aren’t even human? Hell, Google is basically DARVO in corporate form, so it’s likely they’d make you pay for a bot click, admit it IS a bot, but tell you that maybe an AI agent clicked your ad to scrape the data and recommend you in its AI-generated content, so you have to pay for that click anyway.

As of June 3, 2026, agentic traffic has grown so rapidly that bot/artificial users now outnumber human users on the internet for the first time in history. What was that I said about inflated numbers?*** Do with that what you will.

How it will affect publishers (you should read this one, even if you’re not a publisher)

I feel strongly that people who aren’t publishers or who don’t work for publishers should read this section, mostly because there is a sort of collective Daria-volleyball-game-hand-raise from those of us who consume media when changes like this are made that affect media in drastic ways. I get that, because it can feel like, okay, but that’s not me? Or my problem? So why should I care? And also because publishers/the media aren’t exactly always doing the right thing these days, or they’re owned by big bucks billionaire bozos, so it’s like, again, why should I care?

It’s because what affects publishers does affect you. It won’t right away. But you’re a frog in a pot of warm water set to boil: By the time it affects you, it’s way too late to fix it.

The research shows that when people engage AI Overviews, they generally read the summaries but don’t generally click any links. This means that AI Overviews summarize the content for us, giving us the feeling that we “get the gist” without having to read (or pay to read) the full article. So we don’t click on the publisher’s link to the full article because, like I said, we get it, right? (Never mind that those summaries can be dead wrong, not the point, but still.)

When we don’t click on the link to the publisher’s article, we never make it to the publisher’s website.

When we don’t make it to the publisher’s website, they lose website traffic.

When publishers lose traffic, it means they lose the people who could see the ads on their site to click on the ads.

When they lose clicks on the ads AND if they lose traffic, they lose ways to sell their ad inventory to advertisers.

When they lose traffic, they need to find ways to make up for it. One option is to pay for Google’s ad inventory so they can show up as a citation in the AI Overview and cross their fingers that people click (which is not common). Other options include paying for ad delivery on other platforms as well.

When they have to spend more on ads because they’re losing their own ad revenue (because fewer clicks because fewer site visits).

When they don’t bring in enough revenue, there are layoffs. This means lower-quality reporting because staff are stretched too thin, it means less thorough or simply less fact-checking, opening them up to misinformation that would further erode their credibility, it means fewer (if any editors), it means more AI-generated content that, again, can contain misinformation and harm credibility.

And after all of this, they eventually close. Or are bought by people with a distinct agenda.

When they close, that means there are fewer publications (and more unemployed people) to deliver quality information and ethical journalism. It also means less access to free information, which has been an increasing problem, meaning that only those who can afford it⁴ will be able to be informed by less biased, more objective reporting and information OR they can settle for what Google gives them.⁵

Pictured: Left - Brave search engine results when asked about Google’s most recent announcement and its implications on search. Note the dates, which are all recent. Middle - Same day, next minute search on Google’s search engine with the same query. Also on the News tab, which is supposed to default to the most recent news from publishers, but instead buries the most recent publisher opinions of this change in favor of better press. Right - The same query on Google’s primary SERP—more recent results, but still months old and, importantly, pre-announcement. One might conclude that Google is strategically showing only the narrative they want shown.

When Google controls the flow of information, it’s not a problem until it is. Do you want continuous access to diverse information, or do you want to live in an environment where the only information available to you is owned by, idk, Chef Boyardee?

Oh, and also: When publishers close, or when they have less traffic, it means fewer places for ads to be delivered. When ad inventory is lower, ad costs are higher. When ad inventory is lower, ad platforms like Google or Meta will deliver more ads more frequently to anyone they can, however they can. Would you like to scroll Instagram where only every fourth post is an account you actually follow and NOT an ad?

Publishers are a critical, load-bearing part of our digital ecosystem as we’ve built it. Impacts to them ARE impacts to the rest of us.

How it impacts all of us as general humans who search for things on the internet

Basically, by creating this sort of circular loop where Google just points you to Google, it turns the “lobby of the internet” (as Google has been referred to) into the “padded room of the internet.” You’re stuck inside with a lot of questions about how you got there and how you get to leave.

There is what should be the most obvious impact: The fluctuating quality of information provided by AI. Gemini was famously bad at delivering information when it launched AI Overviews, to include telling users to eat rocks or put glue on pizza to keep the cheese from falling off, all pulled from Reddit shitposts it didn’t identify as shitposting.

The argument can be made that it has improved, that it is always improving. And I think that’s a very sweet, glass-half-full way to describe the situation. But the reality is that, even if it’s always improving, it means that right now, before it improves, it’s at risk of being dangerous, uninformed, misleading, or even just incorrect. If a car is being tested and it randomly explodes once every 100 times it’s turned on, but then we tweak it so that it’s only once every 1,000 times it’s turned on, we don’t put it on the road and justify it by saying, “But look at the vast improvement! And we’re making more progress every day!” We wait until it doesn’t fucking explode, because of the danger it poses to everyone when it does, whether or not they were driving the car.

And even if you believe that the dangers are overblown (they’re not, and you’re wrong), here are just some general examples of incorrect information that I have personally recently experienced:

A new report out of 404 Media exposes how brands are now spamming subreddits with fake “recommendations” of their brand in order to show up in AI search results, since several LLMs, including Google, have partnerships with Reddit where they pay Reddit to scrape their data. The particular example used first in the report is from a peptide company in r/biohackers…so…an unregulated product that someone consumes in their body. Where could the problem possibly be in that??

And this will keep happening, especially on Reddit, because it’s not even considered an outlier or even a problem–instead, it’s a new “marketing discipline” called AI Optimization (AIO) or Generative Engine Optimization (GEO). It’s a job that people are getting paid for, to optimize brands to show up in AI recommendations…and this is one of the most effective tactics, to spam highly scrape-able sites with manufactured (fake) reviews. (For my SEO comrades, it feels a little bit like the keyword stuffing of the aughts.)

Reddit would assure you that it has been dealing with bot and AI spam for decades, that it is adept at identifying and regulating or blocking it. It would tell you that this is why Reddit is great: Users can also identify spam and downvote it, meaning that the upvotes have the best, most quality information. Two things, though: First, remember that Reddit is profiting directly from AI scraping their data, so there’s not exactly a lot of incentive to like, eliminate constantly more sophisticated spam techniques since they get paid either way. This might feel counterintuitive; an argument can be made that they have an investment in even greater vigilance against spam, except it’s…not really necessary. Why? Because the companies to which they are providing access to that data don’t really give a shit if it’s manufactured. They just need the data to feed the machine to feed it to the rest of us.

We now verifiably know that AI scrapers do not read for quality, they read for answers to the question. In a report by Semrush, most AI citations from Reddit actually had fewer than 20 upvotes. The robots are not reading the most popular content on Reddit, policed by the community…they’re reading the volume of repetitive information and deciding that that is consensus. And this isn’t just Reddit: The Atlantic found that Shopify published 60 listicles on “the best e-commerce platforms,” all of which (surprise) rank Shopify as the best one. Figma is doing the same, and ClickUp might be the most egregious cited in the article, publishing a whopping 300 blogs about the “best project management software” and making ClickUp the best in each one.

You might think, “Okay, but this is just table stakes marketing. Of course brands write content like this and rank themselves highest–it’s Content Marketing 101 for the middle funnel. How is this any different from SEO strategy of the last 10+ years?” And look: You’re on the right track, it DOESN’T seem that much different on its face. What MAKES it so much different is the sheer VOLUME of the SAME EXACT TYPE of content–a sin that may have previously decimated their SERP rank in the days of SEO–is what proves that this isn’t being written for HUMAN BEINGS. It’s being written for AI BOTS who crawl the internet looking for, again, SHEER VOLUME OF CONSENSUS. When 300 blogs arrive at the same conclusion, that seems like a fairly solid consensus to most humans…but we are trained at least at some point to consider the source and the end goal. Language Learning Models aren’t.⁶

Now could they one day be interested in the quality (or, Jesus I can’t believe I’m even saying this, reality) of the data, because people don’t want to engage in an uncanny valley where they don’t know if what they are reading or being recommended is real or make-believe? Sure. But not only will the damage already be done to such a degree that clean-up will be a very expensive (read that as “mostly not worth it”) task, it’s also just not going to be a priority any time in the short-term future. AI companies compete with other AI companies but they also know that, at the end of the day, they are each other’s best and most reliable allies. They don’t care that the data they are scraping is AI because the point has never, ever been to improve the lives of others, it has always been to make as much money as they possibly can on this tech arms race before it all implodes. It’s dot-com all over again.

Maybe you still don’t know why you should give a shit. Maybe you think that you will be able to easily decipher what is real from what is fake. While our very human tendency to act out Icarus over and over again is a blog topic for another time, I’d love for all of my readers to do me a favor for the purposes of this blog and imagine a tomorrow where you got it wrong. Where your socialization into capitalism is running in the background and you accept something a robot says (or a social media post says) as truth because it seems plausible to you, and you’re not a dumb person. But seeming plausible and being factual are two different things. And like it or not, just like all systemic issues, the idea that “might makes right,” or more present-day, “if they made all this money, they must be doing something right”⁷, is an idea that has been ingrained in us since our youth, and it is a HARD one to separate ourselves from***. Similarly, we’ve repeatedly been told that because advanced technology LIKE LLMs or AI bots are capable of consuming vastly more information than we ever can or ever have, they must be right, or at least, they must know better than us.

But even if we have a knee-jerk tendency to believe that technology knows more than us, we need to push back against this idea that volume is the same as quality (we are all familiar with the phrase “quality over quantity”, but the technocrats seem hellbent on making us believe the opposite even though we know it’s not the preference). We know that AI systems favor volume over quality. We also know that AI systems are basically a black box to users: We know they work, we don’t know HOW or WHY they work (and anyone who says they do is lying to you, or you’re lying to yourself, please go read this book and then tell me that again). We know they train on our data, but we don’t know how they’ve been directed or how they grow. So how do you know that the AI models won’t be designed specifically to direct you to buy products from nefarious actors, or to ONLY consume information designed to slowly change your mind and get you to sympathize with bad actors (ie: a propaganda machine)? This isn’t a sci-fi dystopia: We already have a MULTITUDE of examples that AI information is being manipulated before it reaches us.

Two examples of this come from the same origin user: A horror artist decided to see what he could make AI believe, just as an experiment. Enter the terms “Parasitic Encephalization” and “Dranunculus graviditatis,” two fictional terms he created and assigned definitions to and got the models to believe, for a substantial period of time, that they were real. (As of this writing, Gemini still believes that the latter is a real thing that happens. It is not.) When bots are trained on inaccurate, patently false, made-up, manufactured, or highly manipulated information, they will pass that on as fact. And until it is caught and corrected, no one knows except the bad (or curious) actors who fed it that information. Believing anything a bot tells you without verifying is just like that kid citing his dad’s opinion as fact. It’s like deciding you’re going to do whatever Donald Trump tells you to do, even if what you think he’s saying sounds maybe untrue, maybe dangerous, maybe problematic, but he’s the President, so he has more access to more information than you do and you should simply trust that he is making decisions according to that information acting in your best interest. 

I hope you see how fucking insane that sounds. And I hope you see the parallels between this and high control group system behavior (for another blog).

An optimist might look at all these examples and say, “Well yeah, but they were all found and corrected eventually,” and they’d be right. I can’t deny that. But as I’ve said previously: False information can be dangerous until we know it’s false. Waiting for it to be found does not mitigate disaster. Again, it’s putting a new car on the road knowing that there’s a specific color that is likely to explode, but that we won’t know which color explodes until it does a few times and then we can be certain, but a lot of people are already dead.

There is a scene in the immaculate final season of Hacks where an AI technocrat is trying to convince Ava and Debra that “AI is here to stay,” that it is “the inevitable future” so they need to get on board or be left behind. Ava’s response:

To draw another parallel: AI is not an inevitability like recession indicators are lagging indicators and not leading indicators, as in, they simply reflect what is currently happening and start to form predictions on what will happen if nothing changes. A few corporate mass layoffs are lagging indicators. But a leading indicator can end up bein the decision you make next after seeing the lagging indicator. The recession doesn’t happen because of those few corporate mass layoffs–the recession happens when hundreds or thousands of other businesses look at those few mass layoffs and panic and start tightening the belt in anticipation of things getting worse. When everyone tightens their belt, it creates a domino effect. If everyone held still and waited, it’s possible to avoid a recession entirely. It’s also possible to create one from your panic. It’s the difference between holding your breath and waiting for a predatory animal to ignore you or not notice you and pass you by–you might get away with it, you might not. But if you scream and run, your chances of survival are only as good as how much faster you are than what’s chasing you.

AI takeover or AI replacement or basically-mandatory-AI-adoption-integration is not an inevitability. It only is if people are forced into it and do nothing to question it or push back against it. Even AI acceleration doesn’t have to be an inevitability. We can slow this down and insist on using it only for common good. It only becomes an inevitability if we acquiesce, if we allow the corporations that have shaped our user behavior for decades to shape this next frontier of our behavior as well. 

Businesses like Google are trying to force us into this place.⁸ They are trying to control the news cycle and they are inflating reports to control and push a narrative that tells you that you are the outlier even if you simply have QUESTIONS about moving forward. The reported actual AI adoption that we see coming out of independent researchers is vastly different than the reported adoption we see coming from companies who need us to use their AI in order to make that skrilla. People will only keep moving in this direction at this speed, in greater numbers, if Google keeps trying to herd us like cattle to force us into that direction. Google doesn’t want you to know you have other options, they don’t want you to see the exit: It’s the entire reason they pivoted their algorithm to put your other options below the fold and favor their own search engine pages. You can and should push yourself to scroll, to verify, to use it differently than the way they insist you should, the way they are changing their interface to make you use it. At a certain point, these companies don’t make these decisions and innovations because they make sense for the world–they make these choices because they believe they are too brilliant to fail, that their every idea is a winner because they had some big bets and big wins already. But they’re not the director, they’re not the main character, and you don’t have to play the part just because they gave you the script. 

For more on your options to move away from this forced enshittification and circularity, check out my last post on alternatives to using Google as a search engine.

¹i am a human and i have an opinion and i bought the pillow and it’s fucking amazing. i still would have liked to find out from other humans before i bought the damn thing.

²no word on whether Google Scholar will be prioritized but i’m guessing only if there isn’t anything to sell

³it’s worth noting that a lot of reports have come out in the last year or so about how PMAX is not a black box anymore because…there are some metrics that actually get reported on now. to me, that being the standard for whether it’s a black box isn’t, like, the gold standard, but i digress

⁴also worth noting: there has, of course, been a massive uptick in journalism that gets put behind paywalls in the last few years. closer observation reveals a very substantial difference in the type of journalism that gets paywalled…namely journalism that leans left or center, with some center-right or let’s call it “college-educated right” like The Economist. outlets like Fox News, Breitbart, Newsmax, and The Daily Wire remain free and accessible to everyone because they have people bankrolling them hard. the dots are pretty close here for people to connect them, but this is where it affects us most: if the only fr.ee, accessible journalism is “performance”-based right-wing opinions, we can’t be surprised when public opinion tides turn to the right.

⁵which i hope has been established that this can change at any point in time depending upon what is best for google’s own business model and has absolutely nothing to do with whether the information is true, accurate, or vitally necessary.

⁶which is to say, they aren’t YET. they CAN be trained to do this. there is, however, zero incentive for companies like Google, specifically, to train LLMs to examine the source/end goal because they make more money when users think less critically.

⁷this might be the most pervasive, most insidious assumption that the vast majority of us have been socialized to make. this singular assumption is what led me to my career and finding out the absolute fucking farce that it was is also what led me to not only burn out, but leave marketing entirely. it’s what made me want to write this newsletter and re-launch our podcast. it is patently false, and we should KNOW that from failed mega-investments like the metaverse. yet it persists, this idea that companies wouldn’t spend insane amounts of money if they didn’t know what they were doing, or that if they are making money hand over fist, they must know what they’re doing. i’m here to tell you firsthand: it’s wrong. it’s wrong. it’s wrong it’s wrong. I have had clients who are household names, who have been mentioned in these newsletters and the one thing they all have in common is leaders at every single level, to include the tippy-top, who make decisions based purely on vibes and ego and bonus structure, and not on any kind of data or math that makes sense to anyone else.