Latest / Retail Media Breakfast Club / What Is Your AI Agent Buying? [New Columbia & Yale Research]
Transcript
- [Weds 8/28] What Is Your AI Agent Buying? [New Research] === [00:00:00] If I were to ask my husband, my mother, and my best friend to pick me up a new pair of slippers at the mall, chances are they'll bring me wildly different products. My husband will grab whatever's on sale. My mother will [00:00:15] pick up something practical and durable, and my best friend might choose based on style and something that matches my personality. [00:00:23] Each person would use their own criteria shaped by their relationship with me and their personal shopping [00:00:30] habits. Now, imagine those shoppers are AI agents, Claude Chat, GPT, and Gemini, each with their own built-in preferences and decision making patterns. A new research study [00:00:45] from Columbia and Yale researchers gives us the first real look at how AI agents actually shop when left to their own devices. [00:00:54] The findings should make every brand and retailer rethink their assumptions about the AI-driven [00:01:00] commerce feature that's coming to us quickly. [00:01:03] [00:01:04] Before I jump in, just ~some quick background on the methodology of this study because it is very unique and I think,~ [00:01:05] ~but first, here's ~some quick background on the methodology of this study because it is very unique. So this team of researchers who were five researchers. Across [00:01:15] both a private AI company called My Custom ai, which builds custom AI models for enterprises as well as academics from Columbia and Yale. So it is an academic research study and What they did was build a sandbox model that replicated the [00:01:30] e-commerce shopping journey in allowing them to fully control variables like layout prices, reviews, badges without interfering with a live retailer.com site. Then they, [00:01:42] Evaluated three frontier [00:01:45] models, including Claude Sonnet 4G PT, 4.1, and Gemini 2.5 flash, and they tracked how agents from the various models acted within this sandbox environment. All these variables were very controlled, [00:02:00] allowing them to isolate exactly how these. Shopping agents behaved in different scenarios. [00:02:07] That's important context for what we're gonna talk about now. [00:02:10] So first of all, reviews are extremely important [00:02:15] to all of these AI models. Just like consumers, really, AI models are looking at both review count and review quality, and the study was able to quantify. The impact of the number of reviews by [00:02:30] estimating how much a seller could actually increase their product's price if they were to double the number of reviews. [00:02:37] So this is called, uh, pricing headroom, or a price equivalent trade off, and they found that if you [00:02:45] doubled the number of reviews, Claude Sonnet. Would allow you to increase your product price by 19.4%. With GBT 4.1 it was 37.4%, and with Gemini 2.5 flash [00:03:00] 17.2%. So put simply, if you currently have a hundred reviews and increase. [00:03:05] To 200, you could charge 19 to 37% more of your retail price and still get selected by these AI models [00:03:15] at the same rate. So this is super interesting because. We've always cared about reviews. We've always cared about ethically getting reviews within all of the regulations in terms of service of different retailers. [00:03:28] But now within the context of [00:03:30] LLMs shopping for us, it is even more important to prioritize acquiring reviews and good quality reviews even more so. Number two, the sponsored product problem. Here is a very interesting finding. AI [00:03:45] agents actively avoid sponsored products across all models. [00:03:50] Tested. I'll just say that again. AI agents actively avoid any products that are sponsored, regardless of [00:04:00] which model we're looking at. [00:04:01] This is profound and it's a real bias against advertising that's going to have big implications for retailers as I've, as I've spoken about, a existential threat [00:04:15] to retail media as a business model for retailers as well as a brand trying to figure out how to stand out. In the marketplace. I'm not gonna dig too much into this topic today 'cause it deserves its own entire [00:04:30] episode and article. [00:04:31] So make sure that you're subscribed to catch my deep dive into this topic number three, market concentration risk. Another concerning finding isn't about individual products, it's about market dynamics. AI agents [00:04:45] consistently concentrate selections among a narrow set of products with some brands never getting selected despite being viable options. And just one little tidbit from this study, one of the interesting ones was an Amazon private label brand, like [00:05:00] their Amazon Basics brand not being selected. [00:05:03] In, a final assortment, because this has been a bear for a lot of large brands trying to fend off. Upstart sellers, unbranded [00:05:15] brands. You know, the keyboard mash brands that you see on Amazon sometimes that come directly from Chinese, manufacturers specifically. So that might be kind of good news if you're an established brand well-positioned. [00:05:28] But it does [00:05:30] have obviously the converse effect of innovation being stifled for new or niche products they're gonna struggle to break through because AI really likes to play it safe with selecting, um, known [00:05:45] quantities. So this is obviously going to be a double edged sword. When your time, budget and team are already maxed out. Keeping up with retailer [00:06:00] platforms and media requirements can feel like a never ending labyrinth. That's why even the best brands bring in an execution partner. [00:06:09] Acosta Group makes retail media execution simpler. They stay ahead of every [00:06:15] platform. Update tech shift. And media spec, so your campaigns actually show up and perform without draining your team. [00:06:23] Let Acosta group handle the chaos behind the scenes so you can keep building the brand. Check out [00:06:30] Acosta Group and tell them Retail Media Breakfast Club sent you. That's A-C-O-S-T group. [00:06:38] [00:06:40] And number four, AI optimized content. Mixed results here, this is [00:06:45] pretty surprising. AI agents don't uniformly prefer AI generated content. In about 25% of cases, AI preferred AI generated. Uh, Content in specific categories like [00:07:00] iPhone cases, there was a 9% improvement in market share with the Claude model for AI generated product content. [00:07:09] But there's some failures in other categories like washing machines. There was a negative impact across all [00:07:15] models when they were looking at AI generated PDP content versus human. Driven PDP content. So there's no universal AI friendly content template is what the conclusion is [00:07:30] here, [00:07:30] what this means right now. So there's three immediate implications that I'm seeing. One for brands review acquisition becomes as important as advertising spend. This is an area to double down on, whether you're talking about humans, looking at your product [00:07:45] pages or AI LLMs [00:07:47] for retailers, your own curation of, uh, the recommended products and featured product selections are more influential than ever [00:07:58] And for everyone in this [00:08:00] ecosystem, the sponsored product model faces a fundamental challenge when AI agents systematically avoid ads. The entire retail media revenue model needs rethinking. [00:08:09] But as I said, I have more thoughts on this. I don't think it's the end of sponsored product ads as we know it. [00:08:15] Make sure that you tune in next week for a continuation of that discussion. But I just wanna zoom out and look at the bigger picture for a second. This research captures AI agents operating in today's grid-based website centric shopping environment, [00:08:30] but that's also the biggest limitation of this research study. [00:08:32] This. Study replicates the E-commerce SERP layout that we've used for 20 years. But we've also gotta recognize that consumer behavior is evolving beyond traditional [00:08:45] browsing. There's other ways that we're interacting with brands and retailers online today. [00:08:49] One is with vertical video commerce, so TikTok shop Instagram shopping that bypasses search grids entirely. Impulse purchases are based on creator recommendations, [00:09:00] and you go straight to a product page and you check out. You're not browsing and discovering on a retailer app or website. Number two is agent to agent shopping. [00:09:09] There's protocols emerging like MCP that will make website navigation obsolete [00:09:15] when you're actually using an agent. So instead of an agent actually browsing product grids, AI agents will directly access retailer inventory feeds and product data. So whether retailers are building towards this scenario right now is a little [00:09:30] unclear. [00:09:30] But when it happens, concepts like grid position, which is something I didn't discuss in this deep dive, but it's very much a big piece in the research. Paper grid position is sort of a meaningless concept. And then finally, voice enabled shopping. I might [00:09:45] lose a few people with this, but I think that we're getting another shot at this with Amazon's Alexa Plus and Tesla's integration of grok, Alexa and Siri. [00:09:56] Certainly didn't crack voice commerce, right away. But [00:10:00] that technology is improving and I think there's good use cases for reordering and things like that, which are gonna obviously operate completely outside of visual grid layouts. [00:10:11] ~The brands and retailers who win won't be those who game the current system, but those who build authentic product quality and customer satisfaction and trans. That translates across all these emerging shopping modalities, reviews, ratings, product differentiated. Reviews ratings product differentiate matter whether an AI agent is analyzing a grid or processing an inventory feed, but the tactical optimizations around positioning and badges.~ [00:10:11] ~I'm just gonna redo that conclusion. ~The brands and retailers who win won't be those who game the [00:10:15] current system, but those who build authentic product quality and customer satisfaction. That translates across all these emerging shopping modalities. [00:10:24]