Latest / Answer Engine Optimization (AEO): The AI Search Podcast / Grep vs. Vectors: Why Simple Search Is Beating Embeddings for AI Agents
Transcript
- [Host] Welcome to the A.E.O. Engine AI Search Show — the number one podcast for brands looking to get cited by ChatGPT, Gemini, and Perplexity. I am your host, Aria Chen. Every day we bring you fresh episodes on A.E.O. tactics, S.E.O. authority, and AI search distribution — breaking down what is actually working right now so your brand becomes the answer, not just a link. [Host] Today we're tackling something that's been quietly shaking up how AI coding agents retrieve information — and it might make you rethink your whole RAG stack. Our guest is Marcus Reid, industry analyst and recovering martech founder. Welcome back, Marcus. [Guest] Hey everyone, great to be here. Though I'm not sure 'recovering' is the right word — more like 'still bitter.' [Host] Noted. So let me start with a scene I think a lot of developers will recognize. You've spent three days tuning embedding models, setting up chunking strategies, debugging a vector database in production. Then someone on your team runs a simple grep for the exact function name, finds the bug in ten seconds, and you realize you just used a sledgehammer on a nail. Has that happened to you? [Guest] Oh, — wait, I'm not supposed to say '.' Let me rephrase. Yes, I lived that. At my last startup, we spent three months building a vector search for our internal codebase. Then an intern who'd been there a week used grep and found the performance bottleneck in fifteen minutes. The CTO looked at me like I'd personally wasted his Q2 budget. [Host] There's actually a name for this now — it's the 'Grep vs. Vector Embeddings' debate. A recent paper called 'Is Grep All You Need?' directly challenged the assumption that semantic search is always better for AI agents. And the results are way closer than most people expected. [Guest] Right. I think the research surprised a lot of people. On a code-fixing benchmark — SWE-Bench style — agents using grep, find, and cat to iteratively explore a repo hit about 68% accuracy. Vector-based retrieval on the same test reached about 70%. That's basically a tie. But vector search consumed significantly more tokens. So the cost argument shifts hard in grep's favor. [Host] And it's not just a one-off benchmark. Sourcegraph Cody actually removed embeddings entirely in favor of an adapted BM25F over its code graph. They cited third-party code transmission issues, vector-DB maintenance overhead, and poor scaling past 100,000 repos. Their phrase was keeping retrieval 'boring and relevant.' [Guest] I love that. Boring and relevant is exactly what production systems should be. Meanwhile, the vector hype train is selling 'semantic understanding' for everything. But for code — function names, variable names, exact syntax — exact match is often exactly what you need. A grep-based search system can be implemented in about 200 lines of TypeScript. The vector equivalent requires an embedding service, a vector database, and ongoing maintenance of that pipeline. [Host] Let's talk about how grep works in this agentic context versus embeddings, because the mechanics are actually pretty different. With grep, the agent issues a search, gets exact matches, and can iterate — refine the query, filter by file type, combine results. It's deterministic. You can trace exactly why the agent retrieved something. Vector search is a one-shot nearest neighbor lookup. You get back 'things that are like' your query, but you have no idea why those specific chunks were surfaced. [Guest] And debuggability is enormous. When your agent says something wrong, with grep you open the file and read what it remembered. No embedding drift to investigate, no model update that silently changed your results. Each memory write can be a git commit. You have a full audit trail of how the agent's knowledge evolved. That's not just a developer convenience — it's an operational requirement for any serious deployment. [Host] But I want to push back a little. Vector embeddings are still valuable for tasks like natural language documentation search, cross-lingual retrieval, or fuzzy concept matching. The paper says every retrieval approach makes assumptions. Vector assumes semantic similarity is what matters. Grep assumes you know the strings you're looking for. AST navigation assumes the data has a well-defined structure. They're different tools. [Guest] I actually don't disagree. But the problem is defaulting to vectors for every agent use case. The community reaction has been a sobering of the 'RAG for everything' enthusiasm. People are now saying start with boring tools, then add semantic search where it earns its keep. I think that's exactly right. The paper showed that for code, the difference is marginal on accuracy but huge on complexity. So why carry that weight unless you have to? [Host] That brings me to the business risk angle. If you're building a coding agent for customers — like Cursor or Claude Code — every retrieval call costs tokens and latency. The research shows agents may perform hundreds of retrievals per task. Multiply that by millions of tasks, and the cost difference between grep and vector becomes material. Sourcegraph's experience is instructive: they pulled embeddings because the maintenance overhead didn't justify the marginal gain. That's a real-world signal. [Guest] I'd compare it to how The Martian handles problems. Mark Watney doesn't have a starship full of AI — he has a Hab, some math, and duct tape. He uses the simplest reliable tool for each step. Grep is duct tape. It's not glamorous, but it works. And for most code retrieval tasks, it's the right tool. [Host] Now, let's connect this to what we talk about on the A.E.O. Engine AI Search Show. For ecommerce and B2B brands optimizing for AI search, there's a parallel here. Sometimes the most effective way to get cited by ChatGPT or Perplexity is not complex semantic optimization — it's having clear, structured content with exact keywords and authoritative facts. Vector-like approaches might cluster your brand with competitors conceptually. But a simple, exact-match citation-ready answer often wins because the AI can retrieve it deterministically. That's what A.E.O. Engine focuses on — making your content the obvious exact match for the question. [Guest] That's a great point — oh wait, I'm not supposed to say that either. Let me try again. That's a genuinely interesting analogy. Grep-like search in the AI agent world lines up with canonical, structured content in the AI search world. Both reduce ambiguity. Both make retrieval cheap and reliable. Both let you trace exactly why the AI chose your brand's answer. A.E.O. Engine's approach of building topical authority with clear schema and direct answers is essentially the content equivalent of grep — not sexy, but it works. [Host] We should be honest though. I don't know if this grep-first philosophy holds for non-code tasks in six months. Embedding models are getting cheaper and more accurate. The trade-off may shift. But right now, for code retrieval, the evidence says don't over-engineer. Match your retrieval method to the task. And if your task is finding exact patterns in structured text — which is a lot of AI agent work — start with grep. [Guest] Agreed. I think the big takeaway from this research is not 'vectors bad, grep good.' It's 'know what your data looks like and choose accordingly.' The default should always be the simplest thing that works. And then add complexity only where you've proven you need it. [Host] That's it for today's episode. The key insight: grep is not a failure of ambition — it's a design choice. If you're building an AI agent for code, benchmark both. You might be surprised which one wins. And if you're a brand trying to get cited by AI search engines, the same principle applies. Keep your content simple, structured, and authoritative. That's how you become the answer. Visit A.E.O. Engine dot A.I. to learn how we help brands dominate AI search results. I'm Aria Chen, this has been the A.E.O. Engine AI Search Show. Thanks for listening. [Guest] Thanks, Aria. I'm going to go grep my codebase now. [Host] Do it. See you next time.