Latest / AI Fire Daily / #48 Robin: Stop Wasting Tokens - How Graphify Gives Claude Code a Brain and Slashes API Costs by 60% 💸
Transcript
- 0:00Asking an AI to trace a single code base request
- 0:02cost 200 ,000 tokens. Then by giving the AI a
- 0:07simple map first, that cost dropped to 80 ,000
- 0:10tokens. Yeah, that is a massive drop and wasted
- 0:13computational effort. I mean, we're literally
- 0:15changing the starting line here. Welcome to the
- 0:17Deep Dive. I'm really glad you are joining us
- 0:19today. We have a pretty fascinating shift to
- 0:21talk about. We really do. You know, I still wrestle
- 0:24with prompt drift myself, especially when I ask
- 0:27an AI to analyze a massive project and it just
- 0:30gets hopelessly lost. Oh, absolutely. It's incredibly
- 0:33frustrating when it just spins its wheels. Exactly.
- 0:36So today we are exploring Graphify. It's an open
- 0:38source tool that basically gives cloud code a
- 0:41reusable memory. Which is something developers
- 0:43have been desperately asking for. Right. We'll
- 0:46cover how it builds a map of your repository.
- 0:48We'll look at the three -pass system it uses
- 0:49to understand your files. And we'll see how it
- 0:52fundamentally changes the way AI interacts with
- 0:54code bases. Okay, let's unpack this. We know
- 0:58AI gets lost in big projects. Why does that actually
- 1:01happen? Well, mainly because it totally lacks
- 1:03inherent structure when it begins. Right. Normally,
- 1:06cloud code searches file by file. It spawns these
- 1:09little explore agents, opens folders and just,
- 1:12you know, builds context from zero. It's kind
- 1:14of like trying to navigate a new city by walking
- 1:16down every random alleyway to see where it goes.
- 1:18Yeah, exactly. It's chaos. Versus just looking
- 1:21at a subway map before you leave the house. Right.
- 1:23And GraphDefy fixes this by turning the repo
- 1:26into a knowledge graph first. It builds that
- 1:28subway map. Why does crawling a repository manually
- 1:31eat up so many tokens? Because the AI is wasting
- 1:35context windows just trying to figure out the
- 1:37shape of the project before it even begins to
- 1:39answer the actual prompt. So it builds a map
- 1:41first, saving the AI from wandering blindly.
- 1:44Yes, exactly. And the way it accomplishes that
- 1:46is really methodical. It uses a three -pass approach,
- 1:49right? How does it start? Pass 1 focuses purely
- 1:51on the code. It uses a tool called TreeSitter
- 1:54to read the code structure locally. Classes,
- 1:57functions, imports. And this pass is completely
- 2:00deterministic. Wait, let me stop you right there.
- 2:02What does deterministic mean here? It follows
- 2:04strict logical rules without any AI guessing
- 2:08involved. Got it. So it just reads the actual
- 2:10mathematical connections. Right. No LLM hallucination
- 2:14yet. Beat. Then pass to handles audio and video.
- 2:19It uses faster whisper to transcribe media files.
- 2:22Like internal demo videos or meeting clips. Yeah,
- 2:26exactly. Those transcripts are added directly
- 2:28to the map. Wow, that is incredibly useful for
- 2:30capturing raw institutional knowledge. It's a
- 2:32total game changer. Then we hit Pass 3. This
- 2:36is where the large language model steps in for
- 2:38documents and images. Like PDFs and markdown
- 2:40files. What's fascinating here is how it blends
- 2:43rigid logic with AI interpretation. It understands
- 2:46the business logic in a design PDF and connects
- 2:49it back to the code from Pass 1. If Pass 1 is
- 2:51free and local, why do we even need the LLM in
- 2:53Pass 3? Code is rigid and easy to map, but human
- 2:57context, like design PDFs, requires AI to interpret
- 3:00the meaning. It reads code rules, exactly, then
- 3:02uses AI for the messy human stuff. Exactly. It
- 3:05creates a complete picture. So once that map
- 3:08is built... What does this web of data actually
- 3:11look like in practice? Right. It breaks down
- 3:13into three core parts. Nodes, edges, and communities.
- 3:16Let's define those for everyone. Nodes are the
- 3:19distinct entities, a file, a function, or a document.
- 3:22Okay, isolated pieces of information. Mm -hmm.
- 3:25Edges are the lines connecting those individual
- 3:28nodes, the actual functional relationship. On
- 3:31communities. Communities are organic clusters
- 3:34of highly related nodes, like neighborhoods in
- 3:36our subway map. The developers ran a really great
- 3:39demo of this. on the Open Design Repository.
- 3:43Yeah, it's a huge open source code base. Perfect
- 3:45stress test. Graphify scanned 203 files in about
- 3:48six minutes. And it created 197 distinct nodes
- 3:52plus over 3 ,400 edges. That is a massive web
- 3:57of dependencies. Yeah. And it found 109 communities,
- 4:01right? Yeah. It highlights surprising connections.
- 4:04It even identifies these things called God nodes.
- 4:06Ah, the highly connected critical parts of the
- 4:09repo. Whoa. Yeah. Beat. Imagine scaling that.
- 4:13Imagine seeing God nodes in a code base you've
- 4:16worked on for years but never truly understood.
- 4:18Beat. It completely exposes the hidden, fragile
- 4:22architecture of your project. What practical
- 4:24value does knowing the communities give to the
- 4:27AI? It allows clawed code to isolate the specific
- 4:30neighborhood of code it needs to solve a problem,
- 4:33ignoring irrelevant files entirely. Nodes are
- 4:36things, edges are connections, communities are
- 4:39specific code neighborhoods. That perfectly summarizes
- 4:42it. It's why it beats traditional search. Since
- 4:44we're talking about memory, listeners are probably
- 4:46wondering how this differs from GraphArea. They
- 4:49do sound similar, but the core difference is
- 4:51that Graphify does not use embeddings. Let's
- 4:53clarify that. What are embeddings? Translating
- 4:56text into numbers so AI can find similar underlying
- 4:59meanings. Here's where it gets really interesting.
- 5:01Graphify maps actual existing structural relationships.
- 5:06GraphArea searches giant unstructured document
- 5:09collections. Exactly. GraphRag finds things that
- 5:12sound related. Graphify reads exact, deterministic
- 5:15relationships that already exist, like code imports.
- 5:19It's basically RG Lite for repositories. Yeah,
- 5:22mapping reality, not probability. So if I have
- 5:2510 ,000 messy policy PDFs, I shouldn't use Graphify.
- 5:30Correct. GraphRag is for massive, unstructured
- 5:33text. Graphify is for structured systems where
- 5:36you want to know exactly how A connects to B.
- 5:39Graphify map structured systems. GraphRag searches
- 5:42giant piles of loose documents. Spot on. You
- 5:45have to choose the right tool for the job. Let's
- 5:47get back to the practical payoff. Time, money,
- 5:50and keeping this map alive. The token savings
- 5:52are very real here. In that test, asking Claude
- 5:55to trace a design request, the standard run cost
- 5:57200 ,000 tokens. And 150 ,000 of those were wasted
- 6:01just on explore agents. The AI was just frantically
- 6:04bumping into walls. Right. But the Graphify run
- 6:06took only 80 ,000 tokens total. That is about
- 6:0940 % of the original cost. Now, I do want to
- 6:11warn listeners not to expect the massive 70 times
- 6:14token savings hype you might see online. Yeah,
- 6:17the internet loves to exaggerate. Always. But
- 6:1940 % is still massive for big projects. And setup
- 6:22is incredibly easy. You just ask Cloud Code to
- 6:24install it. You just run slash graphify dot.
- 6:27And there's this really cool dash dash obsidian
- 6:31flag. Which turns the map into an obsidian vault
- 6:33for personal notes. It's so fun to visually explore
- 6:36your code base that way. But code changes daily.
- 6:39How do we keep this a living repo? You run Graphify
- 6:43hook install. It rebuilds the map automatically
- 6:46on every single commit. If my team pushes 50
- 6:49commits a day, won't updating this map get incredibly
- 6:52expensive? No, because the structural code updates
- 6:54don't use the LLM API. They just use the local
- 6:57free parsing tool. The hook updates the map locally
- 7:00and for free on every commit. It's completely
- 7:02sustainable. So what does this all mean? Graphify
- 7:06isn't just a cost -saving trick. No, not at all.
- 7:08If we connect this to the bigger picture, it's
- 7:10a fundamental shift in how we work with these
- 7:12tools. We are moving from treating an AI like
- 7:15a search engine that has to rediscover your project
- 7:18every morning to giving it a permanent living
- 7:20memory. This raises an important question about
- 7:23the future of development. It gives the AI genuine
- 7:26situational awareness. Think about the obsidian
- 7:29export feature. If Graphify can instantly map
- 7:33the complex logic of a massive software project
- 7:36and turn it into a visual vault, actually wait.
- 7:39What happens when we start pointing these deterministic
- 7:41mapping tools at our own personal hard drives,
- 7:44our personal notes, or our scattered thoughts?
- 7:46Two -sex islands. That is a pretty wild concept
- 7:49to think about. Try pointing Graphify at your
- 7:51messiest project this week just to see the map
- 7:53it generates. Thanks for joining us on this deep
- 7:55dive.