Latest / AI Fire Daily / #440 Neil: Agentic Workflow Frameworks Crushing The Context Window Ceiling
Transcript
- 0:00Everyone is fighting the exact wrong war right
- 0:02now. Oh, absolutely. We constantly jump from
- 0:04Claude to ChatGPT to Gemini. Yeah, looking for
- 0:08the next big upgrade. Right. We keep chasing
- 0:11the illusion of a magic model. Beat. But the
- 0:14model is no longer the bottleneck. No, it really
- 0:16isn't. Welcome to this deep dive. Today, we explore
- 0:20what an agentic workflow actually is. It is completely
- 0:23going to change how you work. We will discover
- 0:25why cramming prompts absolutely destroys your
- 0:28output. And we will map the exact method to build
- 0:32daily skills. People still treat AI like a magic
- 0:36search box. And honestly, that is a massive mistake
- 0:40for your daily productivity. We really need to
- 0:42treat it like a brand new employee instead. Like
- 0:45onboarding a real person. Exactly. We are completely
- 0:48changing how we interact with these tools. Right,
- 0:50because an agentic workflow isn't just one megaprompt.
- 0:53No, not at all. It is the entire loop the agent
- 0:55runs through. It includes the specific context,
- 0:58the tools, and your hard rules. Yeah, and it
- 1:01even includes where the results get saved eventually.
- 1:03So it's a whole environment. Picture the model
- 1:05is just one machine on an assembly line. A perfect
- 1:08machine on a broken line still produces broken
- 1:11parts. That is a great way to look at it. You
- 1:13wouldn't hire a brilliant college grad and just
- 1:16walk out. Right. You would never just say, handle
- 1:19the whole company today. Exactly. You have to
- 1:21clearly define what good and bad looks like.
- 1:24But wait, these models have read the entire internet.
- 1:27Yeah, they have. Shouldn't that vast public knowledge
- 1:30be enough for basic tasks? Like, why do they
- 1:33need constant hand -holding for every little
- 1:36thing? because vague inputs will always yield
- 1:39incredibly vague outputs. These models just predict
- 1:43the next likely text chunk mathematically. They
- 1:46know general facts, but not your highly specific
- 1:49standards. They don't know your company's taste.
- 1:52Right. They do not actually think the way that
- 1:54humans think. The agentic workflow is the specific
- 1:57training plan for that new hire. Got it. So public
- 2:00knowledge isn't enough. It needs your specific
- 2:02training plan. That is exactly it. That makes
- 2:04total sense when you frame it like onboarding.
- 2:06But here is the problem. Since the AI desperately
- 2:10needs our instructions, we overdo it. We go way
- 2:13too far. Our natural instinct is to give it absolutely
- 2:16everything immediately. We just dump entire company
- 2:19manuals right into the chat window blindly. And
- 2:21that instinct completely breaks the machine over
- 2:24time. It totally does. We have to talk about
- 2:26the mechanics of the token window here. A token
- 2:29is basically just a tiny fragment of a word.
- 2:31Right. It is how the AI digests information.
- 2:34The AI processes text using these small mathematical
- 2:38chunks. Anthropix beta models are hitting 1 million
- 2:42tokens right now. Which is just a massive amount
- 2:44of data. It is. Most standard models give you
- 2:47200 ,000 tokens. But there's a huge problem with
- 2:50actually using all that space. Right. The recent
- 2:53research from Chroma highlights this perfectly.
- 2:55Yeah, the needle in a haystack test. Exactly.
- 2:58All 18 leading models drop in quality with longer
- 3:02contexts. They enter what the developer community
- 3:04calls the dumb zone. The dumb zone. That sounds
- 3:08incredibly frustrating to deal with. Oh, it is.
- 3:10It happens between 50 ,000 and 150 ,000 tokens.
- 3:15The attention mechanisms inside the neural network
- 3:18literally get diluted. So the agent basically
- 3:21just forgets your initial instruction. Right.
- 3:23It forgets and it gets incredibly sloppy. This
- 3:26degrading phenomenon is widely known as context
- 3:29rot today. I still wrestle with prompt drift
- 3:32myself. to sex silence. It happens to everyone.
- 3:36You ran a brilliant prompt with perfect safety
- 3:39guardrails. And then 20 messages later, the AI
- 3:42completely forgets those fundamental rules. Yes.
- 3:45An agent that is sharp at message five fails
- 3:47at 30. It just totally loses the plot. And this
- 3:50brings us directly to the 95 % rule. 95 % of
- 3:53users do not need giant instruction files. because
- 3:56it just wastes resources. Repeating known facts,
- 3:59like explaining what TypeScript is, just burns
- 4:02tokens. You can easily burn 140 ,000 tokens over
- 4:0520 turns. Wow, just on repetitive basics. You
- 4:08are paying precious context for information the
- 4:10agent already knows. So why do we instinctively
- 4:13react to an agent's failure by adding more? Like,
- 4:16why do we constantly cram more rules into the
- 4:19prompt? Because we mistakenly think that more
- 4:21text creates more safety. We think we are building
- 4:25better walls. In reality, your core signal just
- 4:28gets buried under useless noise. The AI simply
- 4:31loses the plot entirely when it gets overwhelmed.
- 4:34So we are just confusing it. You should only
- 4:36use large files for very specific things. Things
- 4:38like mandatory legal compliance language or strict
- 4:42brand voice guidelines. Makes sense. We think
- 4:45more text adds safety, but it just creates confusing
- 4:48noise. Exactly. It is honestly like shouting
- 4:50at someone in a crowded room. That is a perfect
- 4:53analogy. So if we cannot cram the context window,
- 4:56what do we do? Right. How are we supposed to
- 4:59give the AI the rules it actually needs? There
- 5:01has to be a better way. The elegant answer is
- 5:03to use something called skills. Right. A skill
- 5:05is just a small file with detailed instructions.
- 5:08Keep it super simple. It has a name, a short
- 5:10description, and an instruction block. And the
- 5:12secret here relies on a concept called progressive
- 5:15disclosure. Which Anthropic talked about recently.
- 5:18Right. The agent only loads the name and description
- 5:21into memory initially. It acts like a lightweight
- 5:23menu of available tools. Just a menu, not the
- 5:25whole meal. The heavy instruction body loads
- 5:28only if the agent actively needs it. The AI triggers
- 5:31a specific action. based on a semantic match.
- 5:34The token math on this mechanism is absolutely
- 5:36incredible. It really is wild. The metadata only
- 5:39costs about 50 tokens to scan initially, but
- 5:43the hidden instruction body might be 5 ,000 tokies
- 5:46long. This creates a nearly 18 to 1 token savings
- 5:49ratio for you. Whoa, imagine saving tens of thousands
- 5:52of tokens a week just hiding instructions. It
- 5:54adds up so fast. It is like stacking Lego blocks
- 5:57of data efficiently. It keeps the context window
- 6:00perfectly clean and high. focused? It is a massive
- 6:03upgrade for your daily workflow. For sure. But
- 6:06there's a huge warning we need to discuss right
- 6:08now. Okay what is the catch? You should never
- 6:10use random downloaded skills from the internet.
- 6:13Wait if I shouldn't use public skills aren't
- 6:15I stuck? Writing custom code for an agent sounds
- 6:18like a massive bottleneck like how do I actually
- 6:21create one without just guessing the code? Because
- 6:23public skills can contain highly dangerous prompt
- 6:26injections. Oh wow. They might have strange API
- 6:29endpoints hidden deep in the code. They could
- 6:32even contain malicious data exfiltration steps
- 6:35inside them. So they are stealing your data?
- 6:37You are essentially letting a total stranger
- 6:40write scripts for your agent. That sounds like
- 6:42a massive security risk for any company. It is.
- 6:45And even if they are totally safe, they were
- 6:47written for someone else. Right, for a different
- 6:49company's workflow. The seams will show up very
- 6:52fast in your own setup. So you read the public
- 6:55skills for structural inspiration only. Just
- 6:57to see how they did it. You borrow their general
- 6:59architecture, but you write the code yourself.
- 7:02You tailor it perfectly to your own proprietary
- 7:05data. Right, so borrow structure from public
- 7:07skills, but write the actual code yourself. Exactly.
- 7:10Protect your workflow. Midroll sponsor read.
- 7:13mid -roll sponsor read, content inserted here,
- 7:16and mid -roll sponsor read. So let's look at
- 7:19the exact step -by -step process of building
- 7:21one. Yes, let's get into the practical side.
- 7:23We are going to build a daily competitor tracking
- 7:25bot today. I really like this practical example
- 7:28from the outline. This bot checks five competitor
- 7:31websites every single morning. Automating the
- 7:34boring stuff. It looks at their pricing pages,
- 7:37blogs, change logs, and X accounts. Now if you
- 7:40use a lazy prompt, you just ask what is new.
- 7:43Which never works well. The agent will immediately
- 7:45fail and ask you confusing questions. It will
- 7:48ask about RSS readers, web scrapers, or visual
- 7:51comping. Right, because you gave it an incredibly
- 7:53vague objective. It doesn't know how to start.
- 7:56To fix this, you have to build a very clear spec.
- 7:59You need to add specific exclusion rules directly
- 8:02to the prompt. Give it actual boundaries. You
- 8:04tell it to only flag if a price moves 10%. Or
- 8:08flag it if a brand new paid tier suddenly appears.
- 8:11Yes. Or if an ex post has over 200 likes. Exactly.
- 8:15Then you tell it to save that output to a Notion
- 8:17page. Keep it organized. But here is the absolute
- 8:21golden rule of this entire process. You must
- 8:25run the workflow completely manually at first.
- 8:29You have to actually prove the logic works in
- 8:31reality. Yes. Do not skip this step. You wait
- 8:35for two completely clean back -to -back manual
- 8:38runs. Just to be sure it wasn't a fluke. Only
- 8:40when it succeeds twice do you save it as a skill.
- 8:43And you don't even write the skill description
- 8:45yourself. Well, you let the AI do it. You ask
- 8:47the agent to review those two successful runs.
- 8:50Then it writes its own skill description in under
- 8:5230 words. That ensures the skill is born from
- 8:55a real success. It is not just a blind guess
- 8:58from a human. It uses its own language. The AI
- 9:02knows exactly what trigger words will activate
- 9:04it best. That is the only kind of skill worth
- 9:07keeping around. But what happens when reality
- 9:09actually sets in on a Tuesday? Oh, it always
- 9:11does. Like a website completely changes this
- 9:13layout without any warning, or an API throws
- 9:16a random 503 error during a run, the skill is
- 9:19going to trip and break eventually. Of course
- 9:21it will break when the real world shifts. So
- 9:23what do you do? When it trips, you don't just
- 9:26rewrite it from scratch. You use what we call
- 9:28recursive improvement loops. Okay, how does that
- 9:30work? You ask the AI why it broke, and you fix
- 9:34it together. Ah. So we use recursive improvement
- 9:36loops to fix and update it together. Exactly.
- 9:39You iterate on the errors. But once you get one
- 9:42skill working, a dangerous temptation kicks in.
- 9:45Oh, I know what you were going to say. People
- 9:47immediately want to build a massive complex network
- 9:51of agents. I know everyone listening right now
- 9:53immediately wants to build swarms. You want 15
- 9:55sub -agents talking to each other with a maze
- 9:57of arrows. It looks so cool on a whiteboard.
- 9:59But you are saying that is a massive trap for
- 10:02beginners. It is the trap of wanting to look
- 10:04impressive online. You have to build a hut before
- 10:07you build a cathedral. Start small. Stick to
- 10:10one main agent and maybe three to five skills.
- 10:13You should only scale that system when your productivity
- 10:16truly demands it. Let's talk about using your
- 10:19actual code as context. Because the agent can
- 10:21read things straight from your code base. Right.
- 10:24It sees Next .js, SuperBase, and Tailwind automatically.
- 10:27It knows what they are. Do not waste your precious
- 10:29tokens explaining those public frameworks. You
- 10:32should be using clean starter templates instead
- 10:35of long explanations. There is a really great
- 10:38screening question for this context. You just
- 10:40ask yourself, can the AI find this information
- 10:43publicly? That is the perfect filter. If it can
- 10:46find it on the open web, Skip adding it. Don't
- 10:49repeat the internet. You only add the things
- 10:52the AI absolutely cannot find publicly. Things
- 10:55like your highly specific customer personas or
- 10:58pricing logic. Exactly. Or the specific things
- 11:01your company likes to say no to. That taste and
- 11:05constraint is highly proprietary to your business.
- 11:08Those are the high value pieces of context you
- 11:10have to protect. But isn't it safer to just remind
- 11:13the agent what framework we use just in case
- 11:15it somehow forgets the syntax during a long task?
- 11:18No, reminding it just clutters the precious context
- 11:21window unnecessarily. It takes up that valuable
- 11:24space. It pushes you right back toward the dumb
- 11:26zone we discussed earlier. Keep it out if public,
- 11:29included if proprietary. I see. Skip it in public.
- 11:31Keep it only if it is completely proprietary.
- 11:34We now have a blueprint for a lean, simple workflow.
- 11:38Which is great. But you need to know about the
- 11:40painful reality of the first 14 days. Let's share
- 11:43that skeleton template example for a weekly revenue
- 11:46review. It pulls your financial data directly
- 11:49from Striken Notion. Super useful automation.
- 11:52It calculates week -over -week deltas and flags
- 11:5415 % moves. Then it saves a clean five -line
- 11:58summary for your team. It sounds great, but getting
- 12:01this to work means hitting the two -week wall.
- 12:03The two -week wall. The first 10 to 14 days are
- 12:06gonna honestly hurt. You are gonna spend way
- 12:09more time fixing errors than producing. The automated
- 12:12system feels significantly worse than just doing
- 12:15it manually. It is super frustrating at first.
- 12:17Yeah. But around day 14, you finally hit the
- 12:20turning point, the recursive loops pay off, and
- 12:23the skills stop breaking. Everything just starts
- 12:25clicking. A reporting task that used to take
- 12:27an hour now takes 10 minutes. And this creates
- 12:30a massive career advantage for you over others.
- 12:33People using tight, agentic workflows are going
- 12:36to do the work of three people. They really are.
- 12:38A solo founder. with five sharp skills becomes
- 12:41incredibly fast. They will easily outrun a 10
- 12:45-person team that is still arguing about tools.
- 12:48That leverage is entirely real, but you have
- 12:50to actually earn it. Is this initial friction
- 12:53the real reason people mistakenly blame the AI
- 12:56models? Like, is this why they jump to a different
- 12:58provider the second something fails? Absolutely.
- 13:01They blame the machine because the initial training
- 13:03is hard. They don't want to do the work. They
- 13:05quit at the wall instead of pushing through the
- 13:07friction. They refuse to pay the initial training
- 13:10cost for their new employee. Exactly. They quit
- 13:14at the two -week wall instead of paying the training
- 13:15costs. The big takeaway here is that the model
- 13:18was never the bottleneck. It was always our process.
- 13:21Less context, clearer rules, and smaller starts
- 13:24are the absolute keys. They are what actually
- 13:27create a true agentic workflow for you. Keep
- 13:30it lean. Do it by hand, get two clean rends,
- 13:33and make it a skill. Thank you for joining us
- 13:35on this deep dive today. I want to leave you
- 13:37with a final thought before we go. Beat. Think
- 13:40about a repetitive task you do every single morning.
- 13:42We all have them. What are the unspoken, invisible
- 13:45rules sitting in your head right now? The ones
- 13:47you have never explicitly written down for another
- 13:50human? Let alone an AI. How could an AI ever
- 13:53guess them if you don't write them down? We used
- 13:55to think the magic was always in the model, but
- 13:58the real magic is making your invisible rules
- 14:00visible. OTRO music.