Latest / AI Fire Daily / #112 Robin: Claude Code + Hermes Can Build an AI Agent That Keeps Learning

Transcript

#112 Robin: Claude Code + Hermes Can Build an AI Agent That Keeps Learning · AI Fire Daily

• 0:00
  You know, you spend hours, literally hours teaching
• 0:03
  an AI exactly what you want. Oh yeah. You feed
• 0:05
  it all your specific rules and formatting preferences.
• 0:08
  Right. It learns your unique style perfectly,
• 0:10
  you know, during a long, intense session. It
• 0:13
  really feels like you finally have the perfect
• 0:15
  digital assistant. Until you don't. Exactly.
• 0:19
  Because then you close your laptop for the night
• 0:21
  and the next morning it just starts from zero.
• 0:24
  all over again. It has complete and total amnesia
• 0:27
  about your previous work together. Which is,
• 0:30
  I mean, a deeply frustrating experience for anyone
• 0:32
  building serious systems. Yeah, it really is.
• 0:35
  So welcome to the deep dive. Today we are analyzing
• 0:38
  a very specific architectural setup. We are looking
• 0:42
  closely at Claude code paired with Hermes. Right.
• 0:45
  And our mission today is exploring how this setup
• 0:48
  actually solves AI amnesia. because we are going
• 0:52
  to explore how dividing labor fundamentally changes
• 0:55
  your workflow. You essentially split the daily
• 0:57
  work between a dedicated builder and an operator.
• 0:59
  Yeah, and they are linked together permanently
• 1:02
  by a shared memory layer. Right, and that memory
• 1:05
  connection allows you to create a self -improving
• 1:08
  AI agent. A specialized agent that learns from
• 1:11
  its own failures automatically over time. It
• 1:14
  actually gets better and more efficient while
• 1:15
  you're sleeping. It really is a massive paradigm
• 1:18
  shift for automated workflows, you know? You
• 1:21
  are moving away from having one massive AI do
• 1:24
  everything for you. We are splitting the actual
• 1:26
  work into highly specialized software roles.
• 1:29
  Exactly. You get a digital ecosystem that accumulates
• 1:32
  real wisdom over time. Beat. Well, let us define
• 1:35
  the clear division of labor required here first.
• 1:37
  Right. You need a strict boundary between these
• 1:39
  two distinct software tools. Giving each tool
• 1:42
  one clear job creates a much stronger system
• 1:45
  overall. So first, we have Claude Code acting
• 1:48
  as the dedicated system builder. You sit at your
• 1:50
  keyboard for focused, active building sessions.
• 1:54
  You give it complex instructions and review what
• 1:56
  it builds carefully. You test the immediate result
• 1:59
  and make necessary structural changes together.
• 2:01
  Right, like it could build a website, a script,
• 2:04
  or complex business automations. You use it to
• 2:07
  create the entire system architecture from scratch,
• 2:09
  basically. But you know, it has one really annoying
• 2:12
  weakness on its own. The amnesia. Exactly. When
• 2:16
  your active session ends, the useful project
• 2:18
  context disappears completely. Claude is absolutely
• 2:21
  brilliant at building complex software systems
• 2:24
  from scratch. But it lacks permanence in its
• 2:26
  standard default form today. Right. It simply
• 2:29
  forgets the nuanced technical decisions you made
• 2:32
  together recently. And that is exactly where
• 2:34
  Hermes comes into the picture for us. Hermes
• 2:36
  acts as the dedicated, always -on operator in
• 2:39
  this unique setup. Once that initial system exists,
• 2:42
  Hermes keeps it running independently forever.
• 2:45
  You can run Hermes locally on your own desktop
• 2:47
  machine, right? Yeah. Or you can deploy it remotely
• 2:49
  on a dedicated cloud server. So you assign it
• 2:52
  schedule jobs and let it monitor background work.
• 2:55
  Right. And it handles the daily grind without
• 2:57
  needing constant supervision. But its primary
• 3:00
  weakness is handling heavy coding work completely
• 3:03
  from scratch. Right. You would not use Hermes
• 3:06
  to build serious software systems initially.
• 3:09
  That specific difference in their fundamental
• 3:11
  capabilities really matters here, I think. Yeah,
• 3:14
  there is naturally some overlap between the two
• 3:17
  distinct AI tools. I mean, quad code can automate
• 3:20
  some basic daily tasks in a limited way. And
• 3:24
  Hermes can technically work with complex code
• 3:26
  when it really has to. But you get much better
• 3:29
  results respecting their individual core strengths.
• 3:31
  Exactly. You build the system first using quad
• 3:34
  code entirely. Then you hand it over to Hermes
• 3:36
  to keep running daily. Beat. You know, it is
• 3:40
  a fascinating dynamic. I like to think about
• 3:42
  it like commercial real estate, actually. Oh,
• 3:44
  how so? Well, Claude is the high -priced architect
• 3:47
  who designs the office building. OK, yeah. The
• 3:49
  architect draws the intricate blueprints and
• 3:52
  eventually leaves the site. Right. They are not
• 3:54
  sticking around to check the daily plumbing issues.
• 3:57
  Exactly. Hermes is the 247 property manager keeping
• 4:00
  the lights on constantly. The property manager
• 4:03
  handles the daily maintenance and routine building
• 4:06
  security. They have completely different skill
• 4:08
  sets and totally different daily priorities.
• 4:11
  You would not ask your expensive architect to
• 4:13
  fix a leaky pipe. Right. But what happens when
• 4:15
  the building actually needs a structural upgrade?
• 4:18
  I mean, does Hermes ever try to rewrite? Claude's
• 4:21
  core code if it spots a bug. Well, usually Hermes
• 4:24
  just spots the bug and gathers the diagnostic
• 4:27
  data. It flags the underlying problem so the
• 4:30
  original builder can address it. It knows its
• 4:32
  own limits when it comes to massive structural
• 4:35
  changes. Got it. Hermes flags the issue, but
• 4:38
  Claude... does the actual remodeling. Exactly.
• 4:42
  Because the architect and property manager have
• 4:44
  very different technical skills. We really need
• 4:46
  to see how they manage real projects today. Let
• 4:49
  us look at them operating out in the wild. Website
• 4:51
  monitoring is a really great starting point to
• 4:53
  understand this. Yeah, so Cloud Code builds the
• 4:56
  initial site and the complex backend infrastructure.
• 4:59
  It builds the detailed sign -up flow and any
• 5:01
  surrounding automations required. But building
• 5:03
  the actual website is really only the first part
• 5:05
  today. Right. The site needs to be closely watched
• 5:08
  and maintained carefully afterward. So after
• 5:10
  the public launch, Hermes takes over the ongoing
• 5:13
  daily monitoring work. It watches for new user
• 5:16
  signups and email open rates constantly. It monitors
• 5:20
  conversion rate changes and looks closely for
• 5:22
  broken pages. It watches for sudden unexplained
• 5:25
  drops in important daily performance metrics.
• 5:28
  And flags them for Claude. So websites are a
• 5:31
  great example of monitoring static digital infrastructure.
• 5:34
  But what if the required output is something
• 5:37
  highly creative instead? Oh, that is where it
• 5:39
  gets really fun. Right. Like, can an operator
• 5:42
  like Hermes actually help with something totally
• 5:45
  subjective? Yes. This setup gets even more interesting
• 5:48
  with video content creation. OK, walk me through
• 5:50
  that. So Claude Code builds a custom video editing
• 5:54
  skill for you initially. You define exactly how
• 5:57
  the video cuts should feel dynamically. Like
• 5:59
  you define how fast the individual scenes should
• 6:02
  move along smoothly. Right. specific editing
• 6:05
  choices you want the system making. Then Hermes
• 6:07
  looks closely at what happens after the video
• 6:09
  launches. And it does not just check the basic
• 6:11
  public view counts alone. Right. It goes deeper.
• 6:14
  Much deeper. It takes the time series data from
• 6:17
  your video analytics dashboard directly. Oh wow.
• 6:20
  Yeah. It analyzes second by second viewer retention
• 6:23
  across the entire published video. It sees exactly
• 6:26
  where human viewers lose attention. and lead
• 6:29
  immediately. It notices exactly where your vocal
• 6:32
  delivery slows down too much. It spots precisely
• 6:35
  where a section feels too long for viewers. Low.
• 6:38
  Imagine an AI analyzing second -by -second human
• 6:41
  attention drops to direct its own future edits.
• 6:45
  Wait, hold on. Are you saying the AI is literally
• 6:47
  watching the timeline data? Yes, exactly. It
• 6:50
  is analyzing the data to see where humans actually
• 6:53
  get bored. Yeah, it is a profound shift in how
• 6:55
  we think about creative feedback loops. It uses
• 6:58
  that precise engagement data to direct its future
• 7:01
  edits. Right. Hermes creates a short technical
• 7:03
  brief explaining what should change next. And
• 7:05
  that brief goes directly back into Claude code
• 7:07
  for immediate improvements. So each finished
• 7:09
  video creates valuable new information for the
• 7:12
  next one. This system learns what holds human
• 7:14
  attention and what completely fails. The exact
• 7:17
  same operational structure also works perfectly
• 7:20
  for client operations. Oh, like building deliverables.
• 7:23
  Yeah, Claude builds the technical deliverables
• 7:25
  you sell to your clients regularly. And then
• 7:28
  Hermes helps with everything that happens after
• 7:30
  the final client delivery. It tracks client activity
• 7:33
  and watches for unfinished administrative tasks
• 7:36
  closely. It catches operational issues early
• 7:40
  before they become much bigger problems. So Claude
• 7:42
  builds what the client needs initially to get
• 7:46
  started properly. While Hermes helps keep the
• 7:48
  entire business operation moving forward steadily.
• 7:51
  But there is a crucial foundational rule for
• 7:55
  all these new workflows. Which is? None of these
• 7:57
  complex systems will work perfectly on day one.
• 8:00
  Right, obviously. You build the first version
• 8:02
  and let Hermes run it alone. Just observe it.
• 8:04
  Yeah. You fix the obvious weak points and stabilize
• 8:07
  the system completely. Only then do you add more
• 8:10
  complexity to it safely. Start with one stable,
• 8:13
  simple workflow before building the next one.
• 8:15
  You have to establish a really solid foundation
• 8:17
  first every single time. But... You know, is
• 8:21
  this workflow entirely dependent on these specific
• 8:23
  tools? Like, can you swap out Claude code for
• 8:26
  something else entirely? Yes, absolutely. You
• 8:28
  just need a capable builder and a reliable operator.
• 8:31
  You could use different models if they support
• 8:33
  the same architecture. Makes sense. ChatGPT or
• 8:36
  Codex can build. The split roles are what matters.
• 8:38
  Yeah, exactly. Beat. But there's an obvious problem
• 8:41
  with this entire theoretical setup, right? What
• 8:43
  is that? Well, Hermes writing a video brief for
• 8:46
  Claude is actually completely useless. Ah! It
• 8:51
  is entirely useless if Claude cannot actually
• 8:54
  read the brief. Yes, the communication gap. How
• 8:57
  do we get the property manager's notes delivered
• 8:59
  reliably? How does the architect actually see
• 9:02
  what needs to change next? Well, by default,
• 9:05
  these AI tools do not share any knowledge natively.
• 9:08
  Right. A conversation on your phone with Hermes
• 9:10
  stays totally isolated forever. It will not magically
• 9:13
  transfer over to Claude code later on. And the
• 9:16
  exact same thing happens in reverse during active
• 9:18
  building sessions. You spend an hour explaining
• 9:20
  a complex project to Claude code. And Hermes
• 9:23
  does not automatically get any of that essential
• 9:25
  background context. So you end up explaining
• 9:28
  the exact same thing twice to different tool.
• 9:31
  I have to admit, I still wrestle with prompt
• 9:33
  drift myself. Oh, it's the worst. Constantly
• 9:36
  repasting context into a fresh window is an exhausting
• 9:40
  daily process. It really is. It often feels like
• 9:43
  you're taking two steps forward and one back.
• 9:45
  Yeah, it is a very common frustration for most
• 9:48
  people building things today. Right. But that
• 9:50
  is exactly where AI agent memory comes into play
• 9:53
  perfectly. Both of these distinct tools connect
• 9:55
  to the exact same shared memory layer. You can
• 9:59
  conceptualize this memory layer as one shared
• 10:02
  digital notebook. Commies can write something
• 10:04
  new into this shared notebook absolutely anytime.
• 10:06
  And Claude code can read that stored information
• 10:08
  later when it builds. Claude can also save a
• 10:10
  new structural decision for Hermes to use. They
• 10:13
  basically leave permanent notes for each other
• 10:15
  to find later. To connect them securely, Hermes
• 10:18
  just needs your unique API key, which is basically
• 10:22
  a secret code letting programs securely talk
• 10:25
  together. Exactly. You just make sure both tools
• 10:28
  use the exact same memory ID. And there are a
• 10:30
  few different memory providers available on the
• 10:32
  market today. Memono is probably the easiest
• 10:35
  place for you to start experimenting. Because
• 10:37
  it is incredibly fast, simple to set up, and
• 10:40
  totally free. ZEPP is highly useful for tracking
• 10:43
  how information changes over time naturally.
• 10:47
  It tracks what was actually true before information
• 10:50
  was updated recently. Letta gives you much more
• 10:53
  direct visibility into the memory itself. Oh,
• 10:56
  so you can closely inspect how that memory actually
• 10:59
  gets updated internally. Yeah, and SuperMemory
• 11:01
  is highly useful for a simpler local machine
• 11:04
  setup completely. You do not need to overthink
• 11:07
  this specific part initially today, though. No,
• 11:09
  just pick one memory provider and connect both
• 11:11
  of your tools together. So after setup, you should
• 11:14
  test the connection with something incredibly
• 11:15
  simple. Oh, I love this test. Right. Tell Claude
• 11:18
  your favorite food is fish and chips. Tell it
• 11:21
  your second favorite is warm truffle pizza. Then
• 11:24
  you open Hermes and ask for your top favorite
• 11:26
  foods. If Hermes gives the same exact answer,
• 11:30
  the shared memory works perfectly. You now have
• 11:32
  tools carrying context across different complex
• 11:35
  jobs effortlessly. They can finally learn from
• 11:38
  each other in real time constantly. Does setting
• 11:43
  up this memory layer require ripping out existing
• 11:47
  memory systems? I mean, does it break the internal
• 11:50
  memory inside Claude or Hermes? No, not at all.
• 11:52
  It is designed to be highly modular and completely
• 11:55
  non -destructive. It integrates smoothly without
• 11:58
  replacing their core internal operating systems.
• 12:01
  Right, so this memory layer just sits peacefully
• 12:03
  on top of your existing tools. Exactly. Once
• 12:05
  they share a single brain, the entire system
• 12:08
  changes fundamentally. It moves from just running
• 12:10
  tasks to actively learning daily. It starts learning
• 12:13
  from its own mistakes over time without prompting.
• 12:16
  Right. Imagine Hermes is running a workflow and
• 12:18
  something fails completely. Maybe an important
• 12:20
  client email gets missed during a busy week.
• 12:23
  Or maybe a conversion rate suddenly drops for
• 12:25
  no apparent reason. Hermes inspects what happened
• 12:27
  and figures out the actual root cause. And then
• 12:31
  it updates the specific instructions it follows
• 12:33
  for next time automatically. That behavioral
• 12:36
  correction gets saved into the shared agent memory
• 12:38
  forever. For example. Hermes saves a new rule
• 12:42
  about your active clients. Always verify the
• 12:45
  current client status before sending the weekly
• 12:48
  report. That important lesson does not disappear
• 12:50
  after one quick passing conversation. It becomes
• 12:53
  a permanent, reliable behavior for the workflow
• 12:56
  going forward. The system improves silently through
• 12:59
  its normal daily background work. Hermes sees
• 13:02
  the flawed result and you correct the mistake
• 13:04
  quickly. That specific lesson gets stored and
• 13:07
  reused for the future automatically. The entire
• 13:10
  workflow becomes much more reliable over long
• 13:12
  periods. It keeps accumulating small corrections
• 13:14
  instead of repeating past failures constantly.
• 13:17
  The full improvement loop looks exactly like
• 13:19
  this in practical terms. Cloud Code builds the
• 13:21
  system and Hermes runs it daily. Hermes learns
• 13:24
  on the job and the shared memory keeps the lesson.
• 13:27
  Then Cloud Code upgrades the underlying system
• 13:29
  when it is needed. You are not manually rebuilding
• 13:32
  the workflow every single time anymore. Right.
• 13:34
  And it happens completely organically over long
• 13:37
  stretches of time. You know, this mirrors human
• 13:39
  neuroplasticity in a really fascinating way,
• 13:42
  conceptually. It really does. Termes handles
• 13:45
  the small daily synaptic adaptations and routine
• 13:49
  operational adjustments. While Claude handles
• 13:51
  the major foundational learning and deep structural
• 13:54
  changes required. It is a beautiful division
• 13:56
  of cognitive labor in software design. It mimics
• 13:59
  exactly how our own brains handle new habits
• 14:02
  daily. But who decides when a small daily tweak
• 14:05
  is not enough? Like, who decides when a massive
• 14:08
  Claude upgrade is actually required? Well, when
• 14:11
  a specific error persists constantly, you must
• 14:14
  step in and escalate. Hermes flags the recurring
• 14:17
  failure, demanding a much deeper a technical
• 14:20
  fix from the builder. Okay, so Hermes hits a
• 14:22
  hard wall, and you tag Claude back in. Yeah,
• 14:26
  that's exactly it. Before you hand over the keys
• 14:28
  to this fully autonomous loop, though, you need
• 14:30
  stripped guardrails to prevent quiet background
• 14:32
  disasters from happening. The system could quietly
• 14:35
  drive off a cliff without them installed safely.
• 14:37
  Yeah, there are five practical tips for setting
• 14:39
  this workflow up safely today. Let's go through
• 14:42
  them. Tip one. First, you must intentionally
• 14:45
  split your AI models by operating cost. You want
• 14:48
  to use your absolute strongest model when Claude
• 14:51
  is building. That is where you want stronger
• 14:53
  reasoning and much fewer mistakes. But you should
• 14:56
  run Hermes on a cheaper or local model instead.
• 14:59
  Hermes just handles the daily repeat background
• 15:02
  tasks for you tirelessly. This strategic split
• 15:05
  keeps your expensive computational reasoning
• 15:07
  focused where it matters most. Exactly. Tip two
• 15:10
  is to design the handoff from the very beginning.
• 15:13
  You need to explicitly tell Cloud Code that Hermes
• 15:15
  will run the system. So Cloud then creates clearer
• 15:18
  instructions and much cleaner outputs for it.
• 15:20
  It creates simpler modular commands that Hermes
• 15:23
  can call easily anytime. Tip 3 is to enforce
• 15:26
  strict self -verification on all tasks performed.
• 15:30
  Right. You add a simple instruction to every
• 15:32
  single important task generated. You tell the
• 15:34
  system to verify the result before marking the
• 15:37
  task complete. This one simple verification step
• 15:40
  catches a surprising number of silent failures.
• 15:43
  Like if Hermes updates a text file, it must open
• 15:46
  and check the file. If it changes a live web
• 15:48
  page, It confirms the page actually works. It
• 15:51
  sounds basic, but it works. But can an AI reliably
• 15:55
  verify its own work like that? I mean, can an
• 15:57
  AI that just made a mistake catch that same exact
• 16:01
  mistake? Well, it checks the final output state,
• 16:04
  which reveals its own earlier mistakes. It forces
• 16:07
  the model to objectively re -evaluate the actual
• 16:10
  finished product. It's not flawless, but asking
• 16:13
  it to double -check catches a shocking amount
• 16:15
  of errors. Yeah, it really does. We're going
• 16:19
  to take a brief moment right here. This deep
• 16:21
  dive is supported by the dedicated team at Framework.
• 16:24
  When you are designing complex systems, your
• 16:26
  physical environment really matters. Absolutely.
• 16:29
  Framework builds beautiful, height -adjustable
• 16:31
  desks for deep, highly focused work. They use
• 16:33
  premium solid wood and incredibly quiet dual
• 16:36
  motors inside. It is the perfect physical foundation
• 16:38
  for your daily building sessions. You can find
• 16:40
  their entire lineup at frameworkdesks .com. Now,
• 16:44
  let us get back to the system architecture discussion.
• 16:46
  Okay, so tip four. Tip four is establishing strict
• 16:49
  approval gates for overall system safety. Low
• 16:53
  -risk background work is perfectly fine on its
• 16:55
  own, mostly. But you must require human approval
• 16:59
  for any potentially risky actions. Like sending
• 17:02
  external emails or spending real money should
• 17:05
  require your explicit permission. Deleting important
• 17:09
  files or publishing content publicly should need
• 17:12
  your human eyes first. You never want an AI executing
• 17:15
  those specific actions completely unchecked.
• 17:18
  Definitely not. Tip 5 is setting up clear escalation
• 17:21
  protocols for strange recurring errors. Let Hermes
• 17:24
  actively call Claude code when it needs deep
• 17:27
  technical help. Hermes might notice the exact
• 17:29
  same error appearing every single week. It hands
• 17:31
  the complex problem back to Claude for a deep
• 17:34
  structural fix. It gets the underlying system
• 17:36
  updated and continues running smoothly afterward.
• 17:38
  You can also create a dedicated coder profile
• 17:40
  specifically for this escalation process. It
• 17:43
  acts as an emergency contact when the operator
• 17:45
  gets totally stuck. Right. Let us briefly synthesize
• 17:49
  this entire deep dive together right now. The
• 17:51
  era of massive monolithic AI chats is quickly
• 17:55
  shifting away. We are definitely moving into
• 17:58
  modular AI workflows right now today. You carefully
• 18:01
  separate the complex roles into a dedicated builder
• 18:04
  and operator. You bridge their natural gap with
• 18:07
  a shared persistent memory layer. This creates
• 18:10
  a digital ecosystem that actually accumulates
• 18:13
  valuable wisdom over time. It never starts from
• 18:16
  scratch every single morning for you again. But,
• 18:18
  you know, this new architectural setup leaves
• 18:19
  us with a truly fascinating question. Yeah, if
• 18:22
  we are building autonomous systems where the
• 18:25
  operator manages its own memory. It carefully
• 18:27
  audits its own failures and summons its own architect
• 18:30
  to upgrade its code. Are we rapidly shifting
• 18:33
  away from being the active creators of our work?
• 18:35
  It's wild to think about. Are we essentially
• 18:38
  becoming HR managers for highly capable synthetic
• 18:41
  employees. Two sec silence. Wow. Thank you for
• 18:45
  taking this deep dive with us today. Keep questioning
• 18:47
  the automated software systems you use every
• 18:49
  single day. Keep exploring what is actually possible
• 18:52
  in this constantly evolving technological landscape.
• 18:55
  It is moving fast. We will see you next time
• 18:57
  on the deep dive.