Latest / Superlinear / How to Build AI Agents Like Dots, Instinct, Muse & Grok Bot

Transcript

How to Build AI Agents Like Dots, Instinct, Muse & Grok Bot · Superlinear

• 0:00
  Christine: A few days ago, OpenAI launched Dots, and it feels like the continuation of a much bigger trend. Personal agents are getting much easier to use. So right now we've got Dots, Muse, Instinct, Grok Bot, OpenClaw, Hermes, and much more.
• 0:16
  Christine: Some are polished products you can start using immediately, and some give you much more control and let you build your own set-up.
• 0:23
  Christine: But once you have all these options, the question becomes when should you use which one?
• 0:29
  Christine: And if you do want to build your own agents, what is the right way to set that up and run it reliably in production or at scale? So that's what we're going to figure out today on Superlinear.
• 0:40
  Christine: What are the different layers you can reach for? What should you build yourself versus take off the shelf? And what changes once you go from experimenting with one agent to actually running these systems for real? Today's episode is also a very special one we're very happy to have the founder of Agent37, Vishnu Krishnaprasad here.
• 1:02
  Christine: And Vishnu has seen a lot of agents get deployed in the real world. So I'm very excited to hear from him what actually works or breaks in practice.
• 1:12
  Vishnu: Hey guys I'm Vishnu, founder of Agent37 dot com.
• 1:15
  Vishnu: And in Agent37 we basically build sandboxes to run agent harnesses.
• 1:22
  Christine: I was exactly wondering with so many popular agents out there, like which ones are the most popular, you know, which ones are good at what you know, which one should you use for which use cases. Maybe we could walk through a few of them.
• 1:38
  Vishnu: So the way I would classify these harnesses agentic harnesses, so Hermes and OpenClaw, I would put them in one category. These are like what I would call personal agents. So let's say you have a start-up idea and you're trying to build a agent that talks to your customers like Dots or Muse or Instinct these would be great starter harnesses for you to use and build on top of.
• 2:01
  Vishnu: The reason being they come with all the channel integrations, right?
• 2:06
  Vishnu: So, for example, you probably want your users to be able to talk to it on iMessage or Telegram or, WhatsApp or any of these platforms.
• 2:13
  Vishnu: So these harnesses come with it by default and so it's much easier to plug and play and build an custom experience on top of it.
• 2:21
  Vishnu: The other category I will put is Pi.
• 2:23
  Vishnu: Pi is very minimal. I think this is something only developers who are building their own harness, for example, OpenClaw is built on top of Pi as a harness. So it's really great and if you have a very minimal niche use case that you want to use a harness for and you don't wanna build your own harness this is great it's like imagine you have a start-up and you're building something in a niche like I don't know like legal space or government space or something and you want like great control over the harness in terms of what it does what it loads into the context window, how it manages memory this is what I would start with and then in the coding agents, category where say you're building a startup idea where I don't know like Greptile which reviews your code right?
• 3:17
  Vishnu: It needs to check out your code base, it reviews your code and so on then Codex, OpenCode and Claude Code, all of these fall into the category of coding agent harnesses, which would be great to start with for those kind of use cases.
• 3:29
  Christine: Cool. And since Dots came out yesterday or by the time we launched this episode a few days ago where does Dots fit in?
• 3:41
  Vishnu: Dots definitely fits into the personal agent category. Yeah. So everything Dots Muse dot AI Instinct Grok Bot all of them pretty much are in the personal agent category so the starting harness if you wanna build a custom experience like that I would probably pick Hermes or OpenClaw for that.
• 4:00
  Brandon: But it's really for like you use Grok Bot, you use Instinct, you use Dots, Muse, but you don't build on Dots and Muse whereas you can actually build on OpenClaw, Hermes something else So I think that's like that's like an interesting distinction.
• 4:17
  Vishnu: That's right. Yeah, these products are for consumers to just use it Even if you're not trying to build a product by yourselves, if you're a user, super user, where you want a control over your harness much more hands-on and you want control like you wanna look at the files, you want terminal access and all those things, then definitely Hermes or OpenClaw is a far better option than something like Grok Bot or Instinct or yeah or Dots
• 4:43
  Christine: Well at least Brandon you and I have been talking about agents a lot a lot and I also remember that you also had a very interesting mental model how you would describe the spectrum of harness to use when maybe you could share that as well and we could like compare it with this what Vishnu is sharing?
• 5:09
  Brandon: Yeah the way I think about it is I guess like all the way on the bottom end of the spectrum kind of even lower than like if this is a ladder or something.
• 5:22
  Brandon: On the bottom rung is just like an agent loop.
• 5:28
  Brandon: So you like use the agent loop directly.
• 5:31
  Brandon: You do inference, like so we talked about this in an earlier episode, but the agent loop is just LLM does some inference, that inference comes back, there might be a tool call if there's a tool call some code executes and then that gets fed back into the model to do more inference and this keeps going in a loop until the model says OK you're done.
• 5:54
  Brandon: So this is like the lowest level building block I guess. And then if you want to take that and extend it to include like some kind of persistent session or a session at all where there's like history there might be some form of compaction where you're like starting to add on more pieces a system prompt that is more like built-in.
• 6:22
  Brandon: Then like you start to get into kind of Pi territory.
• 6:25
  Brandon: Where Pi is just like an agent loop but it's very configurable and it comes with the session history and like some of these pieces.
• 6:35
  Brandon: And then as you keep going up like the agent SDK from Claude and like these kinds of more these other harness SDKs do a little bit more than Pi out of the box a little less customizable but you start adding pieces like some kind of memory system, I guess Vishnu was saying. But then as you keep going and you get to like Hermes and OpenClaw, like you were saying, OpenClaw has Pi in it.
• 7:03
  Brandon: So what does that add on top of Pi it adds like a gateway so like the ability to communicate to it through Telegram or Instant Messenger or whatever the long-term memory as we already talked about. Yeah
• 7:21
  Vishnu: Yeah I think the big things.
• 7:22
  Vishnu: So when OpenClaw started, the reason it got like super famous, it added the concept of heartbeats you know, where Pi doesn't have a heartbeat, OpenClaw has a heartbeat where every whatever I think default was say five minutes it wakes it up and it like gives a prompt.
• 7:35
  Vishnu: Like the structure of having a soul.md user user.md and waking it up, I think was a key thing. In retrospect, it was a mistake.
• 7:46
  Vishnu: It was an architectural mistake in my opinion because if you look at Hermes agent, it does not have heartbeats. It just manages everything with a cron.
• 7:55
  Vishnu: Because what happened even with our own users, everybody is confused, like what do you use heartbeats for, what do you use cron for and again, cron is an addition on top of Pi, for example. Right? So that and we in fact eventually turned off Heartbeats altogether for all of our users because it was just not, cron was good enough.
• 8:14
  Vishnu: So some harnesses took that decision too, like even Claude Agent SDK does not have Heartbeats but it does have the concept of cron for instance.
• 8:23
  Brandon: Yeah. Yeah. And Thank you. That was what I was looking for.
• 8:29
  Brandon: Like scheduling some kind of advanced scheduling system is what I would kind of call that in the abstract sense, cuz there's like there's cron, heartbeat is like one, maybe, as you're saying, like less useful version of cron maybe there's a use case where it's really important but then there's also like event triggered scheduling which I know is built into OpenClaw and Hermes but is not built into like Claude Code SDK for example where like you have the harness wakes up when it receives an email for example or something like that.
• 9:05
  Christine: So is your philosophy start with the simplest layer of that ladder where you need something and only if you need some more advanced properties or special properties like maybe it's memory, maybe it's proactiveness like scheduling or event triggers, then you move to the next layer where there's something already built for you to take off the shelf.
• 9:31
  Vishnu: Yeah, there is like a rule in you know when you're building good computer science system design which is you always want to start with the absolute minimum you need to and only if your use case grows beyond that should you navigate because historically this is also true for coding right you wanna start with just what you need and what's the absolute minimum you can do to build that and if the use case grows you can mature beyond that.
• 10:00
  Christine: That hundred percent makes sense.
• 10:02
  Christine: And maybe it even is more important in this agentic era because as we're going to build more software with coding agents, the more you build around it that's not relevant often do gets feed into the context window. And then you lose, you get worse attention.
• 10:20
  Christine: So I think this principle that you just talked about, Vishnu, is only becoming more important.
• 10:27
  Vishnu: You're looking for an agentic harness that needs to have good memory by default it is a great one Hermes is I think one of the specialties when you compare it with other agentic harnesses like Hermes or like OpenClaw or Claude Code Codex Grok CLI and many others it does come with in-built memory that is pretty good.
• 10:45
  Vishnu: Recently there were some changes with Claude Code, but up until like maybe two months ago Claude Code did not have any memory of its own beyond what these CLAUDE.md files that you put in your folder, like your code base, or if you have any nested folders you could again put CLAUDE.md there. And so it would only load the context from there beyond like a global CLAUDE.md file. So this was the extent of memory.
• 11:07
  Vishnu: So often I think users, whatever probably some of you may have recollect this, you would talk to it, like in one session you pour your heart out and in the next session it goes back into the same dumb question where you're like, I just explained it to you but then, it's a new thing.
• 11:23
  Vishnu: New session, so new memory.
• 11:25
  Vishnu: So this is how harnesses work.
• 11:27
  Vishnu: Claude has improved it recently they internally within an agent or a computer, they now also manage memories using just Markdown files, which is stored on a per session basis, and the harness actually does refer to it.
• 11:42
  Vishnu: I'm not sure Codex does this just yet so that is like where we are as far as Claude Code's memory goes and the next level of it is maybe OpenClaw.
• 11:53
  Vishnu: When they launched, they were actually one of the pioneers where the memory system was there. You know, it was not like great necessarily, but it was there and the way OpenClaw's memory used to work is I'm not sure if this has changed in the OpenClaw 2.0 version, but the way it used to work is you have like these Markdown files called soul.md and you have like the user.md and memory.md and all these files.
• 12:19
  Vishnu: So any long-term memory that a user has or builds over time, the harness decides that OK, this is maybe worth storing it and so it stores it.
• 12:27
  Vishnu: Or you could maybe prompt your agent, to say hey, save this, it's important to remember to save it. But it was very like plain.
• 12:35
  Vishnu: And memories did not at the time automatically reconcile it did not automatically create.
• 12:42
  Vishnu: And in addition to that, long-term memory, it had a short-term memory where every day of a week it will basically create a folder with a memory log of what happened, like a short summary. And it would load the last two days of memory in the context window.
• 12:57
  Vishnu: And so that was like your bare bones version of memory when it was initially started.
• 13:03
  Vishnu: Now recently they launched something called dreaming which is like automatic consolidation of memory this I think I believe was inspired maybe by Hermes Agent so Hermes Agent since at least the time that I looked into it had a much more sophisticated memory in fact Hermes's memory is not even in a Markdown file.
• 13:22
  Vishnu: They have their own sort of session database and they have their own thing where they store the memory.
• 13:30
  Vishnu: And Hermes started off as a harness that was well advanced in this because not only did it store the memories and its recall was much better than these other harnesses it was also great at remembering the places where it went wrong. So imagine you gave your harness a task and it does X, Y and Z, but maybe it did like steps, like Y one, Y two, Y three, and eventually it went to Z and, you know, it finished the task.
• 13:58
  Vishnu: Some of them may be trial and errors, like it tried something, it didn't work out and then it realized Hermes had the way where it would store these as skills, which is kind of you can think of it as a memory where the next time it runs into this kind of a situation it knows to avoid these steps so that it goes into those errors and uses up your context, you know?
• 14:20
  Vishnu: The thing that it learned during that sessions you know which took up the context window, it saves for future so the next time you do a similar task, it reads the skill and it realizes, oh I can skip this and do Z and go directly.
• 14:34
  Vishnu: So its memory felt a lot more sophisticated beyond just memory of the user talking to it and also like the actions that it did, it kind of self-learned and so even to this day I think from my recent like looking into it, Hermes by far has the most sophisticated memory system to my knowledge where a lot of, at least our customers, what I'm seeing is they're fine with it, they do not need to use some other memory solution on top as long as it's just, we're just talking about the agent's memory with the user.
• 15:11
  Vishnu: Not like an organization like a company brain but something so yeah, that's like the lay of the land.
• 15:16
  Christine: So now we've seen like this whole list and there are like more polished managed products like Dots, Muse and Instinct. But there are also agents like OpenClaw Hermes that expose much more of the machinery and that you can customize.
• 15:33
  Christine: So I'm wondering like maybe this is more of a question for a Vishnu. Have you seen any examples or what are the typical situations when people would reach for OpenClaw and Hermes or build their custom agent instead of reaching for the off-the-shelf products?
• 15:51
  Vishnu: Sure, yeah I think it comes down to the word you use, customization. Like if you're happy with what you're getting from these products like Grok Bot or Instinct or Dots, then I think you should just use that.
• 16:05
  Vishnu: But there comes a point where you become a bit of a power user. Say you want to add your family into a Telegram channel or a WhatsApp channel and you want the bot to answer to them all.
• 16:18
  Vishnu: Right? There is some customization here involved, right? Or maybe you wanna have two Telegram channels or WhatsApp channels and for each of them there is a different topic and you want it to answer something.
• 16:30
  Vishnu: There's an example of a customization that doesn't come out of the box with these finished products.
• 16:34
  Brandon: I just love this like Linux metaphor I also think of like Dots or Muse as like Apple and then to like Hermes and OpenClaw as like Linux, you know?
• 16:47
  Brandon: When you're using Apple products, you're not allowed to configure anything.
• 16:52
  Brandon: You get it as it is.
• 16:53
  Brandon: Too bad. The trade-off is then everything works and it just works. It's so easy to set up.
• 16:59
  Brandon: It's frictionless and so there's sometimes where you really wanna just have your nice Apple devices and they just work and then like in those scenarios that you're saying, Vishnu, like there are times that you wanna do something custom and special and you just can't.
• 17:19
  Christine: I could also imagine that in kind of use cases or some kind of use cases where privacy might play a role, and maybe you want everything to be local or on-prem, and then that could also be another reason to build your own agents. And I guess you must have seen it a lot too, Vishnu.
• 17:39
  Christine: Maybe people are building products where they embed an agent at the heart of their product and they have a proprietary workflow around it or something?
• 17:51
  Christine: I guess then you can also not use Dots or the other out of the box agents for it.
• 17:57
  Vishnu: That's right. Yeah. If you're gonna build a product on top of it, like let's say you are in some niche, you're a real estate agent, you wanna build a product for real estate people because you have some context in you want an agent to do, definitely you cannot start with any of this, user, based applications like Dots and others.
• 18:16
  Vishnu: You need to go into the harness.
• 18:18
  Vishnu: The, I think the earlier what we were showing the levels of harness, there is Hermes and OpenClaw at the top it's a good starting, harness but if you really wanna go fine-grained then you go to Pi or you go to one of those coding agents. So those are kind of your options if you wanna build a product on top of it. Yeah.
• 18:33
  Christine: OK, that answers a little bit about, you know, the software that we're running, what to choose for.
• 18:38
  Christine: And now I wonder, where does this thing actually live? Like, the first persistent agent I built was OpenClaw back in January, and it literally just lived on my laptop.
• 18:48
  Christine: And that was completely fine for the first version for me just playing with one agent.
• 18:56
  Christine: But if I was building an agent into a product and suddenly it might need to run for every customer and wake up on events and I don't know, keep working when my laptop is closed and maybe even potentially have a lot of agents doing work at the same time.
• 19:11
  Christine: At some point I'll, yeah, just leaving my laptop open is not enough anymore.
• 19:17
  Christine: It's not a good architecture.
• 19:19
  Christine: So I'm curious, like, what are the options from there?
• 19:24
  Vishnu: Yeah, that's a great question.
• 19:25
  Vishnu: So I think most people when they try this, like there's like layers of options so to start with your own computer or your Mac Mini, something that you run it on your it's in your control, it's in home, is probably where most people start. But it's also a little bit risky to run these harnesses in your own computer because if something, if it gets prompt injected or if it gets, hacked, then your all your other data is still in that same computer. So you haven't had a chance to like segregate and play around with it till you build trust and see how it goes so but it's the easiest way to get started for sure but I think eventually you'll probably wanna move off of it and especially if you're building a product on top of it the next option you have is you could for example use something like Hetzner or there's these are virtual private server, options that you have. There's a bunch of them like Hetzner and there's a few others like even AWS for that matter you can rent a EC2 instance and do it Azure has this similar thing but it's a little bit hands-on heavy, meaning, it's you have to go to the UI and you have to create a machine and then you have to hand manage this it is fine maybe if you're just playing by yourselves for your own computer instance and you don't want to manage like your own local thing if you're gonna build a product on top of it I would definitely recommend to move to the next layer which is like renting a sandbox or a computer on something like Agent37. There's other options too but this is one of the key places where we specialize which is you can basically rent like, purchase a sandbox that runs, it's like a computer that runs for a user, individual user and it has its own like files, its terminal, browser and everything.
• 21:33
  Vishnu: And it can be up and running all the time or you can sort of also put it to sleep if, let us say, you wanna optimize for cost after a period of like no activity so these would be like the three layers that I would say like running it on your own.
• 21:49
  Brandon: Yeah and Vishnu the I think when I was first trying to understand like what even is a sandbox?
• 21:57
  Brandon: It's like a new term and maybe since you're deep in this like can you just talk through like if I if I'm an engineer and I'm like familiar with like a virtual machine versus like a container or maybe you can just talk through that briefly too then like what is a sandbox related to those things? Yeah.
• 22:18
  Vishnu: Yeah, certainly I think most people when they think of computer, they think of a computer on their home, right?
• 22:23
  Vishnu: Like a PC sitting in but a virtual private server is another name for a computer, but that is not something, sitting in at your home or your place, but it is sitting on the cloud somewhere and the key trick is a virtual private server may not be like from your perspective. You can access it online, you can go there you can boot it, reboot it, and you have a terminal access and everything. It you may even have a desktop where you can click around but it may still be one large server, like a host, and most companies, like including AWS and others even us you have a one big bare metal and then we slice it into these VMs virtual machines and then we give a slice of it to every person because it's just easier to manage and there's some benefits of doing this so that is what a virtual private server or a VM is what you typically hear Sandbox is synonymous to that.
• 23:26
  Vishnu: Is sometimes a smaller slice like for example, for running just an agent like Hermes Agent or Open Claude Code, you probably don't need a full-on computer like a Windows or a Mac or a Linux with all the things that comes with the operating system. Right?
• 23:42
  Vishnu: Cuz the operating system comes with many things, you know.
• 23:45
  Vishnu: It comes with sometimes Microsoft Word or the equivalent in Linux.
• 23:50
  Vishnu: It may come with like a file browser.
• 23:53
  Vishnu: It may come with many other things.
• 23:55
  Vishnu: But imagine you're building a product where you really are just building let's say Grok Bot for real estate agents, right?
• 24:03
  Vishnu: So you probably don't need all the other things you know that comes with it.
• 24:06
  Vishnu: You just want a really lean kind of computer that is able to run has files, has a browser and is able to run some harness that you choose.
• 24:16
  Vishnu: This slightly lean version is called a sandbox which is another term but yeah so that's like a computer, a virtual machine which is on the cloud or a VPS it's a synonym and then sandbox is potentially a slightly leaner version of that because it's cheaper than renting a full computer.
• 24:34
  Brandon: Yeah. And I'm so curious I'm sure some of this is like your secret sauce so you can't share but if you can, like how is it possible that you're able to make the sandbox so cheap?
• 24:50
  Vishnu: There's certain characteristics that are true when you run an agent harness versus when you run other programs. So imagine you're running a game like first-person shooter game, your RAM is going to be pretty highly filled that's an example.
• 25:05
  Vishnu: CPU maybe not as much, it's gonna spike up and down.
• 25:08
  Vishnu: GPU maybe quite a bit.
• 25:10
  Vishnu: Versus when you run something like let's say you're building your app and you're doing a lot of compilation, right? Like a use case where you're compiling it. So generally it's not so RAM heavy but it's probably gonna be very high in CPU heaviness.
• 25:26
  Vishnu: Like you're doing a lot of threads because you're doing a lot of computations similar to that as maybe mathematical computations you're running a Python program right?
• 25:33
  Vishnu: It's gonna and then there is harnesses where if you look at the workload it's mostly just waiting.
• 25:43
  Vishnu: And it's a loop and it's mostly just waiting for the LLM to return a response.
• 25:46
  Vishnu: Right?
• 25:48
  Vishnu: Like the like if you look at the life cycle of it and like a like a real hundred year timeline of a person who is a harness it wakes up and you know it does some things and then it waits for five years for messages to come from LLM and then it wakes up, it does some more activity and it waits for another seven years.
• 26:05
  Vishnu: So that's how it really is from the perspective of, let us say, what the compute utilization looks like.
• 26:11
  Vishnu: And so we optimized for this by making use of this fact and being able to give more users, you know, the share so that they're all utilized equally, you know, when one person is peaking, the other person is not peaking, and so on.
• 26:27
  Brandon: Obviously, I think something that you didn't mention but is super important also with sandboxes is the isolation, like you shouldn't be able to poke other people's files and things when you're when you're sharing compute and things. So what are some of the maybe challenges or kind of details that people might not realize when they if they go and start building this?
• 26:53
  Brandon: I'm sure you were surprised by some things as you were building your product, right?
• 26:59
  Vishnu: The first best practice is you want to make sure to give one computer or sandbox per user.
• 27:05
  Vishnu: Like if you're have let's say ten users, hundred users, whatever it is sometimes you see startups or products maybe because they're not taught deep enough that you're just using you're running every user's workflow or every user in a single sandbox or in a single computer. That would be a big mistake because then it means like your data can leak from user one to user two which you'd really do not want.
• 27:31
  Vishnu: Right? So like user one may have said, hey create me a presentation showing me how much I'm spending per month.
• 27:37
  Vishnu: Right? And so you could route it to a computer and it has some files. Maybe you're choosing a different profile but you're still running it on the same computer for a different user.
• 27:44
  Vishnu: They could say, oh show me all the files you have in your home directory.
• 27:50
  Vishnu: And no matter how much, like, system prompting you've done, it could be jailbroken and another user could have access to the files that a different user has created. So I think the first best practice that I would recommend is just make sure to have sandbox or one computer per customer that you're using, per user that he's using the reason, by the way, why this pattern I've seen some people do is because they're trying to optimize for cost. They, because when you typically look at let us say the cost of renting a server it's like at least twenty-five dollars to thirty dollars and you realize, oh there's no way I can have hundred users or thousand users for with this kind of cost and so that's why that's what leads to that kind of a design system design which I would advise against.
• 28:36
  Christine: OK and now if I would want to build a new product with an agent embedded in it I wonder how I can start.
• 28:46
  Christine: So maybe Vishnu you may have built something yourself or you may have seen a lot of your users build something.
• 28:55
  Christine: Can you talk through the process the step-by-step process how I should go about building such a product.
• 29:06
  Vishnu: Sure. Yeah, certainly maybe I can take an example of one or two of our customers.
• 29:11
  Vishnu: They all, there are common patterns across, niches of problems that people solve.
• 29:17
  Vishnu: And this maybe, help like relate to the audience as to how they could build this on their own. So for example, we have a case study with these guys Boon Funding they're a AI system that fund raises for your nonprofit organization.
• 29:32
  Vishnu: So it's a very, as you can think, it's a very niche system.
• 29:37
  Vishnu: And so typically when a non-profit organization, when you're applying for fundraising, it's a lot of manual process, right?
• 29:45
  Vishnu: So think about like you have to first of all figure out, filter out who do I even apply to and then once you do narrow it down, you need based on like, okay, are we a good match?
• 29:57
  Vishnu: Do we think, are our chances of getting it as high enough? There is a bunch of things, there's questionnaires that they need to fill in and you need to kind of answer all of those things and it could be long documents sometimes, it's not just a simple Q and A.
• 30:11
  Vishnu: And there's a lot of like back and forth that happens. Historically this has been done by founders of you know the org who kind of sit through and do this.
• 30:20
  Vishnu: So this is, imagine you're a founder who was trying to build like a system where you are building a product for your customers who are non-profit orgs, which is what these guys are doing and they want to automate this, they the first few I believe they started doing it themselves via Claude Code and others helping it.
• 30:42
  Vishnu: But you eventually wanna build a system so that you can scale to a lot of users the manual process of doing something like this is you are probably going through these websites and then collecting the data you need and then you're probably working with your Claude Code and refining the documents and so on and so forth.
• 30:59
  Vishnu: During the process, one of the things that I see a lot of our customers do is once you have a workflow that works for your niche in this case it is a lot of things about refining the right data from the existing source documents and then writing documents in a given format that is right this intuition you get by doing this yourselves manually. But once you do, you can have your, coding agent Claude Code, Codex, whatever her name is, create skills for each of them, right? Like once you've had a successful run and you have something working, just make a skill off of it so that the next time you're running it, you just tell it, hey, use this skill, run it on these files, and then it gives you the output.
• 31:42
  Vishnu: So you wanna get to that point first of all before you can try and automate this and in these case Boon I think I believe they had over a hundred skills for different parts of the process because it is a very complicated process, you know, it is to do it like end to end.
• 32:04
  Vishnu: And so the way to sort of break down the problem is you have an agent at the end of the day that is doing a lot of this work but you want to give it these superpowers of niches to begin with and then you add some system instructions to it that tells it this is when you invoke the skill, this is when you invoke the skill, and so on and so forth.
• 32:27
  Brandon: So we have these individual skills they need to compose together in like different subsets of them in different phases what are the choices you have for how to orchestrate that?
• 32:41
  Vishnu: So the simple option is when you are invoking your agent, you can always inject a system prompt for that purpose. So let's say you have a dashboard where you are now initiating phase three, then you inject a system prompt that says for these tasks these skills are good and so please use this.
• 33:07
  Vishnu: Of course when you include a skill it automatically comes in but what we had found not just in other products too it doesn't always reliably invoke the skill and so it is very useful to add that into the system prompt and give it some more explanation.
• 33:22
  Vishnu: Because skills by design is meant to not bloat your context. It's a one-liner. So that's not often enough for the model to choose the right skill.
• 33:29
  Vishnu: So what we often end up doing is for cases like these where we really wanted to invoke that skill, we just add a system prompt saying for this step when you're here you should use this and so on and so forth That's it. I think a simple and elegant way of going about making sure that the right skill gets involved for the workflow or the phase that you're in the second thing that I've seen some people do is Hermes has a concept of profiles where you can invoke a Hermes agent with a specific profile but for the most part I have not found this to be super useful. I think this has tended to be an overkill for at least the use cases that I've seen it's an option. It's like a more grander option if you wanna hyper-customize it because with Hermes profile you can load a different like user.md, soul.md and so on. So really all this injection.
• 34:23
  Brandon: Same with OpenClaw
• 34:23
  Vishnu: Yeah. So you see, yeah, that's how I would think about this how I would think about designing that.
• 34:28
  Christine: What exactly do you tell your agents?
• 34:30
  Christine: Like, OK, these are the different phases.
• 34:32
  Christine: Whenever you go through these phases, you have to trigger, send a notification or send a message.
• 34:38
  Christine: Is that the instruction?
• 34:40
  Vishnu: There's two things.
• 34:40
  Vishnu: First, the skill is not a one-time thing that you build and forget it is a refinement. At least from a lot of startups that I've seen, it is always a refinement the pattern that I see being followed is you have a skills repository, GitHub repository. You may create the first draft, but as you see errors in process again, initially I would say the human will look at these, feed it to their, coding agent and say, we saw these errors, they used these skills.
• 35:08
  Vishnu: What went wrong? Can you refine these skills?
• 35:10
  Vishnu: And so the skills are a continuous repository of the refinement. And typically you can do a Canary deployment. Sorry, go on, Christine.
• 35:19
  Christine: Is that a phase in your workflow as well?
• 35:21
  Christine: Is that a retro phase?
• 35:23
  Vishnu: No. This so the phase itself is how a customer moves from their request all the way from phase one, phase two, phase three to phase four, which is maybe your submission of, this thing. But this is like a outside of that. There's a new process that you are doing as a founder of your product who is trying to refine this whole, parts of these phases where you are refining, maybe you're re-doing the workflows, maybe you're adding humans or removing humans.
• 35:47
  Vishnu: This is kind of runs parallely to that.
• 35:49
  Christine: OK. It's separate.
• 35:49
  Christine: It's not part of the agentic workflow that, recursively improves itself. OK.
• 35:56
  Vishnu: No. Correct. This is just to improve your product and the workflow itself That happens out of the box.
• 36:01
  Brandon: And I really wanna go deep into like how the skills are refined, but maybe that's out of scope for this conversation, cuz it's already insane. But it just curious, like do you see that with folks that you've talked to or yourself, are you manually going back to those skills and working with the agent and just kind of getting a gut check like, oh this feels better?
• 36:28
  Brandon: Do people have like eval suites that are set up that are more formalized?
• 36:32
  Brandon: And what tools are people using for that if you have seen that? So
• 36:37
  Vishnu: Evals is definitely not something you should start with to begin especially when you're just creating a product.
• 36:42
  Vishnu: It has to be you trying it out, you feeling it out.
• 36:46
  Vishnu: However, once you have done that cycle two to three times, for example when I was in Amazon, we did write evals for a lot of things and it was based on certain signals but evals comes to it is when you have already a very good idea of what you would do manually.
• 37:06
  Vishnu: That's when you actually write an eval.
• 37:08
  Vishnu: You could, again, in a large company typically it's more process heavy than common sense.
• 37:14
  Vishnu: So and so sometimes people start with evals without even doing it themselves and not having the intuition. In my opinion, that often leads to a waste of time, which I have seen a few times happen Yeah, my recommendation would be start with it. Once you are reasonably comfortable with these are the signals and this is how it goes, how you evaluate it, then you write an eval.
• 37:38
  Vishnu: And once you have an eval, it becomes very easy to refine the skill right? Because you just run a cron job or so and say if the evals are performing poorly, go here's the GitHub repository, create a Git Push, refine the skill in Git Push there's also some places where they were already doing this and this was also my recommendation, is to use a human.
• 38:01
  Vishnu: For example, a lot of websites where you need to apply for these grants it's very hard to use a bot to computer use and get those skills. So on these my recommendation was and this is what I believe they're doing even now just have a human consultant hire them, ten, fifteen dollars, twenty dollars an hour.
• 38:24
  Vishnu: And their job is to click on computers, like the computer use part is humanized, and they go get every question copy-pasted into a form and add it in.
• 38:33
  Vishnu: And so really your agents are kind of giving work to the humans when it needs, like, OK, I need all the questions for it.
• 38:42
  Brandon: It's the meat proxy.
• 38:42
  Brandon: You have the human listening to the instructions by the agent and giving the data back to the agent.
• 38:49
  Brandon: That's cool. Yeah, I think that's like an interesting tool in your like sandbox of tools. OK.
• 39:00
  Vishnu: This is a very common pattern too I've seen.
• 39:02
  Vishnu: Even we do it. So in our own sort of we have our own internal SEO tool that we've built which sort of writes blogs and it writes its own blogs and it does its own thing. But there are places where we need a human where it delegates the task to a human and one of my consultancies that I've hired.
• 39:23
  Vishnu: Their job is to look at our dashboard and see what job the AI has given and then do that job And so often I may come review it and look at what is happening here but the AI is giving them the job that they need to do.
• 39:38
  Brandon: So what's an example of like one part of your SEO workflow that gets delegated to a human?
• 39:46
  Vishnu: Yeah. It's very convenient and I don't know if you guys have heard of this start-up called Rent a Human I was especially excited for it.
• 39:55
  Vishnu: I actually tried integrating them into my workflow but I realized I would rather have a single human that I hire on a long term basis than a new human every time because then there is accountability and trust. I could just say, hey, what's up?
• 40:10
  Vishnu: Like, why is this not working or so on, rather than a new person. And so eventually I decided I should just interview and hire the people and then have that one person do the thing
• 40:22
  Brandon: It's like rediscovering why you hire people at your company instead of just have like little contracts transactional.
• 40:32
  Vishnu: Yeah. I'll actually give you guys one more example.
• 40:34
  Vishnu: This might be very useful for someone who is applying for startups or applying for grants and so on.
• 40:38
  Vishnu: So for a long time when OpenClaw and Hermes came up, I was trying a lot to have everything be done by my agent.
• 40:46
  Vishnu: I'd I think I've spent so many countless hours debugging like browser use tools to try and do it.
• 40:53
  Vishnu: Eventually I gave up and I said fuck it I'm gonna hire a human to do this.
• 40:57
  Vishnu: So the human's job was basically the step one which is go to the websites, get all the questions, put it in a document. AI does the rest of it.
• 41:06
  Vishnu: And then the human's job is also the submission part.
• 41:09
  Vishnu: They will go submit.
• 41:09
  Vishnu: It's blind copy paste, they don't have to use the you know that much of a thought process there and I think I got into a bunch of accelerators before I got into YC a bunch of my VC calls and everything were booked and I would just look at it and go ohh, OK, I must have applied it so then I had a separate email for this person. I would go check.
• 41:35
  Vishnu: I was like, ohh, that's why this is happening.
• 41:36
  Vishnu: OK, cool. So parts like that I think you can definitely automate with a combination of your workflow with the agent that's running and delegation to humans.
• 41:45
  Christine: So we had create the skills, define the phases, in a workflow it's identified what the phases are when you when the human input is needed.
• 41:56
  Christine: And I guess maybe there's a trigger notification that the human needs to provide input.
• 42:02
  Christine: And then what is the next step?
• 42:03
  Vishnu: The at least the way I've done it and I've seen some of my start-ups customers do it is it's just a dashboard. The human goes in and logs in every day.
• 42:15
  Vishnu: If there is tasks for them, they do it and they complete it. And the whole workflow is just tracked in another dashboard.
• 42:20
  Vishnu: It's just regular code. There is nothing special about it.
• 42:24
  Vishnu: You don't need to use n8n or any workflow builder for something as simple as this
• 42:29
  Christine: What dashboard are we talking about?
• 42:31
  Vishnu: So it's a custom dashboard you would build.
• 42:34
  Vishnu: Something that's vibe coded by the product creator.
• 42:36
  Vishnu: It's an admin dashboard that you have where you can see how many tasks are waiting on the human, how many are, in one process.
• 42:42
  Brandon: Yeah. When you're trying to take this and it's like, OK, I have this workflow now I wanna get the human out of the loop more How do you decide, like, do I run it in Pi, do I run it in Hermes where do I run it?
• 43:05
  Vishnu: When you're building a product, there's certain characteristics of a product which you can map to, you know, how you choose this given agent.
• 43:12
  Vishnu: So let's say you have a product where it needs to have good memory, meaning it needs to remember the conversations that it had with its users.
• 43:20
  Vishnu: Great examples of this are of course like things like Grok Bot, Instinct, and so on, right?
• 43:24
  Vishnu: You may have told two months ago that hey my aunt Sarah is coming to town and today you don't wanna have to restate that. Right?
• 43:30
  Vishnu: So this is cases like this.
• 43:34
  Vishnu: You may need it to proactively do some things, like every day check your email and things like that.
• 43:41
  Vishnu: And so you want a simple starter product.
• 43:43
  Vishnu: If answers for things like this is yes, then you probably wanna start with a like a really high level harness like Hermes or OpenClaw because these come out of the box for you.
• 43:51
  Vishnu: On the other hand, let's think of a product like, let's say you are building a web developer tool product where the job is code reviewing agent, or you're building a product where you're gonna automatically monitor what's happening in your production systems and if something is going wrong, if there's a lot of latency or errors, you need to figure out what is wrong in the code, fix the code and then push to production, right? Like there are many cases of products where coding is the primary use case.
• 44:19
  Vishnu: If the answer to that is yes, then you probably wanna stick to one of the coding harnesses that are more, used to that. And within this, of course, if you're really wanna use an open source model and not wanna be tied to Claude or OpenAI, probably OpenCode there's a third use case where, let's say you're building something, let's say you're building a harness yourselves. Let's say you want fine-grained control over the harness and the harness you need to really control how much system context gets into the harness, the prompt of the, harness itself I've heard about cases where I think what is that?
• 45:02
  Vishnu: There's a law company that is building a harness for parsing documents and then making certain decisions.
• 45:10
  Vishnu: And so there you really want fine-grained control over what is in the system prompt of the harness loop and you sometimes want to change the behavior there.
• 45:20
  Vishnu: So then you probably either build your own harness from scratch or you start with a extremely minimal harness like Pi, where you don't care about memory, you don't care about cron jobs, you do not really coding, but you do want control over the behavior of how the harness behaves, right?
• 45:36
  Vishnu: So then Pi is like a great harness to start with.
• 45:38
  Vishnu: So it kinda depends on the product you're building and what characteristics is true about your product.
• 45:45
  Christine: So, first step again, identify what the actual manual process is, build intuition by doing it yourself with an agent and then you have to choose which agent to use for your use case. And Vishnu you just gave us a really great framework to choose the right agent.
• 46:08
  Christine: And then after that you create the skills and then you define the different phases where you might need humans in the loop you could have built a dashboard where you could monitor the different statuses in each phases and what's blocking. And then the last step well the last part in this first step is create the system instructions when to invoke the skill. For example, invoke this X skill during that phase three. OK, and that's basically the whole step of building your agentic workflow.
• 46:46
  Christine: Cool, that's super helpful. I feel like I got a better sense now if I would want to build a product with an agent at the core of it, how I do it from scratch.
• 46:55
  Christine: From building the product to deploying it and then maintaining it.
• 47:01
  Christine: All right. I also know that you had another example prepared where you actually already like take a lot of this work out of the hands and it was something related to templates.
• 47:13
  Christine: Is that right, Vishnu?
• 47:15
  Vishnu: Right. Yeah. So we do have templates.
• 47:17
  Vishnu: So what we have seen is with a lot of our customers, they often fall into one or other one or two categories that of products that they want to build.
• 47:26
  Vishnu: So what we have done is we have actually created GitHub examples with these templates. So on our documentation if you see it's the let's say you want to build a clone of Dots that was recently launched by OpenAI. Or you wanna launch Instinct or something similar to Grok Bot or a niche or Muse and we have many more examples here for every harness there is you could just go in and copy paste this prompt and give it to your coding agent and it will just do it for you. You know, it will create like a starting template. From that point on you can basically revise it.
• 48:02
  Brandon: So yeah, since Dots just came out good job getting this up quickly this is like the day after Dots was announced can you show us what it looks like if you build your own Dots?
• 48:17
  Vishnu: Sure, yeah. Actually, first off, let me show you what Dots looks like, because I don't know if everybody has had the chance to like even play around with it.
• 48:23
  Vishnu: So for example, I have this dot.
• 48:25
  Vishnu: I've named it as Pip and it's really just like a single chat interface where you can chat with it and it's kinda like meant to be your friend which does things for you.
• 48:38
  Vishnu: Every user basically has their own computer.
• 48:41
  Vishnu: And this is like a must-have for any of these agentic products like, Muse Grok Bot, a dot and all those things. And it needs to be able to have its own browser, its own files, its own desktop, its own memory and so on and so forth.
• 48:53
  Vishnu: Do you see the setting up PIP's computer?
• 48:55
  Brandon: OK, cool. Cool.
• 48:58
  Vishnu: So I actually just went in and I created my own starter template. And so we have a very similar example.
• 49:09
  Vishnu: So you can, for example, connect to Gmail, Calendar, Notion. I'll skip this for now.
• 49:13
  Vishnu: And then you can start chatting.
• 49:16
  Vishnu: So. OK. Hey, Pip. So I'm gonna say open a browser and go to Google dot com. And
• 49:28
  Brandon: Look there's Google.
• 49:29
  Vishnu: Yeah, there's Google and we'll also give it search for Bob Marley So the basics of basically creating an agent like this is that you need a harness first of all, running.
• 49:51
  Vishnu: So in this example that we've used, we've used Hermes agent as the harness that is like the driving factor.
• 49:56
  Vishnu: So when you're messaging back and forth, really what is happening is it is going to the Hermes agent that is installed that is running inside the user's computer. So I as a user I just signed up over here and in my computer over here you know it has its own browser, it has its own files and so it is basically running these things. And so Hermes agent for example is able to control browser. It's a feature of Hermes agent.
• 50:25
  Vishnu: And so it is opening its browser and it is basically as you're saying things it is doing it.
• 50:31
  Vishnu: So and of course you can take control, you can close, you can search for something else and so on.
• 50:37
  Vishnu: But this is like the basics of an agent like this.
• 50:43
  Vishnu: There's more to it most of these products also give you like, default connections. Because when, let's say you want your agent to go to Gmail and then check the last twenty days of email and see if any investors have, emailed me and put it in a Google sheet.
• 50:58
  Vishnu: Say you wanna do something like that or you wanna build a product something like that.
• 51:02
  Vishnu: So you probably don't want to have the agent use browser because browser use is still very in its early phases. It's not as great as humans using it.
• 51:11
  Vishnu: And so what a lot of products do is they give you connectors like this that you can hit connect and then it connects to your Gmail and it uses the MCP or the APIs with it to do most of those tasks.
• 51:21
  Vishnu: So in general if you can have a user use one of these connectors in Connect, that is a better user experience but that's like the basics of you know how you would build a product like this.
• 51:33
  Vishnu: And the rest of it is just nice fancy UI.
• 51:36
  Christine: I think the useful thing for me after this whole is that building an agent and deploying it isn't just one decision.
• 51:42
  Christine: There are a lot of trade-offs and a lot of different options there's the question of what agent stack should I use, do I want the polished managed product or do I need something more configurable like OpenClaw or Hermes and do I actually need to own a lower layer because the agent itself is part of what I'm building? So the principle I keep coming back to is don't rebuild a layer just because you can.
• 52:09
  Christine: Be really clear about what you actually need to own and take the rest off the shelf when it's generic plumbing. So yeah, thank you so much for sharing all the knowledge and sharing the examples, Vishnu.
• 52:20
  Christine: These were very concrete examples and they were really fun to listen to too.
• 52:24
  Christine: So yeah, thank you for joining us today and for the listeners, if you're building agents and you found a pattern or failure mode or lesson learned that we missed, please send it to us.
• 52:37
  Christine: We're Superlinear and we hope to see you again next time.