Latest / 4kd.ai / Ep. 8 | Building an Agentic Business

Transcript

Ep. 8 | Building an Agentic Business · 4kd.ai

• Jack BeVier (00:00) Hey everyone, welcome to forked.ai, your real estate investing AI podcast. back this week with my good friend David Moses of Avenue Group Real Estate up in the Michigan area. and Dave, we were man, we were talking just a second ago. It has been a minute. We've we let too much time here elapse in between the last last time we talked. I think just like summer schedules and I had I traveled, I was in Eastern Europe for a couple years, and so we have been David Moses (00:29) Nice. Jack BeVier (00:30) delinquent, but but the world marches on and much has changed and a lot a lot is happening. So we're gonna have a a catch up episode here to get folks up to speed on what's been going on for the past month or so. so what's been going on with you, man? What is what is the latest in in your AI progress? David Moses (00:50) Yeah, so my universe has pretty much been around you know, focusing on efficiencies, focusing Jack BeVier (00:55) You know, focusing on efficiencies. David Moses (00:57) on trying to figure out where the ball is getting dropped the most and try to fill that in. that that really was my focus for a long time. getting workflows that read documents, workflows that understand you know how to enter things, how how to get data from one place and put it in another place in a complete and, you know, fully detailed way. And I think we we've done a pretty decent job of that. And then we kind of hit this wall where things Jack BeVier (01:15) to enter things, how to get data from one place and put it in another place in a complete and you know fully detailed way. And I think we've done a printing job of that. And then we kind of hit it in the wall where I think that's David Moses (01:35) got implemented but didn't get like they they got they got Jack BeVier (01:41) Yeah, ninety percent. David Moses (01:41) ninety percent there and not actually put out into the world. and then you know we ran into these things where if it didn't handle everything, we felt like it was not ready for deployment. So we've been working on workflows. we use the architecture we have, if you remember, is essentially we have Claude that is connected using these custom connectors that we built to our data. So basically our Mongo databases, and it can even we created these workflows in N8N that allow us to that allow Claude to just create API calls. and fire them through directly to each of our CRMs and PMS systems and you know and what have you. Jack BeVier (02:28) Which is and and you and you mirrored each of those in Mongo. So you're so you're working you're analyzing the data that's within Mongo, but then you can but then the the workflows will push directly back to the database to update that data, correct? David Moses (02:44) Correct, correct. And that can be done in in Clawed. And then we also started building individual agents in Hermes. originally it was OpenClaw. We kinda abandoned that because OpenClaw was not quite as it just everything kept spilling over. Like you'd you'd be in one thread trying to discuss a particular workflow and there would be this kind of overflow from other things you were working on. And it would, you know, i There were times where it would like completely dismantle a workflow thinking that it was wrong because it thought the objective was this other, you know, workflow and it was clearly spillage over from another another thread. So we use the communication is through Discord. And so that helps us organize each workflow or each kind of subject that we're working on into a separate thread and then keep those things that the idea is to keep them siloed. but that wasn't it w wasn't working. So so we when we switched to Hermes, we found most of that just went away completely. It still had some context from you know, some cross workflow context. It was still learning things from from you know, the di various threads that we were in, but it wasn't it wouldn't like grab that context and use it in a particular prompt to try to update a workflow and it was work you know, where it was working on the wrong workflow. So Jack BeVier (04:09) So explain to me the like I haven't played with Hermes yet. I've heard of the agents, but I haven't used them at all. So my understanding is and I and I am building some foundation because I want to bring agents into the system yet, but we really haven't, like to speak of yet. And but I've been hearing more about her you know, Hermes agents specifically. And my understanding is that you need to Yeah, that that you need to document your s basically if you know, I think I it leads me to the conclusion that like documenting systems and processes is back. because you know, an agent needs context in order to in order to perform a function. And so if you point an agent with no context at a workflow, it'll, you know. I you use this like, you know, it'll be like a twenty two year old. It'll like be okay. But if you gave it context and you give it like an SOP manual and it understands the vocabulary and understands the nature of the business, it'll probably do a pretty good job. What so what have you seen? David Moses (05:07) Yeah, I think I think that's that's spot on. I I would I would say that you write you write an SOP manual as though you are focused only on showing them what the right way to do things looks like. you know, I found it's very difficult for humans, I think, to train AI because we kind of operate our focus is my god, this is broken, something's wrong. you know, the bear's gonna eat me kind of, you know, back of our brain kind of kind of thing that we you know we focus on the negative, but it's really difficult to train AI like that because ha AI knowing what not to do isn't as helpful. AI really needs to know what a successful end looks like. and it it learns and performs significantly better when you focus it on, you know, This is what a this is what an example of what a correct, you know, a correct output would look like. and so that's been tr tricky. It's been really tricky to try to, you know, build that into our our our agentic workflows and because that's, you know, I I think Claude does a significantly we're you know, when we're on Hermes, we're using now we're using chat five point six soul, which is like, you know, cause we can tie it to the OAuth token and not pay for Tokens. You know, we don't we don't wind up having to pay for you know a ridiculous amount of tokens. It'll just use you know, it'll throttle it a little bit, but it'll it'll work. Jack BeVier (06:43) Are you so have you built up how does how does Hermes work? Explain to me how it works. Like are you writing all of your agentic workflows inside the Hermes software? Is that where you're doing the training? Or are you pointing your Hermes software at like an obsidian vault with a bunch of notes in there? How how does all that how does it how does it get the context? David Moses (07:02) So for building the workflows themselves, the you know, the platform we still use for the most part is N80, but we are using, we are moving some workflows to just being Python scripts, you know, right on you know, right on our EC2 server. But in terms of the training part of it, like, you know, what does like how do we do things, so to speak, we use this kind of kind of the architecture is this. You have A Mongo database that is our collection of knowledge about anything and everything that an that an agent would need to know or want to know to build our workflows, like how to make this API call, what the endpoints are, you know, what the deep link URL addresses, you know, how they're formatted, and anything and everything it lives in this in this Mongo collection. And each each document they call them documents but each each item in the collection has one field that is just a description kind of like kind of like an MD file but but kind of stripped down just like here's an example or here's here's a description of what this you know what this knowledge you know this piece of knowledge talks about yeah and is this re and re and Mongo Jack BeVier (08:13) Means. Is this are we in we in Mongo or are we in in Hermes right now? David Moses (08:18) well so th this stuff Hermes interacts with Mongo. And that's where it will it will reach into this knowledge library and whenever it's trying to do something to get you know get context on how to do what it's doing. So there's one Mongo has a feature that allows one field in each document to be automatically embedded semantically. So basically it can do a semantic search of this particular field across all these documents and figure out which pieces of knowledge are relevant to what I'm trying to do. And then it can use those things and that's how it kind of that's it's like long term memory. But it ha it has its own each agent has their own MD files and maintains their own kind of memory of what they've talked about. So that's like it's just a series of MD files that it keeps just locally within within the same really on the same server that that they live on. And then it has then they have the context of the actual thread, you know, the thread itself, so they can look through anything we've talked about in that specific Discord thread. and then it has this long-term vault of knowledge for how to build things. And the way that kind of knowledge vault works is after each session, or at least once a day, the agent will suggest, they'll they'll actually submit to A workflow, they'll suggest new nuggets, new new pieces of knowledge. And then there's another agent that will review all of those nuggets of knowledge to decide is this something we don't already know? Like is this something that's not already in our knowledge base? if if so, you know, if it if it are it's already in there, hey, we already have it. Is there any way that this new piece of knowledge enhances something we have? If so, it'll update it. or is it something that is, you know, Brand new. And if it's something brand new, okay, first let's check that it's correct. And then if it's correct, we it it adds it in. So there's like separate workflow that kind maintains our long-term knowledge that all of the agents can access. So all of the agents may have access to like, okay, here's how you go and find the like all of our API keys live in a vault, right? Not every agent has permission to use them. How those permissions work and how to access them in the vault and what to look for. All the instructions are in Mongo. The vault is on something called vault porton. So that would be when the agent when an agent is saying, Okay, I need to make this API call and I don't know from my own with my own soul MD file or whatever other MD files they have, I don't know inherently how to do this. So I need to go and find out. They go and they find out and they learn, okay, I gotta go to this vault, I gotta search here. And you know, that's how that's essentially how they that's how that's how our long term versus short term memory works. It's It's not perfect by any means, but it's but it it try it it does a good job of keeping things relatively consistent as you have multiple people working on creating and modifying and fixing workflows. Jack BeVier (11:19) So where do you like, where do you write the SOP to start with? Like how do you teach how do you actually teach the agent what you want it to do and then teach it how to do that? David Moses (11:28) So how I do it now is I do it. So I will have I each person has their own agent. And each person, not everybody in the company, but each one of our leaders has their own agent. And so whatever they have to do, whatever process they're trying to accomplish, they simply tell the tell their agent what they're trying to accomplish. And they kind of painfully walk through explaining it to the agent what the steps are. And they kind of just walk through doing it. And then the agent will actually go and do it. They'll try to make API calls to update certain fields. If it can't find a field because that endpoint doesn't exist or that endpoint doesn't deliver enough information for them to do what they need to do, it'll it'll say, okay, here's what I'm missing, here's what's missing. And then that becomes the part the human has to, you know, like an example would be our agents for a long time, they couldn't go log into the bank account and download the bank state. So that became, you know, when when the agent needs to look through the bank statement for, you know, you know, kind of pre-reconciliation purposes or to try to identify, you know, whatever it is the workflow, whatever is the agent was trying to do, you know, they we know, okay, well, somebody needed to go in and download the bank statement and save it here. That was the human part of it. And so that process of reconciling a bank account, let's say, had some agentic port Portions to it and it had some human portions to it. But the way we kind of trained it, the way we kind of built the manual for it was to literally walk through doing it with the agent, and the agent would remember every single step of it and figure out I I don't know that there's a better way around that, but I'd love to hear if there if there was. But by the agent figuring it out, there was this component to it that you know, it we naturally went through the progression of figuring out what endpoints were available, what weren't, what data was there, what data wasn't there. All of that kind of happened organically through the process of building the SOP because the the agent could make a bunch of different calls correctly and incorrectly and pull a bunch of data from a bunch of different endpoints very, very quickly in order to figure out, okay, this is the right way, this is the right endpoint to use, this is the right place to go to get this data. And They we we wound up in a much more wound up getting to where we wanted to get to significantly faster. Jack BeVier (13:52) So you're taking the approach of these agents are really like a personal assistant for each of the chairs that you have in your organization. David Moses (14:01) Yes. And and then it's be it becomes so the workflow candidates become the things that people do repetitively that the agent already knows how to do. So th when you just say to an agent, hey, I need you to figure out why I have an automation that it's a deterministic a automation does not have any agent involved in it. All it does is it takes an invoice that is submitted from a vendor through our, you know, through property meld, which is like a you know portal for handling work orders at at properties, they submit their invoice there, and the system just the system just pulls out the work order and enters the invoice into QuickBooks. So in order to do that, in QuickBooks, we have each owner that we manage for or each entity that owns a property that we own, you know, is a customer. And then each each property that that entity owns is a subcustomer. Each unit in that property is a subcustomer of that. And then each work order is a subcustomer of that. So all that has to be mapped to the IDs that are in App Folio and the IDs that are in property meld. And so what this does is it kind of Jack BeVier (15:11) Which you didn't which you did in Mongo. David Moses (15:14) which we did in we we mirrored it, we mirrored all these things in Mongo so it would only have to look one place to get all these IDs. but initially, you know, we had to, well, I guess way initially, we had to manually. literally go in manually figure, you know, this is the app folio ID for this property or this owner, whatever it was. so then we created this workflow that will automatically take the vendor or take take the it does an API call to pull the vendor's invoice, which includes the unit ID, the work order ID, you know, when I say ID, like the IDs that are in the system, not, you know, our work order number. and it would feed that it would basically match those up. with what's in QuickBooks and then enter the invoice and attach it in. And so it could be paid out of QuickBooks. And we have an agent that oversees that. And then the agent, whenever that workflow fails, the agent says, okay, well, what's going on here? this is a new unit. Or this is this is a new vendor. No. And then it knows to build the, you know, knows how to create the subcustomer in QuickBooks or create the vendor in QuickBooks or do whatever, you know, fill in whatever is missing. and so that became the candidate for because that was that was actually built before it was built by AI, but it was built before agentic AI really existed in N Aden. These were just deterministic workflows that were, you know, dumb workflows. but now we have an agent and we just say to the agent, Hey, go in and figure out what's broken whenever you get an error. Go in and figure out what's broken. And we had to would walk through it every, you know. It would it would look and see, okay, well, it's missing a QuickBooks ID. here's who's missing the QuickBooks ID. It's this unit, it's this work order, it's this whatever's missing a QuickBooks ID. And then it would, and it it it knows how to fill in those blanks. So because the agent knows how to do that, now we've created, you know, we're creating a workflow or augmenting that first workflow just to say, you know, when this ID is missing, like we know what the mistakes are, so when this ID is missing, this is how you fill it all in. Jack BeVier (17:11) fascinated. Are you building does each agent know what the other agents are doing? Or are they all David Moses (17:19) At pres Jack BeVier (17:19) silos? David Moses (17:20) the at at n and this is this is I guess a drawback to Hermes, but they are all siloed. not that they can't, you know, quote unquote talk to each other, because they can see if they're given permission to see, they can see the interactions the other agents have with everything. but they the way Hermes kind of builds it is they are the agents are fairly well siloed. it's it It is they can create their own sub agents and and those that whole ecosystem of agent and all their sub agents is clean and and consistent and and you know w works really well. But once you have, you know, you don't have a lot of one agent learning from another or understanding what another's, you know, kind of work through, that's probably a good, you know. That's probably a good thing to augment in the future is the at least the ability when an agent gets stuck on something to at least be able to ask the question, you know, to the other agents to say, W you know, how do I do this? right now it has that that knowledge base that I talked about that is a shared knowledge base. And it's a permissioned knowledge base too. So it knows, you know, hey, you know, i if you're an agent for this, you know, these agents have these permissions. these agents don't which it's just my leadership team right now. So they're really the only things that are you know, siloed in terms of permissions from the knowledge base are things that are for me personally. Are you are you technically Jack BeVier (18:42) Are you doing are you setting up like cron jobs so like they're doing a certain task on a daily, weekly, hourly, webhook basis? David Moses (18:50) Yes, they you know, most of our repetitive workflows are done in N8N and they're on schedules. it's just more robust. It's more you know, but now we have the one like our mirror workflow is just a Python script on a cron job. But the agents do have like I have an agent that will go into my because I use it's on a Mac. So I put the ma I I attached my Apple ID into the Mac. So it actually looks through my photos to see if I have any receipts and pulls them out, reads them, figures out what they're for, and either, you know, creates a reimbursement request or it, you know, my agent just knows how to do that and he and he checks once a day to see if there are any receipts. he keeps my personal books. He'll he'll attach it to whatever card in QuickBooks, find the transaction in QuickBooks and attach, you know, literally upload because I do my personal stuff in QuickBooks. So it uploads that receipt to, you know, to that transaction in QuickBooks. and that's it's it's pretty cool. I think I've ha mine pretty well built out. I want, you know, I'm just trying to encourage my team to do to do the same. Like you use it for your, you know, personal stuff. It'll look through my WhatsApp messages. It looks through my, you know, it'll tell me if I haven't responded to someone important in a, you know, in in in a certain period of time. Like the other day when I you sent me a text message and I I didn't I didn't respond for a while. Actually Rufus, my agent, actually sent me a text which says, Hey, Jack sent you a message and you never responded. And that that's actually how I figured out that I forgot to respond to you. Jack BeVier (20:20) man, this is really interesting. So like you're using so this is really like this is your new open claw. It's just got a little bit, you know, a little bit tighter guy guardrails on it. And guardrails enough that you're that you've become you're you're very comfortable giving it right access to David Moses (20:33) Okay, you're pretty comfortable access to Jack BeVier (20:37) all of your source databases. David Moses (20:40) Yeah, it it is, you know, I mean, like look, I think prompt injection is a real thing. You know, I think that there are a lot of security type concerns, but I think for every security concern that it raises, there are at least two that it solves. And and I I I mean that by like You know, my agent operating on Soul five point six can read an email and know that it's a phishing email better than Microsoft systems can. You know, and you know, Jack BeVier (21:09) Yeah, I hear that. Yeah. David Moses (21:10) and and there are, you know, when when I see a when when I get an alert that there's been a a login in my Netflix, like it knows not to tell me that. It knows I don't like no one cares. Jack BeVier (21:25) Yeah, yeah. David Moses (21:26) But but when it But when it but when it detects that there's been a login in my bank account, you know, and it knows I'm up north, it sent me, it literally sent me a message saying there's a a login in your bank account from a new IP address and I know you're out of town. Is that you? So it's I feel like there are some things that it solves that it there there are some security type things that it solves that are much more practical and real to me than some of the you know, kind of abstract sorts of potential risks that it that it raises. Yeah, that's super interesting. Jack BeVier (22:03) Man, that's super interesting. Okay. So I'm trying to attack the same problem. I want to get agents into my workflows to handle the easy stuff, right? The stuff that's just like very, you know, well, the stuff the anything that's like purely if then statements should just be an automation, right? And then the things just above David Moses (22:24) Yep, Jack BeVier (22:25) that where like the patterns are very clear and or or like there's a lot of like just diligent legwork that needs to happen. I don't want to pay a human to do that. I th you know an agent can do that frankly just better and cheaper, faster, better and cheaper. so the so here's the way that I'm attacking it. So we are putting all of our we've you know, Salesforce is our system of record. And so it's all my data points are in Salesforce. I'm you, you know, you got what you've got your equivalent of Mongo for me. That's that's Salesforce. So everything's defined in Salesforce. That's where all the, you know, that's where all the fields are defined. And in those field definitions, we are backfilling, you know, a little bit of the description, right? A little more context, right? Trying to just trying everywhere to give it a little bit more context, right? Like so so that it doesn't, it's just not, you know, because it'll make an assumption, right? If it's called loan amount, it assumes, yeah, it you know, it knows something about that. But if the field ID or the descriptor ID is something vague or you know doesn't fully understand it, it'll start to it can it can make more mistakes, right? So we're trying to like bulk up the description of even the field definitions. And then on top of that, I've got kind of like object. Right, like all my concepts. So a lead comes in. it's basically anything in an object oriented software that is defined by an object, right? So a lead comes in, a quote gets created, an opportunity gets created, a re you know, for us a loan object gets created. I've got a, you know, a sale of a portfolio of loans. all of those objects get defined. And so we're going in and we're providing more context to. the nature of those of the basically the nature of the business and the different concepts memorialized by objects in in my Salesforce architecture. And that's another set of markdown files. And obviously those objects have the fields associated with them. So those relationships are can now be understood because the object has some description and the fields have description. And these fields sit on those objects. So now the agent knows a little bit more, right? David Moses (24:32) Mm-hmm. Jack BeVier (24:32) And then on top of that, now I want to define roles. So I've got you know, I've got whatever, you know, I've got a loan officer, I've got a loan officer assistant, I've got a loan processor, I've got an application review specialist, a mortgage pricing specialist. All of those are are roles that I have defined in the context of my business, and they interact with certain objects and fields. So I've mapped all of the roles. I actually start I so I I started, I just literally uploaded my org chart and was like, here's my org chart, here's all the humans. You see them in as users in the Salesforce database. Map this org chart to David Moses (25:11) The source question of these topics. Jack BeVier (25:14) those users and make sure that everything reconciles, right? Like tell me where you see any mismatches. And so there was like, you know, a couple of mismatches and some titles needed to be updated, right? So now I've got, you know, my I've now I've got my roles defined also in Markdown objects. and then on top of that. Then I'm putting my SOPs on top of that, right? So I feel like I'm building like a like a pyramid, right? Like so I've got my SOPs, which are what the things that the roles do. And at the moment, we are in the process of redefining a lot of systems and pro or a lot of processes, right? Because I'm now able to write our own custom software. So like I, you know, everything that we did before six months from now, we're doing it slightly differently. You know, with all the data entry eliminated and all the automations that can be built, built all the APIs sucking in information, sucking that information in directly as opposed to going to websites and collecting information, just bringing it all and presenting it in one place, right? So that actually dramatically changes the SOP, right? Because before it's go David Moses (26:08) Right. So that actually dramatically changes the SOP, right? Because before Jack BeVier (26:12) to this website and now it's the data is just presented to you. So I could just delete, you know, that whole I can just delete, you know. So I'm trying I'm going through the SOPs that we currently have. And like every time there's a verb that uses a computer, creating, you know, just pulling that data in and just delete it, you know, delete that sentence from the SOPs. And so then I'm in at the end of this, this is a this is going to be a many month process. But the idea is at the end of this, my new SOPs with all the redactions of all, you know, the of the of the automations and things that I've built will be much smaller and they will only be the things that humans do. Right. Like I, you know, the only the things that require a human will be the resultant SOPs. And anything that's like basic, like I don't know, kind of like a basic analysis, or you know, logic that I can either be hard-coded into an automation or executed by an agent because you know the business logic is like pretty well defined. and so I'll have a set of agentic SOPs that do the all the busy work. And then I'll have a resultant list of human SOPs. And I then you and then you plug in humans to the resultant list of SOPs. And so the that that's so that that SOP layer is right now in flux. And it's gonna be in flux significantly, I think, for the next six months. And then I top down just wrote some just just talked about the business, right? I just like, you know, gave it like. Here's our, you know, here's our history, here's our business philosophy, here's our credit, you know, philosophy on credit. you know, here's how how we see the macro backdrop, here's some things that we've learned over the years, and just trying to give it as much business context as possible. The idea being that any agents that we do put into these workflows will now have down to the nitty-gritty detail, you know, data you know, data record level context up to the where we think interest rates are going, you know, basically. and like, and if if it has all that context and everything in between, it should the resultant work that it should do should be should be pretty damn good. and so I'm going from like, I'm not giving like at the moment, I'm not giving anybody agents. I'm trying to build this pyramid such that I can start grabbing SOPs at the end and and and creating job descriptions basically that are just purely agentic. And then And then, you know, and then redefining the job descriptions of what my people do as purely human things that that don't have any agentic, you know, or that that are best done by humans, I'll put it that way. David Moses (28:37) Mm. Jack BeVier (28:39) so anyway, that's that's been my approach to the problem. It's like a and and all of that is memorialized in markdown files that are within our Salesforce system because that's where we're that that is, you know, that is our data base. So that's our approach to it. David Moses (28:49) DNF base. So that's our approach to it. I'm really I'm really curious. Jack BeVier (28:53) I'm really I'm really curious about whether I'm like trying to build a castle too big like like too big of a castle all at once and I should be getting my hands dirty a little bit with agents in the meantime and learning. So you're giving me some pause here on on my approach. But anyway, that's that's that's where we're doing that's where we're biting off right now. David Moses (29:15) So your organization, how many how many people total? Jack BeVier (29:18) A hundred and sixty. David Moses (29:19) And then and then when you when you add in you know the people that have to regularly interact, you know, with you know with the company, like when when like how many how many customers, how many vendors? Jack BeVier (29:36) two hundred and fifty loans a month. David Moses (29:38) Okay. So I I would say you have like two two things. Number one, I wonder I don't know if you have the time for it, but it might be cool to kind of think about starting like a a a a digital twin on the on the you know, kind of like on the outside of the organization and and under under the premise basically just say, okay, if I were starting the company from scratch based solely on like I'm gonna make it a purely agentic, purely digital company, right? Understanding that it's not there yet. Like we we know that agents we it gets smarter every every day it seems, it gets it gets every you know, week it gets smarter. But it's not there yet where you can have a completely agentically run, you know, lending company. It's it's highly unlikely. So Jack BeVier (30:36) Well, yeah, the the problem is the th the thing is the customers, right? Like in theory, you could. It just when you look at the pipeline and like cause we've we've we've like dabbled with this idea. The problem is that like the percentage of the pipeline that could go through without ever having to speak to a human, I just think that everyone overestimates what how big of a percentage that is. I think it's just a tiny percentage, right? Like so like what I'm building could David Moses (31:03) You mean buy in from the customer side? Because there's not buy in on the Jack BeVier (31:09) it the world's just not that clean. I don't know. The world's just not that clean. Everyone's got a little bit of a situation, a little bit of a tweak. The documentation isn't quite correct. things change, you know, just like, you know, we we need to we need to orchestrate a pivot. And like at some point along the the loan process, everyone needs a little bit of critical thinking and problem solving. And it's just a very like In theory, yes, if I had an ideal customer and a very, you know, you know, who was who completely 100% had their David Moses (31:36) Yeah, this is five percent. Jack BeVier (31:38) together and they had a very cookie cutter vanilla deal and everything came in clean and the title company did their job, like it's it just doesn't happen. Like it never it just never lines up that way. So like I'm building all the I'm building all the pieces such that like when that piece comes in And that piece gets clean, it just happens, right? Like if if if a piece will get clean, but I but I don't believe that I could ever actually create well, sorry, with this generation, right? At the moment, I I'm I don't think I'm like could could repla could replace a job description even. I can't even replace a job description. I'm just taking chunks of the pipeline off the table that are like box, box, box, check like check, check, check, check, check, check, process, right? Like David Moses (32:22) Or easy stuff. Jack BeVier (32:22) I just won't have a human I won't need a human to go check check check check check check process. But dude, that's like one in ten files for them, right? And for each job description, it's different files, right? Like a step goes cleanly for this guy on this file and then gets held up for two weeks over in in the next phase. And then or or vice versa. David Moses (32:39) Mm. Jack BeVier (32:40) so like I don't think that I don't think that a purely agentic loan like lending company would be very successful. I think you'd David Moses (32:46) I don't think so either. I and I don't think the question is whether it would be successful now. The question is whether you know, whether it'll get there. Like whether whet whether whether two years down the road it would it would be there. but the but I think the exercise Jack BeVier (33:00) Well, so what I'm building though, like what I what I'm building though is the idea is that it is an agentic lending company. Like every component of our business that can be done by a by AI and agents, it can or is done by AI and agents, but then I have but it but in every but in every step I still have a we need to speak to a human. and and and I and so I'm gonna and so none of my humans are going anywhere. Like there I I need all of them. I'm just hoping that we can handle, you know, fifty or a hundred percent more capacity with the same people because the easy stuff just go just runs through and then everybody else ends up doing the hard work on the hard files. David Moses (33:39) So yeah, I mean that's and the hard work is Jack BeVier (33:42) Critical critical thinking, problem solving, pivoting, risk judgment, you know, just like risk analysis and judgment. There's a ton of that. There's there's that in every phase. David Moses (33:49) But but the hard the hard work is not But is that the time consuming work? Like that's the hard work. That's the difficult work. That's the stuff that really takes, you know, expertise, experience, intelligence, but it doesn't but is that stuff like so I did Jack BeVier (34:05) Well, y I'd I'd what my hope is that they my hope is that they spend the extra time doing that well, right? Like, hey, when you're just trying to like shuffle files through, sometimes you just get a little lazy because you're busy and you don't pick up the phone and have that twenty five minute conversation that you should to make sure you make the optimal risk judgment call for David Moses (34:21) that twenty five minute conversation that you should to make sure you make the optimal risk judgment company. Jack BeVier (34:26) the company. And you just you just shuffle it through and, you know, pass it along. And and I'm hoping that this extra time allows them to spend the appropriate amount of energy on the high value tasks. And I still think that there'll be a net savings of of time so that we could handle more business with the same team. but I'm hoping it is both like faster for the customer and of a higher quality work product, you know, all around, right? Like better customer experience and better risk management decision and faster process because You know, because we're using better tools to, you know, where where they make sense and not you know, where they where they make sense and only where they make sense. David Moses (35:08) So what's the first customer facing thing you're going to or do you already have one that's agentic that is customer facing? Like this is something this is an agentic, you know, workflow or something. What's that? Jack BeVier (35:18) Just launched. Yeah. Yeah. So like right now when you submit a DSCR applic like literally just now in the past like two weeks. like when you when you go you can go online and you can start your DSA DSCR application online and auto and instantly get a quote on your DSCR loan, which is not agentic. That's just an API. Like that's just sending data to a pricing engine and pulling data back. And then you can say like, yep, let's move forward. And you can swipe your credit card. Again, none of this is AI. and then you can actually start uploading documents and you can get instant feedback on those documents that, like, hey, you're missing a signature on page seven of your operating agreement or whatever. David Moses (35:55) Cool. Yeah. Jack BeVier (35:57) that is a gentic. That that is, you know, that is OCR and then analysis by an agent. and so the the the the new the borrower portal that we just put out there has doc analysis and really underwriting built into it. and so that our people are then only just reviewing exceptions, right? If the operating agreement looks good, they don't even click on it. It just there's a green there's a th operating agreement thumbs up. We're like, Great, we're not even gonna click on that PDF. We're only clicking on the red boxes and the yellow question marks, you know? David Moses (36:28) Yeah. And and that has has it already sped up the process? Jack BeVier (36:33) it's literally like two weeks out. I mean it's got to. Like it it should, you know. If the people use it. I mean, dude, for a long time we've been, we were we were testing stuff alongside people and we were like, hey, just hey, you know what? Believe it, right? Just like send if it, if it, if if you if the if the underwriting tool says that the loan file is good, just send it through. Dude, a 40% of the team just couldn't. Like they just they still clicked David Moses (36:56) Yeah. Jack BeVier (36:57) on the PDF. I said, Don't click on that PDF. And the next day they were just like, I'm gonna click on Like this, you know, they just couldn't, you know, couldn't bring themselves to to actually trusting the thing. So like I yeah, this trust building process is gonna be really a big part of this. Like we're you know it's David Moses (37:12) Th that's the honestly that's that's the that's where the agents have helped my leadership team the most, is that like they would just innately go look for a document in a folder, which with our folder system for the with our file system over the last, you know, decade, two decades, whatever, that could be a that could be a lengthy endeavor, even even if everything is where it's supposed to be. Let's just assume for a moment that the document's not exactly where it's supposed to be, or someone created a folder and named it this instead of that. Jack BeVier (37:48) Mm-hmm. David Moses (37:49) But just, you know, operate under the assumption that humans were involved with most of this. And and just just that's what they would do. They're aut automatically their mind would go to the and it's when I gave them the agent, I was like, stop looking for things. Okay. Just just don't you need to look for emails. You need to look for f files that are in folders. You need to look for all these things. Stop looking for it. Just ask the agent, see if the agent can find it for you. And Lo and behold, that like, because they were able to just ask and get an answer, like it was they they began to then it was kind of like the back door to to to trust for them. It was like, I've got something I've got to solve. I know this is a problem. How can the agent help me? And then they would just ask the question. And then I I feel like they were, they've they're getting to the point now where they know when to trust and when. question, which I think is really important. When you have deterministic workflows, you can trust them. You should trust them. It's not worth your time to not trust them. When you have agentic workflows, there are things that it will miss. One out of fifty, one out of, you know, there there are things that it will miss. And knowing when to trust it and when not to trust it, and knowing what outputs, you know, that it gives you that you can rely on or not, God, there's like a feel for it. It's almost like like it's not just obviously wrong anymore. It used to be like it would just give you, I would just type in a prompt and it would tell me something and I was like, that's You know, I would just know. Now it's it's much more subtle, but I I my fear is that people who are getting involved in it now and learning now. are never going Jack BeVier (39:37) Mm-hmm. David Moses (39:38) to the obvious And because they never Jack BeVier (39:39) Mm-hmm. David Moses (39:40) get the obvious they're gonna miss the really subtle So yeah, Jack BeVier (39:42) Yeah, I hear that. David Moses (39:43) yeah. So I I felt like the the agent because like they're asking the agent questions that they absolutely know how to do it themselves. They know what they're looking for. They know where to find it. And so when the when the agent says I can't find it, I looked here, here, here The person knows, well, you looked in the wrong frickin' place. It's not there. It's over in this. And so, you know, I'm I'm sh hoping that that will kind of train both the agent and the human. What for the human, it's what can you trust the agent to do and what do you need to teach the agent? And the agent, it's what do you need to learn, what's the success look like. and then and then having it be fun, you know, having it be, you know, like I gave every single agent the personality of the humans like you know favorite TV show or movie or whatever. So it would like respond like mine mine is Rufus and he responds like George Carlin's character from Bill and Ted's Excellent Adventure. And so he'll say like most excellent and he'll say like you know like and he'll come up with things that you would think Rufus might say. And it makes it you know Jack BeVier (40:52) That's hilarious. David Moses (40:53) it's silly but it makes it fun. And and like that's the thing. They make it fun and give it and and just Jack BeVier (40:58) And give it David Moses (40:59) kind of the default should be ask the agent. Just ask the agent, right? Because this is imagine this is a real human. They're sitting there, they're waiting for things to do. If you don't give them things to do, it even if it's not your money paying them, you're gonna feel like money's being wasted. You're gonna like that person sitting there saying, Hey, how can I help? and you're not utilizing that help, you're gonna f you're you know you're gonna f feel wasteful. And think of the agent that way. Like anything that the agent can do for you, you should be asking the agent to do for you. And at the at at the leadership level, they have a lot of access. How you push that down the levels where you need to start, you know, really kind of getting specific with what they can and can't see. You don't want the agent to be able to like, hey, go look at Jack's email and, you know, Tell me if he's pissed off at anybody. You know, like you hold Jack BeVier (41:54) Ha ha ha. David Moses (41:56) it. What's what does really Jack BeVier (41:57) Well, so like w I I'm curious, like you've been using agents with your more like more senior people. have you p have you had any agents actually doing like block and tackling work? And like and why not I would and I would I might argue that like, you know, the the person who's you know, the sixty five thousand dollar white collar person probably has the most use for for an agent helping them to perform their daily functions. Have you pushed it down to that level yet or replaced any workflows at that level yet? David Moses (42:29) Not yet. Not I I don't have any agents beneath my leadership team level. Not yet. not that I well besides besides Carl. Jack BeVier (42:34) Any reason? Concern or is it like a concern about a particular thing or? David Moses (42:38) B beside Carl. Carl Carl but Carl's my brother and he's working on AI workflows, so he has to have access to everything and I completely trust him. So so that part is you know, but everybody else it's i i it's there there's no real good reason. I just haven't gotten there yet. but but it is to some degree you know. My organization's not huge. It's not, you know, it's not it's not a gigantic organization. So I may not actually get to the point where I create one agent for each person. It may be more like one agent for a specific department, you know, where I'll have all of my community managers will have a community manager assistant agent. And it just saves me on this is what they can access, this is what they can't access, because I only have to build it out for one agent. And then we have to monitor one agent to to make sure that, you know, things aren't, you know, that they're not able to access things that they shouldn't access. So and I also don't know that it's absolutely necessary to have, you know, like if everybody is talking to, you know, Claudette in my, you know, like if everybody's talking, like I've got a an agent called Rhoda. And I get so I guess Rota is rolled out to be because I do have one VA that can actually communicate with with with Rhoda, the agent. and Rota can go into it's it's a an agent that is financial, you know, so it's it's in QuickBooks. So they they can go and go through and pre reconcile bank accounts. They can tell, you know, they can tell which bank accounts have been reconciled through what dates, they can say, know, they can look at what transactions were applied to the wrong class or no class at all, or the wrong division, or no division at all, you know, those types of things. and I you know, I don't know how helpful it's been for her. It's only been a a couple of weeks, but I so I guess I I guess I will augment my prior response and say I have rolled it out at least to one person who's not at the leadership team level. Jack BeVier (44:37) Are you you you've mentioned QuickBooks a number of times and talking about like the different workflows that you've created. Are you are they actually bookkeeping? Like is an agent actually bookkeeping for you? David Moses (44:47) For me personally, yes. All my Jack BeVier (44:49) Me personally. David Moses (44:50) yes. For me personally, all my bookkeeping is done by my agent. beyond that, we have so we manage the books of, you know, dozens of associations. Jack BeVier (45:00) When you say for you personally, you mean for Avenue Group Real Estate? David Moses (45:03) No, for me person my personal like my bank my personal bank account, my personal credit cards, my personal stuff. Like it will look at all my receipts, my emails, everything and it will categorize everything that needs to be categorized. It'll it can see when I transfer money in or get a transfer from a bank account, it knows what bank account that is. It knows, hey, apply this as a contribution or a distribution, whatever it is from that particular entity. So my personal bookkeeping is done. Jack BeVier (45:29) So it's it's you like you have a you you have a personal David Moses (45:32) Personal QuickBooks account. Jack BeVier (45:32) QuickBooks account. David Moses (45:34) Yeah, I have a QuickBooks account that's just my personal books because I mean I haven't found any other Jack BeVier (45:39) Maybe. David Moses (45:40) personal financial software that can handle a balance sheet. Like it they just don't have balance sheets. It's just like, you know, yeah, your car, your house, and all your securities, Jack BeVier (45:49) Yeah, yeah. David Moses (45:50) but nothing else. You know, like my own. Yeah. Michael Jack BeVier (45:52) It's really interesting. I just I don't do that, but it's really interesting. I it makes sense makes a ton of sense. I get it. Why not? So basically like you you can print and then like that's your PFS. Like you can David Moses (46:00) Yeah. Jack BeVier (46:00) press press print on your QuickBooks and that's your PFS. David Moses (46:03) I I can't I mean I wouldn't do that because I'm in real estate and so it's gonna give the book value of all my real estate. Yeah, it's Jack BeVier (46:08) Yeah, yeah, yeah. No, at book. Yeah, it's it's your book yeah, yeah, it's your book PFS. Yeah. David Moses (46:12) correct. Yes. Is it the the you know the the Yes, exactly. I I I haven't figured Jack BeVier (46:14) That's cool. And then you mark and then you mark to market. David Moses (46:18) that part out yet, but I'm sure I can do it and just do a journal entry that's a mark to market journal entry. So I can actually get my you know, but it does I'm I actually am live working on my PFS as a separate workflow. So what it will basically maintain like it'll look every day and see what the rents are, which w units are vacant, what you know, so it'll have all my like if I sold a property, if the buildings of a specific like an app folio, it it'll run a balance sheet and if the buildings of particular property are zero, then it knows that property I don't own it anymore for whatever reason. So, you know, don't put it, you know, take it off the PFS. I'm building that out now. I haven't finished it yet. But Jack BeVier (46:56) But you haven't done but Avenue Group Real Estate, it hasn't you haven't had it do bookkeeping there yet. You haven't had any of the agents do bookkeeping there yet. David Moses (47:01) So not Avenue Group Real Estate. What it does do though, so we have like a a few dozen communities that we manage the books for, HOAs, so forth. So every document that every document gets processed now through an agentic workflow. And even though there's a human who actually clicks the button to put it into the books, it will figure out what it's for. It'll figure out if it's associated with a work order, like a vendor bill that's associated with a work order. It'll associate the work order. It'll figure out what GL account to put it to based on, you know, that it looks at that vendor, all the you know, previous bills from that vendor for all the communities. It'll look through because each community has its own GL account, chart of accounts. So it'll look through and it'll identify and almost almost 100% accurately identify the right GL account. It'll identify the right bank account. So if they're if if what they are paying for is a capital improvement and they want it to come out of their capital improvement bank account, or they have a special account for like a roof replacement or something like that, they'll have it'll it'll guess it'll guess correctly the bank account that it comes out of. and it will go to the manager to approve and the accounting you know, the our our accounting person to approve. And when they both click approve, it enters into the Jack BeVier (48:25) And when they prove it entered, David Moses (48:27) So i in a sense it's doing the bookkeeping. It's just not the the last leg, right? There's a le person who clicks on it and says, Yes, this is correct. Jack BeVier (48:34) That's amazing. David Moses (48:35) So and then what's really cool about it isn't like that part of it doesn't take all that long. You know, because it doesn't take all that long, they do make like a deck how many people do make mistakes. This doesn't really make mistakes anywhere near the level that they were making mistakes. And I mean, maybe they would make mistakes like four out of a hundred. You know, this makes mistakes like you know, one out of a thousand and they're very predictable, very obvious mistakes. So but it was doing it just does wonderful things in terms of detail. Like if a as an example, if if there's a utility bill that has gas and electric on it, separates those things out. If there's a bill that Jack BeVier (49:13) How does it input the data? Do you just give it like ac o online access to the to your utility account and say like hey, run on the first of the month, go in there and like look at all the PDFs and pull it down and David Moses (49:23) We either scan or save the bill. We we either scan it Jack BeVier (49:26) Can David Moses (49:27) some some human needs to either open the mail and scan it in to a and then and then the workflow takes from there. Everything that comes into a particular email address pulled it from there. Or it's something where they have to save it to a folder, you know, or for or save it and forward it to that email address. either of those things will work. And it will pull it and process it and then I you know you know, basically c categorize it, code it, figure out what community it's for figure out what whatever it is, right? So that's all that's all working pretty well. and but it's so much more detailed. Like if it has utility bill Jack BeVier (50:01) Yeah. David Moses (50:01) has a five dollars processing fee or service fee or late fee or whatever, it breaks that out. Jack BeVier (50:09) Yeah. David Moses (50:09) And it'll it'll catch things in the bill that people weren't catching. Like, hey, this this bill has a spike in usage, you know, or this bill has no usage at all. You know, and like those things on a water bill, like that's an indication that someone's not living there. If it's a zero water bill or very, very low usage. You know, and then the other side, if it's a spike, it could be an indication there's a leak. And that's the kind of stuff where as the manager, like we're responsible for catching that stuff. And so now it can this through this whole process, it'll it'll catch it. And then what we're our first forward, our first customer facing dashboard will be to the treasurers of these communities that we manage where they can get the dashboard. They can see all their documents. They can see a description and like all the analysis that was done on each document. You know, like there are a series of questions that we ask. If it's a utility bill, you know, it it'll Jack BeVier (50:54) the community stuff you manage they can get the data documents. They can see a description like all the analysis that can run on each document. You know, like there are a series of questions. Yeah. Do you think we go, you know it it'll you know David Moses (51:10) It'll give us interesting tidbits. Hey, the last utility bill was this much, you know, the usage was this much. The the previous year same period was this much. you know, it it'll Jack BeVier (51:19) Yeah, same period. David Moses (51:20) that no, that is a mostly deterministic workflow. Jack BeVier (51:24) I was gonna ask, okay, or would you would you hard code all that business logic in? Is that it so it's any it's an innate end flow? David Moses (51:30) It's yeah, it's an NADN flow. and it it is but it does have an agentic component to it, 'cause the the agent is the one that makes the Jack BeVier (51:38) Looking back. David Moses (51:38) yeah, that makes the final decision on the GL code and the bank account. Those are the two key, you know, key things. but Carl you know, my brother does a very he does a pretty good job of keeping the tokens efficient. not so much for the cost because it's not really that expensive, but for the speed. the workflows go a lot faster if he kind of like has these deterministic parts of the workflow that narrow things down for the agent. So the agent's only picking from, you know, so it's not picking from every GL account that's in our PMS. It's only picking from the G GL accounts that are available for that, you know, that are of that type and available to to for that community. things like that, which kind of narrows it down. Jack BeVier (52:22) Yeah, we've that's really interesting. We've found that like for like the complex logic systems, like adding some rule based component to the prompts r help really helps with speed and and hallucination, right? Like you said, it just narrows it down. Like using some things like it's doesn't have to run through like the entire j you know, menu of possible options because it's narrowed it down to four. gets one of four that you know wrong, much fewer than if it's analysing twenty five ideas. Yeah. David Moses (52:53) Yeah, that's right. It but you you have like we we have the same thing. You have so many examples of a human making a decision on how to code an invoice that the agent as long as you just narrow down, you know, hey, this was the vendor, this was the community, and then here's a description of what this bill was, right, it'll code it pretty damn well by giving it those, you know, by By giving it access to those correct examples, I mean it's wildly accurate. It I I'm I have yet to come up with a mistake that it made that wasn't either one, a scanning error, or two, a reasonable judgment call that I would have made a different call. Jack BeVier (53:40) Mm-hmm. David Moses (53:41) You know, where where it it coded something, it said something was a repair, like maybe. only part of the roof was replaced. So we called it Jack BeVier (53:48) Mm-hmm. Yeah. David Moses (53:49) a repair. And it's like, you know, or or it called it an improvement. And I mean we're like, may I maybe I would have called that a repair. I probably wouldn't call that an improv. But like those are judgment calls. Those are not, you know, those aren't things where the agent like, you know, said it was a swimming. You Jack BeVier (53:57) you know, those are things for the agent know, set it Call it plumbing. Yeah. Yeah. David Moses (54:03) know, those are those are things I could live with. you know, in in terms of but but yeah, that like I said earlier, the positive example, like giving it here's what success looks like. is so so much more effective than telling it what it's doing wrong that I just you know it's almost almost like you don't give it y you know it's like if I say, Don't think about an elephant, right? It's like, what are you thinking of? You're thinking of an elephant, right? Like this it's very it's almost impossible for you not to think of the thing, you know, and and and That's like the how the human brain works. It's it's pretty similar to how these neural networks actually function too. It's like if you give it if you focus it onto the mistakes it's making or the you know, then it's going it it's not going to perform as well as if you focus it on the actual, you know, it's like if you're if you're it was an analogy. I'm trying to remember who said it. I think it was Simon Sinek. he was talking about you know, w a a downhill skier, right? or a a cross country skier. It you know if you're if you focus Jack BeVier (55:08) It mean you know, if you're if you David Moses (55:11) on if you if you're looking at the trees, you're gonna hit the trees. Right. But focus Jack BeVier (55:14) Yeah, yeah. Yeah. David Moses (55:15) on the path through the trees and don't look at the trees, you know, you're you're you'll get through just fine. And so that's Jack BeVier (55:20) Yeah, yeah. Gotcha. David Moses (55:21) that's essentially the same thing. You just have to, you know, don't focus it on the mistakes, you know, to say avoid this or avoid that. It's focus it on here's what correct looks like. Jack BeVier (55:30) Super interesting. All right, David. It was a that that time went very quickly. It's a pleasure, David Moses (55:36) Yeah. Jack BeVier (55:36) as always. I gotta get back out there, get back into it. David Moses (55:39) All right. Jack BeVier (55:41) so hey, I'm looking forward to it. I will talk to you very soon. Hope David Moses (55:44) What are you gonna work on for next week? Jack BeVier (55:47) What am I working on for next week? I have I'm picking up okay. So I'm picking up some projects in our property management and real estate side of things. So essentially, like I need to I need to connect my pipeline management, property management and construction management softwares together into I'm using Postgres instead of Mongo, but into something, you know, a a data lake that is similar to what you have done and making sure that all those APIs are. plugged up correctly and that the business logic that that the that the system understands the business logic of real estate as it goes through the pipeline so that I can start to plug I think really you know try trying to walk a version of the path that you are in terms of being able to have agents help with the bookkeeping, help with pre-po coding stuff, identifying sis you know errors in the system, making sure a house doesn't get left behind because someone just missed an email. You know, and I lose two weeks because someone hasn't thought about that address in the past two weeks because they missed an email and it fell off of a report. Just trying to tighten all those, you know, get those systems talking to each other and tighten start to tighten up all those things. Like right now, I literally do a bi weekly audit of those three systems against each other in a meeting of like four people. And that's what we do, you know. And it's, you know, it serves to also keep us on the same page with respect to all the inventory because we go address by address, but you know. That's not a super efficient, you know, but it ends up, you know, and we and we can't we that's what keeps us from like letting balls drop. But if I miss one of those meetings, it's a problem because, you know, then I go a month without that. And then those, you know, when a ball drops, it's it starts to get pretty expensive. So anyway, I've got this vision of a system that links all of, you know, that links those three systems together in a database. And then I and and I think I should I think I need to bust out a Hermes agent and say let and And start playing with it because I kind of left my open claw behind. I haven't really I haven't touched it much in the past like six weeks. and David Moses (57:46) You you can ask your open claw agent to build your Hermes agent. That's what I did and it did and it would it did a great job. It it will. It will. Just just I just told Jack BeVier (57:50) yeah? It won't it won't hurt its feelings. It'll be okay with it. David Moses (57:57) it, Hey, you know, do you I'm not sunsetting you. You you're you're gonna still get, you know, all the compute you want. You're gonna be super happy when this is, you know. But Jack BeVier (58:06) And then you downgraded it to a to a an inferior token, an inferior API token. David Moses (58:12) But but you but I so so you're focus like and you probably get a lot more leeway. Like the the stakes are lower there, right? I mean like the money might be you know high in terms of what you can save or what you can make on that part of it. But if you f if you screw something up, y there's no client, like there's your properties, right? Yeah. You know like a client. Yeah. Jack BeVier (58:31) Yeah, yeah, yeah. Yeah. And I'm o and I'm still gonna run it manual myself in the meantime. Right. I'm just trying to I'm trying to make some of my processes redundant through through, I don't know, custom software and agents. Yeah. David Moses (58:43) Yeah, bust out that Hermes agent. You're gonna be very, very happy. I'm gonna I'm gonna see if I can do Jack BeVier (58:46) Yeah, I gotta do that. David Moses (58:47) some show and tell on that condo, the the condo documents, the the invoicing. So I can Jack BeVier (58:52) yeah. David Moses (58:53) I I'll show it to you next week and you can critique it and tell me, hey, well it'd be cool to do this, be cool to do that. and Jack BeVier (58:59) Excellent. David Moses (59:00) and I I'll have it I'll have it ready for us to look at. It's probably it's probably ready now. Jack BeVier (59:05) Excellent. All right, guys. Sounds good. Hope everyone got something out of that. thanks for tuning in to forked.ai. Hope everyone has a great week. See ya. David Moses (59:13) Thanks. Take care.