Latest / The AI Ecosystem Podcast / Is the era of integrations over? Saving tokens and keeping our humanity in the AI era
Transcript
- Bob Moore: Greetings, welcome back to another episode of the AI Ecosystem Podcast. We seem to never run out of things to talk about here and we've got a bunch of really fun topics lined up for today. Bob Moore's CEO Co-Founder at Crossbeam here with our Chief product Officer, Lindsey DeFalco, who's gonna kick us off. How are you doing, Lindsey? Lindsey DeFalco: Great. Hope everyone is doing fantastic as well. yeah, I'm ⁓ I'm excited about what we're gonna talk about today. It's like a a bit of different topics that we've touched on in ⁓ in past episodes, but ⁓ you know, as we say every time everything changes week to week. So ⁓ there's there's even more to talk about. so ⁓ I think the the first thing we wanna we wanna get into ⁓ is concept of ⁓ what integrations are now, which we've talked about a bit before. and how that's changed with the fact that ⁓ you know you have two mcp servers working together in in an AI platform or in another mcp client and that in itself constitutes an integration ⁓ and kind of all the mechanics around that so ⁓ you know the the fact that what ⁓ what kind of defines how these ⁓ Bob Moore: Mm-hmm. Lindsey DeFalco: Different MCPs work together is really just text, whether it's in the form of a skill, a plug-in, and then there's also a lot of like ⁓ cost optimization ⁓ exercises that that people are running, a lot of products popping up in that space. ⁓ and just ⁓ like a lot of different variables to ⁓ to think about and consider both as a builder and a consumer of this this new wave of of integration. So is that a an accurate kickoff, Bob? And and what what part do you want to start with there? Bob Moore: Yeah. Yeah. That's good. So you use the phrase what integrations are now. Maybe a good place to start is just wrapping our heads around what integrations were before, like what that even meant. ⁓ like to me, I feel like it it can mean a lot of things. ⁓ but traditionally speaking, it's when two technology products, typically software products, can talk to each other. ⁓ and usually that's achieved by Lindsey DeFalco: Mm. Yeah. Mm-hmm. Bob Moore: either bi-directional or at least one directional APIs. But then there's this tricky business, which is APIs are code, but they are also data pipelines. And those pipelines typically need to extract data, transform the data in some way, load it into some kind of destination that then knows how to process it and create some kind of value. A classic example would be like the crossbeam ⁓ widget inside of Salesforce, ⁓ you know, the cross beam copilot. So Basically when you're inside of Salesforce, all of a sudden you're an authorized user and Salesforce knows how to go talk to Crossbeam and get a bunch of data out to serve up to you to use in that context. Integration, great. These can be backend only, they can be front-end and back end, they can be all kinds of things in between, but they typically are code, and that code typically needs to be executed somewhere. And there's typically some kind of data payload or processing or something. So the question is always who's gonna host it? ⁓ How is it powered? Who's in charge of the engine that drives it? It feels heavy. Yeah. ⁓ and also who's gonna build it? Of course, yeah. Like even in that example we just gave, like you could argue, hey, it sounds that sounds like something Salesforce should build by tapping into the CrossFeam APIs. But also there's Salesforce that are gonna build ⁓ they're gonna build our integration, especially when we were a a Measly startup. So we have to build it, and then they have to provide a means by which other people can build stuff into their stuff, which is its own additional layer of tech. So the important thing being Lindsey DeFalco: gonna build it. Yeah. Bob Moore: Integrations, the way that technology products interact with each other and talk to each other used to be like pretty heavy-duty engineering and infrastructure investments. And even the companies that tried to simplify that, like the Zapiers of the World, or you know, on the more enterprise side, like the Treys or the Workados ⁓ of of the world, It's still at the end of the day. ⁓ Zapier is great for like simple consumer use cases, but when it gets complicated, it gets too complicated. And then for Trey or Wiccado, like there are companies that pay those businesses millions of dollars a year. Like it's not like you're avoiding the enterprise software problem and getting something that's cheap and easy by doing it at scale. Like these things tend to be custom and annoying. So so now in the AI world, maybe I'll toss it back to you. Can you state more plainly and maybe even give an example of like what an integration Lindsey DeFalco: Yeah. Bob Moore: Looks like right now. Lindsey DeFalco: Yeah. And and also just to touch on like since this is, you know, the AI ecosystem podcast, the the kind of like partnership side of the house here too is something I think a lot about because in the world that Bob was just describing, the ⁓ teams had to involve a lot of different people to get to a point where they both understood not just the business use case, but also what was technically possible. And that has always felt like a very kind of long pull on the tent of ⁓ you know, you'll often have partner teams that are like, hey, there's an opportunity here. Our use cases match. We have customers in common. And then you bring in the tech team and it's like, ⁓ our API doesn't do that. ⁓ or, you know, like, ⁓ we can't ingest data. We can only push data. ⁓ and so that kind of leads me into what they look like now. and their simple basic form. an integration now can literally be ⁓ few paragraphs of text telling you how to use two different MCP servers. And this is where things get start to get pretty crazy. So ⁓ only do you get to skip the you know, O does our technology support this ⁓ question because you know anyone can ask Claude, hey, does your MCP server ⁓ do this? You know, what are the tools available? What are the capabilities available? Like what can you do? ⁓ but really if you you know if you look at like a a ⁓ a product like a Plugins and Claude are a great example of what an integration today is. So a plugin is literally multiple connectors, which are MCP tools, and then a set of skills, which are these written text instructions. And so you have this, you know, ⁓ programmatic data access into these two different tools. ⁓ like Bob said that used to need to live in either one tool or the other tool. ⁓ but now that ability to access that lives in this ⁓ this other third party tool. So in this example, Claude. ⁓ and then you have this set of written instructions that are ⁓ you know, maybe saying something like, ⁓ if you wanna go target this account, then you need to go ⁓ hit Salesforce's MCP server and get ⁓ account information. You need to hit CrossName's MCP server and pull the ecosystem data. ⁓ and then you need to, you know, submit this lead to your your PRM and you know use their MCP server. And so that's the world we're in now where if if MCP servers ⁓ you know and I I'll use that as a foundational example. We won't get into what this all means with agents and everything right now too, but it it's literally just a text file. ⁓ like this text file and the fact that these in these MCPs exist like are an integration. Bob Moore: Yeah, so it's it's the fact that the L LMs can reason, I think gets rid of a ton of the code that like if you think about those ETL pipelines, the extract transform load, the extraction part kind of ⁓ out of the reasoning that the LMs can figure out in real time around how to hit the APIs and get the data out that they need and the transformation is also something that the LLMs are exceptionally good at doing, kind of even within their own logic layers, a lot of times without even writing code. ⁓ or without writing code that is visible to the end user. ⁓ So you end up with kind of like the E and the T parts, there's kind of the getting the data out and getting the answers out in the way you want, being things that are actually kind of developed on the fly every single time it's needed as needed just by talking to the APIs. And then the load part is just there's like a receiver on the other end that you can also just toss these arbitrary payloads to. ⁓ And it also either Is sending it to an API or to an MCP in a very structured format that it knows how to process in a deterministic way, or it's tossing it into some chat agent on the other side that's got an MCP behind it that can further transform the data into exactly what it needs. And it's like, yeah, bing, bang, boom, ⁓ it's done. And yeah, you can make these these plugins now that are multi-party integrations where you're combining data from a whole bunch of different sources and creating some net new value proposition. But again, yeah, it's just like as long as everybody's got. really good abilities to get data in and get data out that the LLMs know how to interact with, which MCP helps with, as CLIs and and ⁓ you everything else that's kind of like a gateway to interacting with their data models. You can build anything. And then that anything is mostly text. And I think that's where like my brain starts to melt a little bit with ⁓ I immediately go to the business implications of this and just Is there is there a way to monetize integrations anymore? Is there a way to build scalable marketplaces for integrations or plugins? It feels like right now with Claude, you know, there's not there there's kind of like their baseline plug in universe that we're a part of. I get this terminology mixed up. What's the cross be if you go into the anthropic marketplace like hit connect cross beam, is that considered a plug in or is that a ⁓ it's a connector, Lindsey DeFalco: It's a connector. So you have Multiple one or more connectors can exist in a plugin alongside one or more skills. So it basically just like in it creates the connective tissue in a structured way that is the integration. Bob Moore: So Got it. ⁓ the crossing connector is allowing people to do the extract, the LLM is allowing people to do the transform. ⁓ And then if you had plugin where you had another connector, like, hey, I want to push this into Salesforce, ⁓ and you've got a Salesforce then the load can also be handled. So you build a plugin that basically constitutes what would have been an integration before. With with no code, with just some text. It's like pull it out of here, make it look like this, push it into there. Okay, so great. So that actually frames the question of like a plug in marketplace then, currently my understanding of plugins is that there's not really across company lines style marketplaces or best practices, but within a company you can build plugins that are kind of shared across team members. Is that kind of the state of the art right now? Lindsey DeFalco: Yeah, there so you can you can look they've started doing plugins, browsable plugins. So again, we'll use, you know, ⁓ Claude as this example. And this is like there's an anthropic and partners ⁓ plugins mm, I don't know, li I'll call it a library. ⁓ and one thing I've seen that is interesting, so most of these are just like single single companies. So it's like, you know, there's an intercom one. There's a Figma one. There's an Airtable one. and this theme of having some sort of like marketplace to hold these skills or plugins has been popping up. So you'll notice a lot of companies also have like skill libraries on their website. And it's you know, some that they've built in-house, but then also people can you know, submit to have their skill listed on ⁓ on in the skill library. So like Figma is a good example of this. They if you go to their marketing website, they have like a whole library of skills where you can just go again download this like text is instructions about ⁓ you know what ⁓ ⁓ the LLM should be doing with your specific connection. ⁓ yeah so I do think it's interesting because it's like what is yeah what's an integration marketplace if everyone has MCPs. ⁓ It's like an MCP marketplace plus like a skill and plugin marketplace together. that's that's what you need. I don't know what does to like actual integration marketplaces. You are muted. My favorite thing to say post-2020. Bob Moore: Amazing. ⁓ it is ⁓ it's this it's this interesting thing where it's and it's not less valuable, it's more valuable. So it's it's significantly more valuable and more robust and more resilient, less likely to break, requires less maintenance, creates enormously more value, and is so incredibly thin that it's almost impossible to monetize against. I mean like the MCPs you can monetize against, but I guess it just is It makes me really curious about where the where the monetization in this universe is gonna go. And maybe this is why, you know, among other things, we see this like expectant compression of software multiples and ⁓ it's like what we're able to charge for the amount of value that gets delivered by software products ⁓ in the AI era is just like those exp that bar is gonna be so incredibly high and Yeah, things like these integrations that you ⁓ you maybe used to be able to like in an enterprise environment, we've had people, you know, pay us hundreds of thousands of dollars to build a single integration for a single company. ⁓ and it I I can't fathom that happening five years from now. ⁓ and for most companies, probably much less than that. ⁓ so there's this whole like slice of the GDP of the way software and especially cloud software gets monetized that's just gonna like poof go up in smoke. Unless somebody really, really creatively figures out how to charge for it. ⁓ and the best guess I have is just making MCP really expensive. and and trying not to charge for the integration part. Lindsey DeFalco: Yeah. And that's actually so that point you made is is interesting too because there's ⁓ you know, ⁓ everyone's all different things with credits and tokens and ⁓ you know, however they're charging for their AI functionality, but ⁓ there really not a ton of consistency ⁓ in market around like whether or not people do charge for their MCP or don't. ⁓ and seen like an interesting pattern where, you know, ⁓ basically a lot of companies decide to charge on data in. ⁓ Bob Moore: Mm, yeah. Lindsey DeFalco: As opposed to data out. And then there's companies that it makes more business sense for them to charge for, you know, data out as opposed to data in. ⁓ and you enter this weird ⁓ I mean, I think everyone has been anyone who procures anything has been complaining about cost predictability of actually procuring SaaS products now with ⁓ you know, with all the unknowns around credits and tokens and all of that. and it introduces a similar question with the fact that you know, you are Bob Moore: Great point. Yeah. Lindsey DeFalco: Your integration is also within this other tool that has its own cost structure. So, what I mean by that is like we get a lot of questions with our ⁓ our MCP server that we have. We'll get a lot of questions that are actually questions for Claude on how they bill and how they access MCP tools and you know what it means cost-wise for their team to be using Claude. and sure they're using Crossbeam's MCP server when they're, you know, when they're using it, but ⁓ the question is actually not not for us. It's about you know Claude usage. And so you you in one of these ⁓ these plugins and say you've got five MCP connectors in there to do one thing, maybe it's a skill, whatever. ⁓ you what's the cost of running that skill? ⁓ and how can you ⁓ how can you actually estimate that if some of the companies charge for MCP access, some don't, you don't know how many tokens it's using in Claude. ⁓ And I think we're still in this like super early phase of what the monetization actually looks like with all of this to your point. ⁓ and it's so unpredictable right now. Bob Moore: Yeah, it is also we got into this I won't call it an argument, but I'd say like a a clarification internally with our own teams recently where we're figuring out how to charge for our MCP and we charge for it when you use it to take data out of the platform. Now we've always charged to take data out of the platform. And somebody internally brought up, but hey, ⁓ I forget the example. That they use it. It was Gong or somebody, right? ⁓ there's plenty of of companies. It's like they don't charge anything for the MCP. but those companies have a business model where they charge you when the data's going in. So like, ⁓ it might have been post hog too, something like that, where it's like ⁓ you send data points, yeah, you send data points into them and they ingest it and they get you on the way in, and then they give you, you know, you can go create your value on getting the data out ⁓ when you're done working on it. Lindsey DeFalco: Mm. Figma post hoc, yeah. Bob Moore: We do the opposite. We let you send an unlimited amount of data in. We don't charge you for it. ⁓ we let you ingest through all these complex ⁓ integrations. ⁓ there's the there's that word, who knows how complex they'll be in the future. ⁓ we let you ingest all this data, we don't charge for it. ⁓ it's only when you go and actually turn it into some kind of value and pull it out downstream. So the other challenge is that there's enough confusion in the market that buyers are going to look ⁓ r just Buyers will get less sophisticated about why MCPs are being charged for by different companies than they were about why they were paying money for those companies in the first place in a pre MCP world. And I think paying paying for data in versus data out versus tokens or just the amount of ⁓ kind of compute that has been applied against whatever it is that you're ⁓ you're kind of asking that platform to do. it's it's gonna there's gonna be a lot of head scratching ⁓ conversations while everybody wraps their minds around this stuff. ⁓ and we're starting to we're starting to feel it now, right? Which is like, do we have an announcement to make or is our like MCP officially ⁓ super live and awesome, or we just released some cool stuff? Is there a ⁓ we're we're in a zone now where the MCP is like kind of live and in prime time for Crossbeam, right? Lindsey DeFalco: Yeah, yeah. And I think like the ⁓ you know, the questions where it overlaps with another pricing model are always interesting to me because I think those were way more isolated when it was, you know, one to one integrations and it's a conversation like again, the partner conversation was always, okay, what tier is this on for you? ⁓ okay, so you know, it does it make sense to have this on a different tier and having the tier mismatch almost of like, ⁓ our free accounts can use this integration, but you have to be on a superpaid tier. Bob Moore: Mm-hmm. Lindsey DeFalco: And it's like that that same thing kind of ⁓ still exists, but it's just in in the cost modeling. So you know, looking at again, like if you're in in Claude and and trying to figure out ⁓ you know, what's Claude gonna charge me plus what is CrossMeam gonna charge me ⁓ for the MCP, like what does this actually like cost to roll out the tool? that I think is still like the that's gonna be like the trickiest thing to navigate in the market right now. I know you were you were talking to me a little bit about the cost of using all of these different models in general. And I don't know if you want to get into that, but I thought it was interesting. It's like it people everywhere are trying to hack this and maybe it's not hacking. you know, maybe it's using it as intended. But yeah, I'm I ⁓ I think the folks would love to hear about that. Yeah. Bob Moore: Yeah. Yeah, that's like ⁓ value engineering. ⁓ it's it's this very funny thing where I was with a group of friends who are all kind of engineers by training turned entrepreneur types a week or so ago. And at first we were talking about this now fairly well known I guess that you can install between your actual Claude code instance or ⁓ whatever the interfaces which you interact with your LLM the LLM itself and it's called caveman. ⁓ the idea is that it kind of wraps these instructions around every query that you send in that say that basically have an answer you like a caveman. And when I say like a caveman, it's so that like your LLM will talk to you and say as few words as humanly possible in order to express the information. ⁓ and it's like there's there's an episode of The Office where Kevin is doing this, I forget why, but it's like ⁓ why use more word when less word good? ⁓ and just compressing what you get out as much as possible. And what's interesting is that it doesn't actually impact or degrade the quality of the the thought. or the analysis or the conclusions that get reached by the LLM. But what it does do is cut down the number of output tokens ⁓ that get generated because what it's actually serving up to you at the end of the day as a result of its reasoning just is is less because it's more condensed. and it's kind of wild. Some people see like 30% plus compression in the amount of output tokens consumed as a result of using this thing, which is pretty wild. and then interestingly of them brought up this thing that I was not aware of, which is that there's a there's a plug-in where ⁓ you can actually put it on the other side, which is in between what you type in and what actually gets sent to the LLM, that basically without you seeing it or needing to worry about it, cavemanifies what you have typed in. ⁓ if you write some kind of big verbose paragraph that is ⁓ unnecessarily descriptive or repetitive What I what this thing is doing is it's sending that paragraph to a very, very inexpensive model to clean it up, shorten it, condense it, and kind of make it express what you're trying to express in as few tokens as possible. And that cuts down on the input tokens ⁓ that are actually going in the other side. So, you know, when you use these two things in combination, it's this funny thing where We as humans want to express ourselves, me especially, by rambling on and saying too much and being too verbose. And then ⁓ the LLM does what it does. And maybe it ⁓ you know, normally would try to match your tone. ⁓ but instead coming out the other end, you get the caveman stuff back. So it's like this, you know, caveman in, caveman out buffer, ⁓ that when you do these things in combination, you might be able to like chop your chop your LLM bill in half, depending on what kind of work you're doing. Like you can't Lindsey DeFalco: Mm-hmm. Bob Moore: ⁓ make code as a caveman. But you can have variable names that are like, you know, specific like explicitly ⁓ never more than five characters long and you can ⁓ you know do other things. The the minification of code is a real thing that's been around for a while. ⁓ but now instead of bandwidth and speed reasons, it's for token consumption reasons. And ⁓ that's a pretty that's a pretty wild thing to think about because there's a ton of that value engineering going on. Lindsey DeFalco: Yeah. Yeah. And I think what what's interesting to me about it too is like when we talk about ⁓ what integrations used to be and what they are now, there is a level of ver variability in terms of how people use it that didn't exist before. So like the flexibility, ⁓ you know, if you ask me how much, you know, ⁓ someone's gonna use a specific MCP tool or you know, our MCP server, it's like, I don't know, how do they talk to Claude? are they asking, you know, give me Bob Moore: Yeah, totally. Lindsey DeFalco: fifty accounts. Okay, now give me these five accounts. Okay, now let's dig into this one account. you know, it's a a lot of it's just how people are interacting with these models, how they talk to them, you know, how well it's interpreting it, how much it knows to actually ⁓ hit the right tools. And some of that is, you know, tool design as well, which I know, Bob, you went a bit of a it wasn't even a rant, whatever. like tool discovery. ⁓ don't forget what you're calling it. Like not AEO, but Bob Moore: ⁓ yeah, totally. Yeah, yeah, yeah. No, yeah. This is another like yeah, it's like ⁓ kind of like growth engineering your way into making the L L Ms always think your tool is the right tool for the job, just to steer maximum ⁓ requests in, yeah, yeah. Lindsey DeFalco: Yeah. Yeah, yeah, exactly. And like, I don't know if anyone's been, you know, talking to Claude or Chat GPT before and you're like, why is it calling that skill? Why is it do what did I say? ⁓ that made it do that thing. So there's more variability and we're seeing what people are doing to and control it. and I think it's also leaving just a ton of ⁓ you know, confusion this stuff in in the market right now because there are no there's no standards yet. And we haven't had this level of variability in both like usage and pricing that's even like possible before. So Bob Moore: Yeah. And I I don't think that anybody is necessarily good at this yet either, including the foundational model companies, because there this article going around a week or two ago that some some calculation had been done that if you're using like a ChatGPT Pro plan or Codex Pro, as soon as you reach the point where you have actually consumed eleven percent of what you're technically allowed to consume while you're on that plan. Lindsey DeFalco: Yeah. Bob Moore: ⁓ open AI is unprofitable on your customer relationship. So their their margins on what they're selling inside these plans are so bad that they're basically like negative 89% ⁓ gross margins, assuming absolute consumption. Now, because of the way they limit when you can consume tokens and the you get these five or six hour windows during which like if you're doing too much you get cut off and they assume people are sleeping and you know there's there's all these ways in which it's very, very difficult for you to actually reach the 100% mark. ⁓ and you're seeing moves from these companies also to not allow you to like ⁓ ⁓ via APIs ⁓ ⁓ a count against that quota, right? Like all of that is marginally billed at like the the going rate for tokens as opposed to being part of a plan. but does beg the question, I've had this I've had this thought, I'm sure this would be a violation of ⁓ some you know, Anthropic's terms of service or something, and we're not doing it. ⁓ if anyone from Anthropic is listening. ⁓ but I was very tempted and curious about we have so many employees that are on this Claude Teams plan, and they have a certain amount of tokens that are available every six hours. And a lot of them, you know, actually I I would be so bold as to say all of them sleep every night. ⁓ and there are these hours where like, consuming zero percent of what theoretically is available. So how hard would it be to install something that's like at the application layer inside of everybody's terminal that is like a router that sits between Claude code or even the Claude desktop application ⁓ and actual thing that gets sent to the LLM that ⁓ goes and basically sends requests out through different people's accounts based on who has the most quota left available. And then with any remaining tokens, like goes and does fun generic stuff or runs out ahead of our roadmap or does all kinds of other kind of compute and research tasks that maybe are of commodity value but might be interesting or or or move us ahead in some way. ⁓ basically like back in the day when you used to be able to install like a Chrome extension that would mine Bitcoin for you ⁓ when your processor was dormant. It would just like pin your processor until you move the mouse and started using your computer again. Kind of like that thing where coding projects where the context you need is usually just within the scope of the ticket or issue or pull request that you're working with. ⁓ It really doesn't matter that it's my Cloud account that's doing it. Like I could use anybody's. ⁓ there's probably a lot of people that aren't engineers that in theory you know, ⁓ have access to a bunch of tokens that could be getting used for that. So I have to wonder, while it's not something that I've seen be commercially available, ⁓ are people developing these things? Is there like a black market for these like token optimization platforms? Because what you get in your plan is extremely negative margin for the model companies. But when you when you blow out your numbers in your plan, you hit your weekly limit or your limit within your six hour window and you absolutely need to buy more and you buy tokens at the margin, you're paying out the nose for them. Like the the rate at which you spend money for those tokens that you're kind of paying the going rate for as opposed to the effective rate per token you're paying from within your plan is astronomically more. So what you really want to do is squeeze these plants for everything they're worth. And I think that's part of part of what I'm getting at is like when people run out of their plan quota, it should just be able to fall back and consume from somebody else's. And I feel like you could build software to internally route for that. ⁓ in the same way, by the way, there are whole companies out there now that have raised a lot of money that are helping people reduce the amount of token spend that they have overall. And I think what those companies are doing is basically just a black box of the caveman to caveman protocol, ⁓ or some clever twists on it. And that's kind of funny too, and almost gets us back to our plug in discussion where it's all just a bunch of text. All everything that I'm even describing here, right? It's like there's some authentication stuff and some weird security implications and ⁓ a little bit of hosting, but like at the end of the day, like this value is just like having some text an idea that's great and some text that works pretty well and making it maybe difficult to copy ⁓ or re-implement that idea and like having the best mousetrap in the moment is like the way you create value now ⁓ which just makes the barriers crazy low. Lindsey DeFalco: Mm-hmm. Yeah. Well, and it's funny, like those ⁓ I mean, I think token reduction in general will probably always be a thing. ⁓ but the the other more nuanced version of that that you're describing, it's like all any of these companies have to do is change their pricing model and then it just like all goes out the window. ⁓ so I mean, yeah, those the black market style companies are definitely just for the ⁓ immediate opportunities. but I do think all of the the token reduction efforts are interesting. I mean, we've been hearing for a while. What there was that ⁓ some stat that came out about like how much saying please and thank you is costing in terms of ⁓ compute and that it's like Bob Moore: ⁓ yeah, totally. That's speaking of things that the caveman mode would probably completely eliminate, Lindsey DeFalco: Yeah. And there's all these, you know, studies on like the tone you use and and ⁓ how to talk to the different models to ⁓ to get the most out of them. And I think a lot of them go against ⁓ how people generally talk to humans, ⁓ which I'm sure we'll have some big psychological study on in like five, ten years. ⁓ the impact of that. But ⁓ yeah, it it feels like a similar ⁓ similar problem, but like ⁓ codified quote unquote. which is like the Bob Moore: Yeah, it's really it I I feel like it will be you people talk about the Gen Z stare. ⁓ I don't know if you've seen any of that, but like there's a there's this thing of like kids that were the argument is kids that were raised by iPads basically. Like there's kind of this extremely distant ⁓ also COVID era, you know, it's like COVID during your formative years, but like you interact with ⁓ Lindsey DeFalco: Mm-hmm. Yeah, yeah. Yeah, covered air. Bob Moore: They're all entering the workforce now, with entry level jobs at your local Starbucks or even in in like a ⁓ you know, office setting. And it's like, yeah, there's like this it's like they're looking past you. It's like they're looking at you as though you are a screen. ⁓ and it's this generational thing because of the technology that was available to that generation at the time. And I'm I know that ⁓ Lindsey DeFalco: Yeah, yeah. Bob Moore: you might be slightly too young to be a millennial. ⁓ maybe you're you might be s I'm an elder millennial, so you're like a solid millennial, yeah. I'm almost I'm almost Gen X, yeah. ⁓ but I claim millennial status keep me to keep me young. ⁓ but anyway, yeah, like we I'm sure, you know, we had our quirks too, but there will be some new generation that's coming up in this post LLM world that yeah, they might be all walking around talking like cavemans to save tokens and interacting with each other. ⁓ because ⁓ Lindsey DeFalco: No, I'm a Yeah, I'm a I'm solidly a millennial. Bob Moore: that is just the way that you end up learning to interface ⁓ with stuff. So instead of staring at an iPad, it is interactive, but yeah, maybe you end up a lot more curt and we all end up sounding like and talking like ChatGPT. There's a lot of ⁓ dashes are gonna be going around ⁓ in the in the future. Lindsey DeFalco: Yeah. ⁓ there's that ⁓ there's like a meme going around that's like, I'm gonna start talking to my partner like Chat GPT. And the example is that one person is like, Hey, did you unload the dishwasher? And the person responds, Yes, I unload the dishwasher, and they're Well, the dishwasher isn't unloaded. And the person responds, You're right to challenge that. And turns out I didn't unload the dishwasher, and that's on me. ⁓ so ⁓ it's already ⁓ already taking over. Bob Moore: Yeah. ⁓ I feel really bad about not doing the dishes, so I've gone out and bought seven new dishwashers, ⁓ and they will they will arrive. I've thrown away our current dishwasher. It can't be recovered. but I have procured several new ones, yeah. Lindsey DeFalco: There's also like Yeah. Yeah. You're right to challenge them. ⁓ yeah, there's also like we haven't ever really touched on this stuff a lot on the ⁓ on the podcast, but there's there's also an interesting ⁓ like ⁓ confirmation of existing biases that exist because of the fact that the all of these models are trained on, you know, existing data and patterns. And so ⁓ one thing I'm actually really interested to see what it looks like, ⁓ you know, in like five to ten years is just like the like the the gender bias biases that are that are baked into it. I read this like ⁓ think piece the other where somebody was editing their resume ⁓ and ⁓ they're a woman and resume like very like you female name and it was editing it as such. And then they created a new account and they changed their name from like Jennifer to James, did the exact same resume and it was completely different. And like all of the language was like much stronger. ⁓ and there's all of these like everything that all of these models are trained on have all of the inherent, you know, ⁓ biases baked them, whether that's like race, gender, all the all the same diversity issues that think have existed in tech are getting Bob Moore: ⁓ interesting. Lindsey DeFalco: and that's where I'm like, because they're getting amplified right now. but I'm curious if it also is like codifying them in a way that ⁓ allows to be ⁓ more more ⁓ or something. Like I I think there's a way that this actually could be beneficial for these existing, but not yet. Like right now I think it's just that people don't don't realize that and they like ⁓ and that's kind of ⁓ ethical dilemma think with AI is everyone thinks it's a neutral party. Bob Moore: Yeah. Lindsey DeFalco: And it's not a neutral party. It's trained on a very specific set of data. Bob Moore: Yeah. If Yeah, it is like it's it's almost like the ⁓ the minute that the models came out and became mainstream in society, then they immediately have this impact ⁓ on ⁓ the in we express ourselves and we communicate as society because it's doing all of that cognitive lifting for us. So whether it's like having a hard conversation with your partner about why you didn't watch the dishes or it's writing your resume or whatever else, like It is we as humans are already falling victim to it doing all the work. So it's almost like whatever whatever society was and whatever ⁓ good, bad, and ugly characteristics of society exist at the moment at which ⁓ the models get trained, yes, those those things that are true about society, which is which is far from perfect, get baked in and almost like locked in to the way that the models express themselves. And then from then on out it's just like a photocopy of a photocopy. Lindsey DeFalco: Mm. Bob Moore: and we have seen, if you remember in the very early days of I this with Google where it was very clear that they had attempted to overlay some guardrails ⁓ to attempt to avoid these biases try to infuse like a certain ⁓ level of the destruction of stereotypes from appearing inside of images that get generated or inside of text that get generated. ⁓ the problem is like when you overlay that onto probabilistic stuff, you just it it doesn't actually do a great job of creating positive output. It just takes you into like crazy nonsense town really quickly because you have this massive probabilistic model. And then what basically amounts to again, it's all just text, like a system query with a paragraph in it that's like Lindsey DeFalco: Yeah. Yeah. Mm. Bob Moore: You know, if you're gonna and by the way, if you generate an image of ⁓ you know, a person with these characteristics, then make sure that you're you're not ignoring the fact that this issue exists and then boom, all of a sudden every single character that you generate looks exactly the same. ⁓ or like there there are these other weird artifacts that actually make the LLM less good at the things it's supposed to be good at, ⁓ independent of of those biases. So it's like Lindsey DeFalco: Yeah. Bob Moore: Yeah, it's kind of fucked. ⁓ because you really like you lock it. Are we have I said fucked on this podcast yet? ⁓ but you do you do create this like lock-in moment where unless the reasoning unless somehow you can bake into the LLMs like a values system. and first of all, even coming up with a value system is controversial in and of itself. ⁓ like how do you even get there? Lindsey DeFalco: Yeah. Mm. Exactly. Yeah. Bob Moore: It'll come up with its own value system. ⁓ but I think we're more worried about like don't kill all humans as like a value, ⁓ like a value that it wants to get right that I'm not entirely sure that we've nailed yet. so yeah. ⁓ yeah, yeah, it's like directive one, directive two. Yeah, yeah, yeah. That's exactly right, yeah. Lindsey DeFalco: Yeah, yeah. That's the iRobot? It's like the there's three laws of Yeah, yeah. ⁓ Yeah, I mean I I think we've seen this too with like ⁓ you know, Grok was giving very different responses to things than like, you know, chat GPT and then Bob Moore: ⁓ god, yeah. Yeah, and Gro I would almost argue that Groc ⁓ was on the opposite end of the spectrum from Google. ⁓ like Google, you know, made George Washington a person of color when you generated an image of him. And when you asked Groc about George Washington, it almost had system instructions to make it intentionally inflammatory or like differentiate it by indexing way too far in a direction of having a controversial or unhelpful point of view, which ⁓ Lindsey DeFalco: Yeah, exactly. Bob Moore: It it just it just two different flavors of the same like you can't wrap system queries around this giant, massively ⁓ designed probabilistic, finely tuned, weighted model and expect it to actually ⁓ undermine what's at the root of this thing. It just is gonna give you shittier stuff, right? That's that's that's not actually what the designers want or the people who are trying to make it better want. ⁓ Lindsey DeFalco: Mm. Bob Moore: So it's so hard 'cause it it literally does have a mind of its own. Even if we it's it's it's a hive mind of all of all of our minds, yeah. Lindsey DeFalco: Yeah, it's it's not unbiased, I think is the takeaway. Yeah, what actually Bob Moore: It's not unbiased. I would fully agree with that. It is it is a reflection of if any bias exists in the world, it is an inherent reflection of that bias. Yeah. Lindsey DeFalco: Yeah. ⁓ yeah. And the whole reason that I thought of this is actually because ⁓ you know, you talk about these like cavemen ⁓ tools and yeah, Kevin in the office is an extreme example, but like, you know, stereotypically men speak more directly in, you know, ⁓ email and like professional communications than women. ⁓ and there's all these jokes about like how many exclamation points and you know ⁓ whatever are used. and you know, not a hard and fast rule, but but definitely something that I think is something kind of ⁓ trained ⁓ into women a way. ⁓ but what's interesting about is it's like are certain people more effective with AI just by nature of these ⁓ biases that are like baked into the model how ⁓ like you know how they've been trained to communicate. And you know, whether it's not a gendered thing or or whatever, maybe let's just take like ⁓ people that are direct and people that are it's like Bob Moore: Yeah. Lindsey DeFalco: Are these people that are direct people ⁓ going to be more successful at creating things with AI than people that are ⁓ that are indirect? ⁓ and so it's just like a it's it just points the variability of like again, how you even interact with these models yields different results. and there's no like hard and fast rules about what what is what. So yeah. Bob Moore: Mm. Yeah. That's a great point. Yeah, is there is it inherently ⁓ more expensive and costly and less effective to be a consumer of AI as a person who communicates expressively than it is as a person who communicates directly. And you can you can apply ⁓ gender against that ⁓ if you choose to, but like and I think answer is a resounding Lindsey DeFalco: Yeah. Exactly. Right. Bob Moore: Yes, there is a tax on being a more expressive communicator in the if these cavemen things really work, ⁓ and it sounds like they do, and I have not like kicked the tires on them myself yet, like I know a lot of very smart people who swear by them. ⁓ if they actually are effective, then yeah, ⁓ for all the people out there not using them, which is ninety nine point nine percent of people, you're paying less for AI if you're a more effective direct communicator. ⁓ I won't even say effective, if you're a more direct communicator. Lindsey DeFalco: Mm-hmm. Yeah. Mm-hmm. Bob Moore: Yeah. Crazy. Yeah. Lindsey DeFalco: Which I think again mirrors real life, right? Like ha how much feedback have you given as a a leader of leaders that's like about communication style, right? It's like people that are too indirect or too fluffy or too flowy, it can get away get in the way of, you know, actually getting the point across. so it is ⁓ yet again, modeling real life of like the way that you communicate is important and ⁓ and that's like yeah. Bob Moore: Yeah, I agree. I I I think it's like a you know, I going back to of the things that I have always loved doing in building companies is writing. ⁓ it's this weird, like ⁓ quasi right brained skill set in a extremely left brain oriented ⁓ career that I've had. And ⁓ if I had to throw away all my skills except for one right now, I think I would keep my writing. ⁓ because I I think it's actually the thing, the ability to kind of wrap your head around how to express complex ideas really effectively and ⁓ kind of parse written word in such a way where you get the knowledge and value at the other end is kind of the new old superpower. Like what's old is new again. and we used to have to express these things in in code and in math and other things in order to innovate and invent. Lindsey DeFalco: Mm-hmm. Mm. Bob Moore: And we don't have to anymore. ⁓ and in fact, like the written word is is like the most powerful coding language right now. which is which is kinda wild. like liberal arts ⁓ liberal folks, like here here here we come. ⁓ it's it's time of the day in the sun. Yeah. Lindsey DeFalco: Yeah. Well, and it's like ⁓ I was actually thinking about this a lot lately with, you know, output from ⁓ from AI. ⁓ it's interesting to me because a lot of people just like unabashedly post just raw like Claude and ⁓ I I it's still a very opinion thing on like when that's okay and when that's not okay. ⁓ and Bob Moore: Yeah. Lindsey DeFalco: I was telling somebody I was working on something that I sent that I sent you, Bob. It was like a ⁓ you know, like a a memo that I was writing and I was like, ⁓ yeah, you know, and I used Claude and they were like, You used AI to write a memo. I was like, I think Bob would be disappointed if I didn't use ⁓ Claude. Bob Moore: Yeah. If I get anything that's longer than two paragraphs and I find out somebody wrote it by hand, I'm ⁓ I'm like, Wait, did you d did you d break out the chisel on the stone while while you were at it? ⁓ yeah. Lindsey DeFalco: Yeah. Right. Right. But I but I still have this ⁓ inclination to go through and make it not sound AI. ⁓ and ⁓ you know, I've like from CEOs about whatever they're doing strategically at their company, and I'm like, this literally just reads like you just spit it out with zero, ⁓ zero review. And so, you know, it's like what is ⁓ the art of writing, I think it's still like a split opinion on whether or not people care if it's Bob Moore: ⁓ sure. Yeah, yeah. Mm-hmm. Lindsey DeFalco: You know, like don't care if some people send me straight Claude output stuff if it's we're just like trying to get something done. Whatever, don't rewrite it. but it's like there's this kind of unspoken rule set that is gonna differ at different companies and different use cases on like when it's appropriate to do that and when it's not, and you know, is Claude gonna get good at mimicking that? AI model is gonna get good at mimicking that and then you have to like redo how you talk and you know, ⁓ I don't know. ⁓ Bob Moore: Yeah, I think it just shifts the burden communication the logic why certain hard decisions like what are the hard, non-obvious parts of this and how do we reason out how to arrive at them? Because it's one to have Claude just spit out a memo that gives you like a strategic direction. ⁓ but it's another to go through a very thoughtful process and session with Claude or with a team that is assisted by Claude to work through a lot of very challenging decisions that at the end of the day require human discretion in order to because a lot of times like it's th there's no like it's very rare that there's like a Pareto optimal decision, like no possible reason why an alternative might potentially be better. And this is like asking Claude to like, make a steel man argument or a strongman argument against what you're doing. But you can talk Claude into agreeing with anything pretty easily. And you can talk it into disagreeing with itself. And when it changes its mind, it'll apologize to you. ⁓ and you kind of move on. And this is where a lot of these CEOs get into this like AI psychosis thing of like thinking they're, you know, the greatest business mind ever, 'cause the LLMs keep telling them they are. ⁓ but the reality is like the memo I want to read is not Hey, here's 10 pages of like the obvious stuff that Claude could generate on its own. It's like, here are the extremely hard decisions that basically had to be escalated to human discretion. And here are how how they'd cut. Like how here's how those decisions got made, and here's the logic and reasoning that went into making them. ⁓ And like that's the memo that I will read. And you can have a whole session with Claude about that and come to conclusions and then have Claude write that. five page memo or condense it down to two pages, hopefully. but that's the memo that I want, and it's not the memo that Claude gives you by default when you say, Hey, give me a strategy document. And think that's where like the art of written communication and reading things and like processing how decisions get made is going to move into that world of the non obvious stuff. Like the obvious stuff can be taken for granted. and you get into just like decisioning is the entire thing. Lindsey DeFalco: Yeah. Yeah, that's a great point. the number of times that I'll be, ⁓ know, working on something and it'll come back with like, ⁓ what direction do you want to take this? We can do A, B, or C. I recommend A. ⁓ I think I disagree with it like 90% of the time. ⁓ like, no. ⁓ do C. ⁓ so that is ⁓ and yeah, like, you know, ⁓ this this memo thing I was creating, I was like it it, you know, it didn't take me five minutes. It still took me a good amount of time to work. Bob Moore: Yeah, ⁓ yeah. Sure. Lindsey DeFalco: with ⁓ you know, with Claude on getting to the solution that actually would be the thing. Bob Moore: what a time to be alive. Lindsey DeFalco: Yeah. A little bit ⁓ we we mixed in a little bit of AI ethics in there finally ⁓ Bob Moore: Yeah, it's great. We we have not ⁓ I I've been avoiding ⁓ AI ethics like the plague on this podcast. So it's but it is like it does actually the rubber does really start to meet the road, but it's a very organic way that's like super, super relevant here as we start to talk about like the the nexus of communication and where humans continue to fit in ⁓ and ⁓ what it to use these things and how Lindsey DeFalco: Hm. Yeah. Bob Moore: the nature of you and your identity as a human, ⁓ maybe a direct input as to whether or not you're naturally inclined to be successful in doing that or be like best in class in doing that. Lindsey DeFalco: could we could find out in a few years that there's like a specific type of personality that's like an incredibly ⁓ like eloquent but detailed but long winded, but like it's the perfectly right formula that actually has like the most success. ⁓ Bob Moore: Yeah, well this is it's actually it's funny, I the most argument that I've heard made in that family is from ⁓ the CEO Palantir thinks that that prototype ⁓ who are neurodivergent. So ⁓ folks that are on are on the autism spectrum, folks that have like various forms of ADHD, like people whose thinking process ⁓ is the likely to be ⁓ similar to the models are putting out by default ⁓ and may be contrarian or may have very, very ⁓ reduced ⁓ against Lindsey DeFalco: Hmm. Bob Moore: kind of ⁓ differing opinions ⁓ and communicate very maybe even sometimes brashly. ⁓ like all these things kind of stack up ⁓ and he said he like goes out of his way to try to to hire people that that kind of like sit in the universe of neurodivergency. ⁓ because if you're not neurodivergent, then what why is an LLM not already immediately better than you? like you almost have to go to the point of the conversation we're having, it's like you have to go tell it to or or force it to put you into a place where you're arguing against yourself and creating inherent like contrarianism inside of your discussion in order to get it out of its predictable walls in order to come to conclusions that might actually be differentiated or or be of any value. And I think yeah, his claim is that like neurodivergent folks are like really, really freaking good at that. ⁓ and because of that they just ⁓ they're they're more likely to create novelly valuable things in this era. ⁓ it's an interesting argument. It's interesting. Yeah. Lindsey DeFalco: Yeah, that's interesting. Yeah. Yeah. Cause it is like what is the what is an ideal you know, job description look like for somebody who will only be working with models and not humans? And like does it take away a chunk of the skills that you cared about before? and yeah, I think that that's probably true. But there's a set of skills that are like about you know, whatever it is that has to do with human human interaction. Bob Moore: Yeah. This is Yeah. Lindsey DeFalco: ⁓ that probably change drastically if you just want somebody making decisions, you know, with ⁓ with a model and there there's like a less human interaction, which ⁓ just sounds super depressing because less less human interaction always sounds super depressing. ⁓ but I think it's it's not no human interaction yet. So ⁓ there we go. Bob Moore: Ha ha. Yeah, ⁓ Yes. Yeah. As long as there are humans left. ⁓ I think we'll see I'll take I'll take it as a win, yeah. Hey, there we go, everybody. ⁓ it's been great as always. ⁓ thanks for tuning in and we'll get at you next time on the AI Ecosystem Podcast. Lindsey DeFalco: And on that positive note.