Latest / Sidecar Sync / The Death of Prompt Engineering (?) & OpenAI’s Operator Agent | 57

Transcript

The Death of Prompt Engineering (?) & OpenAI’s Operator Agent | 57 · Sidecar Sync

• 0:00
  Amith: I think this is where people are going to get their first taste of automation, but when the model starts weaving its way into other aspects of their workflow, working for them in various ways, that's going to be, I think, a pretty interesting kind of general purpose capability that will get exciting.
• 0:16
  Amith: Welcome to Sidecar Sync, your weekly dose of innovation.
• 0:22
  Amith: If you're looking for the latest news, insights and developments in the association world, especially those driven by artificial intelligence, you're in the right place.
• 0:31
  Amith: We cut through the noise to bring you the most relevant updates, with a keen focus on how AI and other emerging technologies are shaping the future.
• 0:39
  Amith: No fluff, just facts and informed discussions.
• 0:42
  Amith: I'm Amit Nagarajan, chairman of Blue Cypress, and I'm your host.
• 0:46
  Amith: Greetings everybody and welcome to the Sidecar Sink, your home for all the content you can possibly stand related to artificial intelligence and associations.
• 0:57
  Amith: My name is Amit Nagarajan and my name is Mallory Mejiaz.
• 1:01
  Amith: And we are your hosts and we can't wait to get into another exciting episode with two fantastic and super timely topics in the world of AI and associations.
• 1:10
  Amith: But before we do that, let's take a moment to hear a quick word from our sponsor.
• 1:16
  Speaker 3: Introducing the newly revamped AI Learning Hub, your comprehensive library of self-paced courses designed specifically for association professionals.
• 1:25
  Speaker 3: We've just updated all our content with fresh material covering everything from AI prompting and marketing to events, education, data strategy, ai agents and more.
• 1:37
  Speaker 3: Through the Learning Hub, you can earn your association AI professional certification, recognizing your expertise in applying AI specifically to association challenges and operations.
• 1:48
  Speaker 3: Connect with AI experts during weekly office hours and join a growing community of association professionals who are transforming their organizations through AI.
• 1:58
  Speaker 3: Sign up as an individual or get unlimited access for your entire team at one flat rate.
• 2:10
  Mallory: Start your AI journey today at learnsidecarglobalcom Amith, how is it going today?
• 2:14
  Mallory: We haven't recorded a podcast together in a couple weeks here.
• 2:18
  Amith: Yeah, it's been a bit of time, so I'm doing great.
• 2:20
  Amith: How about you?
• 2:21
  Mallory: I'm doing really well, have some big news on the personal front that I wanted to share with our Sidecar Sync listeners and viewers that I got married on November 9th Not me forgetting already November 9th and then I went on my honeymoon with my husband to Mexico City.
• 2:37
  Mallory: So that's where I've been for the past week or so.
• 2:41
  Mallory: Just got back into the office yesterday.
• 2:43
  Mallory: We had a fantastic time.
• 2:45
  Mallory: The wedding was great.
• 2:45
  Mallory: That was back in New Orleans with all our friends and family, and then we had truly an exciting, amazing few days in Mexico City as well, eating some of the best food ever.
• 2:55
  Mallory: I will say, amit, I don't know if you've been to Mexico City, but the food is fantastic.
• 3:00
  Mallory: Well, I was in Mexico City once when I was a kid, like single-digit years.
• 3:21
  Mallory: I don't remember exactly when, but I days where we didn't think about anything else.
• 3:24
  Mallory: But I'm happy, I will say, to have the wedding behind me.
• 3:29
  Mallory: As fun as it was, it was quite a feat to plan so many little details and when we're already dealing with events, you know, in the world of sidecar it's kind of a crazy thing to also have personal events going on simultaneously.
• 3:41
  Mallory: So happy that November 9th is behind me.
• 3:45
  Amith: Yeah, it's pretty amazing.
• 3:47
  Amith: You managed to pull off Digital Now and then, a couple of weeks later, got married.
• 3:57
  Amith: And if my planner brain was kicking in if I was doing something like that, I might say, hmm well, I wonder if I can combine the two and get a better deal.
• 4:01
  Mallory: No, no, no, you're hearing it right now.
• 4:04
  Amith: We could have had a session just for you guys at Digital Now.
• 4:08
  Mallory: That might sound like my worst nightmare, meath.
• 4:11
  Mallory: You don't expect our first one-year anniversary to be combined with Digital Now in Chicago.
• 4:15
  Mallory: It might have been a good deal to save, but what I will tell you is that it's all relative, because so many people, so many of my friends, said, oh, a wedding must be a lot of details to take care of.
• 4:27
  Mallory: But I said, well, after putting on digital now, honestly the wedding was easy.
• 4:31
  Mallory: Like I hate to say that.
• 4:32
  Mallory: I'm sure for some people it is not easy at all, but not having to coordinate all these sessions and speakers, honestly it was one night, a few hours.
• 4:41
  Mallory: So I would say it's all relative there with the event stuff.
• 4:48
  Amith: Now, mallory, the most important question in the context of this podcast I can ask you about that is did you use AI at all to help you with your wedding?
• 4:55
  Mallory: Oh man, I was not expecting this question.
• 4:57
  Mallory: I don't know that we did.
• 5:00
  Mallory: I hate to put it out there.
• 5:02
  Mallory: I don't know that we used any AI.
• 5:04
  Mallory: My husband is also an avid AI user.
• 5:06
  Mallory: He loves a good chat, gpt and Claude, so maybe he did on his end.
• 5:10
  Mallory: We did write our own vowels, but I explicitly stated we were not allowed to use any AI models to write our own vowels.
• 5:17
  Mallory: So allegedly we should be good there, but no, I don't think we really did in the planning process.
• 5:22
  Mallory: What a shame.
• 5:25
  Amith: But no, I don't think we really did in the planning process.
• 5:27
  Amith: What a shame.
• 5:28
  Amith: Well, maybe not.
• 5:28
  Amith: You know, there are definitely domains where it makes sense because you want to really kind of get into all the details, and so that's.
• 5:34
  Amith: I think perhaps the opportunity with AI is that which makes us uniquely human, is what we want to focus on Right, where we get to spend more of our time because AI takes care of a lot of the other things.
• 5:42
  Amith: So interesting, that's true.
• 5:43
  Mallory: More of our time because AI takes care of a lot of the other things.
• 5:45
  Mallory: So interesting, that's true.
• 5:46
  Mallory: That's a really good segue into our first topic of the day, which is kind of a hard question to ask Is prompt engineering dead?
• 5:55
  Mallory: And then we're going to follow that up with topic two of the day, which is OpenAI's operator agent that should be released soon.
• 5:59
  Mallory: So the first question is prompt engineering dead?
• 6:03
  Mallory: We've obviously all seen how important prompt engineering has become in the AI world, and if you're not familiar with that phrase, prompt engineering it's the art of knowing exactly how to phrase your requests to get the best results from AI models.
• 6:16
  Mallory: It's become a skill that some people have built entire businesses around, so pretty important within the last few months, few years.
• 6:23
  Mallory: But now we're seeing something fascinating the emergence of tools that could automate this process.
• 6:28
  Mallory: Tools like Stanford's DSPY and startups like Zenbase are developing systems that use AI to optimize prompts automatically.
• 6:37
  Mallory: These tools can test thousands of variations, learn from successful outputs and systematically improve results without human intervention.
• 6:45
  Mallory: So, instead of manually crafting prompts, developers can now show these systems examples of good outputs and the AI will reverse engineer the optimal prompt.
• 6:56
  Mallory: So this raises interesting questions about the future of prompt engineering.
• 7:00
  Mallory: If AI can optimize its own prompts.
• 7:03
  Mallory: Will human prompt engineering become obsolete, or will it evolve into something new, perhaps focusing more on defining what good looks like rather than crafting the perfect instructions?
• 7:14
  Mallory: So, amit, this is an interesting question for us at Sidecar because, as you all know, we have an AI prompting course.
• 7:21
  Mallory: So I hope, in this moment, right prompt engineering is not totally dead.
• 7:25
  Mallory: What are your initial thoughts?
• 7:27
  Amith: Well, so we do have our AI prompting course for only $24.
• 7:31
  Amith: People can jump on and check that out.
• 7:34
  Amith: It's a great introduction to AI and the reason we still have it up there is because it's still relevant.
• 7:38
  Amith: And prompting is something that, as like a defined discipline, if you will, probably won't exist.
• 7:45
  Amith: But will you be able to?
• 7:47
  Amith: Will you need to be able to communicate effectively with AI?
• 7:51
  Amith: The answer to that question is absolutely yes.
• 7:53
  Amith: But what's happening with AI and prompting is the same thing that's happened with computing since the very beginning.
• 7:59
  Amith: You know, computing started out with very esoteric and difficult to use ways of communicating with computers.
• 8:05
  Amith: You know the first computers you programmed with like physical switches, and then a big innovation came out and there were punch cards.
• 8:11
  Amith: And then after that there were, you know, some basic ways of inputting text and programming.
• 8:17
  Amith: Languages evolved, but they started off very low level, super, super cumbersome to use with things like assembly language, and then we went into higher and higher levels of languages.
• 8:26
  Amith: So if you think about the history of computer languages, you know back in the 60s we had something called BASIC, which was the first kind of sort of human readable computer language.
• 8:36
  Amith: That really took, you know, significant market share and people started using that weren't like deep computer scientists, and that progress has continued and continued and continued, and computer languages have continued to get easier to use and smarter.
• 8:48
  Amith: What the big deal back in October November 2022, when the chat GPT moment occurred, was that the general public first had a taste of what it's like to communicate at a pretty high level of resolution with a computer through natural language right, and so that first interaction was a major step change for everyone to say hey, wait, you can actually talk to these things without knowing how to code or without knowing the correct sequence of clicks in a particular application.
• 9:16
  Amith: You could just literally type something and get a response.
• 9:20
  Amith: Now, of course, the first versions of that were quite rudimentary, and so our prompt skills were super important, because if you were good at prompting the computer or really the AI model I should say was capable of producing a useful output.
• 9:33
  Amith: But in the earlier days, if you didn't have really good prompting skills, it would be difficult for you to get a whole lot of value.
• 9:39
  Amith: A great example that actually is with imaging models or image generation models.
• 9:43
  Amith: You know, using the early versions of Midjourney through Discord, using the early versions of DALI, were very, very difficult to get anything useful.
• 9:54
  Amith: And then when OpenAI integrated DALI into ChatGPT I think that was about a little bit over a year ago they actually did kind of a meta-prompting layer that what you just described where you say, hey, I want an image that describes you know two people getting married in New Orleans and then flying off to Mexico City for their honeymoon and you'd get something if you prompted the original DALI with that.
• 10:12
  Amith: But what happens now is ChatGPT takes that and goes through a little bit of what I'd call kind of meta reasoning or meta prompting, where it then sends a prompt to the DALI model that was actually written by ChatGPT and all of it to you is totally transparent.
• 10:28
  Amith: But if you actually click on the image you can see the prompting that's going on behind the scenes.
• 10:32
  Amith: What you're describing, mallory, is just more of that.
• 10:35
  Amith: It's going to be more and more and more of this higher order stuff and the models themselves are just going to become smarter.
• 10:41
  Amith: So if you kind of look at that progression that I've tried to paint the picture of, from the earliest days of competing through now, we have had essentially higher and higher taxes that we paid early on in terms and tax in the sense of metaphorical tax in terms of pain or knowledge requirements to interact with the computer.
• 10:58
  Amith: To now, almost nothing.
• 11:00
  Amith: Right.
• 11:00
  Amith: It's super fluid and it's getting better.
• 11:03
  Amith: So I would actually say that the most important thing that you have to really look at if you want to know where the puck is going to be, so to speak, is communication skills.
• 11:11
  Amith: So if you want to be effective in getting what you want from an AI, you have to be good at articulating yourself.
• 11:18
  Amith: Just like if I go to an employee and I say, hey, I really want you to produce this project for me and I do kind of a lousy job of defining the requirements and I don't really provide a lot of detail or I'm kind of ambiguous in the language I use you might not get something super great right.
• 11:33
  Amith: And with AI it's the same thing.
• 11:35
  Amith: I think AI systems are going to be better at coming back to you and refining the knowledge or the ideas with you and then executing on them.
• 11:42
  Amith: But that's my two cents.
• 11:44
  Amith: I don't think prompting as its own skill is like something people are going to be talking about five years from now.
• 11:50
  Amith: But I do think understanding how to communicate with AIs is going to be pretty important, and the AIs are going to get better at understanding our fairly bad communication skills generally as a population, right?
• 12:01
  Amith: So I think that what's going to happen is there will always be people who are at the frontier doing really cool things, that have the greatest skills in this area.
• 12:08
  Amith: Right now, what I would say is zooming back to late 2024, where we are today as we record this episode.
• 12:15
  Amith: Prompting is super important.
• 12:16
  Amith: You have to know the basics, you have to understand what you are trying to do and put it in terms that the AI can understand.
• 12:24
  Amith: Well, you'll get so much more out of it, which is going back to the earlier discussion, why we have an AI prompting course in the fall of 2024.
• 12:33
  Mallory: So, then, it's safe to say that we're all heading in the direction of prompting being baked into these models for us.
• 12:39
  Mallory: But if you want to move quickly, if you want to move right now on this, prompt engineering is still an important skill.
• 12:45
  Amith: That, if you're, better at prompting, you're going to get a significantly different output, and you know the smartest models are already quite good at interpreting what it is that you're trying to do and asking follow-up questions, things like that.
• 13:06
  Amith: But we're going to have AI everywhere.
• 13:08
  Amith: We're going to have AI on our phones.
• 13:09
  Amith: We're going to have AI on our watches.
• 13:11
  Amith: We might have it somewhere else, you know, like on our refrigerator or something.
• 13:14
  Amith: So and a lot of these other environments are not going to have frontier AI.
• 13:18
  Amith: They're going to have OK AI, and so even OK AI is getting pretty good.
• 13:23
  Amith: But I would say that just being thoughtful about what you're asking for and how you interact with the AI will make you a better communicator in general.
• 13:30
  Amith: The kind of weird things that you've had to do, like think step by step.
• 13:35
  Amith: You know that's a prompt engineering tactic that's available and actually is still quite useful in many environments.
• 13:41
  Amith: That's going to go away.
• 13:42
  Amith: You know those kinds of things that are more like how do you kind of seed the model to behave in a certain way?
• 13:47
  Amith: The model is going to figure that stuff out.
• 13:49
  Amith: You're not going to have to be that explicit.
• 13:51
  Amith: But again, if I say, hey, I want the model to write a blog post for me and I'm not very specific in what I ask for I'm going to get you know.
• 15:50
  Amith: Not the greatest response.
• 15:53
  Mallory: So prompting is kind of like searching the web.
• 15:56
  Mallory: Most people will be able to do it without kind of even realizing what strategies they're using, but then some people will be better at it than others.
• 16:03
  Amith: Totally.
• 16:06
  Mallory: And then I also want to loop back to this idea.
• 16:08
  Mallory: You mentioned, amit, of cultivating communication skills on your team maybe not necessarily prompt engineering skills over the next few years.
• 16:15
  Mallory: How do you recommend going about that, especially as it pertains to communicating with technology?
• 16:22
  Amith: Well, you know, probably the most important skill we all can work on is getting better at listening.
• 16:29
  Amith: And so then you might say well, how's that related to AI?
• 16:32
  Amith: You want to just listen to the AI do its thing.
• 16:34
  Amith: And well, I think that the reason I bring that up is listening requires you to be fully present and to think about what it is the other party, whether it's a human or AI, is trying to communicate to you, and then be more thoughtful in the way you respond.
• 16:50
  Amith: A lot of people, their version of listening, is essentially waiting for the other person to stop talking so they can speak, and to the extent that they're thinking much at all while the other person is talking, they're thinking about what they are going to say they're not really activating fully their listening skills, fully their listening skills.
• 17:10
  Amith: And so, in a way, that's super critical here, because if you really pay attention to what the AI is doing, I think you'll be a better partner to it in a sense, and you'll ultimately get a better result from it.
• 17:15
  Amith: But I think it's kind of cool, because what would make me better at using the AI systems of 2026 are probably the same things that would make me a better teammate in 2026 or even today.
• 17:28
  Mallory: That's a really interesting take on it.
• 17:30
  Mallory: We've gotten the question before in the intro to AI webinar that I'll present to you now.
• 17:35
  Mallory: Do you think that using AI models in this way especially as they get more sophisticated and we don't have to prompt them as much, let's say, if we're brainstorming for a new project that we want to launch do you think AI models will in some ways make us lazy, in the sense that we'll do less thinking on the front end because we know we can kind of pass off this duty to a technology?
• 17:57
  Mallory: What do you think about that?
• 17:59
  Amith: Well, I think it's a great question.
• 18:01
  Amith: You know, I think if you broadly look at you know kind of the bell curve of people, you're probably going to see more of that than not.
• 18:09
  Amith: I also think that people who really want to look ahead and say how can they be the best versions of themselves, you know, as a human, you're going to say, well, what can I do with my time?
• 18:18
  Amith: How can I actually use those free cycles to work on the things that I feel like I can contribute the most to, or the things that I get the most value from personally?
• 18:28
  Amith: What is that intersection, right?
• 18:30
  Amith: A value creation and happiness and passion slash purpose.
• 18:34
  Amith: You know, and I think that we have a lot of times when people have talked about that.
• 18:37
  Amith: You know, it's kind of the most generic, like, you know, what do people talk about?
• 18:41
  Amith: Like, hey, where do you want to focus your energy and what do you want to be when you grow up, or where should you go with your career or whatever.
• 18:46
  Amith: I think the most generic skill or the most generic advice is to focus on skills that are at that intersection of you know, what brings you joy and what can create value for the world and in reality, unfortunately, throughout human history, the areas where most people are able to create value, that's economically, you know, viable for them, you know, in terms of income for their, for themselves and their family doesn't usually create a lot of joy.
• 19:14
  Amith: But this is what's interesting here is that with AI, potentially, you know, there is the opportunity to say, hey, let's delegate all this stuff.
• 19:20
  Amith: So therefore, we're being lazy.
• 19:22
  Amith: Um, the flip side is, I do think, that a lot of people are going to be able to switch on a whole bunch of circuits in their brains, categories of jobs created, new types of work that people are doing, but it's going to take people wanting to actually participate in that, you know, and a lot of people are going to say this is awesome, I'm just going to go watch more Netflix.
• 19:56
  Amith: And that's a problem, because if you have that kind of a mindset, you're probably going to quickly find yourself not working, and that's a general societal issue that I have no answer for, but I think there's going to be a lot of that as well.
• 20:10
  Mallory: Yeah, At the end of the day, we're not rats on a wheel, even though maybe some of the tasks that we do can feel like that sometimes.
• 20:17
  Mallory: I think we're all innately creative and how we tap into that is different for everyone, but I agree with you.
• 20:22
  Mallory: I think there's the opportunity for people to be lazy, but there's also a very exciting opportunity in the near term for us to be much more creative, which is exciting.
• 20:31
  Mallory: Moving to topic two today, OpenAI's operator agent, which could be a good title of a movie, I think.
• 20:39
  Mallory: Openai is set to launch an AI agent, codenamed Operator, in January 2025, designed to perform tasks on behalf of users with minimal human intervention.
• 20:49
  Mallory: Some notable capabilities would include booking travel arrangements, managing emails and schedules, conducting online research, completing online transactions and automating administrative tasks.
• 21:01
  Mallory: This AI agent is designed to operate directly on user devices, function within web browsers, use a computer to take actions on behalf of users and process information in real time.
• 21:14
  Mallory: Now it's said that OpenAI plans to release Operator as a research preview through their API for developers in January 2025, so we would likely see a full release a few months after that.
• 21:27
  Mallory: Amit, we've talked about Claude computer use on the podcast.
• 21:30
  Mallory: Now we're talking about OpenAI's operator.
• 21:33
  Mallory: We haven't talked about Google's Jarvis project, but it is similar in the sense that it's an AI agent that can use a computer as well.
• 21:40
  Mallory: We maybe a bit too early predicted 2024 would be the year of the AI agents, but it's looking like 2025 will be the year of the AI agents, but it's looking like 2025 will be the year of the AI agents.
• 21:49
  Mallory: I'm curious what your initial thoughts on this announcement are.
• 21:53
  Amith: So, mallory, I think this is one of the most exciting areas for everyone, because this is a type of agent that's not going to require any customization, configuration or setup.
• 22:03
  Amith: It's just basically saying yes to open AI or cloud or somebody else having access to resources in your computer.
• 22:12
  Amith: That might be to actually control the computer so the operator can actually do stuff for you.
• 22:17
  Amith: You can prompt it, it can go into Excel or Word or Google Sheets and it can actually operate the computer for you.
• 22:23
  Amith: But also, another aspect of this is also just having the computer, having what's on the screen and what you're doing be visible to the AI model, which by itself, even if the AI model doesn't actually take the action for you.
• 22:37
  Amith: There's a lot of value in that and I'll come back to that in a second.
• 22:39
  Amith: But I think this is where people are going to get their first taste of automation.
• 22:44
  Amith: They've been getting a taste of automation in the sense that the model has created a lot of artifacts for them, whether it's blog posts or code or whatever.
• 22:51
  Amith: But when the model starts weaving its way into other aspects of their workflow, working for them in various ways, that's going to be, I think, a pretty interesting kind of general purpose capability, that will get exciting.
• 23:04
  Amith: So you know, some of the use cases you mentioned, I think, are great.
• 23:11
  Amith: One of the ones that I would mention, though, is, before you actually even think about the operator, you know doing things on your computer.
• 23:15
  Amith: If you were to say, hey, I'm just going to let it see what's on my computer and then I'm going to have a conversation with it.
• 23:21
  Amith: So let's say, for example, that, mallory, you were working on our website, which is in the HubSpot platform, and let's say, you were trying to figure out how to do something.
• 23:29
  Amith: So you might say to ChatGPT or Claude hey, I'm running into this problem.
• 23:34
  Amith: I'm using the HubSpot CMS, I am trying to position this particular element over here, but it keeps appearing over in this other spot.
• 23:41
  Amith: What can I do about it?
• 23:42
  Amith: Right, it's like kind of an inherently visual kind of thing that you're trying to explain.
• 23:45
  Amith: Kind of an inherently visual kind of thing that you're trying to explain.
• 23:46
  Amith: You might, if you're particularly advanced in current AI, you might say I'm going to take a screenshot of that.
• 23:50
  Amith: I'm going to drop the screenshot in Cloud or ChatGPT.
• 23:53
  Amith: What if you just said to those apps okay, I'm going to let you see my screen, and then you're not even necessarily prompting it.
• 24:00
  Amith: You're basically like through audio, basically saying and it could be through text as well, of course but you're just essentially saying hey, claude, listen, as you can see on my screen, I'm trying to move this element from here to here.
• 24:11
  Amith: I can't do it, what do you think?
• 24:14
  Amith: And then Claude might respond to you and say well, actually, mallory, you need to click this other option over here.
• 24:19
  Amith: Would you like me to do that for you and Claude does it for you or not?
• 24:23
  Amith: You know you can do it yourself, but even just the idea that you know the computer being given the right to see what's on your screen, or portion of your screen perhaps, is pretty interesting in terms of workflows In the world of programming.
• 24:36
  Amith: There's a massive value creation there, where you know you're working on something in a database or in a website or whatever it is, and the computer, the AI model, can see what you're doing.
• 24:47
  Amith: There's also massive productivity gains Because a lot of times people are going back and forth.
• 24:52
  Amith: They have model open in one window, like chat, gpt, and then the other window, side by side.
• 24:57
  Amith: They're actually doing their work and they're going back and forth, but they're having to pay the tax, right, like as I like to describe it, of describing to the model what's going on and what they're trying to accomplish, whereas if you imagine the model just always being there and you can talk to it in audio, which all these things support, right that could get really interesting, really fast.
• 25:15
  Amith: That's even before the model turns into an agent and takes control of your device.
• 25:20
  Amith: It just gives that model more senses, so it could be more effective in helping you.
• 25:24
  Amith: And then, of course, the layer on top of that is letting the thing actually control your computer.
• 25:32
  Mallory: I think sharing your screen with an AI model is one thing, but you mentioned providing access to files that you have as well.
• 25:40
  Mallory: I'm sure there are people listening to the pod or watching on YouTube who think I'm probably not going to sign up for this operator agent until maybe the masses try it out and let us know if it's safe and secure.
• 25:51
  Mallory: What are your thoughts on that of how to approach kind of privacy security concerns?
• 25:56
  Amith: on that of how to approach kind of privacy, security concerns.
• 25:57
  Amith: Well, this is a little bit of a side note, but I wouldn't necessarily want anyone to think that just because the masses are going to go do something, that it's safe and secure, because there's lots of people who do a lot of foolish things in the world right, both online and in the.
• 26:08
  Amith: You know the world of atoms as opposed to bits, so I think that's one thing to always be thinking about with AI.
• 26:13
  Amith: But I agree, like it's the same privacy and confidentiality kind of question you would ask of yourself before you upload a document to ChatGPT manually.
• 26:24
  Amith: But if you give it access to your full computer and you say, hey, you can do whatever you want to do with my computer, of course, then you're giving it, like you know, sysadmin level privileges to do anything, and so there are probably shades of gray within that.
• 26:38
  Amith: I haven't used this particular tool.
• 26:40
  Amith: There's probably various settings that you have, but ultimately you are giving that tool a tremendous amount of power.
• 26:45
  Amith: But keep in mind the flip side of that is you do that routinely.
• 26:49
  Amith: You first of all use an operating system that's produced by a company.
• 26:53
  Amith: It might be Google's Android on your phone, it might be Apple for their iOS, or it could be the Mac operating system or Windows or Linux or whatever, and fundamentally, just by virtue of running on an operating system, those companies have access to a lot of stuff.
• 27:09
  Amith: A lot of us also use cloud-based storage, whether it's Dropbox or FileShare or Microsoft or Google, so we are accustomed to using a lot of digital services that have very wide access to our digital lives, and so the AI having access to a wider access of our digital lives isn't necessarily, to me, the problem.
• 27:27
  Amith: The question is is the company trustworthy?
• 27:29
  Amith: And that is an open question for all of these businesses, because they're all brand new, they don't really have much of a track record.
• 27:36
  Amith: For all of these businesses, because they're all brand new, they don't really have much of a track record.
• 27:38
  Amith: Openai, anthropic, mistral they all are fairly new companies.
• 27:41
  Amith: Mistral is like a year and a half old.
• 27:42
  Amith: Openai is a handful of years more than that.
• 27:44
  Amith: So I'm not suggesting I think they're not trustworthy.
• 27:47
  Amith: I'm simply saying that one of the reasons I think people are so comfortable with giving Google or Microsoft so much of their data is because these companies have been around a really long time.
• 27:56
  Amith: They're obviously very large.
• 27:57
  Amith: They say that they've done all these things to safeguard your data and they have.
• 28:02
  Amith: But these other companies that are earlier in their journey, younger, less capable in some ways, or perhaps less focused on security, that's where you have to think about it.
• 28:12
  Amith: So it's 100% a topic that you should be thinking about deeply, you know.
• 28:17
  Amith: The thing I would point out is you know, I mentioned operating systems and file sharing tools and stuff like that.
• 28:22
  Amith: I've mentioned this on the pod a number of times and if you've heard me speak, you've probably heard me say this on stage.
• 28:28
  Amith: But a lot of people just let any AI note taker just jump on in to their meetings.
• 28:35
  Amith: It's like, hey, I'm going to have this super confidential meeting, I'm going to talk about whatever, something.
• 28:40
  Amith: I definitely don't want to be outside of the confines of a handful of folks, but I'm going to let them all bring their meat geek and their readai and whatever the hell else they want to bring with them, and I'll think about it.
• 28:51
  Amith: And I don't do that because I'm paranoid about this stuff, but I have no idea who those companies are.
• 28:55
  Amith: Right, I don't know who they are.
• 28:57
  Amith: I don't know if they're on the free plan, which means the data you provide.
• 29:00
  Amith: Typically, if you're free, is essentially you contributing to that company's future, or if they're buttoned up, things right.
• 29:09
  Amith: So I think we make these decisions.
• 29:11
  Amith: There's this dissonance between reality and our perception of reality that we just kind of block off, and so I think people need to be aware of all this stuff.
• 29:19
  Amith: I'm not saying don't use these tools.
• 29:21
  Amith: What I'm saying is is just be have a consistent approach to saying, hey, I'm going to think about how much diligence I do before I use Read AI or Meet Geek or any one of these other tools.
• 29:32
  Amith: In a similar fashion, with a tool that's going to have access to your computer, think about it a little bit, because everybody and their cousin is going to have a similar tool, like the operator tool and like Anthropix tool that has access to your computer.
• 29:44
  Amith: Everyone's going to have that.
• 29:45
  Amith: It's going to become super commoditized and you'll find stuff for free.
• 29:49
  Amith: A lot of that free stuff will probably be malware and you know there's going to be a lot of collateral damage, unfortunately.
• 30:10
  Mallory: And that's from bad actors.
• 30:11
  Mallory: But even with companies that have good intentions you're going to have problems.
• 30:14
  Mallory: So use the to Blue Cypress, our parent company, and saying you know, we really want to try out this operator thing and Blue Cypress as a whole will have to kind of go through this decision-making process of will we allow it, what kinds of actions will we allow operator to take potentially, or another tool, who will be at fault, right, if operator does something incorrectly?
• 30:34
  Mallory: So I would like to hear a little bit initially about what you think that thought process will look like in a few months when, as a business leader, you have to make that decision.
• 30:43
  Amith: Well, I'm going to look at it from my perspective.
• 30:45
  Amith: I'm going to look at it as let's try it out with a handful of people who are a little bit more savvy, who are a little bit more cybersecurity conscious.
• 30:53
  Amith: We do a lot of work around Blue Cypress to give people basics in cybersecurity, but some people are obviously both a lot more knowledgeable but also a lot more thoughtful like day-to-day about that type of stuff.
• 31:04
  Amith: A good example is the person who uses a VPN when they're on the airport Wi-Fi versus the person who's like whatever.
• 31:11
  Amith: The latter case, that's not going to be one of our early adopters.
• 31:15
  Amith: So we kind of have a good sense of that.
• 31:17
  Amith: Our company's not super big.
• 31:18
  Amith: We have a pretty good idea of who our people are and what their skills are, so we'll probably start with a fairly small number of such users and then expand it.
• 31:27
  Amith: There's risk with everything in life.
• 31:28
  Amith: The easiest thing is to say I will mitigate my risk by not using any AI at all.
• 31:33
  Amith: Using any AI at all, Of course, then you're creating the biggest risk of all, which is the fact that it's actually not even a risk.
• 31:39
  Amith: It's a fact that you're going to be out of business pretty soon.
• 31:40
  Amith: So I think you have to balance these things, and so for us, I sense that we will probably operationalize it gradually, particularly because this is a pretty big potential security hole if you don't manage it reasonably well.
• 31:55
  Amith: I also think that part of it has to be just going back to education.
• 31:59
  Amith: People don't spend time thinking deeply about these things as much as maybe we'd all like them to do in theory, but we also want them to do their jobs right, and so you know we need to provide guidelines and policies to people to say look, this is what you're allowed to put in your computer, this is what you're not allowed.
• 32:15
  Amith: With our work-issued laptops, we can control what's allowed to be installed on there, so people can't just install garbage.
• 32:22
  Amith: But at the same time, people use a lot of personal devices that an IT department doesn't have control over.
• 32:27
  Amith: So you might feel all safe and secure because IT has policies on your work laptops, but then someone might be logging in on their home computer or whatever, and they might have computer use turned on and that computer may have access to a terminal window, to something in the corporate environment, or that particular computer might just have access to files that are business files.
• 32:48
  Amith: Right, that happens all the time, and how do you control that?
• 32:51
  Amith: The short answer is you really can't.
• 32:53
  Amith: So you have to double down on education with your team.
• 32:57
  Mallory: What do you think about controlling or I shouldn't say controlling, but creating guidelines around what types of activities these AI agents could be used for, because that can be so vague as well.
• 33:07
  Mallory: You can't quite possibly list out all of the activities it can be used for, but what do you think about that?
• 33:12
  Amith: Well, I think we should be trying to do is to give people good examples and bad examples.
• 33:16
  Amith: So, the more we are able to say, look by example, we're going to show you these are good use cases, and here's why these are the reasons that these are good use cases.
• 33:26
  Amith: Everything has a little bit of risk to it, or sometimes a lot of risk, but the reward is such that it is a good example of a good use case and we provide examples of vendors and we have agreements with vendors like OpenAI or Anthropic or whoever that we say, hey, we trust these vendors, so use only them.
• 33:43
  Amith: That way, you have guardrails on the tool use to some extent and then also share bad examples.
• 33:49
  Amith: Share, like the thing that I just mentioned, which is just like random note taker tools popping into your Zoom call or whatever.
• 33:56
  Amith: Don't allow that.
• 33:57
  Amith: Don't feel bad about declining entry to your Zoom meeting to a AI notetaker.
• 34:03
  Amith: It's not like you're denying access to a person, so it's just.
• 34:08
  Amith: I think stuff like that needs to be constantly.
• 34:11
  Amith: There's no end to the education process.
• 34:13
  Amith: This world is moving so fast.
• 34:15
  Amith: You've got to invest in educating your people and policies.
• 34:18
  Amith: Policies are going to be a little bit different by organization, but there's some general purpose kind of common sense things.
• 34:23
  Amith: I think the biggest thing is just everyone needs to understand these tools at some level and if they have no concept whatsoever and they view it as just a magical machine, you're not going to have good results, you're going to have a problem.
• 34:44
  Mallory: We know Microsoft owns 49% of OpenAI, so we will surely be seeing something like an operator agent pop into Microsoft Copilot, probably sometime within 2025.
• 34:49
  Mallory: But based on what we've seen thus far, copilot tends to be a little less strong than what we see come out of OpenAI, at least directly, at least initially.
• 34:57
  Mallory: So what are your thoughts there?
• 34:59
  Mallory: Should associations kind of wait until they see this appear in Copilot?
• 35:03
  Mallory: Should they start experimenting beforehand?
• 35:05
  Mallory: What do you think?
• 35:07
  Amith: You know, copilot within the Microsoft Office world and just in general, across all the Microsoft products, I think, is a super interesting thing.
• 35:14
  Amith: It is an AI within the context of your existing work environment.
• 35:19
  Amith: So if you sign up for Copilot, it has access to your SharePoint, it has access to your OneDrive, it has access to all your stuff and so, yeah, clearly, because it's coming from Microsoft, you're going to look at it and say that they're going to be compliant to the same level of security and standard that they have for all of your other stuff.
• 35:38
  Amith: So I think that's probably a reasonable perspective to have.
• 35:41
  Amith: My experience with the Microsoft Copilot first generation has been both positive and negative.
• 35:47
  Amith: It's generally not as capable as ChatGPT or Cloud or even Gemini, but it is quite useful to have that AI sitting in your Word document or sitting in your Excel document, and this new wave of capabilities that they've announced over the last couple months are quite exciting.
• 36:03
  Amith: I think it's great marketing right now because it's not something that we actually see in practice people are using, but the opportunities are immense.
• 36:12
  Amith: They're on the right track with it and Microsoft is famous for this, but it usually takes them three major iterations of a product before it really is the product people want, and what I like about that is they go out there in the market and they put something out there and see what happens.
• 36:26
  Amith: You know people talk about like minimum viable product and lean startup mindset and all this stuff as if it's some brand new invention that people out of like really smart young people in Silicon Valley came up with it.
• 36:35
  Amith: Well, microsoft's been doing this since 1975, and a lot of other people have done similar things before them.
• 36:43
  Amith: So the key to it is just to understand where these products are in the life cycle.
• 36:45
  Amith: You're buying into a vision more than you are buying into a set of capabilities today.
• 36:50
  Amith: So you know, we're a big Microsoft shop.
• 36:52
  Amith: I think they're doing a really good job with what they're doing, but they're doing it at a scale that is really unprecedented.
• 36:57
  Amith: So part of the reason the limitations exist is that they are rolling it out for, you know, millions and millions of users in a way that's a lot deeper than a consumer product like ChatGPT that doesn't have to consider the context of, like, your whole SharePoint and things like that.
• 37:14
  Mallory: Well, that is a wrap, everyone on episode 57 of the Sidecar Sync podcast.
• 37:19
  Mallory: We will see you all next week on holiday week.
• 37:22
  Amith: Awesome.
• 37:23
  Amith: Thanks for tuning into Sidecar Sync this week.
• 37:28
  Amith: Looking to dive deeper?
• 37:29
  Amith: Download your free copy of our new book Ascend Unlocking the Power of AI for Associations at ascendbookorg.
• 37:37
  Amith: It's packed with insights to power your association's journey with AI.
• 37:41
  Amith: And remember Sidecar is here with more resources, from webinars to bootcamps, to help you stay ahead in the association world.
• 37:49
  Amith: We'll catch you in the next episode.
• 37:51
  Amith: Until then, keep learning, keep growing and keep disrupting.