Latest / Protected (by Corsair) / AI Agents Are Taking Action for Your Business. But Who's in Control?
Transcript
- 0:02[SPEAKER_00]: Welcome to Protected by Corsair USA.
- 0:06[SPEAKER_00]: This podcast is for business owners, executives, and professionals.
- 0:10[SPEAKER_00]: We want to understand cybersecurity and technology best practices.
- 0:14[SPEAKER_00]: We talk about real world security threats, facing businesses today.
- 0:19[SPEAKER_00]: What you can do to protect your people, your data, and your business.
- 0:23[SPEAKER_00]: And how technology, including AI, is changing the cybersecurity landscape.
- 0:27[SPEAKER_00]: On Rick Valdas, founder of Corsair USA, and in each episode, we'll have straightforward conversations about cybersecurity, managed IT, and the technology decisions that can help you keep your business protected.
- 0:41[SPEAKER_01]: Hi, I'm Brian Gilchrist, and today we're continuing our conversation about artificial intelligence and cybersecurity.
- 0:47[SPEAKER_01]: We've been talking about AI that can answer questions, right, documents, analyze information and help employees get things done.
- 0:55[SPEAKER_01]: But what happens when AI doesn't just give you an answer?
- 0:58[SPEAKER_01]: What happens when you give AI the ability to take action?
- 1:01[SPEAKER_01]: There's a big difference between asking chatGPT, right me an email, and giving chatGPT or another AI system, the permission to actually send the email.
- 1:11[SPEAKER_00]: That's right, Ryan.
- 1:12[SPEAKER_00]: I can understand how someone that receives a huge number of emails might use AI to help review, prioritize, and draft email responses, but actually sending the email or accessing your Google workspace or Microsoft 365 environment is another thing.
- 1:31[SPEAKER_00]: And that's what we're talking about today.
- 1:33[SPEAKER_00]: AI agents.
- 1:34[SPEAKER_00]: So, Brun, let's start with the basics.
- 1:38[SPEAKER_00]: When we say AI agent, what are we actually talking about?
- 1:41[SPEAKER_01]: The simplest way I can explain this is traditional AI generally gives your information.
- 1:46[SPEAKER_01]: An AI agent can be given a goal and then takes steps to accomplish that goal.
- 1:50[SPEAKER_01]: It can potentially interact with other software, retrieve information, make decisions,
- 1:55[SPEAKER_01]: to perform tasks.
- 1:56[SPEAKER_01]: So instead of here's the answer, it's here's what I think you want done and I've gone ahead and done it for you.
- 2:02[SPEAKER_00]: That's where cyber security policies and procedures are going to come into play because we're not just protecting information.
- 2:08[SPEAKER_00]: We're trying to protect against malicious actions.
- 2:11[SPEAKER_00]: So let's say a company has an AI agent on their website to help the sales department respond quickly.
- 2:18[SPEAKER_00]: Agent has been configured as follows.
- 2:21[SPEAKER_00]: So when we receive a qualified lead, create the customer in our CRM, send a welcome email, schedule and meeting, and notify a sales person, the AI agent isn't an employee sitting in a desk.
- 2:33[SPEAKER_00]: It doesn't call in sick.
- 2:34[SPEAKER_00]: It doesn't need to take breaks, and it works 24, 7.
- 2:37[SPEAKER_00]: So from a business perspective,
- 2:40[SPEAKER_00]: sales department time to potentially generate more revenue with less staff.
- 2:44[SPEAKER_00]: That sounds pretty cool, doesn't it?
- 2:46[SPEAKER_00]: It does, and it could save somebody a lot of time.
- 2:48[SPEAKER_01]: But let's think about what that agent now has access to.
- 2:52[SPEAKER_01]: To create the qualified lead in the CRM, and obviously,
- 2:55[SPEAKER_01]: has access to the company CRM to send the email, that has access to the email.
- 3:00[SPEAKER_01]: This schedule of meeting has access to the calendar.
- 3:02[SPEAKER_01]: It might have access to other customer information and other systems as well.
- 3:06[SPEAKER_01]: That's why I think businesses need to stop thinking of AI agents, simply as software.
- 3:11[SPEAKER_01]: If an AI agent can make decisions and take actions, we should start asking, what authority does it have?
- 3:17[SPEAKER_00]: Here's the question, I think every business owner should ask.
- 3:21[SPEAKER_00]: What happens when AI gets it wrong?
- 3:23[SPEAKER_00]: An AI agent doesn't have to be malicious to create a cybersecurity problem.
- 3:29[SPEAKER_00]: It can simply make a mistake.
- 3:31[SPEAKER_00]: Imagine that an agent is supposed to issue a $500 refund.
- 3:36[SPEAKER_00]: but because of a bad instruction, bad data or some unexpected situation, it issues a $5,000 refund.
- 3:44[SPEAKER_00]: With traditional software, we generally know what the software's program to do, but with an AI agent, there can be a layer of interpretation and decision making involved.
- 3:54[SPEAKER_00]: So, at some point, the question becomes, who approved the action?
- 3:58[SPEAKER_00]: If no one approved it, then you have to ask, why it did we give the agent that authority in the first place?
- 4:04[SPEAKER_01]: Employees already have access to these business systems.
- 4:07[SPEAKER_01]: They have email, they have Microsoft 365 or Google Workspace, the CRM, and perhaps the accounting system.
- 4:13[SPEAKER_01]: What happens when an AI agent gets access through that employee?
- 4:16[SPEAKER_00]: Well, that's a good point.
- 4:18[SPEAKER_00]: If an employee has access to 10 systems, and we give an AI agent access to those systems through that employee's credentials, we've potentially created an automated pathway into all 10 systems.
- 4:31[SPEAKER_00]: And like I mentioned before, the AI agent doesn't get tired and take a break or go home at five.
- 4:36[SPEAKER_00]: It doesn't take a vacation and can potentially execute tasks much faster than a human, which is great when it's doing the right thing.
- 4:47[SPEAKER_00]: Let's turn this around.
- 4:48[SPEAKER_00]: We've been talking about an AI agent making an innocent mistake.
- 4:52[SPEAKER_00]: What happens if an attacker gets controlled of the agent?
- 4:56[SPEAKER_00]: That's where the stakes become much higher.
- 4:59[SPEAKER_00]: An attacker doesn't necessarily need to compromise every system individually.
- 5:04[SPEAKER_00]: If they can manipulate the agent or the information the agent relies on, they may be able to influence what the agent does.
- 5:12[SPEAKER_00]: So instead of attacking the software directly, they could potentially attack the AI's instructions or the information it's processing.
- 5:21[SPEAKER_01]: And now we're entering an area where concepts like indirect prompt injection become important.
- 5:26[SPEAKER_00]: No, you're gonna have to explain that in English.
- 5:28[SPEAKER_00]: Sure, let's say an AI agent is instructed to review incoming documents to email.
- 5:33[SPEAKER_01]: One of those documents contains hidden instructions.
- 5:36[SPEAKER_01]: designed to manipulate the AI.
- 5:38[SPEAKER_01]: The AI reads the document.
- 5:39[SPEAKER_01]: It follows the malicious instruction.
- 5:41[SPEAKER_01]: And suddenly, the AI is doing something it was never supposed to do.
- 5:45[SPEAKER_00]: So the attack doesn't necessarily look like break into the computer.
- 5:49[SPEAKER_00]: It's more like convince the computer to do something it shouldn't.
- 5:53[SPEAKER_00]: I mean, there's another issue here that I think business owners are going to need to understand immediately, which is trust.
- 6:01[SPEAKER_00]: If an employee receives email from someone, they don't recognize.
- 6:06[SPEAKER_00]: But what if an AI agent receives information from another AI system?
- 6:10[SPEAKER_00]: Does it automatically trust it?
- 6:12[SPEAKER_01]: Yes, that's definitely a problem.
- 6:14[SPEAKER_01]: We have to be careful about creating chains of trust.
- 6:17[SPEAKER_01]: Agent A talks to Agent B.
- 6:19[SPEAKER_01]: Agent B accesses a database.
- 6:21[SPEAKER_01]: Agent B sends information to Agent C. Agent C takes an action.
- 6:25[SPEAKER_01]: Now you've created an automated chain and no one may actually be watching the entire chain.
- 6:31[SPEAKER_01]: That's why AI security isn't just about protecting the AI.
- 6:34[SPEAKER_01]: itself.
- 6:35[SPEAKER_01]: It's about understanding the entire environment around me.
- 6:39[SPEAKER_00]: Okay, so let's bring this back to something business is already understand and that's called least privilege.
- 6:45[SPEAKER_00]: An AI agent should have only the permissions it needs to perform its job and nothing more.
- 6:52[SPEAKER_00]: If an AI agent schedules appointments.
- 6:56[SPEAKER_00]: Does it really need access to payroll?
- 6:58[SPEAKER_00]: If it answers customer questions, does it need the ability to delete customer records?
- 7:03[SPEAKER_00]: And if an AI agent can create a payment should it be able to approve that payment?
- 7:08[SPEAKER_01]: That's where separation of duties comes into play.
- 7:11[SPEAKER_01]: The same internal controls that are applied without technology, you don't want the same automated system creating and approving financial trends.
- 7:18[SPEAKER_00]: which brings us to something I think is going to become very important human approval, especially for a high-risk actions.
- 7:26[SPEAKER_00]: You might allow an AI agent to draft an email, but require a person to approve it before it's sent.
- 7:33[SPEAKER_00]: You might allow an agent to prepare a payment, but require a person, a human to approve the payment.
- 7:40[SPEAKER_00]: You might allow an agent to identify
- 7:48[SPEAKER_00]: determine what happens next.
- 7:49[SPEAKER_00]: We're not saying don't use AI agents, what we're saying is don't give them unlimited authority.
- 7:55[SPEAKER_01]: True.
- 7:56[SPEAKER_01]: AI can be extremely useful, but useful doesn't mean it should have unrestricted access.
- 8:01[SPEAKER_00]: Okay, so let's say I'm a business owner listening to this.
- 8:04[SPEAKER_00]: I don't have a huge IT department and I don't have a cyber security team.
- 8:09[SPEAKER_00]: What should I be doing?
- 8:10[SPEAKER_01]: Well, Rick, I would start with five questions.
- 8:12[SPEAKER_01]: Question one, where are we already using AI?
- 8:14[SPEAKER_01]: Don't assume, you know, talk to your employees.
- 8:16[SPEAKER_01]: You may discover there are already using AI tools.
- 8:19[SPEAKER_01]: You didn't know about question two.
- 8:21[SPEAKER_01]: What information can those AI systems have?
- 8:23[SPEAKER_01]: access, customer information, financial information, low-E information, company intellectual property, question three, can the AI take action?
- 8:31[SPEAKER_01]: There's a big difference between reading information and changing information.
- 8:35[SPEAKER_01]: Question four, what permissions does it have?
- 8:38[SPEAKER_01]: And finally question five, what happens if AI makes a mistake?
- 8:42[SPEAKER_01]: That's the question company's often
- 8:44[SPEAKER_00]: I like the idea of an AI inventory.
- 8:47[SPEAKER_00]: We've always been encouraging businesses to keep track of devices and applications.
- 8:52[SPEAKER_00]: So now they need to know what AI they're using and track that just as they would, any other software application.
- 9:01[SPEAKER_00]: And that's particularly important because AI can enter and organization very quickly and quietly.
- 9:08[SPEAKER_00]: an employee could sign up for an AI service in five minutes.
- 9:12[SPEAKER_01]: And suddenly you have a new application handling company information.
- 9:15[SPEAKER_01]: As we stated in our previous podcast, that's why policy matters.
- 9:18[SPEAKER_00]: A lot of small businesses here, the word policy and immediately think, okay, here comes a 40-page document.
- 9:25[SPEAKER_01]: It doesn't have to be.
- 9:26[SPEAKER_01]: A practical AI policy might simply answer what AI tools are employees allowed to use.
- 9:32[SPEAKER_01]: What company information can they put into those tools?
- 9:34[SPEAKER_01]: What information is prohibited?
- 9:36[SPEAKER_01]: When is human approval required?
- 9:38[SPEAKER_01]: Who is responsible for approving new AI?
- 9:41[SPEAKER_01]: applications and what happens with something goes wrong.
- 9:44[SPEAKER_01]: That's something a small business and actually implement.
- 9:47[SPEAKER_01]: It's not hard.
- 9:48[SPEAKER_01]: Exactly.
- 9:49[SPEAKER_01]: Cyber security doesn't have to mean making everything complicated.
- 9:53[SPEAKER_01]: It's simply identifying risks and putting reasonable controls around them.
- 9:56[SPEAKER_01]: Brian, do you think AI agents are going to become more mainstream?
- 10:00[SPEAKER_01]: I think they are.
- 10:02[SPEAKER_01]: And becoming part of the technology landscape.
- 10:04[SPEAKER_01]: The question isn't really whether businesses will use
- 10:11[SPEAKER_00]: I'm frequently trouble shooting software issues.
- 10:14[SPEAKER_00]: So many technology companies don't want to answer the phone.
- 10:17[SPEAKER_00]: They bury their phone number on their website, make it hard to find, and they force customers to use an email form or to create a ticket or use a chatbot.
- 10:29[SPEAKER_00]: And if you're fortunate enough to get somebody on the phone, then you have to go through VoiceJail where
- 10:34[SPEAKER_00]: The AI agent is prompting you to provide an account number and explain why you're calling.
- 10:39[SPEAKER_00]: I can't think of one time where it has not been a hassle to get someone on the phone, unless you have their direct phone number.
- 10:47[SPEAKER_00]: So while it's evident that AI agents are in place, it's still not as good as having a human answer the phone and connect you to the right person or department.
- 10:57[SPEAKER_00]: I'm not saying don't use AI.
- 10:59[SPEAKER_00]: Quite the opposite.
- 11:00[SPEAKER_00]: A.I.
- 11:00[SPEAKER_00]: could be a great productivity tool, but the more capable A.I.
- 11:04[SPEAKER_00]: becomes, the more important cyber security becomes.
- 11:08[SPEAKER_01]: I agree, the more we ask A.I.
- 11:09[SPEAKER_01]: to do for us, the more we need to control what it's allowed to do, but I also agree that there are some situations where human interaction provides a much better customer experience.
- 11:20[SPEAKER_00]: So let's leave our listeners with a simple way to think about this.
- 11:24[SPEAKER_00]: AI that answers questions is one thing.
- 11:27[SPEAKER_00]: AI that can take action is something else.
- 11:31[SPEAKER_01]: And the question isn't simply, is the AI accurate?
- 11:34[SPEAKER_01]: The question is, what happens at the AI as well?
- 11:37[SPEAKER_01]: And what damages can it do when it's wrong?
- 11:39[SPEAKER_01]: That's the cybersecurity question.
- 11:41[SPEAKER_01]: AI agents are going to change how businesses operate.
- 11:44[SPEAKER_01]: They can automate repetitive work, improve productivity, and help businesses do things that previously be acquired a lot more time in people.
- 11:52[SPEAKER_01]: But with that capability comes responsibility.
- 11:55[SPEAKER_01]: No way your AI can access.
- 11:57[SPEAKER_01]: No way it can do.
- 11:58[SPEAKER_01]: The minutes permissions and require human approval for important decisions.
- 12:02[SPEAKER_01]: And don't wait until something goes wrong to figure out what your AI is connected to.
- 12:06[SPEAKER_00]: I could curb Ron, that's excellent advice.
- 12:09[SPEAKER_00]: I'm Rick Valdez, and I'm Brian Gilchrist.
- 12:11[SPEAKER_01]: Thanks for listening to Protected Bike, or Siri USA.
- 12:14[SPEAKER_01]: If you're wondering how secure your business really is, visit corecentercms.com and register for a free cybersecurity assessment.
- 12:21[SPEAKER_01]: It's a simple way to identify potential risks and get a better understanding of where your business stands.
- 12:26[SPEAKER_01]: Until next time, stay informed, stay secure, and stay protected.
- 12:30[SPEAKER_01]: Thanks for listening to Protected Bike, or Siri USA.
- 12:33[SPEAKER_01]: Thanks.