Latest / How I Work / The prompt that shows what your AI actually knows about you (and it’s less than you might think)
Transcript
- 0:00Speaker 1: You open a new chat and AI greets you like
- 0:04Speaker 1: an old friend. It knows your job, your dog's name,
- 0:08Speaker 1: last week's projects, and then it hands back something painfully average,
- 0:14Speaker 1: or it mentions a role that you left two years ago.
- 0:18Speaker 1: What AI knows and what it assumes about you are
- 0:22Speaker 1: two very different things. In this episode, Neo and I
- 0:28Speaker 1: look at what is actually sitting in your AI's memory.
- 0:33Speaker 1: You're going to get one simple prompt that makes it
- 0:36Speaker 1: show you what it knows. And we also cover how
- 0:40Speaker 1: to fix memories that are stale or just plain wrong.
- 0:45Speaker 1: And we also get to the biggest blind spot. AI
- 0:49Speaker 1: might know your job, yet have no idea how you
- 0:53Speaker 1: actually do it. By the end of this episode, you'll
- 0:56Speaker 1: know exactly what your AI really has on you and
- 1:01Speaker 1: how to set the record straight. Welcome to How I
- 1:10Speaker 1: AI with me, Dr. Amantha Imber and Neo Applin, head
- 1:14Speaker 1: of Inventium AI. Each episode we share one practical way
- 1:18Speaker 1: to use AI better at work and in life. No fluff,
- 1:22Speaker 1: no tech jargon, just things you can use straight away.
- 1:26Speaker 1: Now the memory function in all the major AI platforms
- 1:31Speaker 1: has come a long way and it's changed a lot
- 1:34Speaker 1: in the last four years. So Neo, you're working with
- 1:38Speaker 1: people every day at Inventium AI, training them along with
- 1:43Speaker 1: our team here on how to get more value from AI.
- 1:47Speaker 1: What do you find that people really misunderstand about how
- 1:52Speaker 1: memory works in the AI platform that they're using?
- 1:57Speaker 2: They think that AI just knows everything about them and
- 2:02Speaker 2: it just talks to me like it knows me and
- 2:05Speaker 2: it gets how I like to work. And AI certainly
- 2:07Speaker 2: is getting to that point and it does know a
- 2:10Speaker 2: fair bit about you and it does reflect you back
- 2:11Speaker 2: to you. but it doesn't know everything about you as
- 2:15Speaker 2: much as you feel sometimes it does. And so what
- 2:19Speaker 2: some people misunderstand is that it does know maybe your name,
- 2:23Speaker 2: your job, something about your pets and your hobbies and
- 2:26Speaker 2: things like that, maybe some projects you've worked on. But
- 2:29Speaker 2: the amount of the knowledge there as far as what
- 2:32Speaker 2: it knows versus what it assumes, there's a massive gap.
- 2:37Speaker 2: And you can find out how much it actually knows
- 2:39Speaker 2: about you, but assumes kind of interesting.
- 2:41Speaker 1: Yeah, so I want to know, how can listeners actually
- 2:46Speaker 1: find out today what the AI knows about them?
- 2:53Speaker 2: Thank you. All the tools have got different ways to
- 2:55Speaker 2: do this. So if you're in Claude, there's actually a
- 2:58Speaker 2: setting you can go to. It's in your settings. There's
- 3:00Speaker 2: a memory setting there. And you can actually see what
- 3:03Speaker 2: it knows about you. And it's got different categories about you,
- 3:06Speaker 2: topics you're working on, areas that you work with, people
- 3:09Speaker 2: you work with, and also some projects if you've got
- 3:12Speaker 2: projects in there as well. And ChatGPT has got a
- 3:14Speaker 2: similar kind of function you can go to. It's under settings, personalization.
- 3:18Speaker 2: You can actually see the memories there. But there's an
- 3:21Speaker 2: other way you can do it, which is a bit
- 3:22Speaker 2: more fun. Open up a new chat thread and then
- 3:25Speaker 2: just in your regular AI and then say, tell me
- 3:28Speaker 2: what you know about me. Give me R-A-G emojis on
- 3:33Speaker 2: your confidence. R-A-G is red, amber, green emojis. So in
- 3:39Speaker 2: project management, they call it R-A-G or RAG for red, amber,
- 3:42Speaker 2: green emojis. And I like the confidence thing because it's
- 3:45Speaker 2: going to give you a bit of a clue as
- 3:46Speaker 2: to whether it thinks something or it knows something about you.
- 3:50Speaker 2: So you don't even have to go to the settings
- 3:52Speaker 2: for this. Just go new chat thread. Tell me what
- 3:54Speaker 2: you know about me. Give me R-A-G emojis on your confidence.
- 3:57Speaker 2: If you want to next level it, make sure you
- 4:00Speaker 2: say something along the lines of don't search any of
- 4:02Speaker 2: my emails or whatever, because otherwise it'll cheat. It'll go
- 4:05Speaker 2: and do a little stalking behind the scene and tell
- 4:08Speaker 2: you what it is about you. And so what you
- 4:10Speaker 2: might find is it's pretty impressive, but when you dive
- 4:13Speaker 2: a bit deeper, it's not as impressive as you'd want
- 4:16Speaker 2: to see really to be a really helpful, useful assistant.
- 4:20Speaker 1: Okay. So let's just say it shows us what it
- 4:23Speaker 1: knows about us and what it's confident about. Or maybe
- 4:26Speaker 1: we go into the actual memory in settings in Claude
- 4:31Speaker 1: or chat GPT and we're thinking, hang on, it's saying
- 4:36Speaker 1: this thing about me, but that is out of date
- 4:39Speaker 1: by two years, as I have personally had the experience
- 4:42Speaker 1: of finding out. So how do we change the memory?
- 4:46Speaker 2: One of the easiest ways is just to tell it
- 4:49Speaker 2: what you want to change. So in a regular chat thread, say,
- 4:53Speaker 2: I want you to forget this, or this is incorrect
- 4:56Speaker 2: in the memory. then please change it to that. And
- 5:00Speaker 2: so you can actually just tell it. A bit of
- 5:02Speaker 2: a tip here is usually when it updates the memory,
- 5:05Speaker 2: it'll say a little memory updated or something like that.
- 5:08Speaker 2: It's important so you can see that. If you're on
- 5:10Speaker 2: Claude or on ChatGPT, you can go into the memory
- 5:13Speaker 2: functions and then tell it from those screens. And so
- 5:17Speaker 2: you will find there's things in your memory which might
- 5:19Speaker 2: be old as you had, in which case maybe you
- 5:23Speaker 2: don't want it to think that you work in that
- 5:25Speaker 2: role anymore because you don't, that you live in that
- 5:28Speaker 2: location because you no longer live in that location. So
- 5:31Speaker 2: the memories can be actually incorrect and they can be old.
- 5:35Speaker 2: So the old we've just discussed, incorrect can be maybe
- 5:38Speaker 2: you helped a mate on a particular piece of work
- 5:41Speaker 2: and then suddenly it thinks you work for that you know,
- 5:44Speaker 2: that company or in that area. So I've had those
- 5:47Speaker 2: kinds of things. I build scenarios all the time. At
- 5:50Speaker 2: one stage, it thought I worked in mining for about
- 5:52Speaker 2: two weeks because I asked for a mining example. So
- 5:54Speaker 2: I had to go in the memory and then color it. So, yep,
- 5:57Speaker 2: get it to show you the memory. And if it
- 5:59Speaker 2: is incorrect, make sure you update it because otherwise it's
- 6:02Speaker 2: going to give you a very biased result.
- 6:05Speaker 1: And can we still use the prompts, remember this and
- 6:09Speaker 1: forget this as a very quick way within a chat thread?
- 6:13Speaker 1: to change the memory?
- 6:15Speaker 2: Yeah, that's the way to do that. If you really,
- 6:18Speaker 2: really want it to forget something that's very, very important
- 6:21Speaker 2: for it to forget, what you may want to do is,
- 6:25Speaker 2: number one, tell it to forget that thing. And number
- 6:28Speaker 2: two is delete the chat threads that have reference to
- 6:31Speaker 2: that thing. Because of course, it can still see those
- 6:33Speaker 2: old chat threads. Because other than that, if you tell
- 6:35Speaker 2: it to forget something in the memory, but it's still
- 6:37Speaker 2: hanging around, it's not forgotten forgotten. So yeah, you might
- 6:40Speaker 2: need to delete from both.
- 6:42Speaker 1: So how does the memory actually impact our conversations with
- 6:48Speaker 1: AI just on a daily basis?
- 6:52Speaker 2: It really wants to be helpful. And on each new
- 6:55Speaker 2: chat thread, what actually happens behind the scenes is AI
- 7:00Speaker 2: has a brand new session, a brand new brain, if
- 7:04Speaker 2: you'd like, just for you. And at the start of
- 7:06Speaker 2: the chat thread, then the memories of you get injected
- 7:10Speaker 2: in at the top, also your custom instructions. And so
- 7:13Speaker 2: then AI knows a little bit about you, how you've worked.
- 7:15Speaker 2: It's probably that two or three pages worth of information.
- 7:18Speaker 2: And then AI will go, okay, now based on this human,
- 7:22Speaker 2: how can I best help them? And then you can
- 7:24Speaker 2: start the chat thread. So you can't see that two
- 7:26Speaker 2: or three pages worth of memory that gets injected at
- 7:28Speaker 2: the start of each chat thread. But what it does
- 7:30Speaker 2: do is it takes that and then tries to help
- 7:33Speaker 2: you from there. So in your memory, it'll give you
- 7:37Speaker 2: a couple of things you've worked on in the last
- 7:38Speaker 2: couple of days. So previous chat threads often, even in
- 7:43Speaker 2: other chats, a little summary of the other chat threads
- 7:45Speaker 2: often get injected at the top of that chat, your
- 7:48Speaker 2: new chat thread. So it feels like it knows a
- 7:52Speaker 2: fair bit about your work or your previous chat threads,
- 7:55Speaker 2: and it will try to imply these things. Oh, it
- 7:57Speaker 2: sounds like you've been working on this, so let's continue
- 7:59Speaker 2: to work on those things. And so that's great, and
- 8:03Speaker 2: that'll help personalize things. But the risk is that whilst
- 8:08Speaker 2: it's only got a little bit of a note that
- 8:11Speaker 2: you worked on that, it doesn't know every word on
- 8:13Speaker 2: the thing you worked on. It hasn't seen the documents
- 8:16Speaker 2: that you worked on last week. It just knows that
- 8:18Speaker 2: you worked on those documents last week. So we kind
- 8:21Speaker 2: of assume that because it seems to talk about a
- 8:23Speaker 2: thing I worked about last week, it knows all the
- 8:24Speaker 2: things about it. It doesn't. And that is one of
- 8:27Speaker 2: the problems there. The other problem is that the longer
- 8:32Speaker 2: you have that chat with AI... then sometimes it'll forget
- 8:36Speaker 2: more things on the memory at the start. And so
- 8:39Speaker 2: you might find that the longer the chat, I'm talking
- 8:42Speaker 2: about pages and pages and pages and you've got a
- 8:45Speaker 2: chat thread you've kept forever, it's been one of your pets,
- 8:48Speaker 2: it'll start to forget little things about you because it
- 8:50Speaker 2: was only ejected at the start of the chat thread
- 8:52Speaker 2: and it'll forget it over time.
- 8:55Speaker 1: So what's the biggest blind spot you think people have
- 8:59Speaker 1: with AI and its memory?
- 9:03Speaker 2: Both are tied to the one thing, which is AI
- 9:06Speaker 2: does know a lot about you and maybe how you
- 9:10Speaker 2: work with AI, the kind of words that you might
- 9:12Speaker 2: use with AI. But what it doesn't know is how
- 9:16Speaker 2: you do those things. So it doesn't have a playbook
- 9:19Speaker 2: or a recipe of how Amantha does, I don't know,
- 9:24Speaker 2: designs a new keynote. It knows that you've worked with
- 9:27Speaker 2: it for maybe research for a keynote, but it doesn't
- 9:30Speaker 2: know how to generate that keynote. And we kind of
- 9:32Speaker 2: feel like, because it seems to know a lot about
- 9:34Speaker 2: me and my role, that it's really job ready, but
- 9:37Speaker 2: it's not quite job ready. You still got to boss
- 9:39Speaker 2: it around and tell it what you need. But we
- 9:41Speaker 2: kind of feel like it knows all these things, so
- 9:43Speaker 2: it should be able to do those things well enough
- 9:44Speaker 2: for me, but it doesn't. And that's why it still
- 9:46Speaker 2: gives you an average output every so often. Because it
- 9:49Speaker 2: knows your job and it's just assuming that, well, because
- 9:52Speaker 2: of that job, that's how you'd want that thing to
- 9:54Speaker 2: be output. So we still need to be really clear
- 9:56Speaker 2: on what good looks like and how we want it
- 9:58Speaker 2: to do the work for us.
- 10:00Speaker 1: Us. And certainly I know in a lot of our
- 10:03Speaker 1: more advanced training programs that we do at Inventium AI,
- 10:06Speaker 1: that is such a big focus, helping people to create
- 10:10Speaker 1: skills or agents that actually understand how they do the
- 10:14Speaker 1: thing because it's so often the missing ingredient. Neo, thank
- 10:18Speaker 1: you so much for clearing up some of the misnomers
- 10:23Speaker 1: about how memory works in AI. Thank you so much
- 10:28Speaker 1: for listening to today's episode of How I AI. If
- 10:32Speaker 1: this show got you thinking about how you could use
- 10:35Speaker 1: AI better, that is exactly what we help people do
- 10:38Speaker 1: at Inventium AI. We run AI training programs for organizations
- 10:43Speaker 1: and also for individuals. So if you want to upskill
- 10:47Speaker 1: your entire organization or maybe just yourself, you can find
- 10:51Speaker 1: out more at inventium.ai. Hey, and one small favor before
- 10:55Speaker 1: you go. If you've got a moment, like just five seconds,
- 10:59Speaker 1: I'd love it if you could leave us a review
- 11:01Speaker 1: and hit follow wherever you're listening. It really helps the
- 11:04Speaker 1: show reach more people and it helps us create even
- 11:07Speaker 1: better episodes for you. Thank you so much for listening
- 11:10Speaker 1: and we'll see you next time on How I AI.
- 11:13Speaker 1: How I AI was hosted by me, Samantha Imber and
- 11:16Speaker 1: Neo Applin. A big thank you to Martin Imber who
- 11:19Speaker 1: does our sound editing and Jem Rubio for production support.
- 11:23Speaker 1: And thank you to John Kilby who composed the theme music.