Latest / How I Work / 3 signs it's time to start a new chat thread
Transcript
- 0:00Speaker 1: Imagine you're deep into a long chat with AI. And
- 0:05Speaker 1: it starts getting things wrong. Forgetting something you told it
- 0:11Speaker 1: a mere five minutes ago.
- 0:13Speaker 2: Making things up.
- 0:15Speaker 1: And you can't tell if it's you, your prompt, or
- 0:18Speaker 1: maybe just.
- 0:19Speaker 2: The AI is having a bad day.
- 0:22Speaker 1: But what's usually happening is that the chat thread has
- 0:25Speaker 1: run out of short-term memory. And once that memory gets
- 0:29Speaker 1: to about 60% full, the AI starts becoming less reliable.
- 0:36Speaker 1: In this episode, Neo and I talk through how to
- 0:39Speaker 1: know when it is time to start a fresh chat thread.
- 0:44Speaker 2: You will learn the warning science that a thread has
- 0:47Speaker 2: passed its use by date.
- 0:49Speaker 1: Why PDFs chew through your context window faster than almost
- 0:53Speaker 1: anything else.
- 0:53Speaker 2: And what to upload instead.
- 0:56Speaker 1: By the end of this episode, you'll know exactly when
- 0:59Speaker 1: to close a thread and start again.
- 1:02Speaker 2: So that the AI stays sharp. Welcome to How I
- 1:11Speaker 2: AI with me, Dr.
- 1:12Speaker 1: Amantha Imber and Neo Applin, head of Inventium AI. Each episode,
- 1:17Speaker 1: we share one practical way to use AI better at
- 1:20Speaker 1: work and in life. No fluff, no tech jargon, just
- 1:24Speaker 1: things you can use straight away. Neo, today we are
- 1:28Speaker 1: going to be talking about something that might seem on.
- 1:31Speaker 2: The surface quite basic, but we see.
- 1:36Speaker 1: Hundreds and hundreds, if not thousands of people get this
- 1:39Speaker 1: wrong when we are training people in workshop rooms. And
- 1:43Speaker 1: that is, when should you open or start a new
- 1:47Speaker 1: chat thread? So, Neo... What are your top tips on
- 1:52Speaker 1: how do we know when it's time to start a
- 1:54Speaker 1: new chat thread?
- 1:56Speaker 3: The biggest tip is if the AI seems to be
- 1:59Speaker 3: doing strange and wacky things, then maybe it's time to
- 2:02Speaker 3: start a new chat thread. Because going back a step,
- 2:05Speaker 3: the reason is AI doesn't have an infinite memory for
- 2:09Speaker 3: each of your chat threads. They've got a technical thing
- 2:11Speaker 3: called a context window, which is it can hold a
- 2:14Speaker 3: certain number of tokens or words in an entire chat thread.
- 2:18Speaker 3: And that's everything you say, everything it says, all the
- 2:20Speaker 3: documents that both of you are creating, and also every
- 2:23Speaker 3: thought that it might have behind the scenes that it
- 2:25Speaker 3: doesn't show you kind of fills up this short-term memory.
- 2:28Speaker 3: And so once the memory gets full, or at least
- 2:31Speaker 3: starting to get full, science is saying about 60% full,
- 2:35Speaker 3: then you'll find that hallucinations happen more often and the
- 2:38Speaker 3: AI is more likely to get things wrong. So if
- 2:40Speaker 3: you're finding it's making up stuff, it's getting things wrong,
- 2:44Speaker 3: ask yourself, have I been talking with this chat in
- 2:46Speaker 3: this one chat thread For an awful long time, maybe
- 2:49Speaker 3: it's time for me to move off to another chat thread.
- 2:52Speaker 1: So when you say 60% full, how do we know
- 2:56Speaker 1: what the total amount is? What does that look like?
- 2:59Speaker 1: And is it different across the four major AI platforms?
- 3:03Speaker 3: And wouldn't it be amazing if they had a bar
- 3:05Speaker 3: chart going, awooga, awooga, you've just hit 60%. I'm about
- 3:08Speaker 3: to start to lie to you. No, it doesn't do that.
- 3:11Speaker 3: Isn't it quite annoying? each of the different four platforms
- 3:15Speaker 3: does have a different context window. The short-term memory thing
- 3:18Speaker 3: is different in each, and it's changing over time. I'm
- 3:21Speaker 3: hoping by the time you hear this podcast, you'll go, oh, wow,
- 3:24Speaker 3: it's even better than it was yesterday. They don't generally
- 3:27Speaker 3: come out and tell you how big these ones are,
- 3:29Speaker 3: but you can even ask your AI, how big is
- 3:32Speaker 3: your context window? And then even say, how many words
- 3:35Speaker 3: would that kind of equivalent be. For some of them,
- 3:38Speaker 3: I've got more of like a 200-odd A4 pages worth
- 3:42Speaker 3: of text. And it's kind of hard because is it
- 3:44Speaker 3: A4 pages with double-sided? Is it like wall-to-wall letters and numbers?
- 3:48Speaker 3: It's kind of hard to kind of get that in
- 3:50Speaker 3: our head. So I don't mind the term of 200
- 3:52Speaker 3: pages for some people. And others, if you're on Gemini,
- 3:57Speaker 3: they're just rocking it. You're like 1,000 pages worth of
- 4:00Speaker 3: text or something like that. And so each of them
- 4:01Speaker 3: are different. And these things change over time. But there
- 4:05Speaker 3: are really Two big principles here, which is if that
- 4:09Speaker 3: limit is getting full, and they've done a lot of
- 4:11Speaker 3: science on this, which is once you start hitting about 60%
- 4:14Speaker 3: of this context window getting full, then it's more likely
- 4:17Speaker 3: to make stuff up, hallucinate, and get things wrong. Either
- 4:21Speaker 3: forget something you've asked. or misinterpret something in the entire
- 4:25Speaker 3: conversation before. And so it's a good idea then to
- 4:28Speaker 3: start a brand new chat thread. Obviously, if you're 100%
- 4:31Speaker 3: full on that context window, every new word you put
- 4:34Speaker 3: at the bottom means the word at the top is
- 4:35Speaker 3: going to drop off what it can say. And so
- 4:38Speaker 3: don't go forever, which is why in all our trainings
- 4:41Speaker 3: and here in our podcast, we always say start a
- 4:43Speaker 3: new chat thread for pretty much every time you come
- 4:45Speaker 3: to AI and hopefully you won't even hit this problem
- 4:47Speaker 3: in the first place.
- 4:49Speaker 2: Now, another thing on context window.
- 4:52Speaker 1: So you say that maybe at the time of recording,
- 4:54Speaker 1: which is September 2026, it's maybe about 200 pages of text,
- 4:59Speaker 1: but also each page and each file type is not
- 5:03Speaker 1: created differently. So for example, something that I hit yesterday,
- 5:08Speaker 1: I was trying to create a mega project around health.
- 5:12Speaker 1: I was uploading all these different health documents that were
- 5:17Speaker 1: all PDF documents. And you heard me whinging to you
- 5:20Speaker 1: that I had hit my context limit.
- 5:23Speaker 2: Which I rarely do because I'm pretty careful.
- 5:25Speaker 1: About how I use AI and, you know, just managing
- 5:31Speaker 1: what is going in so that I don't hit those limits.
- 5:34Speaker 2: But I did because it was all PDF documents.
- 5:37Speaker 1: So are there little hacks that we can use so
- 5:41Speaker 1: that we're not filling up that context window as quickly,
- 5:44Speaker 1: like with the types of files that we're uploading?
- 5:47Speaker 3: Absolutely. The first hack is every time we come to
- 5:50Speaker 3: AI for a new purpose, go new chat. There's no
- 5:53Speaker 3: point putting in, I don't know, health information or querying
- 5:56Speaker 3: and learning about health with any kind of company learning,
- 5:59Speaker 3: querying about company type stuff. They're definitely two different chats there.
- 6:03Speaker 3: So that kind of helps at the start. The other
- 6:05Speaker 3: is the files you upload do make a difference. And
- 6:09Speaker 3: so you'll find that markdown or text files are awesome.
- 6:15Speaker 3: And this is why people love markdown files, which, by
- 6:17Speaker 3: the way, they're just text files with a couple of
- 6:20Speaker 3: little clues as to first styling and whether it's a
- 6:22Speaker 3: bold and things like that. So if you see a.
- 6:25Speaker 3: md file, don't be scared. It's just a text file.
- 6:27Speaker 3: And that's great because it's pretty much it's all awesome
- 6:30Speaker 3: stuff and not a lot of filler. If you're looking
- 6:32Speaker 3: at things like PowerPoints and Word documents, then often the
- 6:36Speaker 3: AI has a tool just to extract the text and
- 6:39Speaker 3: the structure from those things. And so they can actually
- 6:42Speaker 3: be okay as well. PDFs, as you found out, are
- 6:45Speaker 3: the absolute killer. And the reason for that is PDFs
- 6:47Speaker 3: have been built for a printing format or viewing format,
- 6:51Speaker 3: not an understanding format. And so AI kind of knows
- 6:54Speaker 3: this because like if you ever selected a bunch of
- 6:57Speaker 3: text in a PDF, then often you'll get like tables
- 6:59Speaker 3: coming out weird and things like that. So AI can't
- 7:01Speaker 3: just go select the text in the PDF and copy it. Instead,
- 7:04Speaker 3: what it does is it does that, but it also
- 7:06Speaker 3: takes screenshots of every page of your PDF sometimes. And
- 7:10Speaker 3: then it needs to read that screenshot. This takes a
- 7:12Speaker 3: lot of tokens. So PDFs are the worst, absolutely the worst.
- 7:16Speaker 3: If you can do some kind of like a, extraction
- 7:18Speaker 3: from your PDF to a text file or a Word
- 7:21Speaker 3: doc or whatever first, oh my God, that will save
- 7:23Speaker 3: you a lot of tokens. But ultimately, only upload what
- 7:27Speaker 3: you need to for that particular chat thread. Don't go
- 7:30Speaker 3: nuts and give it all the documents in the world
- 7:32Speaker 3: because of course you'll fill that earlier. And the other
- 7:35Speaker 3: is particularly with PDFs. If you can somehow get the
- 7:38Speaker 3: text out of it in a different way, you'll also
- 7:40Speaker 3: find you won't hit that limit as much.
- 7:42Speaker 2: And I'm sorry about using the PDF.
- 7:43Speaker 3: That really sucks for you.
- 7:44Speaker 2: Yeah, it certainly did yesterday.
- 7:47Speaker 1: Now, before we go, because I know we are going
- 7:49Speaker 1: to come back in a future episode and talk about
- 7:51Speaker 1: some more advanced tips for how to close a chat
- 7:55Speaker 1: thread so that you're capturing all the goodness and really
- 7:59Speaker 1: easily exporting it into a new chat thread, particularly if
- 8:02Speaker 1: you have hit that 60% window or if you're just
- 8:04Speaker 1: finding that the AI is starting to get a little
- 8:07Speaker 1: bit dumber.
- 8:09Speaker 2: But in a nutshell, what are the big things that
- 8:12Speaker 2: you would.
- 8:12Speaker 1: Like people to take out of today to be very
- 8:16Speaker 1: clever about when to start a new chat thread?
- 8:19Speaker 3: Number one, every time you come to AI for a
- 8:22Speaker 3: new purpose, particularly at a new day, go to a
- 8:24Speaker 3: new chat thread. Chats are not there to be crafted
- 8:28Speaker 3: and kept for forever. You're there to be used, thrown away, used,
- 8:32Speaker 3: thrown away. So think of them more like a tissue.
- 8:34Speaker 3: If you use it for one purpose and then it's
- 8:36Speaker 3: gone kind of thing. So yeah, if it's important, then yes,
- 8:39Speaker 3: get it to download a document out of your chat
- 8:41Speaker 3: or something like that. So absolutely number one. Number two
- 8:45Speaker 3: is just keep a mind on how long you've been talking.
- 8:49Speaker 3: And so you'll know I've been using this for lots
- 8:53Speaker 3: of documents, lots of chats, lots of ins, lots of outs.
- 8:56Speaker 3: And so therefore you'll go, this feels like it's a
- 8:59Speaker 3: longer chat and then maybe it's time for me to
- 9:02Speaker 3: wrap this one up. The third is if it starts
- 9:05Speaker 3: hallucinating and lying to you and getting things wrong, that's
- 9:09Speaker 3: an absolute canary in the coal mine for you. So yeah,
- 9:12Speaker 3: you might go, yep, this is definitely time for me
- 9:14Speaker 3: to start wrapping these ones up. So yeah, there's no
- 9:18Speaker 3: easy way to say, oh, it's 60% or it's 100%.
- 9:20Speaker 3: It's more like, do I feel like it's been a
- 9:22Speaker 3: long time? Maybe it's time for me to wrap these
- 9:24Speaker 3: things up, get that document out, get the output, whatever
- 9:27Speaker 3: I need to do, and then start a new chat thread.
- 9:30Speaker 2: Amazing.
- 9:30Speaker 1: And I will just add wherever possible, use markdown or
- 9:33Speaker 1: text files so that you don't crazily eat up your
- 9:37Speaker 1: context window.
- 9:39Speaker 3: Those PDFs are a killer. Also, images can be quite
- 9:42Speaker 3: hard as well. So if you can upload a bunch
- 9:44Speaker 3: of images, it does take a lot of processing time
- 9:46Speaker 3: for it to understand the images. So yeah, pick the
- 9:49Speaker 3: documents that you're going to upload and you'll realize a
- 9:52Speaker 3: little bit more length in your speech rather than having
- 9:55Speaker 3: to wrap it up earlier.
- 9:58Speaker 2: Neo, thank you so much.
- 9:59Speaker 1: And we will be covering in a future episode in
- 10:03Speaker 1: the next few weeks, some more advanced tips about how
- 10:06Speaker 1: to neatly wrap up chat threads so you can take
- 10:09Speaker 1: all that great knowledge into a fresh one with a
- 10:12Speaker 1: fresh context window. How I AI was hosted by me,
- 10:16Speaker 1: Samantha Imber and Neo Aplin. A big thank you to
- 10:19Speaker 1: Martin Imber who does our sound editing and Jem Rubio
- 10:22Speaker 1: for production support. And thank you to John Kilby who
- 10:26Speaker 1: composed the theme music.