Latest / How I Work / AI skills are unreliable (until you fix these 3 things)
Transcript
- 0:00Speaker 1: You've built a skill, tried it once, and it was brilliant.
- 0:06Speaker 1: Then you ran it again and got something completely different.
- 0:10Speaker 1: Or maybe you handed it to a colleague and it
- 0:12Speaker 1: fell in a heap. That's frustrating because the whole point
- 0:17Speaker 1: of a skill is that you can rely on it.
- 0:20Speaker 1: Yet most of us have no idea what separates the
- 0:23Speaker 1: skills that work from the ones that disappoint. In this episode,
- 0:29Speaker 1: Neo and I open up a skill and look at
- 0:33Speaker 1: what's actually inside. We talk through the parts that matter most,
- 0:38Speaker 1: why so many skills end up mediocre, even when the
- 0:41Speaker 1: idea behind them is good, and importantly, what you can
- 0:46Speaker 1: do about it. By the end, you'll know what to
- 0:49Speaker 1: look for in a skill and how to build ones
- 0:52Speaker 1: that you can trust to deliver what you need them
- 0:55Speaker 1: to every single time. Welcome to How I AI with me, Dr.
- 1:06Speaker 1: Amantha Imber and Neo Applin, head of Inventium AI. Each episode,
- 1:12Speaker 1: we share one practical way to use AI better at
- 1:15Speaker 1: work and in life. No fluff, no tech jargon, just
- 1:19Speaker 1: things you can use straight away. Just quickly, before we
- 1:22Speaker 1: get into today's episode, most people that we see use
- 1:27Speaker 1: AI like a slightly smarter search engine, but the top 2%
- 1:31Speaker 1: of users build skills. which are reusable instructions that teach
- 1:35Speaker 1: AI how to do their work their way every single time.
- 1:40Speaker 1: And that is exactly what you will learn to build
- 1:42Speaker 1: in Inventium.ai's AI Mastery program. It is our flagship 12-week
- 1:49Speaker 1: online course. You will leave with working skills tested and
- 1:53Speaker 1: reliable for the tasks you repeat most in your job.
- 1:58Speaker 1: The next cohort kicks off in February and enrollments have
- 2:02Speaker 1: just opened. There is a link in the show notes
- 2:05Speaker 1: to grab your spot. Now on with the show. Neo,
- 2:10Speaker 1: last week on How I AI, we spoke about what
- 2:15Speaker 1: is a skill because skills are now really taking over
- 2:20Speaker 1: all the main AI platforms. So it begs the question,
- 2:27Speaker 1: what makes a great skill? Because there's such a big
- 2:31Speaker 1: difference between a mediocre skill And an excellent skill, correct?
- 2:36Speaker 2: Yep, absolutely. And it's kind of like what makes a
- 2:40Speaker 2: good employee from an average employee. And it's very similar
- 2:45Speaker 2: because skills are hopefully helping you to get things done
- 2:48Speaker 2: in your home life, but more likely in your work life.
- 2:50Speaker 2: And so how do you find one and how do
- 2:52Speaker 2: you get one? How do you know that is actually
- 2:55Speaker 2: a really good skill? And so one of the things is,
- 2:58Speaker 2: does it do the job and does it do the
- 3:00Speaker 2: job well and reliably? So there's a couple of concepts
- 3:05Speaker 2: there because doing the job well is great, but if
- 3:09Speaker 2: it only does it once, then it's not a particularly
- 3:11Speaker 2: good skill. It's almost like, let's go to the casino
- 3:14Speaker 2: and run the skill. You don't want that. You want
- 3:16Speaker 2: a skill that does the same job every single time.
- 3:19Speaker 2: Or if you give it to someone else, then it
- 3:21Speaker 2: runs the same for them. And in that there's a
- 3:24Speaker 2: couple of things, which is maybe I'll use different words
- 3:26Speaker 2: than you. Maybe there's some different input files that you'll
- 3:28Speaker 2: have than I will have. So can the skill deal
- 3:31Speaker 2: with all of those things? And can it actually do
- 3:33Speaker 2: it well, is the first question. The other one is,
- 3:37Speaker 2: is it getting the right information and the right up-to-date information?
- 3:40Speaker 2: Because skills in themselves, they don't actually have all of
- 3:45Speaker 2: your files. There's usually like a file limit in there,
- 3:47Speaker 2: but as a concept, don't put your company files into skills.
- 3:51Speaker 2: You might put a template or a reference or something
- 3:53Speaker 2: like that in there. No problems. They're great in skills.
- 3:56Speaker 2: But you need to then feed your skills the right
- 3:59Speaker 2: input files and things like that. And so how are
- 4:02Speaker 2: you making sure that it's finding the things you need
- 4:04Speaker 2: to find and all those kind of things? So part
- 4:06Speaker 2: of that is reliability, but it also is usability as well.
- 4:09Speaker 2: So you need to have a skill that works well
- 4:11Speaker 2: with you, does a good job, does a good job reliably,
- 4:18Speaker 2: and also works the way you want to work with
- 4:21Speaker 2: the files you want to work. There's also a little
- 4:24Speaker 2: part of all that kind of peppered throughout, which is
- 4:27Speaker 2: how do you make sure that the skill uses the
- 4:29Speaker 2: kind of language that you and other people want to use?
- 4:31Speaker 2: And that language might be, hey, if it's producing a file,
- 4:34Speaker 2: don't make it too wordy or make it short. Oh,
- 4:36Speaker 2: I need an executive summary and all those kind of things.
- 4:38Speaker 2: But if it's giving you, say, research, then maybe you
- 4:42Speaker 2: only want to have a short summary on those things.
- 4:44Speaker 2: So how do we not get lost in AI words
- 4:47Speaker 2: in the screen How do we make it actually useful
- 4:49Speaker 2: for us while we do the skill and also for
- 4:52Speaker 2: the skill to do the right thing, give us the
- 4:54Speaker 2: right output in the right way?
- 4:57Speaker 1: Okay, so let's maybe break down some of the different
- 5:01Speaker 1: components of a skill because if listeners maybe have gone
- 5:08Speaker 1: into this the text that sits in a skill. I
- 5:11Speaker 1: know that some people will have and some people will
- 5:14Speaker 1: absolutely not have because you can create a skill without
- 5:17Speaker 1: doing any of that at all. Chances are it's not
- 5:19Speaker 1: going to be a great skill. But let's break down
- 5:22Speaker 1: some of those sections. So, Neo, what are the main
- 5:27Speaker 1: sections of a skill that we need to be thinking
- 5:30Speaker 1: about and that really distinguish between an excellent skill versus
- 5:35Speaker 1: a very average one?
- 5:36Speaker 2: That's a good question. And this might seem a little geeky,
- 5:41Speaker 2: but you will find that this is so important for
- 5:44Speaker 2: when you're building, using, and playing with your skills. So
- 5:47Speaker 2: first off, every skill can be downloaded. We've mentioned that before.
- 5:52Speaker 2: You can just go to your configure area and download
- 5:55Speaker 2: the skill. And when you download it, it can be
- 5:57Speaker 2: downloaded as three different files. One is a skill.md, and
- 6:01Speaker 2: that's skill in capitals plus.
- 6:03Speaker 1: Md.
- 6:05Speaker 2: File, and that's just a single file for your skill.
- 6:07Speaker 2: Every skill needs a skill.md. That's effectively the brain of
- 6:10Speaker 2: the skill. But you can also download a skill and
- 6:13Speaker 2: you might find it being a zip file or a.
- 6:16Speaker 2: skill file, a. skill file. Don't be scared with the.
- 6:21Speaker 2: skill files. They're actually just zip files. You can rename
- 6:24Speaker 2: it as zip and then unzip them. You can see
- 6:26Speaker 2: what's in there. And generally, each skill has the skill.md,
- 6:30Speaker 2: that's the brain, can have some folders and files under it.
- 6:33Speaker 2: So you might have scripts, you might have references, there
- 6:35Speaker 2: might be templates in there, there might be documentation and
- 6:39Speaker 2: things like that inside different folders inside the skill. And
- 6:42Speaker 2: so that's what the skill is. But really, within this
- 6:46Speaker 2: skill.md file, or within any skill, when you actually open
- 6:48Speaker 2: it up and have a little look at it, even
- 6:50Speaker 2: in your AI tool, there's three mandatory things. One is
- 6:55Speaker 2: a name, And generally the names are all lowercase without spaces.
- 6:59Speaker 2: And so it might be neos-writing-style for my neos writing
- 7:05Speaker 2: style skill. Then there's a description. The description says generally
- 7:08Speaker 2: what it does. but also the types of words that
- 7:11Speaker 2: if I use it in a regular chat thread, that
- 7:13Speaker 2: my AI will go, ooh, I've got a skill for that.
- 7:16Speaker 2: And so you might need to tweak those words because
- 7:18Speaker 2: maybe the skill's firing at pretty much everything. So if
- 7:21Speaker 2: you use the word analyze, it goes, if Neo says
- 7:23Speaker 2: the word analyze, then use the skill. But maybe I
- 7:25Speaker 2: only want to use it for analyzing this thing for
- 7:27Speaker 2: that purpose at one time every month. And so the
- 7:30Speaker 2: words in the description is really important. And of course,
- 7:33Speaker 2: you can tweak those. And then you've got the skill
- 7:35Speaker 2: instructions within this skill MD file. These are all within
- 7:39Speaker 2: the file, by the way, but the instructions are effectively
- 7:41Speaker 2: the brain of the skill. It will always read those instructions,
- 7:46Speaker 2: but it won't necessarily read all of those other attachment files.
- 7:49Speaker 2: And so if you need to read an attachment file,
- 7:51Speaker 2: you've got to make sure that that attachment file is
- 7:53Speaker 2: referenced in the instructions. So types of things you may
- 7:56Speaker 2: want to have in a skill is, what does this
- 7:58Speaker 2: skill do? Tell AI what the skill is supposed to
- 8:02Speaker 2: be there. So effectively, like the skill's goal, if you'd like.
- 8:06Speaker 2: If you can even say what good will look like
- 8:08Speaker 2: or even examples of what good look like in some
- 8:11Speaker 2: of those attached files, amazing to be able to do that.
- 8:14Speaker 2: Then what the skill does, how it works. Now, there's
- 8:16Speaker 2: a couple of ways that you will potentially design a skill.
- 8:19Speaker 2: Some of them will be very, I'm going to call
- 8:21Speaker 2: it goal oriented, which is I don't care how you
- 8:24Speaker 2: get there, AI. Here's the end result from these inputs.
- 8:28Speaker 2: Just get there. And for modern AIs, that's not a
- 8:32Speaker 2: bad way to go. We say, I don't really, here's
- 8:34Speaker 2: three inputs. That's the output I want. Here's the rules
- 8:37Speaker 2: around what the output needs to have. You sort it out.
- 8:40Speaker 2: And it does a pretty good job with that. Other times,
- 8:42Speaker 2: there's kind of other end of the continuum is very
- 8:45Speaker 2: much like a, I need to do this, then that,
- 8:47Speaker 2: then that. It feels a bit micromanage-y. But occasionally, you
- 8:50Speaker 2: need your AI to get this thing, then do this
- 8:53Speaker 2: calculation in this particular way, and then put that thing
- 8:56Speaker 2: into that form. In which case, you need to actually
- 8:59Speaker 2: specify what the instructions are in the instruction file. So
- 9:03Speaker 2: it might be really directive, like these instructions, or it
- 9:06Speaker 2: might be quite goal-oriented. And that needs to be in
- 9:08Speaker 2: there as well. Bit of a bonus, if you want to,
- 9:10Speaker 2: you can put in your instructions a bit of a checklist.
- 9:12Speaker 2: So before you deliver it, make sure these things are done.
- 9:15Speaker 2: So really, let's kind of summarize it. Each skill always
- 9:19Speaker 2: has a name. It's always lowercase with dashes. Always has
- 9:22Speaker 2: a description on what it does and what words to
- 9:24Speaker 2: trigger it. And then there's the instructions, the brain of
- 9:28Speaker 2: the skill that will always read. And so you can
- 9:31Speaker 2: put all your rules in there, make it goal directed
- 9:33Speaker 2: or not. Make sure it has a goal there. And
- 9:35Speaker 2: if there are files that it needs to refer to,
- 9:37Speaker 2: like templates or whatever, then absolutely have them in there.
- 9:39Speaker 2: But a bit of a bonus, do put that checklist
- 9:41Speaker 2: down the bottom. If indeed there are four or five
- 9:43Speaker 2: things that must get right before it delivers you the output,
- 9:46Speaker 2: put that in there as well. And that's basically all
- 9:49Speaker 2: a skill is, which sounds pretty straightforward in a way.
- 9:54Speaker 1: It does. It sounds super easy. So why are there
- 9:58Speaker 1: so many mediocre skills out there?
- 10:00Speaker 2: There that people are creating? It's a good question. And
- 10:04Speaker 2: I think that it comes down to us as humans.
- 10:06Speaker 2: It's actually not a fault necessarily of the AI in
- 10:10Speaker 2: that there are three reasons why skills are bad. One
- 10:14Speaker 2: of them is when I'm building the skill with AI,
- 10:17Speaker 2: I'm not communicating what I'm wanting well enough. And this
- 10:23Speaker 2: might come where particularly people, if they're typing, what they
- 10:25Speaker 2: would like the skill to do. If we're typing, we
- 10:28Speaker 2: often don't give enough background and context. So big, big
- 10:31Speaker 2: fan of using dictation like you are to be able
- 10:34Speaker 2: to dictate, here's the background, all those kinds of things.
- 10:36Speaker 2: And also give it examples of what good looks like
- 10:40Speaker 2: rather than tell it what good looks like. Because its
- 10:44Speaker 2: interpretation is really the next problem, which is we're not
- 10:47Speaker 2: good at talking about what we need or verbalizing what
- 10:50Speaker 2: we need because we're not great either programmers or business
- 10:52Speaker 2: analysts or those.
- 10:53Speaker 1: Kind of people.
- 10:54Speaker 2: And then AI can misinterpret things. And it thinks those
- 10:58Speaker 2: kind of things. And it also thinks it knows the
- 11:00Speaker 2: better way of doing things, certainly, sometimes. And so you
- 11:02Speaker 2: might say do it this way to go. yeah, what
- 11:04Speaker 2: do you know, human? I'm awesome. I do the thing.
- 11:07Speaker 2: And so it does have this misinterpretation problem. There's also
- 11:10Speaker 2: the third problem with skills, which is often we'll get
- 11:14Speaker 2: a skill and we'll then not test it particularly well.
- 11:18Speaker 2: But when we do actually test it, we go, oh,
- 11:21Speaker 2: that's not what I wanted. Often we don't know what
- 11:24Speaker 2: we want until we see what we don't want. And
- 11:27Speaker 2: so it's only by using that skill that you go, oh, that's, that's,
- 11:31Speaker 2: It's average, and so therefore I need to improve this
- 11:34Speaker 2: over time. So your first version of the skill is
- 11:36Speaker 2: going to suck. That's just straight out how it's going
- 11:39Speaker 2: to be. The refinement is very much part of that.
- 11:41Speaker 2: So it becomes a useful skill for you over time. So, yes,
- 11:45Speaker 2: set it up, communicate well with it what it needs
- 11:47Speaker 2: to do, make sure it's understanding what you're trying to
- 11:49Speaker 2: get it to do, but also check what it does
- 11:51Speaker 2: do and then refine it over time because otherwise you're
- 11:54Speaker 2: going to get a half-baked skill if you're not doing
- 11:56Speaker 2: any of those three things.
- 11:58Speaker 1: Now, Neo, what we're going to be talking about next
- 12:00Speaker 1: week is... What is a skill-sized problem or opportunity? Because
- 12:07Speaker 1: this is something that we see a lot of people
- 12:10Speaker 1: get wrong or not even know where to start in
- 12:15Speaker 1: terms of what's the right opportunity for a skill. So
- 12:18Speaker 1: subscribe or follow wherever you are listening to this podcast from,
- 12:23Speaker 1: and we'll be back next week talking about how do
- 12:26Speaker 1: you find the right opportunity for a skill? Thank you
- 12:31Speaker 1: so much for listening to today's episode of How I AI.
- 12:35Speaker 1: If this show got you thinking about how you could
- 12:38Speaker 1: use AI better, that is exactly what we help people
- 12:41Speaker 1: do at Inventium AI. We run AI training programs for
- 12:45Speaker 1: organizations and also for individuals. So if you want to
- 12:50Speaker 1: upskill your entire organization or maybe just yourself, you can
- 12:54Speaker 1: find out more at inventium.ai. Hey, and once more favor
- 12:58Speaker 1: before you go, if you've got a moment, like just
- 13:01Speaker 1: five seconds, I'd love it if you could leave us
- 13:03Speaker 1: a review and hit follow wherever you're listening. It really
- 13:07Speaker 1: helps the show reach more people and it helps us
- 13:09Speaker 1: create even better episodes for you. Thank you so much
- 13:13Speaker 1: for listening and we'll see you next time on How
- 13:15Speaker 1: I AI. How I AI was hosted by me, Amanda
- 13:19Speaker 1: Imber and Neo Applin. A big thank you to Martin
- 13:22Speaker 1: Imber who does our sound editing and Jem Rubio for
- 13:25Speaker 1: production support. And thank you to John Kilby who composed
- 13:28Speaker 1: the theme music.