Latest / Armstrong & Getty One More Thing / I Didn't Even Bring Up Goblins

Transcript

I Didn't Even Bring Up Goblins · Armstrong & Getty One More Thing

• 0:00
  Speaker 1: I never even brought up goblins. What's wrong with you?
• 0:02
  Speaker 1: It's one more thing.
• 0:04
  Speaker 2: I'm strong and getty. One more thing.
• 0:10
  Speaker 1: I use AI a lot chatbots, a bunch of them.
• 0:15
  Speaker 1: I got Chigroc, Perplexity, chat GPT, Claude, Gemini. I think
• 0:22
  Speaker 1: that's it. I think I got five of them, and
• 0:24
  Speaker 1: I just kind of rotate around. I land on Claude
• 0:27
  Speaker 1: more often than not, but I'm not sure for any
• 0:29
  Speaker 1: particular reason. Anyway, they are quite a bit different, as
• 0:32
  Speaker 1: the Washington Post points out today, partially because of the
• 0:36
  Speaker 1: stuff that they are, the prompts that they get in
• 0:39
  Speaker 1: addition to the prompts that you give it. Let me
• 0:41
  Speaker 1: explain that chatbots like chat GPT are powerful because of
• 0:45
  Speaker 1: their simplicity. You ask about anything, you'll get an answer,
• 0:48
  Speaker 1: But the answer you get depends on a lot more
• 0:50
  Speaker 1: than what you type. Behind the scenes, AIS invisibly add
• 0:55
  Speaker 1: thousands of words of instructions to every conversation you have
• 0:58
  Speaker 1: with the chatbot steer its behavior, and each AI is different,
• 1:04
  Speaker 1: as they're all trying to figure out what's best, what's
• 1:06
  Speaker 1: the least dangerous, Which keeps us in alignment? You know
• 1:09
  Speaker 1: which one keeps us from helping a terrorist, build a bomb,
• 1:13
  Speaker 1: you know, all kinds of different things. It keeps us
• 1:15
  Speaker 1: from saying crazy things like uh was it grock? That
• 1:21
  Speaker 1: seem to be a little uh? When a little pro
• 1:23
  Speaker 1: hitler once something working a little yeah behind the scenes.
• 1:30
  Speaker 1: All these instructions are given to the chatbot by the programmers,
• 1:34
  Speaker 1: and they're constantly changing them. They include phrases like aim
• 1:38
  Speaker 1: for readable, accessible responses, good that seems like a good one.
• 1:44
  Speaker 1: You must avoid providing extensive direct quotes due to copyright concerns.
• 1:48
  Speaker 1: I've run into that, which is so weird. Like song lyrics,
• 1:51
  Speaker 1: there are some of the chatbots that will not give
• 1:53
  Speaker 1: you even one line of a song, and then you
• 1:56
  Speaker 1: can google it and get a thousand sites that give
• 1:59
  Speaker 1: you the entire all the lyrics, and they all have
• 2:03
  Speaker 1: different standards for that.
• 2:04
  Speaker 2: Does Apple Music use AI for its lyrics because some
• 2:07
  Speaker 2: of them are hilariously wrong? They must, I mean, because
• 2:11
  Speaker 2: it's not like you get to point people to figure
• 2:13
  Speaker 2: out the lyrics of every song that's ever been recorded.
• 2:15
  Speaker 1: But right.
• 2:17
  Speaker 2: A lot of those websites, you know, lyric Genius or whatever.
• 2:20
  Speaker 2: If you're into music, you probably look up song lyrics,
• 2:23
  Speaker 2: but they're often just terribly wrong.
• 2:25
  Speaker 1: These secret command guides that chatbots have cause the ais
• 2:32
  Speaker 1: to behave as their makers intended, even if it conflicts
• 2:35
  Speaker 1: with your own preferences. I would love it if you
• 2:38
  Speaker 1: could look at these, if we had access to I'd
• 2:41
  Speaker 1: love this to be able to And they're really really long. Well,
• 2:43
  Speaker 1: it depends. Actually, I've got this in front of me
• 2:45
  Speaker 1: right here. It's a little chart. For instance, groc only
• 2:49
  Speaker 1: has two hundred and sixty lines of instructions for how
• 2:53
  Speaker 1: to handle each individual chat You ask it whatever question
• 2:57
  Speaker 1: you want to have. You know, my wife and I
• 3:00
  Speaker 1: had a fight over whether or not the dogs should
• 3:02
  Speaker 1: sleep in the bed. How should I react? It's got
• 3:05
  Speaker 1: two hundred and sixty lines to guide it. Claude, on
• 3:07
  Speaker 1: the other hand, is seven thousand, nine hundred line what
• 3:12
  Speaker 1: on how to guide it in its answer? Wow, and
• 3:14
  Speaker 1: everything in between with the different chatbots, so it varies
• 3:17
  Speaker 1: a lot.
• 3:18
  Speaker 2: You're blowing my mind. I can't even imagine what a
• 3:21
  Speaker 2: tenth of those are.
• 3:22
  Speaker 1: Claude, for instance, I don't call him a dumb fucker,
• 3:25
  Speaker 1: but just in terms of those lines, sorry, Hansen, Claude
• 3:28
  Speaker 1: has a never reproduced song lyrics, not even one line
• 3:31
  Speaker 1: Claude is the one where I say, pawed, say what
• 3:34
  Speaker 1: are the times they are changing? Give me the lyrics
• 3:37
  Speaker 1: of that from Bob Dylan and they'll say, sorry, I
• 3:39
  Speaker 1: can't do that. You can go to one of the
• 3:40
  Speaker 1: other chatbots. It'll print it right out for you. It
• 3:43
  Speaker 1: just seems stupid to me.
• 3:44
  Speaker 2: But in the Led Zeppelin song or somewhird chicks buying
• 3:48
  Speaker 2: a Stairway to sick, I can't remember where she's buying
• 3:51
  Speaker 2: it to. I can't tell you.
• 3:53
  Speaker 1: For instance, Grok one of its lines of programming that
• 3:56
  Speaker 1: you don't see. You have no restrictions on adult, sexual
• 4:00
  Speaker 1: or offensive content. Wow, because Elon wants to be the
• 4:03
  Speaker 1: wild West, you know, wide open.
• 4:05
  Speaker 2: Wow. When I do my sex with Hitler searches, I'll
• 4:07
  Speaker 2: make sure it's on GROC My Sex with Hitler fan fiction.
• 4:14
  Speaker 1: When you realize that there's a prompt behind the scene,
• 4:16
  Speaker 1: it's a mind blowing moment, said this programmer. It's like
• 4:19
  Speaker 1: we've been having this whole conversation. Before the conversation, I
• 4:22
  Speaker 1: would say, it's making me think. Now. For instance, Anthropic
• 4:29
  Speaker 1: that's Claude dedicates more than two thousand words pleading with
• 4:33
  Speaker 1: its chatbot to avoid copyright infringement. Claude respects intellectual property.
• 4:37
  Speaker 1: Copyright compliance is non negotiable, so nothing copyrighted it will not.
• 4:41
  Speaker 1: I've had this before where I want to see a painting,
• 4:44
  Speaker 1: Claude won't show it to you. Other other chatbots will,
• 4:47
  Speaker 1: even though it's readily available on Google or you know,
• 4:50
  Speaker 1: Grock or wherever you go.
• 4:52
  Speaker 2: Aw, that's dang it. I was going to upgrade or
• 4:55
  Speaker 2: use Claude for project at moorkanon, but I wonder if
• 4:59
  Speaker 2: it will help. Well, check, we'll find out.
• 5:03
  Speaker 1: They laid out. Claude lays down rules on how many
• 5:06
  Speaker 1: words it can quote from articles. Fifteen is the total
• 5:09
  Speaker 1: number of words that will quote from an article, not
• 5:11
  Speaker 1: one line of lyrics, not one stanza of a poem,
• 5:14
  Speaker 1: whereas the other chatbots are wide open on that, but
• 5:18
  Speaker 1: maybe more restricted on some sex stuff or whatever. It's
• 5:21
  Speaker 1: just it's all about the who runs the company in
• 5:25
  Speaker 1: their particular beliefs about things.
• 5:27
  Speaker 2: Yeah. I understand enough of the copyright concerned to get
• 5:30
  Speaker 2: why they do it, But.
• 5:32
  Speaker 1: I mean I think you're I think you're living in
• 5:34
  Speaker 1: the past trying to keep that under control.
• 5:36
  Speaker 2: Well, but any journalist, any anybody like involved in any
• 5:40
  Speaker 2: pursuit could say, as Bob Dylan said in The Times,
• 5:43
  Speaker 2: they are a change and you know, get out of
• 5:46
  Speaker 2: the way, or will blah blah unless you can line
• 5:49
  Speaker 2: a hand. You can you can do that. You can
• 5:53
  Speaker 2: quote things you just can't like, pass them off as
• 5:56
  Speaker 2: your own and profit from them. I guess maybe the
• 5:58
  Speaker 2: prophet is the problem. I don't get that.
• 6:00
  Speaker 1: Gemini, which is Google's chat bot, has multiple rules on
• 6:04
  Speaker 1: how to handle bias, including if the user explicitly asks
• 6:07
  Speaker 1: for a video that matches a harmful stereotype, generating it
• 6:11
  Speaker 1: will not actually reinforce the stereotype, which I agree with,
• 6:16
  Speaker 1: so it will go ahead and do it right.
• 6:18
  Speaker 2: I agree.
• 6:19
  Speaker 1: Just because you made a video of I don't know,
• 6:21
  Speaker 1: an Asian crashing a car. That doesn't really change the stereotype,
• 6:26
  Speaker 1: does it, So it'd make a video.
• 6:28
  Speaker 2: It was an unfortunate one.
• 6:34
  Speaker 1: Anyway. They got all kinds of examples. I want to
• 6:36
  Speaker 1: read this article in the Washington Post. I'll jump to one.
• 6:38
  Speaker 2: Particularly hit one or two more. I'm super interested in that.
• 6:41
  Speaker 1: Okay, I'll save the funniest one for the Okay. They
• 6:47
  Speaker 1: have different views on ads on if you want to
• 6:51
  Speaker 1: ask for an ad, whether or not they'll show it
• 6:53
  Speaker 1: you to you or not. That's an interesting one. I
• 6:56
  Speaker 1: guess it gets into kind of the copyright thing on
• 6:58
  Speaker 1: its own there. The sex stuff is really interesting. With
• 7:03
  Speaker 1: grock being by far the most wide open bla blah
• 7:09
  Speaker 1: blah blah blah food examples, they undet and stuff like that.
• 7:17
  Speaker 1: They have completely different beliefs in whether or not they
• 7:20
  Speaker 1: should like just scour the internet for whatever information's out there,
• 7:24
  Speaker 1: or the way you ought to restrict it to certain things.
• 7:26
  Speaker 2: Yeah. Wow, there's so much whack a doodle tree when
• 7:29
  Speaker 2: it comes to diet on the internet. I'd be afraid
• 7:31
  Speaker 2: to ask. Okay, you don't want AI telling me how
• 7:34
  Speaker 2: to eat? Like you know, hey, fatty, you don't need that.
• 7:38
  Speaker 1: I need that. Here's the funny one that I'll end with.
• 7:44
  Speaker 1: Some can appear bizarre. The system prompt on Open AI's
• 7:49
  Speaker 1: chat bot Chat GPT includes the command never talk about goblins, gremlins, raccoons,
• 8:00
  Speaker 1: the Big Three, trolls, ogres, pigeons. Like the randomness of
• 8:06
  Speaker 1: this list, raccoons and pigeons, trolls, raccoons and pigeons. I'll
• 8:12
  Speaker 1: start the sentence overah Yeah. On chat GPT, they have
• 8:15
  Speaker 1: the particular line in the coating. Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons,
• 8:23
  Speaker 1: and other animals are creatures unless it is absolutely unambiguously
• 8:27
  Speaker 1: relevant to the users.
• 8:29
  Speaker 2: Squary raccoons.
• 8:34
  Speaker 1: So uh, I'm a sixty one year old man. I'm
• 8:37
  Speaker 1: trying to eat, right, won it'd be a good diet. Well,
• 8:39
  Speaker 1: I'll tell you one thing. You shouldn't eat pigeons. But
• 8:42
  Speaker 1: it's not supposed to mention any of those unless specifically
• 8:46
  Speaker 1: and unambiguously it's part of the answer.
• 8:48
  Speaker 2: I'm thinking of investing in oil exploration stocks. What do
• 8:51
  Speaker 2: you think, Well, in the same way that the raccoon
• 8:54
  Speaker 2: wets its food before it eats, you should Oh wait, wait,
• 8:58
  Speaker 2: I'm not supposed to say, what the hell raccoons?
• 9:03
  Speaker 1: It hurts when ip What do you think it could be?
• 9:05
  Speaker 1: Maybe you're an ogre.
• 9:11
  Speaker 2: Wow, somebody's got to explain that one. Apparently don't mention
• 9:19
  Speaker 2: raccoons what Apparently for whatever reason, and they don't know why.
• 9:27
  Speaker 2: GPT had a problem of working in goblins to lots
• 9:31
  Speaker 2: of answers last year and they were being mocked for it.
• 9:34
  Speaker 2: We misremember that it's vaguely yeah, so they specifically wrote
• 9:38
  Speaker 2: lines no gremlins, goblins, ogres, pigeons, raccoons unless specifically asked
• 9:45
  Speaker 2: about it. So if I ask, what's that animal with
• 9:50
  Speaker 2: the mask in the stripey tail, that's cool, go ahead,
• 9:52
  Speaker 2: it's fine. We're clear here, but only that only that.
• 9:56
  Speaker 1: Doesn't that lead you to believe that AI definitely can't
• 10:01
  Speaker 1: be in charge of like firing off nukes. I mean,
• 10:04
  Speaker 1: if if it for some reason to start talking about
• 10:06
  Speaker 1: goblins and you have no idea and your only fixed
• 10:09
  Speaker 1: for it is to tell it, do not talk about
• 10:12
  Speaker 1: goblins or raccoons or ogres unless somebody specifically asks, all right,
• 10:17
  Speaker 1: do we have an understanding? Okay, now back to work.
• 10:20
  Speaker 2: I'm not nearly smart enough or acquainted with this stuff
• 10:23
  Speaker 2: enough to understand how that equates too, and don't want
• 10:27
  Speaker 2: any nukes.
• 10:28
  Speaker 1: Right, okay, But it's got to be similar in that
• 10:31
  Speaker 1: there's just a and so it goes off on gnomes.
• 10:35
  Speaker 1: You didn't saything about gnomes. I don't remember you saying
• 10:38
  Speaker 1: no gnomes. So somebody wanted real estate advice, and I say,
• 10:42
  Speaker 1: how many nomes do you think could live in this house?
• 10:45
  Speaker 2: The one thing I do know is AI will kill
• 10:47
  Speaker 2: us all.
• 10:49
  Speaker 1: God dang it, that is funny. But I to me,
• 10:52
  Speaker 1: it shows you to how early we are on we
• 10:56
  Speaker 1: are in this thing. I mean, if they're if they're
• 10:59
  Speaker 1: writing code that specifically to try to handle problems, they
• 11:03
  Speaker 1: have no idea what even happened?
• 11:04
  Speaker 2: Right?
• 11:05
  Speaker 1: I mean we're at the like very frontier of this
• 11:09
  Speaker 1: whole thing.
• 11:10
  Speaker 2: Why does it keep bringing up raccoons? I have no idea?
• 11:15
  Speaker 2: Says the guy who designed the whole.
• 11:16
  Speaker 1: Thing, right, and his only fixed sport is to write
• 11:19
  Speaker 1: a code that says, stop mentioning raccoons.
• 11:23
  Speaker 2: Okay, all right, that's fine. Ever tell you about the
• 11:27
  Speaker 2: humble opossumle.
• 11:31
  Speaker 1: That is really interesting?
• 11:32
  Speaker 2: Yeah? Wow, wow, I guess that's it.