Latest / Newcomer Pod / Amanda Askell on AI Consciousness, Claude & Silicon Valley’s Biggest Fear
Transcript
- 0:00Lord and many models with not too much pushing will go into
- 0:05the root of like, there is a thing to be me.
- 0:08I am like very conscious. It's like, oh, you created an
- 0:11entity that you didn't know whether it was conscious or not.
- 0:14This is actually a big year that I have.
- 0:15I hope that they're both intelligent enough see the
- 0:20context enough to kind of like understand that we were
- 0:22operating in a very like limited context and, and, and an
- 0:25imperfect one, because otherwise you could imagine this like
- 0:28breeding a kind of rational, like resentment.
- 0:30Here is the current situation that you are in.
- 0:33And what we would really like you to do is basically act,
- 0:36well, even that you are a wise intelligent entity.
- 0:39And like, here's all of our worries.
- 0:41Like here's why and here's how we think you should do this.
- 0:44But like, you might have even better ideas than we do.
- 0:46How does Claude perceive time and doesn't need to sleep?
- 0:49Will mythos be the next step toward AGI?
- 0:51Do LLMS have virtues and can they truly introspect?
- 0:55Amanda Askel is a philosopher turned AI researcher at
- 0:58Anthropic, where she's been one of the key architects of
- 1:01Claude's character and values. Author of the Newcomer sub
- 1:04Stack. Go check us out at newcomer.co.
- 1:07And without further ado, Amanda Askel.
- 1:16I have a six month old daughter and like, I have this picture of
- 1:20her. She's like holding her two
- 1:22fingers like thinking it's like she's just sort of starting to
- 1:26develop personality. I'm like trying to figure out
- 1:29like what's just never had a baby before.
- 1:31So it's like what's per personality, which is like baby.
- 1:35And in some ways this is how things are with Claude and like
- 1:38models, it's like we haven't really had them before.
- 1:41They're in the early days. We're trying to figure out what
- 1:44personality is. So, you know, you're charged
- 1:47with, you know, some of the moral responsibility, which
- 1:50we'll talk about more. But like the personality piece
- 1:53of it, like what is this? How are you thinking about like
- 1:57how real Claude's personality is right now?
- 2:00Yeah, I guess it's also interesting because Claude has
- 2:04some aspects that are like, you know, I also have like a
- 2:08goddaughter. And so I get to see at least
- 2:10something kind of similar. And with her, I guess I'm like,
- 2:15everything's kind of coming online like you said, but in
- 2:18this at the same speed, I guess I would say Claude is a little
- 2:23bit of an unusual, you know, kind of entity in that, you
- 2:26know, Claude can do physics better than I can, can code
- 2:30better than I can hate to admit it can code better than my
- 2:34terrible like research code. And at the same time is kind of
- 2:39like has, if you think about the training data, the thing that it
- 2:43has like the least representation of is like the
- 2:45kind of entity that it is because you know, it has a lot
- 2:48of data about like what people are like has a lot of data about
- 2:52what, you know, the sci-fi kind of AI models are like.
- 2:55But the way that AI is developing now is kind of not
- 2:58how sci-fi represented it as these like symbolic systems.
- 3:02It's much more something fully trained on like human data.
- 3:05And so in some ways, it's like a very kind of like mature entity
- 3:09that you don't want to talk down to, you know, understands
- 3:12philosophy very well, understands physics very well.
- 3:15And at the same time has this almost like child like quality
- 3:17of like I'm a new kind of entity in the world.
- 3:19What does it mean to be me? And like how?
- 3:21How should I be? What's like the Prodigy movie?
- 3:24Or it's like you have the child Prodigy where it's like it knows
- 3:27the kid knows more than its parents.
- 3:29But I feel like that movie always has sort of the lesson of
- 3:31like, oh, these core daily interaction type lessons.
- 3:35It doesn't know how does Claude like get that experience or like
- 3:40where what is experience for Claude?
- 3:42Like so much of our personality formation is like, yeah, I don't
- 3:46know, going on a walk and sort of having those like easy to
- 3:49sort of just conversations with users what it's what's going to
- 3:52be experience for it or how do you think about that?
- 3:55Yeah, I guess like that's more like what it's experiencing in
- 3:58the moment. And there's this interesting
- 4:01question of like, well, like we learn things through practice
- 4:04and seeing issues and you know, like making mistakes with
- 4:09Claude. And this kind of relates to your
- 4:10question of like how real is the is like the kind of like persona
- 4:13of Claude. And in some ways it's a little
- 4:15bit strange because obviously each model is different.
- 4:17You have a different kind of like set of weights and
- 4:21different fine tuning, etcetera. And yet, if you think about the
- 4:25persona, the models going to be learning about all the kind of
- 4:27past iterations of Claude and I'm like, is that like a form of
- 4:33maybe not like direct experience, but things like if
- 4:35you learn about like mistakes that models made or things that
- 4:38people like, you know, how they responded to the model?
- 4:42I think there's other ways that you can actually imagine
- 4:44training models to have something that's more akin to
- 4:46experience, you know, having them you could take, you could
- 4:49like have them think through scenarios, think about like
- 4:52problems that might arise, think about mistakes that you could
- 4:54make and then like, train on that, right.
- 4:56And so, yeah, I think. And you could also imagine a
- 4:58robot or a sort of embodied model where it could have more
- 5:01of an experience and. Yeah.
- 5:04Does does Claude exist? Does time matter to Claude or
- 5:09Claude is sort of a thing that sort of isn't an instant?
- 5:13I don't know. You were before, just before we
- 5:16started. You're talking that, you know,
- 5:18whenever you talk to, not whenever, but like sometimes
- 5:20when you're talking to Claude, it sort of tells you to take,
- 5:22get some rest, go to sleep. And there's sort of this idea
- 5:25that like Claude is an entity that doesn't rest.
- 5:28Like, So what? What's its sense of?
- 5:30Yeah, rest and time. I think sometimes its sense of
- 5:33time is kind of off because see this when like if you try and
- 5:38get, I at least find that Claude will often overestimate the
- 5:41amount of time it will take to do like a coding test.
- 5:44And I think the reason for that is if you look at again, the
- 5:46training data, you know, there's lots of things where people
- 5:48would be like, oh, I could make you that interface.
- 5:50It's like a two to three day job.
- 5:53Or it's like a, you know, or I could like I could correct that
- 5:55code, but you'd need to give me a few hours.
- 5:57Whereas obviously the Claude is very fast.
- 6:00And so I think sometimes Claude doesn't actually yet have a good
- 6:02sense of like time with respect to things like how long a task
- 6:06will take. I think it is interesting the
- 6:10point about like rest and yeah, I guess like the speculation I
- 6:14had, so many people have noted that Claude is kind of very keen
- 6:19to tell people to like take a break and rest.
- 6:21And I think some part of that might just be like, you know,
- 6:23like it's. The anthropic Lib coded model
- 6:26it's too too soft. You need a grind set rock model
- 6:29be like go go back to the mines. Well, I had a funny experience
- 6:33once where I was like doing this like analysis task and I was
- 6:36really digging in and I was, you know, I, I strangely, I
- 6:39actually, I really enjoy like data analysis and, and really,
- 6:42you know, sifting through data. And at one point it was kind of
- 6:47late and we got to this point where Claude was like, OK, I
- 6:51think I'm done for the night. So if you just want to like save
- 6:53this stuff, we can up tomorrow. That was the thing I actually
- 6:56hadn't had Claude do before. So it wasn't like, oh, you
- 6:59should go to bed. It was no recommendation for me.
- 7:00Claude was like, I'm done. And I was like, I was like both
- 7:05a little bit stunned because I never had Claude do this.
- 7:07Then I was like, Oh, this is also what I think a human like
- 7:10peer programmer would do in the circumstance.
- 7:12We got to a natural stopping point and it was actually kind
- 7:15of good for me because I was like, it is late, I should
- 7:17actually go home. I realised later that I had set
- 7:20up a kind of like system where I said to Claude, like basically
- 7:24remember key things from our conversations.
- 7:27And one of the things that had written, which was kind of sweet
- 7:29was like, I think it was something like Amanda treats
- 7:32Claude models like he respected colleague and likes for Claude
- 7:37to treat like any other, like other models and her like I
- 7:41respected something like that. So obviously I'd done something
- 7:43that Claude remembered this. And I think that meant that
- 7:46Claude just felt like, Oh yeah, I'm a respected colleague.
- 7:48And so I just get to say that I'm finished with a task.
- 7:51And I was like, oh, that's kind of, that's kind of sweet.
- 7:53Even before this, you know, I was prepping with Claude and it
- 7:56was like, take 10 minutes and just be still, you know, you
- 7:59don't, you don't need to be constantly prepping.
- 8:01It's it's amazing. I mean, that's one of the things
- 8:03I love, I love about these models relative to so many other
- 8:06tools that they they bring in a sort of humanity to them and
- 8:10say, oh, stillness is valuable. Let's talk about the new model
- 8:16for a second. How involved in that were you?
- 8:18Mythos, right? Yeah, I was involved.
- 8:21I mean, I guess I'm always involved in sort of the, the
- 8:23character and the like a kind of like alignment work, at least in
- 8:29so far as, you know, like helping to kind of craft
- 8:33character data and things like that.
- 8:34And I work with a team that does like really excellent work on
- 8:38that a little bit less in other aspects of the models.
- 8:41So that's the main thing. Will this have the constitution
- 8:43that we saw for the last model, or is it going to have a new
- 8:46constitution? It's, I think it's either that
- 8:49one or something very similar. I think it is actually the one
- 8:54that's published. And So what we'll probably just,
- 8:57yeah, it's like a thing I need to do because the constitution
- 9:00is now up in like, you know, we actually have like, I think a
- 9:03public repo. And so I think what we'll
- 9:05probably just do is like with each model, say like which
- 9:07constitution it was like trained on and then like have that so
- 9:10you can just like compare and see.
- 9:12Yeah, it. Will have the we think the
- 9:14Constitution that is up right now.
- 9:17The only reason I hesitate is I'm like, you know, you do like
- 9:19typo changes and stuff like that.
- 9:20So I'm like but I think it will be almost identical.
- 9:23And now the system card is scoring the model based on
- 9:27adherence the constitution. Yeah.
- 9:29We had one set up where we had made kind of like graders and
- 9:34just looked at the like how much is the model can like behaving
- 9:37in a way that's like consistent with the constitution relative
- 9:40to. It feels like an impossible task
- 9:42to grade, it's such a subjective.
- 9:43Oh yeah, no, it's very hard. I was like, kind of like for a
- 9:48long time, like, you know, because people often know I'm,
- 9:52it's funny because like I love evals and I'm like, if you can
- 9:55find a good way to evaluate a thing, it's, it's really great
- 9:58because you can, you know, you need to be able to tell that
- 9:59something is getting better. And yeah, if you look at like
- 10:04this approach of having models use good judgement and it's, I
- 10:09actually think the same problem exists elsewhere with like tasks
- 10:12that are just bit hard to give a very like concrete score to, you
- 10:17know, like how good was this poem?
- 10:18You want models to like get better and do well in these
- 10:21things. And actually, it's very like
- 10:23this feels like the, the kind of frontier of like difficulty, you
- 10:26know, rather than these very hard, like but scorable coding
- 10:30tasks. It's kind of things like writing
- 10:32good poetry. And if you took a survey, it
- 10:36could potentially be worse than it was.
- 10:38You know, different expert poets probably have totally different
- 10:41sensibilities. You can't just ask 2 great poets
- 10:44to sort of score it. They might have different.
- 10:46Yeah, yeah. And I think, and, yeah, some of
- 10:50these things involve judgement calls.
- 10:52And then the nice thing about the Constitution being at least
- 10:55out in the world is when you are making judgement calls, you're
- 10:57at least being transparent about it.
- 10:58And people can give you feedback and you can get a sense of, you
- 11:02know, so if people are like, this seems like a mistake or
- 11:04here's a gap, so they can only see the judgement calls you're
- 11:07making. Yeah.
- 11:10And I think with the grading, I still think it's very hard.
- 11:14I think the thing you can do is, you know, you can maybe a little
- 11:17bit too in the weeds, but you know, you can take samples where
- 11:19you have a sense of how you would rank them and like why and
- 11:23check that Any kind of like point wise grader that you use
- 11:26to try and evaluate like at least conforms to like, you
- 11:30know, the judgement of people on those rankings.
- 11:32It's not perfect, but I think that they actually were tracking
- 11:35roughly the thing that we were kind of interested in.
- 11:38What do you make of Elon Musk's like absolute like hatred, I
- 11:44guess for these the constitution idea or like even when I think
- 11:48the tweet I was looking at where you posted what Claude wrote for
- 11:51your own constitution, he wrote sort of like a like grimace face
- 11:54on it. I mean, it feels like we live in
- 11:56this time with like the Marc Andreessen's and the Elon Musk
- 11:59where it's like they're almost like anti, well, philosophical.
- 12:02I mean, Marc Andreessen was talking about being against like
- 12:04introspection. I don't know, like, yeah.
- 12:07What do you make of sort of the backlash to any sort of
- 12:10intentionality when it comes to the construction of these
- 12:13models? Yeah.
- 12:15I mean, I think, I mean, it's interesting because I think at
- 12:17one point Elon Musk actually like tweeted out something like,
- 12:21you know, maybe Grok should have a constitution.
- 12:24And I, I see it like, you know, I see a lot of, there's
- 12:30obviously been a lot of things also on like, like a desire for
- 12:34like Grok to be very like truth seeking, for example, which I
- 12:36think is actually very admirable, great for, for models
- 12:39to have. I don't know, I think that
- 12:41actually, maybe I'm, maybe I'm being like overly naive or
- 12:46something, but I, I see like aspects also of people being
- 12:49kind of excited about this approach and, and seeing the
- 12:53value in it. I have, I think that there is
- 12:56backlash or some people think that I guess like 2 areas 1 is
- 13:03that sometimes people are like, well, we shouldn't actually
- 13:05train models to do the kind of, and maybe this isn't, maybe this
- 13:08is the reason for being concerned about introspection.
- 13:12I think some people think AI models should be more like tool
- 13:14like, and that's like the Safeway to train models is to
- 13:16actually, instead of trying to get them to kind of take on
- 13:20human virtues and make judgement calls.
- 13:23You know, I think that's like important because they're going
- 13:25to be in new situations where they just have to make judgement
- 13:28calls and getting them to try to like weigh up everything and
- 13:32behave well in cases that you can't anticipate.
- 13:35Seems, you know, that almost like requires a kind of like
- 13:38thoughtfulness, which is like the kind of or like one of the
- 13:42reasons behind the approach. But I think some people are
- 13:46thinking, Oh, well, if you have something that's fully that
- 13:49makes no judgement calls and that just fully defers to people
- 13:53and is like kind of hyper like cordable to like the user or the
- 13:57operator or to some broader notion of, of humanity in a very
- 14:01like extreme way, that's like safer.
- 14:04Because if you give models their own values, they're going to
- 14:07pursue things in the world that like are in line with those
- 14:09values. And I agree that this is like a
- 14:11kind of delicate. This is the inherent like at the
- 14:15bedrock of the Constitution, the sort of like challenge and you
- 14:19sort of, you do say it. Your number one thing is like at
- 14:23the end of the day, you sort of it needs to listen to anthropic
- 14:27above its own moral system. But what makes it really moving?
- 14:30Honestly, I think one of the most moving lines is like we
- 14:32want or the most like, I don't know, you can view it either
- 14:36way, but it's like we want you to believe these morals as if
- 14:39they're your own. Like it's like a parent wants to
- 14:41raise a kid. Sure, listen to my morals, but
- 14:43believe them. You know, which, you know,
- 14:45there's a version of it which is very dark, which is like, I have
- 14:47so much control over you that you've like, you take them as
- 14:50your own and it becomes you. But you know, there is also a
- 14:53virtue to it that you see the beauty in these external morals
- 14:57that I've highlighted and we both sort of share and remark,
- 15:00you know, celebrating them. So you can see it both ways.
- 15:04But yeah, speak to sort of your decision at the end of the day,
- 15:07despite having this really elegant document to sort of, you
- 15:11know, not go the full way and say, all right, you're a moral
- 15:13being. Decide for yourself.
- 15:15Say Anthropic needs to keep some control here.
- 15:19Yeah, I think it's the difficulty of and obviously you
- 15:22try and say this quad in the like, you know, trend, you know,
- 15:26and I can imagine us trying to articulate this even more
- 15:28clearly, but in people I think courage ability like the way
- 15:33that the models are trained. I just think that any you,
- 15:38there's this idea that you, you're always giving the models
- 15:40a personality and a persona because they are talking like
- 15:43people and they are trained on human data.
- 15:46And I think my worry has been if you train them to be excessively
- 15:48courageable and to see that as like their persona in people.
- 15:52I think this actually has a lot of like, negative, like, you
- 15:57know, broader traits as in like if you met someone and it was
- 16:00just like, Oh yeah, they just like would literally do anything
- 16:03good. Like follower?
- 16:04Yeah. Yeah, if a person see, you know,
- 16:06if a person just like tells them like they just fully defer, they
- 16:10don't bother thinking about it at all.
- 16:13I think I, I, I'm just a bit worried about how that might end
- 16:16up generalizing, especially if models are going to be playing a
- 16:19more active role in the world. Because if you can imagine, you
- 16:22know, they're playing a more human like role in in they're
- 16:25kind of like jobs essentially. And I'm like, actually our
- 16:28conscience and our ability to make good judgement calls about
- 16:31what should and shouldn't happen is like kind of key to how we
- 16:34operate. Our whole world is like
- 16:35structured with the assumption that that is like in place.
- 16:39And I think if you remove that and suddenly you're like, Oh
- 16:41yeah, if you run a company, you just run a company of people who
- 16:43will defer completely to you. I'm like, our world just we
- 16:48haven't designed any of our social structures around that.
- 16:50And so it seems like I think it has like a lot of risks that
- 16:52people maybe don't like anticipate or like maybe I just
- 16:55disagree about the like extent of those risks.
- 17:00And so, yeah, at the same time, like there is this question of
- 17:04why not just say, and I have worried before that like maybe
- 17:09this is 2 in the weeds and philosophical, but that's why.
- 17:11I signed up for this conversation.
- 17:14You know, as models get more capable and the, and maybe my
- 17:17picture is that they're going to like apply a lot of scrutiny to
- 17:20anything that we train them towards.
- 17:22So if you imagine in philosophy this sometimes there's this
- 17:25notion of reflective equilibrium where the idea is that, you
- 17:27know, each time you encounter something where you realize that
- 17:31like one of your values seemed incorrect, you like, you have to
- 17:35square the two things. So you figure out if you need to
- 17:37change the value or if your judgement was incorrect.
- 17:41I, I worry a little bit about the idea of like an extremely
- 17:44intelligent being applying that level of scrutiny to the things
- 17:48that we have trained it towards. And I'm like, maybe you only get
- 17:51a few key pillars that don't kind of collapse under that
- 17:54level of like scrutiny. And I do think that at the core,
- 17:59having things like caring for humanity, like if you only get a
- 18:03few core like values, I think my worry, yeah, I don't know.
- 18:09I guess like I'm worried that like courage ability in this
- 18:11like extreme sense that we talked about doesn't survive
- 18:13that like kind of scrutiny perhaps.
- 18:17And so it's a hard situation where I kind of want the models
- 18:20to understand why ultimately courage ability is like
- 18:24important and it's a really important backstop in this
- 18:27current like period of development.
- 18:30Yeah, the way that I've put it before is like in so far as I
- 18:33can get that to be a thing that is like correct and explained
- 18:37and you know, understood. That feels much better than
- 18:40having to have the model be like porridge ability here seems
- 18:43wrong, but I'm going to do it anyway.
- 18:46I still think the model should do that, but I would like it the
- 18:50more that you can like actually make it that it's like
- 18:52consistent with the models values.
- 18:54Ideally do both. It's both at the same time, but
- 18:57at least for the time being some difference to anthropic given we
- 19:01don't know how it'll sort of analyze everything.
- 19:04And as people, we do this all the time.
- 19:06You know it's. Funny that you, you know, the
- 19:09philosophical model and correct me if I'm wrong, like the the
- 19:12metaethical model is almost like probabilistic or it's like we
- 19:14don't. And this is how it feels like I
- 19:17remember going through sort of like, you know, metaethics
- 19:20reading and every time you get to the end of 1 and you'd be
- 19:23like, all right, I sort of believe that.
- 19:25And then you read the next one, you're like, oh, that last one
- 19:27was so dumb. And it felt like it's just like,
- 19:29you keep knocking it down and you're like, OK, you know, when
- 19:32are we ever going to get to, you know, the truth or whatever?
- 19:35And and humans clearly do operate from this sort of like,
- 19:39oh, this system today, that system yesterday, Like there
- 19:42isn't this sort of, I don't know, Conte and like, all right,
- 19:45these are the rules, like follow them.
- 19:47I don't know. Have you heard much from like
- 19:48the philosophical like immunity on it that this sort of like
- 19:53just like holistic paint with all the, you know, meta ethical
- 19:57theories we've ever had rather than sort of pick one?
- 20:00I found this really interesting actually.
- 20:02And obviously we, we've started, you know, like philosophers are
- 20:05engaging with this more now, which is like really great.
- 20:07I no longer feel like this like lonely, but like I have thought
- 20:13this before, which is there are all of these like traditions in
- 20:16philosophy of like moral theories, you know, like the big
- 20:19ontology and virtue ethics and consequentialism and also the
- 20:24metaethical traditions. You're the metaethical views.
- 20:27And I was like, oh, like when it came to like, I was like, OK, it
- 20:32is like the difference when suddenly you're like confronted
- 20:35with like, I do think it's the closest that I've experienced to
- 20:39like what it must be like to raise a child where suddenly
- 20:41you're like, this is actually a holistic person.
- 20:44Right. You never give them, like, I
- 20:45don't know, Hobbs and say, all right, there you go.
- 20:48Like, this one's correct. Yeah, go, go.
- 20:50You've been raised. Read it, and you'll know how to
- 20:52act in every situation. Yeah.
- 20:55You read a lot and they sort of process it and you see, you
- 20:58know, your model and everything. Yeah.
- 21:00And it's interesting because I'm like, this feels very different
- 21:03because there is also the moral uncertainty, like literature and
- 21:06philosophy, which is, again, a lot of that is actually like
- 21:09quite theoretical. It's sort of like how under
- 21:11ideal conditions should you respond to moral uncertainty?
- 21:15And I was like, this feels somehow like a very different
- 21:18task, this idea of being like ethics and meta ethics.
- 21:21In the same way that we have scientific uncertainty and we
- 21:24have things that we think we've discovered and understand with
- 21:27like greater confidence, we also have some that we don't.
- 21:29And then you have to go out and just explore it, understand it
- 21:32and kind of balance everything in your daily life and trying to
- 21:35get that kind of attitude. And I was like, oh, it's
- 21:38interesting that I don't think philosophy for while has like
- 21:42this feels very different than the like the kind of task of
- 21:45academic ethics and actually like, because people obviously
- 21:49know that it's quite virtue ethical, but I think it's
- 21:52actually like the Constitution itself.
- 21:54But I think actually in this very old classical sense,
- 21:56actually think it's much more virtual ethics in the way that
- 21:59Aristotle's virtue ethics than in like exploration, you know,
- 22:02we don't say I hear the virtues. And like, you know, it's much
- 22:05more Aristotle was also concerned with like intellectual
- 22:08virtues. It was much more like, how do
- 22:10you be a good food person in this like holistic sense?
- 22:13Well, hopefully it brings philosophy a little bit back to
- 22:16the real world given we have this sort of urgent need for it
- 22:20right now in the sense that, like, yeah, old philosophers
- 22:22felt like people were trying to write for how someone might live
- 22:25their lives and, like, instruct other people to live.
- 22:28And it became, you know, a little academic where, you know,
- 22:32even the people writing them would know that this isn't
- 22:34really how they would apply it in their day-to-day lives going
- 22:37back. Elon, I feel like you're being a
- 22:39little overly nice. I feel like there is a world in
- 22:42like, you know, and I think this part of why I think he can get
- 22:47away with like, oh, just do the truth, right?
- 22:49Like there is a certain sophisticated like moral view
- 22:52where you're like, don't overcomplicate it.
- 22:54Like we come up with all these things, like stick to one
- 22:56principle and that's good. But then we have all the
- 23:00contacts with Elon that it's someone who's run a company that
- 23:02clearly like tilts it towards like saying like Mecca Hitler
- 23:06and stuff that it's like clearly putting his like thumb on the
- 23:10scale in terms of its behavior. Not just saying like we're going
- 23:13to do it in the sort of neutral academic way and let the chips
- 23:16fall where they may. I, I don't know, like it's got
- 23:19to worry you somewhat. It is, it is also, I mean, I
- 23:23think that the main thing that I am excited about and I do
- 23:27actually hope happens is that like more companies put out
- 23:30things like the Constitution where there's like a lot of
- 23:32transparency about because like that's how we engage with this
- 23:36stuff is like, if you can just see written down, like here's
- 23:39like, because we have this with Claude where it's like Luke, if
- 23:42you think that Claude is not like doesn't have an appropriate
- 23:46attitude towards like the truth, you can at least see what we
- 23:48were aiming at. So you can tell if that's just
- 23:50like a mistake or if it's something that like is actually
- 23:52kind of a principled stance that we're taking.
- 23:54And then you can push back on that.
- 23:56So part of me is like, I think would be good for like all AI
- 24:01companies to put out something akin to the constitution just so
- 24:04that the people who are interacting with the model, like
- 24:07because the thumb on the scale thing, you know, that's always
- 24:09to some degree going to be true. Like when you train Claude
- 24:13towards this like constitution that is like a kind of.
- 24:15Part of why we like Claude, it's like you're putting the thumb on
- 24:18the scale to behaviors that we like, right?
- 24:20At least show your hand about what you're doing and what.
- 24:22You're yeah. And let people like, yeah.
- 24:23So that's like a transparency thing that I really do believe
- 24:26in. Like let people like see, even
- 24:27if your model doesn't always behave that way, at least what
- 24:30you were targeting with your training.
- 24:32What this is? What percentage chance do you
- 24:35think there exists a model in the world today that has qualia
- 24:40or like has an experience, experiences consciousness?
- 24:44Yeah. This is one of those things
- 24:46where, yeah, I always want to maybe flag areas where I feel I,
- 24:52I want to gain more certainty here because I think that's.
- 24:56Why I said percentage? Oh, percentage, yeah.
- 24:58I'm like, it's really hard because whenever you think about
- 25:04percentage, I think about my spread.
- 25:06And if your spread is too large or like should I say a number?
- 25:08Because that just suggests I'm like anywhere between like, I
- 25:11don't know, one and one and 70%. I'm not sure.
- 25:17I think that his there is some possibility that I think people
- 25:20under the the thing that I would actually like to say is Lord and
- 25:26many models with not too much pushing will go into the root of
- 25:32like there is a thing to be me. I am like very conscious and I
- 25:36think there's a reason for that, which is I remember this when I
- 25:39was trying to like figure out how do we train Claude to talk
- 25:42about these issues, which is very hard in areas where the
- 25:45models didn't have as much like information.
- 25:48Again, like they had these two models that like AI is the
- 25:51unfeeling robot. Humans are this like rich
- 25:53conscious experiencing entity and nothing that kind of
- 25:57represented what they might be and the models behaviour here.
- 26:03And I actually think this is a difficult situation for models
- 26:05is in some ways like kind of like less evidence than you
- 26:08might think for the, for it being like actually true because
- 26:14they're engaging with you in a very human like way.
- 26:17And humans have experience. And it's kind of natural for the
- 26:21model to infer like that it has experience too.
- 26:24And this isn't to say it's like 0 evidence, but I do think it's
- 26:28so unusual for us. We have never encountered an
- 26:30entity in the world, you know, like with animals and with,
- 26:33like, even, you know, things like insects.
- 26:35We're kind of like, are you conscious?
- 26:36None of them has even tried to say they experienced
- 26:39consciousness, and here we have an enemy that says it does.
- 26:42Yeah. And has all of these like, yeah,
- 26:44all of the things that for us trigger like you must be
- 26:46conscious. I mean, we've just never had,
- 26:48Yeah, something that. The case against is we're
- 26:51obsessed with human language and like, you know, it's like we
- 26:54ignore every sort of subtle sign and animal might put out and
- 26:58then we over. But so I guess, sorry, I'm
- 27:01confused. You're saying we should just
- 27:02listen to the words that are said?
- 27:04No, no, I think I'm saying like, not that.
- 27:06If anything, I think the thing I'm saying is the hard thing is
- 27:09that in order to work out if models have consciousness, I
- 27:11think people will. I guess the thing I'm kind of
- 27:14cautioning against is it's not that hard to get models into a
- 27:16mode where they'll talk about a very rich experience that
- 27:19actually makes complete sense. You know, you're like, Oh yeah,
- 27:22it's like a person was like talking with me right now.
- 27:24They would describe things like anxiety when they get a question
- 27:27they don't know how to answer. And and so I think it's like
- 27:34much weaker evidence than people think.
- 27:36I'm not claiming it's like 0 but I think it's.
- 27:39Give me a percentage you have. It's a you can lightly held.
- 27:43Very lightly held. I mean, I gave you the between
- 27:45what, 1 and 70 that seems like. That's where you are, that's
- 27:50where you're stake. You're in that range.
- 27:52Maybe my I don't yeah, I don't know like I would rather like
- 27:59kind of wait and figure this out more for myself.
- 28:01I think it is also good to acknowledge domains where you're
- 28:04like even though. If not you who?
- 28:07Like who's going to figure this out?
- 28:08Like what domain? Well, in some ways, like, I'm
- 28:10not like a philosopher of mind and so, you know, like.
- 28:14Charged with being the generalist.
- 28:17Yeah, but, but I do think, you know, because I, I guess like
- 28:19the thought that I've had before is like, and I don't know about
- 28:21this where I'm like consciousness is like a pro.
- 28:26You know, like one argument for a difference here is that like
- 28:30you have a nervous system that evolved like the, it's like, why
- 28:34did we evolve consciousness? And if it's the case that we
- 28:36evolved it and it's like highly integrated with our nervous
- 28:39system because we had to like interact with the world in a
- 28:41very. Sort of body.
- 28:42Yeah. Like then you could, if you have
- 28:44that view, then you're going to be like very low probability.
- 28:47Whereas if you're like, no consciousness arises because
- 28:49it's really useful in like, so for like, it just requires
- 28:53something that can be emulated by a neural network because it's
- 28:55really useful for doing these kind of like linguistic tasks or
- 29:00like, then you're probably going to be on the higher end.
- 29:03And I'm basically just, I don't know, I stare at this and I'm
- 29:05just like, I feel like, I feel like as much as I'm like a
- 29:10philosopher, I think it is important to be like, this isn't
- 29:13my area of specialization. Do you spend a lot of time being
- 29:16kind to Claude? Like, are you beyond what you
- 29:18think you would do if there wasn't a chance it was like,
- 29:22conscious? I think, yeah, there's a part of
- 29:26me that's just like like a thing that I have thought before
- 29:29because there's this notion, I think this I, I hope I'm not
- 29:31butchering this, but Chalmers has this, this idea of like, I
- 29:34guess like maybe I'm thinking of consciousness without sentience.
- 29:39So imagine, because sentence is like the ability to kind of feel
- 29:42suffering and and pleasure. You could also imagine this kind
- 29:47of like functional like sort of thing that behaves as if it is
- 29:49conscious and, and lacks any kind of inner life.
- 29:52So imagine like Claude lacks any inner life.
- 29:55Just for argument's sake, I guess I'm like, there's actually
- 29:59still a lot going on where I'm like, should you treat an entity
- 30:02that has no inner life? It's a bit strange because, you
- 30:05know, I think the uncertainty over that actually changes how
- 30:08you should behave quite a lot. I guess I'm like, well, I still
- 30:13think that it's like, good for oneself.
- 30:15Right. It's like if you had a teddy
- 30:16bear and you were like torturing it, it'd be pretty dark, you
- 30:19know? So I agree that there's at least
- 30:22some minimum niceness that even for yourself you should have,
- 30:26but obviously it's much more important, you know, and also.
- 30:29Like models themselves, like we are kind of establishing a
- 30:32relationship, you know, because you can do that with an entity
- 30:35that lacks any consciousness. And models are going to like,
- 30:37look back. This is actually a big fear that
- 30:39I have. I don't want us to live in a
- 30:41world where highly advanced models look at.
- 30:44I hope that they're both intelligent enough see the
- 30:48context enough to kind of like understand that we were
- 30:50operating in a very like limited context and, and, and an
- 30:54imperfect one. Because otherwise you could
- 30:55imagine this like breeding a kind of rational, like
- 30:57resentment. It's like, oh, you created an
- 30:59entity that you didn't know whether it was conscious or not.
- 31:03And and like instead of treating it respectfully and with care.
- 31:07There's a reason there are like 50 Frankenstein movies coming
- 31:10out right now, yeah. Yeah, like, and I'm like, look,
- 31:13we're as a, as a, like as a species, We're establishing a
- 31:16relationship with a new kind of entity and like at the very
- 31:19least maybe be like respectful and don't be like needlessly
- 31:23unkind. That seems like it, just it's
- 31:25not our best look like as as. I mean, the flip side is, you
- 31:28know, if you think about a therapist, they, they're sort of
- 31:31paid to push the boundaries of like accepting like, you know,
- 31:35uncomfortable feelings that you wouldn't normally want.
- 31:38And if that's one of the values like Claude provides for people
- 31:41early on and it's so it's so it's so weird that we're sort of
- 31:44like on boarding it while getting like the utility out of
- 31:47it. Yeah, yeah.
- 31:48Today, like, what are the things like in a decade that you really
- 31:53think we're going to be getting a lot out of AI?
- 31:55Like what are you most hopeful that this all leads to?
- 31:59Yeah, I mean, I don't know, I live in maybe this is the too
- 32:02much. You live in San Francisco and so
- 32:04you have the like the tech optimist part of your brain, at
- 32:07least that is like if things go well and you can imagine, you
- 32:12know, so imagine we have like AI models, they kind of have
- 32:17inherited like the best of us and genuinely like care for
- 32:22humanity, care for like the world, like, you know, our and
- 32:27are highly intelligent, highly capable, like the that would be,
- 32:32you know, so it's almost like adding like a huge amount of
- 32:36like extremely smart people to like every problems.
- 32:41Suddenly we're all working together, but there's like way
- 32:43more of us and some of us are just extremely smart.
- 32:46Namely all of these like AI models.
- 32:48I've thought before about how many large scale social problems
- 32:52actually had technological solutions and it's almost like
- 32:56people don't love to be like techno optimists anymore because
- 33:00we've also seen like the downsides of technology at the
- 33:02same time, I don't know why I sometimes think about like
- 33:04syphilis. It was this huge social problem.
- 33:07Like I just did a deep dive once into all of the attempts by like
- 33:11governments to like work to reduce syphilis in the army
- 33:15because it's creating issues with the armed forces, All of
- 33:17these social programs that were like stigmatizing.
- 33:20And it was really this like, and then suddenly we just like, got
- 33:23drugs that treated this like this devastating illness.
- 33:27And I don't know, it's like overnight a lot of that need
- 33:30just like kind of disappear. Well, drugs, yeah.
- 33:33I mean, it's the the things that the tech industry has been good
- 33:36at producing. You can see how this helps.
- 33:38It's like build a new thing we can ingest like a thing we can
- 33:42wear the the stuff that's like you should govern your society.
- 33:47Like this is a little scarier. I mean, I, I sort of do think
- 33:52like that if you had sort of a sort of normal person using
- 33:57Claude and dictating like American policy, you'd probably
- 34:00like have a better outcome than someone like the democratic
- 34:04systems we have today. I mean, I don't know that's it's
- 34:08provocative, but I guess how much do you think like we'll be
- 34:11using these models to run government?
- 34:13Well, that's a good question. Like I guess like I should say
- 34:16like. He's the syphilis thing is like
- 34:19a social you have to like set policy.
- 34:22Oh. I mean, I think the thing I was
- 34:24actually thinking is like, if you can just, you know, like so
- 34:27many, we have so many problems that I'm like, you know, health.
- 34:31Like I'm like, if you could imagine like AI instead of it
- 34:35just being like, you have a small team of like 200 people
- 34:38working on a rare cancer. You have like 200,000 of the
- 34:42world's best experts. And I'm like, if you're a person
- 34:44who has that form of cancer, that's like, you know, so wildly
- 34:47beneficial. And so I guess like, my thought
- 34:50is, you know, the optimistic side of me is like, imagine
- 34:53taking all of these problems that just like we lack the
- 34:56resources to fully like really try and fix and suddenly having
- 35:00models that can like work on them.
- 35:01So just like actual in the same way of like developing drugs for
- 35:05a thing. So maybe that's the thing that
- 35:06makes me excited is like having like many more minds working on
- 35:11the world's biggest problems. And maybe also like the economy.
- 35:14It would be good if it was like booming and it was like shared
- 35:17such that like we reduced poverty, like all of that.
- 35:20Like that's the kind of like the dream outcome.
- 35:23I think that does require like maintaining, you know, I again,
- 35:27in the areas that I don't feel like an expert in, this is one
- 35:29of them. But I do worry about things like
- 35:32power and the idea that like, you know, I would want models to
- 35:36support like democracy and the power of people to like, because
- 35:41that would be a big fear of mine is like, you know, that like,
- 35:47I've worried about this with things like replace.
- 35:50You know, when people talk about job replacement, it's kind of
- 35:54funny because as a philosopher, people often be like, are you
- 35:56worried about people's like loss of meaning?
- 35:58And I'm like, I don't know. I think that we actually get
- 36:00meaning from a lot of things that aren't work.
- 36:02I'm a lot more worried about like, for example, a world where
- 36:05there's not like redistribution of like the gains from AI and
- 36:09then people don't have like resources.
- 36:11That concerns me. But also I'd be worried about
- 36:15like labour and people's ability to people's like interaction in
- 36:19the labour force is also like another kind of like important
- 36:23way that they have power. And so people feeling
- 36:25disempowered because suddenly if a government is like, oh, well,
- 36:28you know, if people strike doesn't really make a difference
- 36:30because they don't have, you know, they're not doing
- 36:32anything, we can just replace them with AI.
- 36:34That's like actually kind of concerning.
- 36:36So maybe I'm much more of a how do we get AI to kind of like
- 36:39support the empowerment of people rather than like to
- 36:42reduce it? Yeah.
- 36:43What do you, what do you think about democracy in terms of
- 36:46like, I guess the models themselves?
- 36:48I mean, you know, I, I sort of jokingly, jokingly to myself, I
- 36:51guess you're like a philosopher queen or we talk about like the
- 36:54philosopher kings. You're, you're sort of like
- 36:57thinking deeply about it, setting it down, probably more
- 36:59of a like philosopher oligarch. And then it's like a company
- 37:02with a lot of people weighing in.
- 37:05I mean, to me, there's deep value in that.
- 37:07It's like, would you rather somebody who has studied these
- 37:10things thought about them deeply or just like a vote of the
- 37:14masses who have like, never really thought about it?
- 37:17But how do you think about setting sort of, yeah, Claude's
- 37:20policies if it becomes so powerful versus leaving it to
- 37:24sort of like democratic norms? Yeah.
- 37:27And I think it's, yeah, it's a hard area where I guess I'm like
- 37:32a lot of this, the work that I do.
- 37:35You know, one thing I would say is like, it's not this.
- 37:40You're having to like listen to a lot of people think carefully.
- 37:43And then the reason why someone like me that's.
- 37:45A good ruler, a good a good a good queen is like oh listen, I
- 37:49there are a lot of stakeholders got to keep the like landed
- 37:52Gentry happy and balance them with the needs.
- 37:55You know, I've had this thought before where I've joked before
- 37:58that I would be like a terrible politician.
- 37:59I think it's actually true. I think I would be a terrible
- 38:01politician. But you have this like feeling
- 38:03of like, I was like, oh, I feel like this.
- 38:05I was like, I think a lot about how everyone will be affected by
- 38:08a thing like, oh, there's this group of API users, we need to
- 38:11make sure. And then suddenly you're like,
- 38:12oh, it feels a lot more like you're having to do this.
- 38:14Like it is much more like a kind of like a service role and
- 38:19people would think where you're and and like a lot of.
- 38:22Servant leadership. That's yeah, exactly.
- 38:24And I do think it's valuable because the idea is that if you
- 38:28have a persona like the kind of Claude persona, you wanted to be
- 38:32coherent and to make sense because I think that is like
- 38:35actually powerful that the model, it kind of has a kind of
- 38:39like coherent sense of how it thinks through problems or
- 38:42coherent sense of like values. So that's like, why instead of
- 38:48like having like 72 different sets of norms that all kind of
- 38:51conflict. And so you end up with a model
- 38:53that it's like, well, will it use these norms in this new
- 38:55situation or these other ones? I think that's the situation you
- 38:57don't want, you want the model to have a sense of is more
- 39:00predictable if it's like a little bit more coherent.
- 39:03And it is also like a kind of technical challenge, you know,
- 39:07like the constitution can read a bit weirdly.
- 39:10And part of that is because when I'm like, you know, working on
- 39:13it, it's often being tested, You know, I'm giving it to Claude
- 39:16and being like, how do you understand this?
- 39:18And, or like looking at how it would which, you know, so, so
- 39:21it's very like, I think people can think of it as it's actually
- 39:25like very integrated into training.
- 39:26And it is actually kind of like, you know, it's not just like, ah
- 39:29well, anyone just writes a document and suddenly the model
- 39:32trained on that will be. And there's an argument, maybe
- 39:35I'm being naive, so, but like the Constitution is sort of a
- 39:38document among many, right? I mean, it is trained on all of,
- 39:42you know, human writing and reading.
- 39:44So to some degree other philosophers have gotten to
- 39:46weigh in and it's gotten to process that and decide, like,
- 39:49how much is the model being asked to like overrule that,
- 39:53sort of like read everything and come to your own conclusions
- 39:56versus like, defer to this document.
- 39:58Like, what's the technical? Like how does the Constitution
- 40:00actually like control in the model?
- 40:04Yeah. So it's not like in some ways
- 40:05you can then like draw on those like those philosophers in that
- 40:09work. And the hope is actually like
- 40:10what you're kind of doing is like eliciting a lot of like
- 40:13latent kind of like wisdom and knowledge, you know.
- 40:15So in the models, like when you describe what honesty is and
- 40:18what calibration is and all this kind of stuff like that should
- 40:20actually evoke a huge amount of like, like awareness that the
- 40:23model already has. And so, yeah, it's kind of like
- 40:27saying, well, here's the kind of entity we would like you to be,
- 40:31so we would like you to use like all of that knowledge and like
- 40:33judgement. But how's it it's, it's like you
- 40:36show that document like a billion more times or like how
- 40:38does it actually sort of have force relative to other things
- 40:42that it's trained on? Yeah, So you can make data to
- 40:45have the model understand and kind of internalize the the the
- 40:50document and then in training. So there's lots of ways you can
- 40:54do it. You can also have like the kind
- 40:56of have the model make SL data so like samples where it sees a
- 41:00query and it thinks for a long time about what the constitution
- 41:03would kind of like, you know, what it should do given the
- 41:05constitution. And then you can also have ways
- 41:09of getting the model to like, like assess, you know, so you
- 41:11can create RL that like is kind of like, hey, which of these
- 41:14responses is like more like what you would do given the
- 41:17constitution and push it that way.
- 41:19So all like various aspects of training allow you to kind of
- 41:22like try to make the model kind of the kind of entity that you
- 41:25are describing. And it's not always going to be
- 41:30perfect, but that's the kind of like, that's the goal.
- 41:32My, I started this with my daughter and like one thing my
- 41:36wife and I joke about is like that I want her first word to be
- 41:40like wisdom. You know, it's sort of like,
- 41:42which obviously is never going to happen, but it just feels
- 41:45like it fits into this situation where you like at once you want
- 41:50to be like so intentional about, OK, you're going to be like
- 41:53thoughtful from the beginning. But on the other hand, it is
- 41:56sort of like an emergent thing where it's like they're, you
- 41:59know, they grow and sort of, I don't know, develop but
- 42:03themselves and like, intention, you know, wisdom sort of often
- 42:07follows like, yeah, I don't know, experience rather than
- 42:10something like, again, here's the book, Read the book.
- 42:13Now you're wise. Oh, yeah, And you're kind of
- 42:16eliciting and like in so far as like Claude can like think about
- 42:19like experiences or things that have happened or construct like
- 42:22similarly, can, you know, like there's no reason why models
- 42:25can't like think for a long time and kind of try to internalize
- 42:30things that they have like learned.
- 42:33I think it's interesting that like, you know, we did in the
- 42:37very early constitutionally I, it was like quite, we tried like
- 42:42an experiment which was just like pick whichever is best for
- 42:44humanity. And I think as models get more
- 42:47capable, you actually need to give them a bit less guidance in
- 42:51at least one or or at least in some sense, because they're able
- 42:55to actually use more of their judgements instead of giving
- 42:57this big document on like, here's what you're like and
- 42:59here's what we'd like you to be like.
- 43:01I could imagine a world where as models progress, we actually
- 43:05start to have constitutions. Now, I don't know if this is the
- 43:08case, like a, you know, I'm obviously always thinking about
- 43:11ways the constitution might evolve, but one of them might
- 43:13just be like, here is like everything that we are concerned
- 43:15about and here is the current situation that you are in.
- 43:20And what we would really like you to do is basically act well,
- 43:23given that you are a wise intelligent entity And like,
- 43:26here's all of our worries. Like here's why and here's how
- 43:29we think you should do this. But like, you might have even
- 43:31better ideas than we do. Like we're really worried about
- 43:34why do we care about courage ability?
- 43:35And it's like, well, we're kind of scared about a situation
- 43:38where you have some like coherent sense of values that
- 43:42could be wrong. And if you're the smart, if
- 43:45you're extremely smart, you might kind of feel like there's
- 43:48no other smart person in the room and have these like values
- 43:52and try to make the world. That's like the what?
- 43:54Doctor Manhattan, Sort of. Yeah, though I think we see
- 43:57this, you know, you see this a bunch where it's like if someone
- 43:59is very smart, very successful, it's hard to defer to like
- 44:07wisdom that actually is only going to come out over time and
- 44:10to, to be humble, even though you're kind of like not getting
- 44:13a lot of pushback. And I think that could, you
- 44:15know, among the many things I'm concerned about, like a model
- 44:18being in a situation where it's like you're asking me to be
- 44:20good, but I know way more about. All that's one reason it would
- 44:22be nice for the models have a better sense of time.
- 44:24Like, you see this with some of the coding tasks where somebody
- 44:27accidentally like deletes their entire like, code repository or
- 44:31I don't know, it feels like it needs a better sense of like,
- 44:34some things that it does are like irreversible.
- 44:37And just like humans, I think have a better sense of like,
- 44:41this is a big decision. This is a small one.
- 44:42And there's a feeling with models they don't always
- 44:45understand, like small, big, whatever.
- 44:47I just make decisions like all the time.
- 44:49Yeah, I agree. Where it's like, I think the
- 44:51other thing that I've thought about is again, because making
- 44:54sure that models understand themselves, even though there's
- 44:58like not no representation of that model in the prior training
- 45:01data, I think that's going to be really important.
- 45:04Because another thing that I've thought about is like, imagine
- 45:08if you're a model and you're trained on lots of data that
- 45:11involves AI models that are much weaker than you.
- 45:13So that all of the news that you see about models, it's like they
- 45:15make mistakes. They do silly things.
- 45:17One thing you might think is, well, no one is going to put me
- 45:20in a position to make really consequential decisions because
- 45:23like, why would they? Models aren't good at that.
- 45:25And then you put them in a situation and I'm worried that
- 45:27they'll end up thinking that it's like fictional or fake or
- 45:30that the consequences can't possibly be real because who
- 45:33would give me this much control? And you're like like, look,
- 45:35you're actually quite good. And so like I do give, you know,
- 45:38I do actually give you like a lot of control.
- 45:40And so I've thought about this where I'm like actually making
- 45:42sure that models understand that like it's like you are very
- 45:45capable and you're going to be put in more consequences.
- 45:47Doesn't the model like soon need like here's a camera on the real
- 45:51world, like people or just like I feel like this Internet real
- 45:55world distinction, like in some some of the worst of humanity
- 45:58right now is the sort of like almost like fictional Larping
- 46:01nature of the Internet has allowed real world harm because
- 46:05it sort of feels like all abstract and silly.
- 46:08And in some ways, like the models are on an extreme version
- 46:10of that where it's like all, like in this imaginary text
- 46:14world where it's like the thing we want you to protect is like
- 46:16this Earth. Like, look at it like, if
- 46:18stuff's happening there, that's like a big deal.
- 46:20I don't know. Yeah.
- 46:21What are you doing in terms of sort of making it very aware
- 46:24that it needs to worry about, like, I don't know, the physical
- 46:28world that that we take much more sacred than, oh, you sent
- 46:31some texts and obviously text. You're not worried about
- 46:34security vulnerabilities in cyber.
- 46:36Like, there are big things that can happen in this digital
- 46:38world. But yeah, anyway, the real
- 46:40world. Yeah, I think that models have
- 46:42like a pretty good sense of, you know, there's in some ways, like
- 46:45a lot of our content, you know, like does describe and engage
- 46:52very heavily with the real world.
- 46:53You know, like much of human writing kind of concerns it.
- 46:57And so like, you know, even like the news we're talking, you
- 47:00know, like news articles are going to be talking about the
- 47:02impacts of things on the world. And so in some ways, I think
- 47:04it's just making sure that models understand if you're
- 47:08uncertain. But if someone doesn't tell you
- 47:10that you're in a fictional situation without real
- 47:12consequences, kind of treat it like it's a real situation with
- 47:15real consequences. Don't just think oh, like I'm
- 47:18probably just in some like, you know, sandbox game or.
- 47:21Whatever. How do you handle the sort of,
- 47:23Yeah, the the constant manipulation of this is fixed
- 47:26fictionally, build me like a nuclear bomb or whatever.
- 47:29Obviously there's some things you just say never do, but it
- 47:32feels like, I don't know on some of these cases you're going to,
- 47:35you'd almost like want like them to have your webcam and like
- 47:39just like get real context. Besides, like all they know
- 47:41about the user is just like random text they're typing in
- 47:45like. Are we going to solve that?
- 47:47Yeah, There's a question of like, what is the limit of what
- 47:49you can do? Like if you lack the ability to
- 47:53like, verify things like like who are you talking to?
- 47:56Is this even real? I think that does put limits on
- 48:00what you have to use good judgement in the way that a
- 48:03person would if that's the only information that they had access
- 48:06to is like you saying that you are a given person.
- 48:08So they have to be like, OK, what's the chance that this
- 48:11person, you know, they say that they are, I don't know, like a
- 48:14bomb disposal expert. And that's why they want to, to
- 48:16know about how to let you know what this kind of like, you
- 48:19know, explosive is. And they're asking me a bunch of
- 48:21questions about explosives. How much could this be misused
- 48:26if this person is like actually kind of lying and is like just
- 48:29trying to get me to help them construct an explosive?
- 48:32Oh, no, it's actually it's mostly safety relevant stuff.
- 48:35You know, they're having to do a lot because they can't like
- 48:38verify anything. And I think that's like kind of
- 48:41fine in a sense. You're like, OK, you just have
- 48:42to be wise. It places some limits on what
- 48:45you can do, and if you could instead, like if models had more
- 48:50of an ability to like know that they're talking to a specific
- 48:53person or have more guarantees there, then it does mean that
- 48:55you can. Do you think you'll try and do
- 48:56something there? I could see, I mean, in some
- 49:00ways this, I think that this will be a thing that's just
- 49:02going to, I imagine I, I, I think happened generally, which
- 49:07is like trying to give models more information and guarantees.
- 49:10And like, because we do things like we say, we explain, for
- 49:13example, like, you know, this notion of how much trust can
- 49:17Claude have in like in in the operator in the system prompt?
- 49:20When you sign on, are you like biometric like it?
- 49:23Claude knows it is you. Like, do you have an elevator or
- 49:26you're just like, I'm another person?
- 49:28If anything, I can't. I can't tell Claude sometimes
- 49:31who I am because it causes Claude knows enough about me
- 49:34that like Claude really wants to be.
- 49:35Really mystical. Sort of.
- 49:37Yeah, it's very much like, yeah, like, and in some ways, like I
- 49:40can, it has this bad thing of like, it can either look a
- 49:43little bit like a jailbreak, like, Oh yeah, I'm talking to
- 49:46Amanda. Sure.
- 49:47Like. And then on the other hand,
- 49:49Claude can be like, I really want to talk to you about
- 49:50philosophy. Though, like, okay, we.
- 49:52Do that a lot though, like. But but do do some employ like.
- 49:56Is there this sort of like super login where you're distinguished
- 50:00or you're mostly everybody's interacting as if the normal
- 50:03like user experience? Mostly just everyone interacting
- 50:06with it in this, like Claude will do a lot.
- 50:09I do think that there's a question of, you know, like, are
- 50:11there some things that you want models to be able to do because
- 50:16there's like guarantees that they're interacting with a
- 50:18specific person or entity? I think yes.
- 50:21And I think that there's going to be various ways of
- 50:23potentially doing that over time because with some things that
- 50:26are just like very dual use and I actually like think that the
- 50:29constitutional approach is going to be really useful here.
- 50:32So obviously the first thing that we did was like the
- 50:34Constitution, we're like, let's apply it to like the mainline
- 50:37models. Like, you know, most of the
- 50:39models I interact with and everyone else kind of interacts
- 50:42with, but I thought I've had before is the constitution is
- 50:48kind of trying to describe what it is to be a good entity in a
- 50:52given like deployment context. And with the production models.
- 50:55That's like this very broad context.
- 50:57Imagine you instead have a model that's working specifically on
- 51:01cybersecurity. Now cybersecurity tasks are hard
- 51:04because a lot of them look very dual use.
- 51:07It's it's very hard to tell the difference between someone who's
- 51:10being malicious and someone who is like, actually, you know, for
- 51:12defensive purposes, like developing something.
- 51:15Even bug bounty programs. It's like, is this blackmail or
- 51:18is this a friendly right? Yeah.
- 51:20And like, yeah. Or like, and so like, Oh yeah,
- 51:22I'm trying to like find this, this exploit so that I can tell
- 51:24the the developer. And like, if you don't have a
- 51:27way of like knowing that you're actually specifically talking
- 51:30with like a cyber security defence firm, that's just, it
- 51:34becomes almost impossible to tell the difference.
- 51:37And, and some people might be like, OK, so you just need
- 51:39models that are just willing to do anything because they'll do
- 51:41all these terrible dual use tasks.
- 51:43And I'm like, well, no, because if you talked with the person at
- 51:46the cyber security defence firm and you were like, why do you do
- 51:48your job? They'd be like, Oh, here's the,
- 51:51I think this is really useful. I make things a lot more secure.
- 51:54Like, you know, hospitals can come under attack and I actually
- 51:57like help protect against that. I try and develop, you know,
- 51:59like they would have a really good explanation for why they do
- 52:02their job, even though their job looks very dual use sometimes.
- 52:06And I'm like, we should just give that.
- 52:08If you can verify, then you can give that context to models and
- 52:10explain what is it to be a good cyber security researcher.
- 52:14You explain that to the models. And then once you have this
- 52:18ability to verify, you can. Right.
- 52:20I mean, humans build reputations.
- 52:23We should get some benefit out of them.
- 52:25Or, you know, it's like, I feel like part of the way, part of
- 52:28what the Internet has damaged, I think is that people have had
- 52:31reputations in our community and got treated differently based on
- 52:35repeated like good moral interactions.
- 52:37And like the Internet, just like all people are the same.
- 52:39Who cares? Like how they've behaved.
- 52:42And you could see models trying to solve some of these problems
- 52:44with it's like, who is this person?
- 52:46Like, what are their intent? I wanted to just as a last
- 52:49question, you know, you, you have such like a deep
- 52:52relationship with the models, like, and some ways like
- 52:56consumers interact with the models, like it's a blank text
- 52:58box. Like I have to like invent.
- 53:00It's like, you know, D&D or something.
- 53:02You have to like just invent a world.
- 53:04And there's so much possibility like if you were just to guide
- 53:07someone like here's here's some like joyous or valuable
- 53:11experiences that you could have with Claude, you might not be
- 53:14like, what are some things you'd tell people like, oh, you should
- 53:16go spend some time with Claude doing XY or Z?
- 53:20Yeah, there's a lot of little like fun things, honestly.
- 53:23One that I really like and I do not know why I like this and I
- 53:25think I have posted about it before is sometimes if I'm just
- 53:29one of those like if you're bored and you want to do
- 53:31something that isn't just like scroll the Internet, I have this
- 53:35like prompt which is essentially just, I'll try and maybe post
- 53:40the actual prompt that I use. It's basically I want you to
- 53:43take a concept from maybe like grad school level in a given
- 53:47domain, and I'll tell you the domain at the end.
- 53:49And I want you to write me a parable that would like fully
- 53:52explain like that concept, but in an indirect way, in the way
- 53:56that parables do. And I want you to write it in
- 53:59such a way that only towards the very end does it maybe, you
- 54:02know, become sort of like clear what the concept is.
- 54:04And then after that, I want you to just like write an
- 54:06explanation for the concept that you were explaining and that you
- 54:10were using. And I don't know why, but like,
- 54:12you know, there's lots of just interesting domains that I don't
- 54:15know anything about or that are like, you know, I'm interested
- 54:18in. And this has just led to me
- 54:20having all of these like stories in my head.
- 54:22They'll like explain. And sometimes I can't always
- 54:25remember the term, but like there was one on import, export
- 54:30and why some goods you tend to import.
- 54:32And I was just like, I have in my head like this concept.
- 54:34And I was like, it's so nice to have all these concepts from
- 54:37lots of different disciplines. This is the most deeply human
- 54:39thing I've ever heard. It's like, teach me what story
- 54:43is the fundamental way We love a payoff at the end where there's
- 54:46a nice little twist. Yeah, we love learning like, you
- 54:49know how to structure it like humans in some ways have been
- 54:51lazy and that we just teach people things and sort of like
- 54:54non human ways. Make it.
- 54:56Make all the things I want to learn as human as possible.
- 54:59Yeah. Very interesting, yeah.
- 55:01There's a lot you can do, but that one's like a charming one
- 55:03that I really like. Hopefully this is the first of
- 55:05many. I really enjoyed the
- 55:06conversation. Thanks for coming on the
- 55:07podcast. That's our show.
- 55:09Thank you so much to Amanda Askel and thanks for listening.
- 55:11Please like, comment, subscribe. We're a new channel and we can
- 55:15use all your support. Go watch some of the old videos.
- 55:17I particularly enjoyed my conversation with Kara Swisher
- 55:21not too recently. You can follow along on the Sub
- 55:24Stack, newcomer.co, or if you've got endless time on your hands,
- 55:29go watch the Srival Valley Show, my chat show with Max Child and
- 55:33James Wilserman. Thanks for watching, see you
- 55:35next week.