Latest / In the Loop AI / We Interviewed The Team Behind OpenAI's #1 Feature

Transcript

We Interviewed The Team Behind OpenAI's #1 Feature · In the Loop AI

• Ejaaz: [0:03] I'm so excited about today's episode. We are joined by Kristina Kaplan and Sameer Ahmed, who both lead memory and personalization at OpenAI. Sameer, Kristina, welcome. Christina: [0:15] Thank you. So excited to be here today. Ejaaz: [0:18] Okay. You are both solely responsible for creating the most valuable moat that's ever been discovered in AI, memory and personalization. In fact, I remember the very moment you guys released the memory feature, because five seconds before that, I was ranting to my friend about how annoying it was that ChatGPT had no recollection of who I am when I opened a new chat. And it was like this light bulb moment went off where I stopped viewing ChatGPT as this tool and more of a friend. And that's an incredibly sticky product experience. Ejaaz: [0:51] But I want to hear about it from your side, what was it like to develop ChatGPT's memory feature? Give us the inside scoop. What was it like to go about creating this vision around memory? Samir: [1:02] I think, thanks for the generous introduction there, Jaz. I think we probably would classify slightly different. And, you know, to be totally candid, memory in ChatGPT predates both Christina and me joining the team. I can rewind all the way back to 2022, ChatGPT comes out, And I'll give you a little bit of an analogy here that I think is a truly apt way of how we think about it. Samir: [1:30] And 2022, sort of like, imagine you had an assistant in a room, you walked in, you asked a question, and it answered natural language. And this was the first time you've seen a computer like answer a natural language and gives you the answer. You leave the room, stops working, your assistant stops working. You walk back in and you say, hey, how's it going? Ask another question. Assistant does not remember who you are. No idea. It's a paradigm shift, but it really is not, you know, the type of experience that you would normally have in a, you know, situation where we're working with an assistant. So fast forward maybe to 2024, it's almost like we gave ChatGPT a notebook and it could write certain things down as you were talking to it. And you would say something, ChatGPT would write it down, would try its best to know what's going on. You leave the room, you'd come back, you'd ask another question, and it would like scour through the notebook to help answer the question it had. And it's a little bit like, you know, maybe the movie Memento, where... You got tattoos on you and you have like some clues, but it's not a perfect picture. So, you know, Christina and I have been thinking like, how can we make this better? There are a lot of people involved here, but that's when, you know, the April memory update. That's around the time we came up with our latest variation of memory. Christina: [2:39] Yeah, so the latest variation of memory, like we had an assistant, a person that had a notebook with information about you, but still very limited. Like might understand your name, but doesn't really understand you or who you are, or really can't carry a conversation from new conversation to new conversation. So the update to memory that we launched in April was really trying to bring a more natural memory to Chachabiti. So taking this like person with, you know, memento style tattoos to be like a real person that remembers you that can like pick up a conversation from where you left off. So that's really what inspired this launch in April is turning Chachapiti's memory into something that's more natural. That is the beginning of what you might expect from like a real assistant that is like the same person in the conversation time after time that you come back and talk to them. Samir: [3:27] Yeah. And that's when I think, you know, that released early April. And that's funnily enough how, you know, we, we, we got in touch because we'd listened to, I think, a podcast you'd done on this. David: [3:40] Yeah, tell us about that, actually. What was it like? So we were talking about memory. I think me and Joss were all very inspired just by what could happen next, what doors this unlocked. You know, in hindsight now, as a ChatGPT user, I'm like, oh yeah, this seems like a very obvious feature to unlock. But before it got delivered, I don't think anyone really knew where the arc of this product was going. Again, now it seems obvious. Maybe you could put us in your shoes as you were listening to us chit-chat about our experiences with memory? What was going through your guys' heads? Samir: [4:12] Yeah, actually, like, Christina had just gone on vacation, and I was just, you know, scrolling through everything on Twitter or looking for different coverage of it, and, you know, the change was not like a visual feature that you could see. And so not everyone picks up on it right away. And I remember someone forwarded me a link to the podcast and I was a little skeptical and I was like, okay, let me read, let me listen. David: [4:39] I listened before Limitless has forked off from Bankless. So it's a crypto podcast. Samir: [4:44] I texted Christina. I said, you got to listen to these crypto bros talk about ChatJPT. David: [4:50] And she was skeptical. Christina: [4:51] I was skeptical. And then Samir basically like bullied me over the course of multiple days to listen to this podcast. He was like, this is like the thing you have to do. It was really interesting to hear you both talk about this launch and to hear, I think like David, you talk about it's a small step, but maybe it's a small step. That's really a big change. And I mean, Ajaz, we like DM'd you on Twitter immediately to talk because we were just so curious, like your reaction was like much stronger. But I'm very grateful to Samir for influencing me listening to this on vacation also Samir: [5:23] Funny story you made us do like a three factor auth we DM'd you and you were like can you prove this yeah David: [5:31] We did a number from our. Christina: [5:33] OpenAI accounts and then I got a mutual friend three factor authentication before Ajaz would talk to us I Ejaaz: [5:41] Didn't believe OpenAI was listening to the Limitless podcast back then But it turns out we say some interesting things. David: [5:47] Yeah, the context for that is that, you know, in crypto, there's a lot of scams. And so when somebody from OpenAI texts you, messages you on a very public social media platform, you're like, what's going on here? Can we color in a little bit about what was the bigger, the broader reception David: [6:02] of memory when it was introduced? Because I think that we're just to kind of jog our listeners' memories, because that was, you know, so many months ago, half a year ago, and time flies. I was skeptical. I honestly can't necessarily remember what I was skeptical about. I know I remember Josh being very excited, but like give us the range of like feedback and reception that ChatGPT users had to the memory feature. Christina: [6:24] This was a change that was mostly under the hood, except we did release a notification to share with people that this change was happening and it kicked people into a conversation that was something like, tell me based on everything you know about me, about who I am and like make it catchy. And I think the reception was twofold. There was like really strong reception, actually stronger than I think we had originally anticipated to that prompt. Like people were really excited to see how Chachi Biti had been understanding them and like really learning about themselves from hearing about Chachi Biti's perception of them. But I think that we were surprised by the reception like a JAWS is, honestly, like, hey, this really changes the experience. Because other than that one notification, there really wasn't much to it visually. And we sort of hypothesized that this natural evolution of memory would be really meaningful. But it was a surprise to us how many people actually like felt that in their experience. Samir: [7:22] And a lot of people will like extrapolate, come to, you know, jump to different ideas from that point in time as soon as they see it. And so it was really, really interesting to just see different takes on this. But, you know, to ground us in like how we think about it, a lot of it has to do with this just a universality of memory and how we can we can sort of meet people where they are. And there's a little element to this around like everyone has a different experience with ChatGPT. If you're, you know, living in San Francisco or if you're living in Indonesia, if you're in Brazil, everyone has a way that they communicate with other humans. And memory helps sort of you know like make the playing field universal so that people all over the world are able to like communicate with chat gpt at the level that they understand and it's not like mapping it to you know how people communicate or computers humans have been talking to humans for tens of thousands of years everyone sort of understands that humans have been talking to computers or interacting computers for 30 years button clicks like widgets and so that's maybe not as natural and not the best way to communicate and tackle higher order concept discussions. Christina: [8:31] Hey Jaws I'm curious to replay a bit of like your reaction because it was very strong and also like how you think about it now a few months later. Ejaaz: [8:40] Okay, so how I thought about ChatGPT pre-memory was it was like talking to a friend that had amnesia. You'd have a conversation, you'd get into a topic about something, and then a couple hours later, it'd be like, hey, who are you? Tell me about yourself. And that kind of broke the illusion. Maybe illusion is not the right word, but it kind of broke the experience a little bit. And so when I saw that memory update on my app, what you described as a kind of small internal under the hood update meant a lot more to me because I knew that over the next few days, I'd be having less of those, hey, this is me conversations and more of just conversations that flowed into something much more bigger, that compounded. David: [9:23] When you guys think about ChatGPT and the evolution of the product, I'm a psych major and I find it very easy to put into metaphors a lot of how ChatGPT works, or at least how I understand it to work. So like the short-term, long-term memory. Like the short-term memory is literally what is the last four prompts that the user and I have just been discussing. So let's retain that information first. But also in my context and my response, and let me also understand a little bit about the deep truths that I know about this user and their interests and their hobbies. And I will frame my response based on like the long-term memory that I understand the user to be. When you guys are thinking about ChatGPT, do you think of it in a cognitive psychology lens? Samir: [10:10] Yeah, we look at it, I think, you know, we try our best to look at some of the prior art here. And in that scenario, it's really helpful to understand like how humans interact with humans and some of the advantages and pitfalls there. And Chachi's memory is today nowhere as good as humans memory and sort of understanding the gist of people, interactions and fidelity and the triggers. So there's a lot that we can draw on from the sort of existing cognitive precedence. Christina: [10:39] It's like a much more natural way for people to interact with with something as well like on the other end of this is a person so a person like I am interacting with Chachi BT I interact with like many other people it's much more natural to interact with something that you expect to be an assistant in a way that you would interact with another person so I think a lot of the like at the end of the day this really comes back to like not Chachi BT but the user like how do people expect something to interact with them and like how do we meet everyone where they are and like what they expect as the user. David: [11:10] I can't exactly remember what I was so skeptical about. Clearly that skepticism has dissolved because I remember just maybe one, two weeks after this feature got released that I realized like, oh, this product gets better the more I tell it about myself. And a lot of. Samir: [11:26] People had reached out to us further down the line, you know, weeks, months, and then mentioned that this is one of their favorite features. And, you know, I would always ask, is it a single moment of magic or is it a slow burn? And, you know, different people give different responses, but for some people it's a slow burn. Like they just realize that things have shifted and like ChatGPT is behaving in a way where clearly it understands them. And I think it has to do with maybe how you're using it. Everyone uses a different fashion. David: [11:55] Yeah. And this feature, the memory feature unlocked, I think what is many people's, not favorite prompts, but I think a prompt that many people have at ChatGPT, which is based on everything that you know about me, who do you think I am? Or what do you think my personality is? Or some like hold up a mirror to me and tell me what you think I am. And so it kind of turns ChatGPT into a diary in a sense. It's like I have like queries and prompts like help me plan a trip that I want to make. But also the trip that I want to make tells Chachupiti a little bit about myself. And so every single prompt, it's like a little bit of just like, here's one more shade of who I am. And it really encouraged me to like, you know, give Chachupiti all my data, tell me all my interests, tell me the secrets that I've never told anyone before. Yeah, talk to us about the arc of how user behavior changed with the introduction of the memory feature? Christina: [12:52] It's a good question. I actually would say my user behavior has changed much more with, well, I think both with enhanced memory, but also Pulse, which came out last week, which is a kind of like natural evolution from memory as well. But I think with memory, like I started sharing with Chachapiti more context. So instead of just going there with a question that I wanted an answer to immediately, like I knew that this context would come up in like future conversations with Chachapiti. So if you share like hey I'm looking for a fish forward restaurant for tonight like help me find that it's kind of unclear to Chachabiti like why am I looking for that am I looking for it because like I'm pescetarian and so it shifted my behavior into be being more along the lines of like hey, I'm pescatarian. I'm looking for a restaurant tonight. Can you like help me find something? And like, I've started sharing facts along the way, as opposed to just like going in with a question like I would in like a traditional search product. And then like, down the line, as Samir mentioned, it's like a slow burn, you see those facts come up in the future. And so now Chachapiti, if I'm like asking about a restaurant, it will suggest like, fish forward pescatarian restaurant friendly restaurants. Christina: [13:58] And, and I can just like expect that now. And so I've started just sharing like facts about myself and contacts that I like expect to be helpful in the future. It seems that this this Ejaaz: [14:08] Approach with Pulse and with the data you guys are amassing is solely meant to kind of help a better user experience for the person using ChatGPT, right? It's like this personalized experience that helps them improve and get better. But I do also want to address the elephant in the room, which is you guys now own so much data, and not just any data, very personal data. I tell ChatGBT everything and very willingly, very consentingly. And I have to ask, what are the plans with this massive honeypot of data? Are there any kind of like thoughts on new products or maybe new experiences that you could do with something like this? Christina: [14:44] Yeah, I mean, I think we're very focused on like helping people achieve really meaningful things in their life. I Definitely, you know, Sam has said to like Chachupiti will probably be the most like sensitive account that people have. And like we take this very, very seriously. But there's also so much to be gained for this. Like I have many personal experiences in the health domain. For example, I was going to get vaccinated for a trip I went on in July and I had already shared some of my prior like vaccination forms as well as my labs that I had done with Chachabiti. Christina: [15:21] And I asked the nurse that I went to go get vaccinated with like, hey, what vaccinations do I need to go to this trip in like a month or so? And he gave me four vaccinations that I needed. And then I also asked Chachabiti like just to check and see what Chachabiti thought. And Chachabiti gave me five vaccinations that I needed. I was like, what is the difference here? And the nurse had looked at my prior vaccination forms and Chachabiti had pulled out that I had low immunity to varicella from a lab I had done early last year and was like, based on your vaccination forms, you have these four that you need. And then you also have low immunity to varicella. So you need to add that one as well. And I think that's an example of like such a sensitive use of my data. Like I have literally uploaded all of my like labs and vaccination forms to Chachubiti, but it's helped with such a like individualized outcome. I wouldn't expect anyone to be able to go look at every single form I've ever shared with them before for every single like doctor's appointment. And so now I know that I can like go to Chachubiti and ask like, hey, I'm going to this doctor's appointment, like what's relevant for that. But at the same time, like I am a user of ChachiBT sharing my like health information with ChachiBT. And I think we take that like very, very seriously and want to make sure that people are able to get a lot of value out of ChachiBT in the way that they want. But... Christina: [16:45] Yeah, at the same time, like really taking data privacy and like security really, Christina: [16:50] really seriously in the work that we do. David: [16:52] So I could imagine some property, some thing like sign in with ChatGPT on this website and I sign and I go to some non-open AI website and I sign in with ChatGPT. And now it's my agent, my LLM, my chat that is piped into this website, my identity signed in with this website. And all of a sudden this website can query is like, hey, what does this user want to see? And my LLM could answer them. These are just my thoughts about what you could do. But there is some notion about like the data and knowledge that ChatGPT knows about me could be extended to the broader internet. And the internet can finally be truly personalized to my interests. When you guys think about the product, how over the mark am I? Samir: [17:37] I think that, you know, if you look at it back to our like assistant intern analogy, interns in the room you go in you talk about some stuff you mention the you know in my case that like I recently had a son who's nine months old now and I come back tomorrow and the assistant's like oh hey like how's it going maybe here's a tip around like what you should expect in terms of like the development cycle and like when they're speaking or they're babbling this is what it means if we extrapolate that you know and we just keep following that arc I would love for that assistant to be able to go into the real world and, you know, do things on my behalf or represent me on my behalf to solve my goals. And it's just, you know, today Chachuiti as a property can deliver so much value. But if you look at the arc of this, there's things that happen all over, all over the internet. And I think like aspirationally, if you had an assistant, you'd want them to be able to go and solve problems all over the internet. Ejaaz: [18:35] Talk to me about how you align an AI towards a human's goals, though, right? Like when I think about like learning David: [18:43] About men for. Ejaaz: [18:44] Us. When I think about meeting a new person, like making a new friend or learning deeper about a person, it takes so many different types of inputs. I'm looking at the expressions on their face, listening to the tone of their voice, and then I'm listening to the actual words that they're saying, not to mention that they have a lot of past experiences that influence the words that actually come out of their mouth. When I think about the chat GPT relationship, I'm just, I'm slamming a bunch of letters. Sometimes they make no sense. Sometimes there are a lot of typos. How do you process that into an AI and say, this is the thing David needs to do, or this is the thing Ejaz needs to build? Christina: [19:21] Well, we are at an amazing research lab. So one thing that's been really special about the personalization team too, is we have like an end-to-end research product team. So we're able to make progress really quickly on the research side based on what our goals are in the product. And we have, as Samir said at the beginning, it really like takes a village to make personalization happen. And we have some amazing researchers working on this problem. Samir: [19:43] And it works the other way, too. From the product end of it, if you pull out ChatGBT Pulse today, you know, you get to the bottom of the page and it will say, you know, curate for tomorrow. And that's effectively, you know, a goal that you could set for tomorrow or something that you want for tomorrow. I mean it doesn't have to be for tomorrow necessarily you could set a longer term aspect like I mentioned like oh keep me up to date with developmental milestones or it could be somewhat you know trivial like I'm a Formula One fan and these races happen like at absurd times in the morning and I'll always say like keep me up to date with Formula One news but do not spoil the race for me otherwise I'll be very angry and there's no like that's that there's no way you could communicate that in any other tool that has existed prior to ChatTNT and get that level of like, you know, captures level of nuance to it and provides me utility in that case. Christina: [20:40] Yeah, I already am doing this with like the curate function and impulse is as Samir mentioned, like I'll say, hey, I'm going to a trip next week. I'm like going on a trip next weekend. Help me plan for my trip. I'm like, that's a goal I have to like plan a great trip to like London for a wedding that I was just at and Chachubuti is like sharing like coffee shops and pescatarian restaurants that I might like and helps me kind of like look ahead and see like what's upcoming in my life. And I'll just like, start sharing things like this is my goal. This is what I'm doing and sort of expect ChatGPT to like meet me where I'm Christina: [21:12] at and help me help me do that better. Ejaaz: [21:14] Let's get into Pulse. This is a brand new feature that Sam Altman announced and credited you both specifically as the next step in memory and personalization. I kind of think of it as like my own personalized ChatGPT. I've gotten into the habit every evening, actually, before I go to bed of talking to it. Samir, you just give an example of saying, you know, tell me about the F1 news, but don't tell me who won because I want to watch the race. I've been doing similar things for all the other tropes of my life. You've explained kind of how it works, but let's pop open the hood. Like what's actually happening underneath that? You've explained memory and kind of like putting together the identity behind David, the identity behind Ejaz. Tell us about this next step of personalization and how it works. Christina: [22:00] Well, yeah, first of all, this is like a feature that took a village and we have an amazing, amazing team that that helped build Pulse. But it really is that next evolution of Chachi BT. So you have an assistant that you go into the room, you start talking to them and now they know who you are every time you come back. But when you leave the room, they stop working. And so you have an assistant that basically like only works in your one on ones effectively. And like that's not someone that I would hire to be my assistant. It's someone that might know me and understand me, but only do work when I'm there. that I generally have to watch them do. Christina: [22:32] And with Pulse, we started to think about how can Chachapiti in understanding you, understanding what's important to you and your goals in life, help you when you're not there. So you don't have to spend all of your time in Chachapiti. And the goal here is not that people are spending all their time in Chachapiti. It's that actually we're more successful when Chachapiti is doing things and helping you and you don't have to be there. So you come back every morning and there's like new value for you to get from your assistant that was like effectively doing work when you were there. So basically behind Pulse is like, we try to understand, you know, what's going to be important to you in the next few days, upcoming. We introduced calendar and email connectors as part of Pulse as well. So not just looking at memory, but also, you know, what's upcoming in your calendar? What's just happened in your email? And how can ChatGPT like understand all this information, understand what's important to you and then help you prepare for your day? Samir: [23:28] It is important to note that, you know, it jazzed when you wake up and you scroll to the bottom of your Pulse page, you know, it ends. And so to sort of reiterate Christina's point there, we are trying to get you prepared for the day, help you accomplish whatever that goal is, whether it's a gym goal or it's a, you know, some sort of event you're planning for, and then, you know, get back to your day. And if we do that then we can take the time in the background with chat gvt can figure out what's important and you know spend the rest of the time that you're not interfacing with it getting work done which is kind of what you would want from most of your assistants yeah Christina: [24:05] Yeah, you can kind of think of it as like while you're asleep, Chachupi Tea is like trying to process all this information you shared with it to understand how it can help you the next day. So then you wake up and there are like things that are ready for you that right now are mostly content-based. So like what can your assistant share with you that's helpful? But you could imagine you wake up and your assistant's like, hey, I wrote this email for you. Do you want to send it? Or like, hey, I, you know, prepared this for you. Do you want to go do that? And so I think the goal here is like becoming more and more like helpful and actionable over time. David: [24:38] This is a side quest of a question. We're going to get off the main quest here. But I've heard the word like overnight, like ChatGPG thinks overnight. That was in some of the release documentation. And you guys have used it a couple times here. Is that intentional or is it David: [24:52] more just like just while you were asleep, while you were away? Or is there actually like maybe there is a lull of when the GPUs are just quiet because everyone's asleep. And so that's actually when we have the extra resources to apply to some of this kind of stuff. Or am I just reading into this too far? Christina: [25:08] We do have some more flexibility overnight, but the current instantiation of pulse is to is to help you prepare for your day. So like you're not there, you're not interacting and engaging with Chachapiti. We're like doing work in the background while you're asleep to help you prepare for your day. Samir: [25:22] Yeah, at least for myself, I always feel like, you know, speaking personally, that when I'm asleep, I'm not I'm not being very productive. Though, you know, sleep is a very important part. And so there's definitely an element to like, if you wake up and someone has worked for you on what matters to you, and that's what you can start your day with, that feels like something that's clearly a step different to how you would operate in the past. Ejaaz: [25:47] I find it interesting how you both describe the Pulse experience, because when I open up my phone, when I open up the ChatGPT app, I have the memory setting, right? And I can go in there and tweak it a bit. But Pulse is kind of like its own app by itself. And so that kind of like bodes me to think or ask you, are these two separate products that are going to like live on their own and go off in completely different directions? Or are they very closely intertwined and maybe they end up merging into some coherent product experience in the future? Christina: [26:21] Yeah, they are aiming to basically like augment the ChatGPT experience as like your cohesive assistant. So I think it is supposed to be like these individual things that are laddering into like your personal assistant. Like your personal assistant needs memory. Your personal assistant needs the opportunity to do work for you when you're not there. And that's like the early instantiation of pulse. And then your assistant needs to understand what you care about and what you're trying to achieve. So it can do those things like on target that's like helpful for you. Samir: [26:52] If you had an assistant who could only do work overnight, but then when you talk to them during the day, didn't remember that or vice versa, when they did work overnight, didn't reference what you discussed during the day, that just wouldn't be, you know, an intuitive experience and it just wouldn't be helpful. So I think like... Foundationally, everything is a function of the memory and our understanding of how Chat2PT understands you. And so I imagine that as it evolves, Chat2PT can take different shapes, but you can envision that your assistant will be there and understand it, regardless of what format you're in. David: [27:33] Samir, you said you'll notice that at the end of your notifications, your morning pulse notifications, that it'll stop scrolling. And reading into that, I kind of see that as we're not here to make you addicted to ChatGPT. We are here to deliver you value and then be done. David: [27:51] Is that a part of opening AI culture? Or is that just something that you said? How far does that ethos kind of permeate through the organization? Samir: [28:01] We've said this in many different forums that ChatGPT is here to help you accomplish your goals and not to hold your time or attention. And I think the way we've designed Pulse sort of mirrors the ethos that, you know, we espouse throughout all of our ChatGPT decision making. Christina: [28:19] I really do think we're like maximally helpful to you if ChatGPT is doing a ton of work for you that's really valuable and helps you do things in your real life. And you're not spending that much time with ChachiBT. So like the hope would be you can share more context with ChachiBT. You can share more information with ChachiBT, but you don't have to be like always coming to ChachiBT with a question that you expect like an immediate response to. And so you wake up, like what I would like out of Pulse personally is like I wake up and then ChachiBT has done a bunch of things that I could have done myself, but maybe like didn't need to spend time on. And then I can just say like, boom, boom, boom, like those are done and get out of the app. Like that would actually be the most valuable to me. So we are very focused on like in an ideal world, ChatGPT is like doing a bunch of work on your behalf because it understands you, it understands what you care about. And then you're getting in there to like help steer it to the things that are most impactful to you. But you don't have to spend all of your time there. David: [29:15] I've noticed that, at least with me and my relationship with ChatGPT, is that I am a producer. I tend to be a producer when I engage with ChatGPT as opposed to Twitter, Instagram. I tweet a lot. And so maybe I'm a producer in the Twitter context too, but I'm also a big consumer. Like I read more tweets than I write. And on Instagram, I'm almost primarily a consumer. I'm just brain writing away, consuming the reels. But when I open up ChatGPT, I do a lot of my writing inside of ChatGPT. And in fact, I do a lot of writing at Bankless, a newsletter or in other contexts. And the next article that'll come out of Bankless. Christina: [29:52] The first half of that article was me actually just starting a prompt with ChatGPT. I was like, wait a second, David: [29:58] My prompt and this article are highly overlapping. And so I've just personally noticed that in my explorations on a line of thought or almost anything, I'm producing something, even if it's not the final form. It's like an anomaly that my prompt in a newsletter is kind of going to be the same thing, but like, nonetheless, the net vibe is I'm trying to produce knowledge and my time spent in GPT is productive. David: [30:27] Once again, not a question here, but just like, I don't know, thoughts, thoughts or reflections on that kind of like dynamic relationship. Christina: [30:33] I think we hear that from a lot of people. And like, it's definitely my personal experience too. When I go to chat GPT, I'm trying to like stay fit and make sure I'm like finding time every day to do so in polls I actually asked for every day I'd like a 10 minute Pilates routine because sometimes I only have 10 minutes and so I wake up and that's ready for me and I can just like jump into that when I'm up in the morning or I'd like to learn more about this thing over time and Chachabiti can like come back and help me learn about something I'd like to prepare for my we have a team like fantasy football league so I have Chachabiti like cooking on my weekly lineup to help me know if like there's anyone I need to sub in from my bench. And I think that's more of like it is productive. It's like productive time that I personally spend in Chachabiti. And like everyone's different. I don't go to Chachabiti to write. Sometimes I go to Chachabiti to code, but we want to help people like do more in their lives in ways that really adds value to them. Ejaaz: [31:28] So it sounds like Christina: [31:30] From what you both are saying, Ejaaz: [31:31] The ultimate vision of personalization, memory, Pulse is to just help the user, help the chat GPT user improve their lives, be that personalized assistant. And I guess the immediate question that jumps to me is, okay, let's take Pulse, for example. Ejaaz: [31:48] What's it doing for you both specifically now? And what could I expect that it would do for me maybe six months from now? What does that trajectory look like? Samir: [31:58] Yeah, at least for me right now, like I mentioned, other than keeping me up to date with Formula One and not ruining races for me, you know, I've been trying to set a number of miles I want to run in a week. And I just have pulse check in on that every day. And otherwise, it's a little bit, you know, out of sight, out of mind, I get distracted with something else. That's like a very simple example. A broader example is, you know, trying to be a better parent and understanding it's like my first kid. And so there's a lot of things you learn, you know, going through the process trial by fire. And so it's been I've been using ChatTBT prior to Pulse on this. And so that's a large part of my life. And it helps me with that. I can imagine the future. It's able to do more, you know, think of the things that ChatTBT is able to solve for you today. There's the entire Internet of information out there. There's things that are in my, you know, in my email and my calendar that could figure out. Who could figure out a way to like maximize my time so that I spend more time with my son on weekends. And so I think if you, you know, as you mentioned, it's about like being able to solve the true goals that people have. And so a lot of them just look like what we would do next to be able to solve that goal. And Christina gave a great example around like, you know, being able to like process my emails and let me know what's important be very helpful for me as well. Christina: [33:20] Yeah, I can pull up my pulse today. So I have my like quick Pilates flow that I get every single day. I did that this morning to kind of like ease into my day and like, and the goal is to stay fit when I'm in like a busy, a busy job. So I think for me, Chachupiti right now via Pulse is helpful to like take things off my plate, like save me time basically every single day. I also have Chachupiti keeping an eye on SF restaurant openings. So I love to like go to a new restaurant. We do team dinners and on multiple occasions in the past month, we've like found a restaurant through Pulse because I had asked Chai Chabiti to keep an eye on like restaurants opening up near our office. So it really, for me, ranges from like very meaningful, like health and personal goals to like. Christina: [34:07] More light, lighthearted, but still like changing things in my real life. So at this point, I've like really changed multiple things I've done. I've done in my life because of Pulse, but I think it's still very limited. Like it's mostly content oriented, as we mentioned right now. And what I would really like Pulse to be doing in six months is queuing up like things that I would have done that Chachabiti can take off my plate. So waking up and saying like, hey, here's a response to this email. Hey, here's a response to this text message. Do you want me to just like go ahead and send it for you or your upcoming trip. I didn't just find a few restaurants. I planned like a whole itinerary for you, an agenda for you while you're there. So I think as our model, like capabilities get better. And then as we learn from user feedback, like that is the trajectory that we're going towards is this like Chachapiti can do more for you when you're not there. And ideally in ways where it can start to like act on your behalf or help you take actions faster. David: [35:02] What I see here, and it's not going to be a perfect analogy, but just like we're where Chachi PT is unlocking parts of like the brain, the tech tree of the mind. Like we've got short-term memory. Now we've got long-term memory. Now it's, you know, continuing to think even when we're not there. So we're just using more parts of the brain or more total capacity. David: [35:23] What are you guys working on next? Like what's the next part of the brain that gets like added on to this assistant that is becoming more and more useful? Christina: [35:32] What would you want from Chachi PT next? David: [35:36] I think I'm kind of ready for it to escape the form factor of a website. I find sometimes like I just won't use ChatGPT because I have such a long thought. They're like, oh, it's going to take me so long to type that out. And yes, I know you have the voice feature, but like I got it. I just sometimes I feel like ChatGPT is a little trapped inside of this kind of blank front end that doesn't get me started. Like when I go to ChatGPT, it's like, tell me what you want. And I'm like, I bet a lot of people want to be told what they want, actually. So there's one notion of this, some people don't know what they want and they need help to get there. There's one notion of the form factor is kind of trapped. It's like a constrained form factor. What do I want, though? I think I could meditate on that and come back with a better answer after a while, though, as well. I don't know if you have any thoughts or reflections on that. Samir: [36:27] One of the things that's most exciting to me is, and you see this today with ChatGPT, is that it's just able to connect the dots between different domains in your life really effectively. And so, you know, I will have my calendar connected to Chachapiti and it'll figure out that I've, you know, I mentioned I need, I want to go running. It's like suggests some spots during the day where I can do it. And it knows like where I am that part of the day. And it's just running routes near there. I've obviously like, I've talked to it a lot about this and asked about this, but the fact that it's able to like blend these things in, I can just imagine, you know, in six months it would Christina: [37:05] Ideally be able to solve problems that, Samir: [37:08] And like connect dots between different parts of my life, whether it's like lifestyle elements, like Christina mentioned, sleep, you know, work elements of it. And so I think that's, that's the most exciting part. And at some point in time, it'll be connecting dots that I can't connect for myself. And that's like, you know, if I could explain that today, I would explain it. But I think it's really, that's the magic of ChatGBT in this case. Christina: [37:27] Yeah. It's already hard for me to do that. Like I have a CGM, like a glucose monitor right now. And I have like various things to track my sleep and I have to like screenshot and then like overlay and also say like, I have these things too that are going on in my life. And there's just so much like data. It's so exciting that people have an opportunity to like own their data and like learn more about themselves, especially like, yeah, I just, I use like so many different things to help me understand like how I can sleep better, mostly stay in shape. And I just don't have time to connect all the dots. So I'm constantly like screenshotting and uploading to ChachiBT, but it would be great if ChachiBT could just proactively like connect all these dots for me. David: [38:07] I have the same experience of like, I just went on a run. Let me take a screenshot of Apple Fitness and tell ChachiBT how I think my RAN went. And then it will have some sort of interpretation about how I think my RAN went from my interpretation. And then also the screenshot of the data, which has to be a terribly slow way of getting to that conclusion. Christina: [38:27] Yeah. Or like when I get a DM spike, it's like, wait, what did I eat at that point earlier in today? It's like the data is all there, but it's not connected. And it's like so hard for people to connect those and like understand their lives. So I personally, that's like what I would want to build for myself. David: [38:45] Yeah. Yeah. Well, I'm sure the app that you use on your glucose monitor, I don't know if you're using levels or not. I use levels in the past. If you like have glucose spike, I'm sure that app would be like, yo, tell me what you just ate. And if we have a trusted assistant actually asking you that question, David: [39:00] that seems like the right UX. Christina: [39:02] I just get the, I just get the, like, your glucose is like way too high right now. And then running between meetings, like trying to figure out later in the day what I ate at that time. Yeah. David: [39:12] Since we have you guys here, I want to ask about a thought, a thought experiment that I've had a theorized product feature, which is. Christina: [39:21] There was this one time, this is going to be a weird example of how to get here, David: [39:24] But when Dolly Parton played at the Super Bowl halftime show, there was a meme that went around on Twitter, which was immediately five minutes after she started playing, the query of how old is Dolly Parton just went completely viral. Because everyone was asking, how old is Dolly Parton? And so that was something that was actually kind of nice to see on Twitter and also in Google. Like they had all this data. It's like all of a sudden, 20 million people are asking the question, how old is Dolly Parton? I would actually love to have this feature in ChatGPT where like I'm asking this query, but like what are other people querying right now? Like what are the trending queries that is uniquely trending in this moment of time? And so right now my relationship with ChatGPT is like kind of one-to-one. I have my assistant. It has me. But like, what about the hive mind of everyone? Samir: [40:14] That's an interesting request, Dave. I think that, you know, one of the most important things about ChatGPT is that it's really between you and ChatGPT. David: [40:23] Yeah. Samir: [40:23] And so I think that's what makes it special, makes it understand you, makes it solve your problem. And you know there's there's probably like you mentioned cases where if chachi witty knew you were watching the super bowl and knew you would be interested in dolly pardon's age then you know ideally you know it would be able to answer that question or think of that question and get an answer for you beforehand but you know if you knew the age it wouldn't have to disturb you with that either yeah it's David: [40:53] Not it's not necessarily about just like it's more about just like understanding the global internet traffic and global demands of everyone and just like all of a sudden the whole earth just got curious about this one thing or or pointed in the loose direction and i think like having like a feedback loop between not just not just me and my assistant but my my assistant like one way to be an assistant to me is to tell me it's like yo here's a trend you might be missing out on it let me inform you on this trend and then you can opt into that trend if you are ready to opt into that trend or not. Christina: [41:27] Yeah, I mean, my honest reaction here is like we have, similar to what Samir said, like really leaned into, it's like you and Chatubiti. And so we have not, like I have not spent a lot of time thinking about like how to connect the dots across different users. Maybe like if that is useful to people, like this is like really interesting to hear your feedback here, but we are very focused on like Chatubiti is like your Chatubiti. It's like completely aligned with you. But as that could be helpful, it is interesting to think about like how does ChachiBT like go into the real world for you. But yeah, so far we're very focused on you and ChachiBT. David: [42:03] Which I appreciate. Ejaaz: [42:05] So here's how I think about ChatGPT. And please don't take offense to this. I think it's like the algorithm for my life. That's how I want to treat it, at least. So I'm giving it all this information and kind of any medium that I interact with, whether it's on a screen, whether it's on a different app, whether it's in person, I kind of like having this assistant around that is able to kind of make me better, critique what I'm doing, or just kind of like guide me in general. You keep using the words assistant when you describe this kind of ultimate manifestation. Like, is it an assistant? Is it this agent? I keep hearing about agents doing things for me and doing my job whilst David isn't looking. And so, you know, being an autonomous version of me. Is that the ultimate realization of what you guys are building? Is that the the end form factor? Christina: [43:00] Yeah, I think so. And actually, this jog is like a memory in my mind from your original podcast when we shipped the update to memory where I think someone's on on the podcast said something like oh chat to bt will be like a digital clone of you and like go be you in other spaces and that is like personally not how I think of it like we do think of chat to bt as like your assistant that is not like I'd be curious like your your thoughts on on this point as well but yeah we think of chat to bt as like your assistant that really understands you and is completely aligned with you, but is like a different entity than you. Samir: [43:35] Yeah. David: [43:35] I think assistant is broad enough that it captures a large, like possible venue of actions. Another concept. Christina: [43:42] That I've used is David: [43:43] Like my representative. Like if I am away, can my chat represent me in a particular context? I know there's like the theorized idea of like there's dating apps. And before I go on a date with anyone, my chat will talk to their chat and those two people will represent their respective users and say like, yo, are we a good fit? And like, sure, that's an assistant too, but also representative as well, like my advocate as well. Again, everything kind of like collapses down to assistant at the end of the day, however. Samir: [44:12] We definitely think about it in the sense of like, you know, your assistant sounds like it's assisting you, you know, not sort of like shooting the basket itself. But I imagine like GPT-5 is just so brilliant and if we're able to scale intelligence so many different domains, and Christina: [44:28] I can easily imagine that like you would have experts working on your behalf. Samir: [44:31] And if you had a dating expert working on your behalf, there is a future where that would simplify things for you as well. But I certainly think that like, you know, assistance is a word we're using now, but we're creating new experiences and we're getting to the point where, you know, ChatGPT could discover new, make scientific breakthroughs. And if they're able to do it on your behalf, that's also, you know, sort of the future you'd want to be in. Yeah, I just Christina: [44:55] Really love representative or delegate it because I think it implies a sense of Chachupiti is like on your side. And that is like mainly what we're trying to like build Chachupiti into is like this thing that understands you, understands what you care about and is like on your side. Like in a situation where Chachupiti is, Chachupiti would act like you would act like with your like best interest in mind. So I really love those terms. David: [45:21] Yeah. And it's also loaded with trust as well. Like I trust my representative. I trust my delegate. Christina: [45:27] Yeah, there should be no like misalignment between like you and your representative, you and your like delegate. I think that's like the main thing we're spending our time on. And I love those terms. But that's the goal is to build ChatGPT into sort of like your representative. David: [45:42] Yeah. As chat gets more intelligent, though, like assistant is very much a subordinate. David: [45:47] Like my assistant is like less successful than I am and less intelligent than I am. But as we get into chat gbt678 all of a sudden that's going to if it's not already it's it's going to be flipped and it's all going to be like a more of a a mentor than it is an assistant. Christina: [46:05] Yeah i don't think i mean there are a lot of brilliant assistants out there and so i don't know that i think of assistant in the same way as you do but i do really love like representative for that for that reason because it really should be like kind of an ex like i think of Chachabiti is like an extension of me I'm like I'm comfortable with this thing like pushing me and really like helping me do what's important in my life and like yeah David maybe Chachabiti is like yelling at you to do the things that you care about because that's what you need David: [46:33] That's how I get motivated you know. Christina: [46:34] Yeah like I need to like push in my life too so I'm definitely more comfortable with like myself like a self-extension of me but not like a copy of me doing that necessarily Ejaaz: [46:46] Okay maybe a hot take but i i don't entirely agree with you guys all three of you on on this one okay what's Christina: [46:54] Your hot take Ejaaz: [46:54] What well it's like yeah i want a representative yeah i want an assistant yeah maybe i want someone to to yell at me in general but the only way this is going is ai gets smarter than us. It knows me better than I know myself. And I want it to kind of put me eventually in a position where we either work together or like, I'm kind of like working with it under like an employee role or whatever that might be. And that's an extreme example. That sounds pretty like doomerism, but I kind of view as that being the ultimate form of AI. And then number two is, I think my AI or whatever this personalization thing becomes, should be able to interact with other people. Because that's how I learn in general, right? Like I talk to people, I have conversations, I share social content, I learn from different books and different things and thoughts and ideas that everyone else has had. And I would like a form of my chat GPT to be able to do that. And it sounds like what we're building right now is very like personalized to you, private, which I love. I come from the crypto background as well. So it's very important to me. But I wonder if there's like some kind of social network that might be able to kind of build upon that. Christina: [48:06] So for the first part of this, you said an employee relationship. Do you mean like you would be Chachabiti's employee? David: [48:14] That's what I heard. Ejaaz: [48:15] Yeah, yeah, in a way. And I know I said that intentionally to make it, because it sounds weird, right? It's like, why would I work for an AI? But I want to push back and kind of say, well, if the AI is smarter than me, is all encompassing, sees all this amazing data, can pull together all these connections, wouldn't that kind of make Christina: [48:32] Sense down the line? Ejaaz: [48:33] Is that weird? I don't know. Christina: [48:38] Yeah, I don't know. I mean, I do personally still think of it as like Chachabiti can be your super assistant. Like it can do more than you can do. But if it's like helping you do more in your life, then ultimately it's like aligned to helping you, you self-actualize effectively. So I mean, how, how it ends up like playing out. I think our hope is that yeah, Chachabiti will basically help you self-actualize the things that you really care about in life and maybe that means doing more than you were previously capable of doing but still like as your representative yeah Samir: [49:11] And if that means it's you know giving you work a jazz and that's your your self-actualization Ejaaz: [49:21] God i'm gonna be the first slave for agi and David: [49:25] You're gonna like it. Ejaaz: [49:26] Christina samir thank you so much for spending the time. I got to say this hour has absolutely flown by. Thank you so much for making the time in what is evidently a very busy time and era for OpenAI. Thank you to the listeners for spending this time with us. It's been a dream to have OpenAI folks, specifically the personalization and memory to come on and dive into the things that we're interested in. And the general hope is that we keep this relationship going. When AGI does eventually become my employer. I want to talk to you guys about that. And maybe that becomes a therapy session. David: [50:00] I have no idea. Ejaaz: [50:02] I have no idea. We're here. Christina: [50:05] Thank you. Ejaaz: [50:07] Thank you. Thank you so much. If you have feedback on this episode, comments, thoughts, disagreements with my view on AI employmentship, please comment, like, subscribe, or send hate mail. And we will see you on the next one.