Latest / In the Loop AI / OpenAI Announces the ChatGPT Agent: Everything You Need to Know

Transcript

OpenAI Announces the ChatGPT Agent: Everything You Need to Know · In the Loop AI

0:03
Josh: So Ejaz you're wealthy you've just won the
0:06
Josh: lotto you have all this money what's one of the first things you
0:08
Josh: do and I know you're probably going to say a lot of answers but there's one particular answer
0:12
Josh: that I want you to say which is you're going to get yourself an assistant right
0:15
Josh: to kind of help you do all the things that you don't like to do they're going
0:20
Josh: to manage your calendar they're going to order your groceries they're going
0:24
Josh: to book your flights they're going to take care of all the essential annoyances
0:27
Josh: that you just don't want to deal with.
0:29
Josh: Now, what's cool about the news from this week is that OpenAI has released something
0:34
Josh: that essentially commoditizes that part of wealth.
0:37
Josh: It gives you an assistant that allows you to do all of these tedious things
0:42
Josh: that you normally don't like to do, but only for the cost of $20 a month.
0:46
Josh: And it's a product called Agent.
0:47
Josh: And here will show up the announcement post that they shared.
0:50
Josh: And what's really interesting about agent is it kind of builds
0:53
Josh: upon a lot of other things that openai has been building right
0:56
Josh: they have a browser feature they have a deep research feature
0:59
Josh: but they haven't created a fully comprehensive assistant
1:03
Josh: and that is what i'm assuming openai is
1:05
Josh: trying to do with their agent feature now the news dropped last week i was watching
1:10
Josh: the live stream ijaz we were we were kind of chatting throughout the whole thing
1:12
Josh: you were slightly underwhelmed because it didn't seem as cool as you wanted
1:16
Josh: i was more like in awe at the user interface i thought it just looked very are
1:22
Josh: you pretty more than anything else?
1:23
Josh: And we're kind of seeing on screen the announcement video.
1:25
Josh: What were your first takes when you saw this, the agent product from OpenAI?
1:29
Ejaaz: OK, so we were both watching that live stream at the same time,
1:32
Ejaaz: and we were sharing our thoughts.
1:34
Ejaaz: Do you remember that one sentence that I had for you, Josh, which was, if this is another,
1:41
Ejaaz: Google Drive connector, another agent that claims to do a ton of things and
1:46
Ejaaz: really it just kind of connects to your Google Drive, I'm going to be really mad.
1:49
Ejaaz: And what are we looking at on our screens right now, Josh? Tell me what we're looking at.
1:53
Josh: That's so funny because I was reading your message. You just sent this about
1:56
Josh: the first minute of the presentation and maybe 10 minutes later,
1:59
Josh: they showed the exact connection that he was describing, which is a Google Drive
2:02
Josh: connector. And what does that mean for the people who aren't sure why that's
2:05
Josh: not super exciting to you? Why is that annoying?
2:08
Ejaaz: So everyone uses email, right? And the most popular email is probably Gmail.
2:12
Ejaaz: And it comes with this suite of different products, right? You've got Google
2:14
Ejaaz: Docs where people write up documents at different company jobs.
2:18
Ejaaz: You've got Google Meet, people do video conferencing. It's a whole thing, right?
2:21
Ejaaz: And when agents started becoming popular, Josh, do you remember one of the main
2:26
Ejaaz: claims to fame was that you could now use AI in your Google account,
2:30
Ejaaz: which means that it kind of becomes chat GPT,
2:34
Ejaaz: but it can reference all your documents, it understands all your Excel sheets,
2:38
Ejaaz: your emails, it writes emails for you, whatever, right?
2:41
Ejaaz: But it ended up being more for PR purposes than what the agent actually did.
2:47
Ejaaz: When you actually tried some of these agents, all it kind of did was,
2:51
Ejaaz: it felt like a very friction filled process very fragmented still josh i still
2:56
Ejaaz: had to tell the ai no i meant i want you to reference this document or i want
3:01
Ejaaz: you to talk about this particular thing and i just ended up doing it all myself
3:05
Ejaaz: so i kind of have a shaded experience with using agents so when i saw this on
3:09
Ejaaz: the live stream i was like oh.
3:11
Josh: God is this
3:11
Ejaaz: The same thing.
3:13
Josh: And is it the same thing? Do you feel it's the same thing? Or is this a slight
3:16
Josh: enhancement? Because it appears to me like it's different. This time it is a little bit different.
3:19
Ejaaz: Okay, I will humor you. It actually, this time actually is a bit different.
3:24
Ejaaz: So you mentioned earlier, Josh, that this is a combination of their operator
3:30
Ejaaz: agent, which is, if I'm not mistaken, a computer use agent, right?
3:34
Ejaaz: So if you imagine it is an AI that can look at your desktop or your laptop screen
3:40
Ejaaz: the same way that I'm looking at it right now, and it can scroll with its own
3:45
Ejaaz: cursor, it can open up new tabs, it can open up documents,
3:48
Ejaaz: it can type your keyboard virtually and write a message, right?
3:52
Ejaaz: And it combines this feature with something called deep research,
3:55
Ejaaz: which you and I have spoken about a lot on this show, Josh, which is kind of
3:58
Ejaaz: like a PhD level researcher for whatever requests that you might have.
4:02
Ejaaz: So in theory, that sounded really good.
4:05
Ejaaz: Well, we just got access to this.
4:07
Ejaaz: I'm just gonna pull up the suite so that people have more context here,
4:10
Ejaaz: because I'm sure a lot of our listeners are kind of like asking, you know
4:12
Ejaaz: can i use this right now the feature just rolled
4:15
Ejaaz: out to all pro users so all anyone that's
4:18
Ejaaz: on the gpt pro plan will get access to this and so i
4:21
Ejaaz: was playing around with this for the last couple of days josh and
4:24
Ejaaz: to answer your question yes it is actually quite useful i kind of use some of
4:29
Ejaaz: the presented prompts so you know kind of like when you haven't typed anything
4:32
Ejaaz: yet they kind of suggest things that you have to type josh and i got it to organize
4:38
Ejaaz: bouquet of flowers to be delivered to my apartment for my girlfriend.
4:44
Ejaaz: And that's all I asked her to do. I was like, you know, off you go.
4:48
Ejaaz: This is not something I want to spend my time doing. So you go do it for me, right?
4:53
Ejaaz: Exactly, right? And so I set it. And then I'll tell you my firsthand experience,
4:58
Ejaaz: Josh, which is kind of funny.
4:59
Ejaaz: I just stared at the screen and I watched it create this virtual browser.
5:05
Ejaaz: So for those of you who are listening to the audio here, kind of imagine a pop-up
5:08
Ejaaz: appearing and it's like a mini desktop.
5:11
Ejaaz: But all the actions, all the movements, all the screen grabbing and all that
5:15
Ejaaz: kind of stuff is completely autonomous.
5:17
Ejaaz: You're just watching the AI do its thing. So it opened up a browser,
5:21
Ejaaz: Josh, and it started scrolling through different links from Google,
5:25
Ejaaz: from GPT, flower websites in my local area.
5:28
Ejaaz: It started using my location tracker to figure out where I was so it could figure
5:33
Ejaaz: out kind of like my vague general address and then figure out what flower shops
5:37
Ejaaz: were near me. It then kind of like started passing through a bunch of different
5:41
Ejaaz: bouquet flowers, all that kind of stuff.
5:43
Ejaaz: Kind of figuring out what's seasonal and what's not, what's within kind of a
5:47
Ejaaz: reasonable price point.
5:48
Ejaaz: And then it gave me a bunch of suggestions. And it said, hey,
5:51
Ejaaz: okay, these are some pretty good suggestions that I've got for you,
5:54
Ejaaz: but I kind of want to see whether I'm on track, whether I'm kind of hitting your vibe.
5:59
Ejaaz: Can you give me some more information to let me know?
6:01
Ejaaz: And so what I basically did, Josh, was I reviewed what it had for me.
6:05
Ejaaz: I scrolled through some of the examples that I had and I said,
6:08
Ejaaz: yeah, I actually kind of like this.
6:09
Ejaaz: And options one and four are kind of like really my vibe.
6:12
Ejaaz: And it said, okay, say no more. And it went away and it started like loading
6:16
Ejaaz: up the checkout sites and all that.
6:18
Ejaaz: And I could see the way it thought Josh, which was like one of my biggest takeaways
6:22
Ejaaz: was it kind of gave me comfort being able to watch it do its own thing.
6:28
Ejaaz: So it took about 10 minutes in this entire process. And by the end of it,
6:32
Ejaaz: I had a pretty good idea of which bouquet I wanted.
6:35
Ejaaz: And it just needed me to fill out some wallet details and off I went.
6:39
Ejaaz: So my initial take was, you've just saved me a lot of time. You did that in
6:43
Ejaaz: 10 minutes where I could have just technically looked away from the screen and
6:47
Ejaaz: got on with my own thing, right? And then come back to it.
6:50
Ejaaz: The other takeaway that I have, and I'm going to take the other side of this,
6:53
Ejaaz: right? Is it still required me to do things, Josh.
6:57
Ejaaz: It didn't know me entirely. And maybe I'm being too hard on the AI,
7:02
Ejaaz: but ideally I want an AI agent that just kind of like knows what I want and
7:05
Ejaaz: gets it done for me, right?
7:07
Ejaaz: And like has access to my wallet and pays for me.
7:09
Ejaaz: We're not quite there yet, but this is a noticeable step change from what we had before.
7:15
Josh: Okay, so good, not great. It'll get you most of the way there,
7:18
Josh: but it won't quite finish the entire thing.
7:20
Josh: And it was interesting hearing the process because during the stream,
7:23
Josh: I was listening to how it works. And you described the combination between operator
7:28
Josh: and deep research, how they're complimentary.
7:30
Josh: I kind of want to unpack that a little bit because operator was the browser-based
7:34
Josh: feature. can control a browser.
7:36
Josh: So now you have this tool that can control a browser, but it can't really read
7:39
Josh: long articles. It's not good at reading long articles. It doesn't have a lot of context.
7:42
Josh: It's not that good at things that deep research was good at.
7:47
Josh: Deep research is good at reading huge amounts of information,
7:49
Josh: compressing it down, coming up with links, and feeding that information back to the browser.
7:53
Josh: So we have this very complementary thing where deep research can't interact
7:57
Josh: with visual web pages, but it can understand a lot of context.
8:00
Josh: An operator can interact with visuals, but it doesn't have the ability to read a lot.
8:04
Josh: And that creates this complementary tool set that we're seeing on screen right
8:07
Josh: now, which is basically what OpenAd calls the agent.
8:11
Josh: It's given a tool set. So the agent spins up a virtual machine.
8:14
Josh: So you can imagine when you prompt in a query, it created this virtual computer
8:17
Josh: that started talking through how to get the bouquet of flowers.
8:21
Josh: It has a text-based browser tool, it has an image-based browser tool,
8:24
Josh: and there's access to its own terminal to create complementary things.
8:29
Josh: So let's say you were for example doing a spreadsheet and you wanted to see
8:32
Josh: what the costs of all the bouquet of flowers are throughout the neighborhood.
8:37
Josh: It can actually do that because it has all these complementary tool sets.
8:39
Josh: So while I can do all of this,
8:41
Josh: Well, it's not completely finished, like you said, you still have to kind of nudge it through.
8:47
Josh: So an example that they used in the live stream was trying to get a wedding outfit for a wedding.
8:52
Josh: And it'll get you kind of close, but then it doesn't quite know the sizing.
8:59
Josh: It doesn't quite know your style preferences.
9:02
Josh: And there's more context that it needs
9:05
Ejaaz: To complete this. It's lacking a bit of context, right? Yeah, exactly.
9:07
Josh: Yes, it's lacking the context and it's lacking the ability to click pay.
9:09
Josh: So we're seeing here in the example, it's stuck on the pay screen because it
9:12
Josh: doesn't have your credit card information and they just don't have that capability stored yet.
9:16
Josh: So I would imagine this is probably just, I mean, this is version one.
9:20
Josh: It will get better. It will have the context. It will get your credit card info.
9:23
Josh: One of the things I loved, and you described this briefly, is when it loads
9:27
Josh: up this browser window, you can actually replay it.
9:29
Josh: So Ejaz, you were at your browser for 10 minutes kind of watching it do its thing.
9:32
Josh: But in the case that you stepped away and you left you could
9:35
Josh: come back and you could actually re-watch all the steps that
9:38
Josh: it took as if you were watching a youtube video which is something that i really
9:42
Josh: liked a lot about this i think of all the things that i liked the most it was
9:46
Josh: the interface it was very pretty it was very easy to understand and it's really
9:49
Josh: funny the example they they were able to just kind of pan through like it was
9:52
Josh: a youtube video see where it made the searches see what it was looking for walk
9:55
Josh: through that step process to see how exactly it arrived at the answer.
9:58
Josh: And that was pretty neat. So, I mean, I haven't gotten a chance to try yet.
10:02
Josh: My account still does not have enabled.
10:04
Josh: Please give it to me. But based on your experience, it seems like it's good, not great.
10:08
Josh: And I think that's a step in the right direction because we went from pretty
10:11
Josh: mediocre to good. And that's a big jump in the right direction. Yeah.
10:14
Ejaaz: So let me unpack my thoughts a little more for you, Josh. Basically,
10:17
Ejaaz: I think it's good for people that have high agency.
10:22
Ejaaz: And what I mean by that is people who know what exactly they need to get done and, you know,
10:29
Ejaaz: maybe what they need to get done requires general access to a browser or and
10:34
Ejaaz: a few different tools will get a ton of value out of this agent product.
10:39
Ejaaz: I'm not saying it's useless. It's definitely way more useful than any other
10:42
Ejaaz: agent that's been released before.
10:45
Ejaaz: For the general public, who is so used to, I feel like at this point,
10:50
Ejaaz: Josh, magic when it comes to the internet.
10:53
Ejaaz: It's like, you know, when you open up an app and you're like,
10:55
Ejaaz: whoa, like snaps of photos that disappear after six seconds.
10:59
Ejaaz: That's so novel, right? Back in the day when Snapchat was created.
11:02
Ejaaz: You know, they want like that kind of like easy to understand kind of like basic
11:06
Ejaaz: level thinking. I don't need to think too much. It just wows me.
11:09
Ejaaz: We're not there yet with agents. But I have a feeling that when you combine
11:14
Ejaaz: agent with deep research as they did today, but with the context of memory,
11:20
Ejaaz: which as you and I have spoken about many times,
11:22
Ejaaz: and that OpenAI has the kind of like forefront leadership on,
11:26
Ejaaz: when you combine all those three things together, then you're going to see that
11:29
Ejaaz: magical experience where all you have to do, well, maybe you don't even have to prompt it, Josh.
11:33
Ejaaz: Maybe you just kind of open your screen and the agent is like,
11:36
Ejaaz: hey, I kind of ordered your coffee for you already.
11:38
Ejaaz: It's actually outside the Korea. I've been tracking him is arriving and I've
11:44
Ejaaz: like ordered your Tesla robo taxi to arrive.
11:46
Ejaaz: So, you know, you'll be ready to go and leave at 730 a.m. or whatever it is to go to the office.
11:51
Ejaaz: That's when people are going to start being like.
11:53
Ejaaz: I will pay hundreds of dollars for this thing. And I won't even think twice.
11:58
Josh: Yes. So we have great examples of people using this so far.
12:02
Josh: I see you have one pulled up right here. I want you to walk me through how you
12:05
Josh: can actually use this thing. Because
12:06
Josh: a lot of these things are kind of constrained by your own creativity.
12:09
Josh: I do a lot of stuff, but I'm not quite sure what I'm able to optimize for.
12:13
Josh: So what are some people using this to optimize for currently?
12:15
Ejaaz: So in this example, this person is using it to shop, which isn't exactly something that I can relate to.
12:21
Ejaaz: I don't do a lot of online shopping, but i have a girlfriend who does
12:25
Ejaaz: and i know that there are many different occasions that she
12:28
Ejaaz: needs to kind of cater and figure out what to buy for and there's so
12:31
Ejaaz: many sites that she needs to browse so in this example over here um you can
12:35
Ejaaz: see like you know she puts in this problem we pause it she goes find a beige
12:39
Ejaaz: trench coat for under 500 any website is fine must be under 500 including shipping
12:44
Ejaaz: must have a belt and double-breasted buttons optional to have a hood but must be detachable,
12:49
Ejaaz: basically all these different specificities that she needs for this comic, right?
12:54
Ejaaz: And kind of reading this kind of gives me a little anxiety because I'm like,
12:57
Ejaaz: oh my god, like I do not have the energy to search for something like this myself.
13:02
Ejaaz: And you see the agent responding being like, understood, I'll start searching
13:05
Ejaaz: for the beige coat under 500 bucks, and off we go, right?
13:08
Ejaaz: So that's happening. It's doing its own desktop thing. And we see it kind of
13:12
Ejaaz: unfolding. It's going through Amazon.
13:14
Ejaaz: It's kind of like going through a bunch of different options,
13:16
Ejaaz: looking at different colors, making sure it fits all the different criteria.
13:20
Ejaaz: And in the meantime, she's timing up another prompt and she's running up another prompt.
13:25
Ejaaz: So she's basically running like a bunch of different prompts for like her own chat GPT discussions.
13:30
Ejaaz: And what she's demonstrating here is she can go back and forth between windows, Josh.
13:34
Ejaaz: And she can like basically multiply her time and effort over a ton of different
13:40
Ejaaz: tasks while she has this agent doing it for her in the background.
13:42
Ejaaz: And if we get to the end of it...
13:46
Ejaaz: As we see over here, we see that it's pulled up a bunch of options.
13:49
Ejaaz: I think it's primarily used Amazon as its main kind of retailer.
13:53
Ejaaz: And then she ends up kind of like making a decision around what kind of coat that she wants to buy.
13:59
Ejaaz: So I thought that was like a really generally accessible or publicly accessible
14:03
Ejaaz: example that anyone can get into.
14:05
Ejaaz: I thought this was pretty fun. So this is something that like really relates
14:09
Ejaaz: to me, right? I'm like getting old.
14:11
Ejaaz: I'm like, okay, I need to, okay, I'm not that old. But like I'm starting to
14:14
Ejaaz: think about major financial decisions, right? I'm like, can I afford a house
14:18
Ejaaz: or am I going to be eternally renting for the rest of my life? Stuff like that.
14:20
Ejaaz: So I'm like, okay, what do I do with my finances? I don't even know the first
14:24
Ejaaz: place to consider or look into.
14:27
Ejaaz: And this demo basically says that it took 20 minutes for this agent to run a
14:33
Ejaaz: task to figure out what a healthy retirement plan might look like or financial
14:39
Ejaaz: plan might look like for this particular user that is demonstrating this example.
14:43
Ejaaz: And what I found really interesting here, and I'm highlighting it on this tweet,
14:47
Ejaaz: is within 20 minutes, it found local tax laws in Vancouver, which is where this guy must be based.
14:53
Ejaaz: It analyzed average monthly spend rates. It calculated savings needed to retire at 30 years old.
15:00
Ejaaz: It researched optimal investment allocations. It found taxed optimized strategies.
15:04
Ejaaz: It built multiple retirement scenarios.
15:08
Ejaaz: And then it created a downloadable presentation with all the results, Josh.
15:11
Ejaaz: This would have cost me $5,000 from a financial advisor and taken weeks.
15:16
Ejaaz: But here I have ChatGPT doing this all for whatever, $50, $100 subscription.
15:21
Ejaaz: And I'm going to bring up the video here where we basically see it kind of go
15:24
Ejaaz: through its thought process and then create a final finished deck,
15:28
Ejaaz: which you see on your screens right now.
15:29
Ejaaz: It looks pretty rudimentary, doesn't it, Josh? Let's be honest.
15:32
Josh: It looks kind of like a... I wouldn't give it a passing grade in terms of style,
15:35
Josh: but perhaps in terms of the actual content. Very impressive.
15:39
Ejaaz: Very impressive. Very impressive.
15:41
Josh: And this third example that we have is pretty funny because just before we started
15:43
Josh: recording, EJs and I were going through what we were going to order for lunch.
15:46
Josh: And it was this kind of intensive process.
15:47
Josh: We were reading off the menu items and choosing what we liked.
15:50
Josh: And it took a few minutes. And this one is a little bit different.
15:53
Josh: This one, the prompt was to order a roast dinner, which I think is very fitting
15:56
Josh: for you to describe EJs because I'm not quite sure what a roast dinner is.
15:59
Josh: This seems like a British thing.
16:00
Ejaaz: It's a Sunday meal, Josh, where the family comes together. It's either roast
16:05
Ejaaz: chicken, roast lamb or roast beef and it is a lengthy process all right and
16:10
Ejaaz: trust me i've watched my mom do it a million times.
16:13
Josh: Well according to this example it does not have to
16:15
Josh: be as lengthy as you imagine it used to be
16:18
Josh: what i'm seeing this agent do is it's actually it's going through
16:21
Josh: a grocery list it's choosing the chicken it's scheduling a delivery time
16:24
Josh: and it's giving you the the prompts in which you would
16:27
Josh: need to just finish filling it out so it's like hey what are the login details what
16:30
Josh: are the credit card information what time do you want delivered but it's doing
16:33
Josh: everything and it's amazing you're watching it click the browser it is going
16:36
Josh: through the thinking it is selecting all the things that you want really impressive
16:40
Josh: stuff so in this case like that solves you your your roast dinner problem where
16:44
Josh: you don't have to worry about it anymore you set this up once it knows your
16:48
Josh: preferences and then you just type in the prompt when you're ready to go and
16:50
Josh: it will place another order
16:52
Ejaaz: You know what's really impressive about this, Josh? I've been on this website,
16:56
Ejaaz: this exact website before actually, Tesco, when I was a university kid.
17:00
Ejaaz: And I remember doing bulk orders because I had like no money back then.
17:04
Ejaaz: So I was like, any kind of like bundle deal, I'll order in bulk and then like
17:09
Ejaaz: store it in the freezer or whatever.
17:11
Ejaaz: There are so many products. It's like going to Costco or whatever.
17:16
Ejaaz: It's Costco here, right? Where it's like a bulk ordering thing. and
17:19
Ejaaz: you know you just spend millions of hours just staring in
17:22
Ejaaz: the eyes being like i don't know what i need do i need this many toilet
17:25
Ejaaz: roll i have no idea this just like abstracts all
17:28
Ejaaz: of that away from you and i just know that there are a ton of people where this
17:32
Ejaaz: would be kind of like super useful for right the other thing was that kind of
17:36
Ejaaz: like popped into my mind is like how relevant does this make like supermarket
17:41
Ejaaz: websites these days right or just like general retailers in general Like,
17:46
Ejaaz: do the Amazons and Tescos, in this example, of the world need to now try and
17:52
Ejaaz: cater to these agents, Josh? Like, how are you thinking about that?
17:56
Josh: Yeah, I think the answer is yes. I think directionally the answer is yes for
17:59
Josh: things much further than just grocery stores.
18:01
Josh: Where if you're building a
18:03
Josh: website, you kind of want a version of your website to be readable by AIs.
18:07
Josh: A lot of websites are visually complex. They have a lot of visual.
18:10
Josh: Well, they're meant to be aesthetically pleasing, but they're not meant to serve
18:13
Josh: data in the most optimized way.
18:15
Josh: So what I've been seeing with a lot of developers they've started doing is they've
18:18
Josh: actually created a .md file, which is a markdown file that just has the raw
18:22
Josh: text data of the website.
18:24
Josh: No way. So when an agent is scraping...
18:26
Josh: No images at all, just the raw text data. Because when an AI is scraping a website,
18:31
Josh: all it wants is the tokens.
18:33
Josh: It doesn't care for the visuals. It can't see in most cases.
18:36
Josh: So the most optimal way of serving these models is just by allowing the robot
18:40
Josh: to come and read your website as it wants to.
18:43
Josh: And I think a similar thing is probably going to happen with delivery,
18:46
Josh: with groceries, with everything is just kind of creating a dual experience, this split experience.
18:51
Josh: There's an experience you build for the human, and then there's an experience
18:54
Josh: you build for the AI model.
18:56
Josh: And the ai model is going to be really boring and plain it's
18:59
Josh: just going to be a massive block of text that has
19:01
Josh: no spacing no prompts it's just text but
19:05
Josh: that's really all the ai model wants it ingests the tokens it sorts them and
19:09
Josh: it pops new ones out and i think this is probably a trend we're going to be
19:12
Josh: seeing with a lot of things is is the user interface the visual element of it
19:16
Josh: is going to matter less and less and i can't talk about this without thinking
19:20
Josh: about an episode that we're going to be filming soon,
19:22
Josh: which is about browsers and AI browsers in general, and how we kind of interact with these things.
19:27
Josh: Because it's becoming more and more clear that the future of engaging with the
19:33
Josh: internet will be requesting something from it and expecting to be delivered
19:37
Josh: that thing without needing to search and go through all the tedious efforts
19:41
Josh: of actively going to the website,
19:43
Josh: searching through the million different SKUs that they have on Tesco, choosing what you want.
19:46
Josh: You could just say, hey, I have this grocery list or hey, I want to make this
19:50
Josh: for dinner. just go find the stuff for me and just make sure it shows about my doorstep by 6 p.m.
19:54
Josh: And that is a really cool trend that I think we are starting to see and we're
19:57
Josh: going to see more as more tools like this get rolled out.
20:00
Ejaaz: I can't help but think that this is going to completely disrupt the advertising industry, Josh.
20:06
Ejaaz: Isn't that like the main way that all these internet companies make all their money?
20:10
Ejaaz: YouTube, Google on search ad revenue, Meta on advertising on all their news
20:16
Ejaaz: feeds and all that kind of stuff. It's advertisers, right?
20:19
Ejaaz: And now I'm trying to think, how do you prove that your advert has made an impression
20:26
Ejaaz: on an artificial intelligent agent, right?
20:31
Ejaaz: To your point, you go from making flashy ads, which were designed for human
20:35
Ejaaz: eyes to see and walk at, to agents that are just reading a bunch of .md text
20:42
Ejaaz: files, as you put it, right?
20:44
Ejaaz: Which is kind of like insane to think about.
20:48
Ejaaz: The other thing that I'm thinking about is I think services are going to look
20:53
Ejaaz: very different now, Josh, right?
20:56
Ejaaz: It's going to become less human-like. So we're going to, you know,
21:00
Ejaaz: we're not naturally going to go on websites.
21:03
Ejaaz: And I kind of have a comment on your browser, a topic that you mentioned.
21:07
Ejaaz: I kind of think of it as like an intermediary step, right?
21:10
Ejaaz: Almost like it's kind of like trying to bridge the gap between humans and AI.
21:15
Ejaaz: Oh, we're used to browsers, scrolling browsers.
21:18
Ejaaz: So, you know, let's add a few AI features to these browsers.
21:21
Ejaaz: And I'm really excited to do that episode.
21:22
Ejaaz: But then ultimately, we're going to
21:24
Ejaaz: end up in this world where it's just advertisers trying to pitch agents.
21:28
Ejaaz: Your maybe your own personal agent to sell something to you and i wonder whether
21:33
Ejaaz: it skews incentives in different ways right maybe you can pay the agent shill
21:37
Ejaaz: a certain product to the individual i wonder how all of that ends up mapping
21:40
Ejaaz: out it's a weird thing to think.
21:42
Josh: About i would actually love to have a specialist on advertising on the show
21:45
Josh: to talk about this that just knows more about it than we do because the model
21:49
Josh: is already starting to break we've seen it a lot with ad blockers now being
21:52
Josh: built into browsers a lot of companies are kind of optimizing out of displaying ads, so
21:58
Josh: On YouTube, a lot of people will just pay for their premium service to not even have to deal with it.
22:02
Josh: So there's this trend towards avoiding ads.
22:06
Josh: Now that these agents are browsing the website, do they insert the ads into
22:09
Josh: the markdown files of the website and hope that they get served?
22:13
Josh: Is the agent going to be able to filter out from there? Because that's this
22:16
Josh: thing called injection poisoning, right?
22:18
Josh: Where you kind of inject these things into the words that it's reading to give it commands.
22:22
Josh: And that was actually part of the presentation that OpenAI had was about the
22:25
Josh: security and defensiveness of these agents.
22:28
Josh: Because when you have an agent that is equipped with your credit card information,
22:32
Josh: all of your personal information, everything about you, and it has read and
22:35
Josh: write ability on the open internet,
22:37
Josh: well, you kind of want to make sure the agent is not giving this stuff up and
22:41
Josh: telling a malicious actor what your address is, what your credit card information
22:46
Josh: is, what all your preferences are.
22:47
Josh: So there's the second side of this conversation, which
22:51
Josh: is the security of allowing an agent to go
22:53
Josh: out into the wild with your entire profile in its
22:57
Josh: brain in the hopes that some malicious actor
23:00
Josh: can't can't prompt inject and create this malicious intention that serves up
23:05
Josh: the information to a website that they're just that's not real there's there's
23:09
Josh: this whole defensible side of this argument to be had also it seems like they're
23:13
Josh: doing a lot of work to prevent this but i guess we'll see as time goes on how it actually plays out
23:19
Ejaaz: Okay, so, so far we've spoken about retail consumption of these agents,
23:24
Ejaaz: Josh. We've spoken about the security.
23:26
Ejaaz: One thing that we haven't covered is kind of like the workforce and the enterprise world, right?
23:33
Ejaaz: Presumably, right, if you can connect an agent to, you know,
23:37
Ejaaz: very neatly use your Google Drive, your Slack, your LinkedIn,
23:43
Ejaaz: your Salesforce CRM, what are you doing from nine to five every day, right?
23:49
Ejaaz: I pulled up this demo here, Josh. I don't know if you can see it.
23:54
Ejaaz: But it's this demo that shows someone using the GPT agent to basically make
24:00
Ejaaz: personalized invite connections to a bunch of different people on LinkedIn.
24:04
Ejaaz: And so it kind of sets the scenario with the prompt saying, like,
24:08
Ejaaz: hey, I want you to basically reach out to individuals that have this kind of
24:11
Ejaaz: a background, that have held this kind of role, that has this many years of
24:15
Ejaaz: experience, and I want you to pitch them on A, B, and C, right?
24:18
Ejaaz: And like throughout the demo, it kind of like goes through, personalizes each
24:22
Ejaaz: request to each individual person.
24:24
Ejaaz: And the first thought that I had in this, Josh, was if you're on the sales team
24:29
Ejaaz: of some company, your job just got automated, dude.
24:32
Ejaaz: Because a lot of the time you spend, if you're a recruitment person,
24:37
Ejaaz: work for a recruitment firm and you're headhunting someone, 90% of their time
24:41
Ejaaz: is spent on LinkedIn. I get a ton of these requests day in, day out, right?
24:45
Ejaaz: And so I'm thinking, well, now your job can be completely automated and it could
24:49
Ejaaz: probably do a much better job
24:50
Ejaaz: and reach thousands of people that you reach over months within an hour,
24:56
Ejaaz: within 20 minutes, within however long this video is, which is two minutes and
25:01
Ejaaz: 36 seconds. So I think that we're going to start to.
25:04
Josh: See a lot of
25:04
Ejaaz: These agents in work environments. I don't know if we're going to have as much insight into that.
25:10
Ejaaz: It's going to be at the discretion of all these different companies as to what
25:13
Ejaaz: they want to reveal to us. But that's something that we should keep our eyes on, Josh.
25:17
Ejaaz: And, you know, I would love to get someone on the show,
25:20
Ejaaz: knows about how it's transforming their own company right at a massive scale.
25:24
Josh: Yeah it's the so there's there's two elements of
25:27
Josh: this agent right it's like the personal unlock that we talked about at the top of the show
25:29
Josh: where now it's this super assistant that you have to take care of all of your
25:34
Josh: yes irons that you don't want to run all the things that are annoying and then
25:37
Josh: you have the professional version of it where it can go and it can make sales
25:40
Josh: funnels and sales calls and it could do a lot of the professional things it
25:43
Josh: can query data it can create nice spreadsheets it can create google slide presentations
25:47
Josh: and granted they don't look that pretty,
25:49
Josh: but I mean, directionally, they're heading there and they will eventually get there.
25:53
Josh: So it's this new trend that is really exciting because it's now here and it's
25:57
Josh: accessible for $20 a month and you can actually go and try it and you could
26:00
Josh: push those limitations, see
26:01
Josh: where the outer bounds lie and see how it can mostly optimize your life.
26:05
Josh: And then as we go, it'll just unlock more and more and more.
26:08
Josh: It'll start taking down your credit card.
26:10
Josh: It'll take down your whole preference stack and just become this supercharged agent.
26:14
Josh: So I think that is OpenAI's agent feature. It's live. It is out right now.
26:19
Josh: I'm hoping my account gets it soon because I really want to use it.
26:23
Ejaaz: Yeah, dude. You know what? I am officially more optimistic than I was at the
26:27
Ejaaz: start of this episode about this thing.
26:29
Ejaaz: I think you've successfully, or rather we're both successfully convinced me,
26:33
Ejaaz: that this is going to become a much larger thing.
26:37
Ejaaz: And I'm just looking at the end of this demo here, Josh, like this screenshot
26:40
Ejaaz: of like, you know, this virtual browser.
26:43
Ejaaz: Even though it's so rudimentary and basic, I kind of get it.
26:47
Ejaaz: Do you know what I mean? it's like running a simulation on my screen and like
26:51
Ejaaz: having it like just set and forget on the top left of my screen and just letting it do its thing.
26:57
Ejaaz: Or like if I'm doing work and I want to watch a stream of something,
26:59
Ejaaz: I can have like a YouTube video like there on the corner of my screen.
27:02
Ejaaz: It's kind of similar to that. So even just me saying that, it's behaviorally getting to me.
27:08
Ejaaz: I'm understanding it subconsciously and I'm like, yeah, you know what?
27:11
Ejaaz: This is going to be fun. So I'm bullish is what I'm saying.
27:14
Josh: Great. I am bullish too. My hot take before we leave, I'm just going to mic
27:18
Josh: drop this is I think the agent is much more impressive and much more important
27:21
Josh: than the AI powered browser.
27:22
Josh: I think the browsers are kind of dumb. The agent hops right over the browser.
27:25
Ejaaz: That's probably where this goes. Dude, that was going to be you.
27:28
Ejaaz: That's going to be my take on the future episode.
27:30
Josh: I was going to show it. We're going to stop before we spoil it.
27:33
Josh: The browser episode is coming soon. As soon as we get access to Comet,
27:35
Josh: we want to use Comet from Complexity.
27:37
Josh: We're working on getting access to it. Once we do, we will have a fully comprehensive
27:40
Josh: episode covering AI browsers.
27:41
Josh: But for now, that was OpenAI's agent feature.
27:44
Josh: I would encourage you if you have a plus plan, which is the $20 a month plan.
27:47
Josh: Go check it out. Give it a try. See where it can automate parts of your life.
27:50
Josh: See where it improves it. See where it detracts from it. I don't know.
27:54
Josh: I'd love to hear feedback. I'm sure both Ejaz and I would love to hear more
27:56
Josh: examples of how you're using it.
27:57
Josh: Because at the end of the day, the only real constraint on this is the creativity
28:01
Josh: that you use when you're using it. So I hope you enjoyed it.
28:03
Josh: That has been another episode. We will be back again soon talking about more
28:06
Josh: AI news in the future. I'll talk to you guys soon.
28:08
Music: Music