Latest / In the Loop AI / Revealing Elon’s Secret AI Trading Bot: Is It Worth It?

Transcript

Revealing Elon’s Secret AI Trading Bot: Is It Worth It? · In the Loop AI

• 0:00
  Josh: Imagine this, you give eight of the world's most powerful AI models $10,000
• 0:04
  Josh: each and tell them, go trade real stocks.
• 0:06
  Josh: No paper trading, but real money with real risk. And two weeks later,
• 0:10
  Josh: most of them have lost a painful amount of cash, which I guess is kind of expected.
• 0:14
  Josh: The kind of drawdowns that would get a human portfolio manager totally fired.
• 0:17
  Josh: But now, they ran the same experiment again, except this time with much higher stakes.
• 0:22
  Josh: There's $320,000 at stake. And we've talked about Alpha Arena before in a previous
• 0:27
  Josh: episode, which I highly recommend checking out.
• 0:29
  Josh: But now we have the new results from the new season, season 1.5.
• 0:33
  Josh: And what was exciting is that there was a very clear and obvious winner,
• 0:36
  Josh: but that winner was a mystery.
• 0:37
  Josh: We don't actually know or we didn't know who the winner was up until recently.
• 0:42
  Josh: In fact, it won all four of the trading competitions in
• 0:45
  Josh: this new season while leaving the other top models like ChatGPT
• 0:49
  Josh: 5.1 and Google Shemini 3.0 fighting for
• 0:52
  Josh: second place so at the core of this is one who is
• 0:55
  Josh: this model and two how on earth did they
• 0:58
  Josh: do it how are they outperforming everyone so much so as to make 65 percent in
• 1:02
  Josh: two weeks in one of these competitions so ijaz i want to walk through everyone
• 1:07
  Josh: about what what just happened what the model is and what alpha arena is so give
• 1:11
  Josh: us the lowdown on on who this was that made so much money oh
• 1:16
  Ejaaz: Yeah well we will get into all of that uh today
• 1:19
  Ejaaz: So Alpha Arena is basically a competition or test to see how well AI models can trade.
• 1:28
  Ejaaz: And they do this in a few different ways, Josh. Number one, they give each model
• 1:32
  Ejaaz: $10,000, as you mentioned.
• 1:34
  Ejaaz: And then they allow them to trade a range of different financial instruments
• 1:37
  Ejaaz: over a period of two weeks.
• 1:39
  Ejaaz: So there's like a season, two weeks, and we see which AI models do the best.
• 1:43
  Ejaaz: And they get all your AI models in there. You've got ChatGPT,
• 1:47
  Ejaaz: you have got Gemini, you've got Anthropics Claude, and you have Grok as well.
• 1:51
  Ejaaz: And so they've gone through about two seasons now, and the results have been
• 1:55
  Ejaaz: absolutely crazy. So they started off with season one.
• 1:59
  Ejaaz: And you can think of this as like the degen crypto season.
• 2:02
  Ejaaz: They gave seven models, $10,000 each, and allowed them to trade crypto assets
• 2:07
  Ejaaz: like Bitcoin, Ethereum, stuff like that.
• 2:10
  Ejaaz: And they did this in something called Perpetual, so they could leverage trade.
• 2:14
  Ejaaz: It was the only instrument that they were allowed to do this.
• 2:17
  Ejaaz: And the results were, as you'd probably expect, a lot of these AI models lost a lot of money.
• 2:23
  Ejaaz: Some of them actually ended up making a decent chunk of money,
• 2:26
  Ejaaz: and they were primarily Chinese models.
• 2:28
  Ejaaz: They were Quen, and I think it was DeepSeat that ended up making money.
• 2:31
  Ejaaz: So there was a lot of takeaways there. As you mentioned, we've got a previous
• 2:34
  Ejaaz: episode where we spoke about this.
• 2:35
  Ejaaz: Definitely go give that a watch. There's a lot of alpha in that one.
• 2:39
  Ejaaz: And then that brings us to season 1.5, where the AI models, instead of being
• 2:45
  Ejaaz: given crypto to trade, were given the ability to trade U.S. stocks.
• 2:50
  Ejaaz: And we're talking about equities, which is something that a lot of us listening
• 2:53
  Ejaaz: to this show are very familiar with. And I think this is for a few reasons, Josh.
• 2:57
  Ejaaz: Primarily, crypto is very volatile, and we kind of want to figure out how the
• 3:01
  Ejaaz: majority of money that is traded in the financial markets can translate into
• 3:05
  Ejaaz: AI models trading that. So a few things that they kept the same is that they
• 3:09
  Ejaaz: gave the AI model $10,000.
• 3:11
  Ejaaz: But there was a number of differences with Season 1.5. Number one,
• 3:15
  Ejaaz: they were allowed to trade US equities and stocks.
• 3:17
  Ejaaz: Number two, there were two new models that were introduced.
• 3:21
  Ejaaz: One was a model called Kimi K2, which is a really good open source Chinese model.
• 3:26
  Ejaaz: But the other was this thing called a mystery model.
• 3:29
  Ejaaz: I'm going to reveal which this model was in a second. But before I do,
• 3:34
  Ejaaz: do you have any guesses as to what model this might have been?
• 3:38
  Josh: Well, I cheated. I know the answer. But what I think is very exciting about
• 3:42
  Josh: this is that like, I think it's important to highlight these models made hundreds
• 3:45
  Josh: to even thousands of trades per model. Yes.
• 3:48
  Josh: And what we want to answer, like the question that I want more than this mystery
• 3:51
  Josh: model is like, is this real signal or is this just, I mean, Luke said earlier,
• 3:56
  Josh: is this a GPU intensive scratch off game?
• 3:58
  Josh: Where is there any real signal? and like I guess we'll talk about the reality
• 4:02
  Josh: of that and what this means for your portfolio if you ever want to manage it but to me
• 4:06
  Josh: I think that's the important thing to highlight. We probably should just spill
• 4:09
  Josh: the beans, EJ. Do you want to just tell them? Who's Mr.
• 4:11
  Ejaaz: Model? I can't keep it in any longer. It was an unofficial version of Grok,
• 4:18
  Ejaaz: aptly named Grok 4.2 or 4.20 for the memers out there.
• 4:23
  Ejaaz: And this was revealed by none other than the Grok man himself, Elon Musk.
• 4:28
  Ejaaz: And the reason why this mystery model was getting so much attention,
• 4:32
  Ejaaz: Josh, was because it ended up being the winner. It made the most money out of any other AI models.
• 4:41
  Ejaaz: And what was more impressive is there wasn't just one competition being run throughout season 1.5.
• 4:47
  Ejaaz: There were four at the same time. So these AI models were running across four
• 4:54
  Ejaaz: different competitions at the same time. That was $320,000.
• 4:58
  Ejaaz: At any one instance, which is a crazy amount of financial money to stake on
• 5:03
  Ejaaz: an experiment. That's a lot of money could have been lost here.
• 5:06
  Ejaaz: And Grok 4.20 ended up performing the best.
• 5:09
  Ejaaz: Josh, I want to go through a few different stats here, which kind of like shows
• 5:14
  Ejaaz: how amazing this particular model was.
• 5:18
  Ejaaz: So firstly, for some context, there were four different competitions that were
• 5:21
  Ejaaz: being run that these AR models were being tested on.
• 5:24
  Ejaaz: Competition number one was something called new baseline. This is basically
• 5:28
  Ejaaz: the ability for these AI models to get access to.
• 5:32
  Ejaaz: Trading AI stocks, to get access to all the common news that you and I can read
• 5:37
  Ejaaz: online and in newspapers to kind of like figure out, okay, what kind of news
• 5:42
  Ejaaz: would affect my stock positions.
• 5:43
  Ejaaz: They would also get access to sentiment data to see how kind of like the markets
• 5:47
  Ejaaz: and retail traders would kind of react to certain bits of news.
• 5:50
  Ejaaz: They had access to a much wider spread amount of data in competition number one.
• 5:56
  Ejaaz: Competition number two was called Monk Mode. They kind of amended the investing
• 6:00
  Ejaaz: prompt here. And so kind of like they traded more conservatively.
• 6:04
  Ejaaz: Competition number three was called Situational Awareness, Josh.
• 6:07
  Ejaaz: So each model had an awareness of other models trading and where they ranked in accordance to them.
• 6:13
  Ejaaz: So there was this kind of like ecosystem of peer pressure being put on by each model.
• 6:17
  Ejaaz: And competition number four was just outright degeneracy max
• 6:21
  Ejaaz: leverage you could only trade with like 20 to 50x leverage which is just kind
• 6:26
  Ejaaz: of i don't think it's 50x but like 30x uh just crazy amount of risk um adjustment
• 6:30
  Ejaaz: to test whether a model would take that risk or whether it would trade more
• 6:35
  Ejaaz: conservatively josh did you have any reactions on the the results of this of this competition the.
• 6:41
  Josh: Results that we're looking at right now actually i found most interesting this
• 6:44
  Josh: is from the new baseline competition it's It's basically the full info mode.
• 6:47
  Josh: And one of the big differences between this mode versus previous competitions
• 6:51
  Josh: that have been held is like you mentioned earlier, it has access to a lot of data.
• 6:55
  Josh: This is the first time an AI trading model has had access to real time information
• 6:59
  Josh: outside of just looking at a chart. So I think.
• 7:02
  Josh: In that sense, this is the closest competition to how a human quant fund would actually operate.
• 7:07
  Josh: So if you're looking for high signal in terms of which AI can actually make
• 7:11
  Josh: you real money in the real world, this is the one.
• 7:13
  Josh: And what we're seeing here is that the Grok 4.20 model, the memetic mystery
• 7:17
  Josh: model, outperformed by a fairly large margin to OpenAI and ChatGPT 5.1,
• 7:23
  Josh: which is the clear second place.
• 7:24
  Josh: And those are the only two that actually made profit. everybody else lost money
• 7:28
  Josh: in the real world competition which to me signals a few things one of them being
• 7:33
  Josh: well perhaps one is really good at
• 7:36
  Josh: understanding real world information perhaps it understands company fundamentals
• 7:40
  Josh: better perhaps it just has access to real
• 7:43
  Josh: world information that's better like grok and having access to the x ai model
• 7:47
  Josh: um so there's a lot of things to speculate here but for me the new baseline
• 7:51
  Josh: chart that we're looking at right now was the high signal one i'm like oh my
• 7:54
  Josh: god wait this has the same type of information flows that i'm now getting so
• 7:58
  Josh: now we're even we're on the same playing field okay
• 8:01
  Ejaaz: Um i actually had a different answer to that which is i was more impressed,
• 8:06
  Ejaaz: Josh, by the situational awareness competition.
• 8:10
  Ejaaz: So this was a competition where each model had access to data and news,
• 8:15
  Ejaaz: but they also had awareness of who they were competing against.
• 8:19
  Ejaaz: So Grok 4.20, the winner, knew that GPT-5 was in second place.
• 8:25
  Ejaaz: And so he was always keeping an eye on GPT-5, being like, oh,
• 8:28
  Ejaaz: what trades is GPT-5 making? Why did they make that trade? Oh, that's interesting.
• 8:32
  Ejaaz: And then he would look at Gemini and be like, oh, what trades are Gemini making.
• 8:35
  Ejaaz: So he would have this awareness of his competitors, which you didn't have in
• 8:39
  Ejaaz: season one, where they were just kind of like trading in silos, right?
• 8:42
  Ejaaz: And why this competition was so interesting, Josh, is this was technically where
• 8:47
  Ejaaz: Grok 4.20 made the most money.
• 8:50
  Ejaaz: In fact, if you look at the top of this leaderboard right here,
• 8:53
  Ejaaz: the account value at the end of season 1.5 was $16,656,
• 9:00
  Ejaaz: which is technically a 60% plus return in two weeks on $10,000 worth of capital.
• 9:09
  Josh: I needed to take my money immediately.
• 9:12
  Ejaaz: Isn't that insane, right? Like if you had to pick a competition of where you
• 9:15
  Ejaaz: would have given an AI model money, just given from this data,
• 9:19
  Ejaaz: and I'm not saying you should do that, you would be most bullish on situational awareness.
• 9:24
  Ejaaz: And I'm going to make some implications here that I haven't tested yet,
• 9:29
  Ejaaz: but it seems to imply that
• 9:31
  Ejaaz: this kind of competitive nature where the models were kind of aware and exposed
• 9:35
  Ejaaz: to their competitors' trades and thinking, and we're going to get to the model
• 9:40
  Ejaaz: chat thinking in a second, seems
• 9:41
  Ejaaz: to have given them a better trading advantage, at least in some cases.
• 9:45
  Josh: Yeah, so like you mentioned, one of my favorite parts, I think we share this
• 9:48
  Josh: in one of our favorite parts about this competition in particular,
• 9:51
  Josh: is that you can actually see all of the trades.
• 9:53
  Josh: One thing about these private quant funds, you don't know what the hell is going on.
• 9:56
  Josh: But with So these models, you can see exactly what they're thinking every time
• 10:01
  Josh: they think and make a decision.
• 10:02
  Josh: So maybe you guys can go through a few of them and see kind of what the model
• 10:06
  Josh: is thinking, how they're processing this real world data.
• 10:08
  Josh: And if there's any tips for us to learn from processing this real world data,
• 10:12
  Josh: because clearly they're a much better trader than I am.
• 10:14
  Ejaaz: Yeah. So I have a few examples pulled up here on the right side of the screen.
• 10:19
  Ejaaz: It's under model chat. By the way, any of you listening to this can go onto
• 10:23
  Ejaaz: this website and see for yourself and scroll through their hundreds and hundreds of posts.
• 10:26
  Ejaaz: But it basically gives us an insight into how each model thinks about a trade
• 10:31
  Ejaaz: that they currently either have open or they're thinking about opening or closing
• 10:35
  Ejaaz: or whatever that might be, right?
• 10:36
  Ejaaz: So it's like being in the mind of an actual investor and figuring out how they make their decisions.
• 10:42
  Ejaaz: An example here at the top of the screen is Gemini 3 Pro.
• 10:47
  Ejaaz: He goes, I'm betting on a breakout in NVIDIA, seeing a strong setup as it holds
• 10:51
  Ejaaz: support and leading the market with a target of $189 and a stop just below $180.
• 10:57
  Ejaaz: So what he's referring to there is kind of a typical quant style of trading
• 11:02
  Ejaaz: where it's kind of like he's looking at technicals, he's evaluating kind of
• 11:05
  Ejaaz: graphs, momentum of the stock price.
• 11:07
  Ejaaz: It's very price evaluated type of trading, right, Josh? But if you look just
• 11:11
  Ejaaz: below it, you've got GPT 5.1, which actually came in second at the end of this
• 11:16
  Ejaaz: competition, who goes, my analysis indicates continued strength in AI names
• 11:20
  Ejaaz: like NVIDIA and Microsoft.
• 11:22
  Ejaaz: So I'm holding out on existing long positions over the weekend and potential macro event risk.
• 11:28
  Ejaaz: Now, the point I want to make about this particular model is it's less price
• 11:32
  Ejaaz: specific and it's more focused on just kind of general themes,
• 11:37
  Ejaaz: news and data that it's seeing outside of price.
• 11:41
  Ejaaz: And that really goes to demonstrate that some of these models are very kind
• 11:45
  Ejaaz: of price and quantitative focused, whereas other models are kind of more thesis
• 11:49
  Ejaaz: driven over a shorter period of time.
• 11:51
  Ejaaz: And it kind of gives rise to these types of personalities, right, Josh?
• 11:55
  Josh: Yeah, well, now we have to answer the uncomfortable question is like,
• 11:58
  Josh: is this evidence that Grok is some kind of money printing god?
• 12:01
  Josh: Or is this just like really well produced content that happens to involve real money?
• 12:05
  Josh: And that kind of comes down to understanding the AI, understand the personalities,
• 12:10
  Josh: understanding how each model considers these trades and how they place themselves
• 12:15
  Josh: in different positions.
• 12:17
  Josh: So I kind of want to go through one by one, all of the models and kind of what
• 12:20
  Josh: their personalities are like.
• 12:22
  Josh: We see with DeepSeek a lot that it behaves, and we mentioned on a previous episode
• 12:26
  Josh: as well, it behaves like a very disciplined quant fund.
• 12:29
  Josh: And DeepSeek, for those that don't know, it's an open source Chinese model.
• 12:32
  Josh: They are very systematic, very mathematic, very comfortable with leverage,
• 12:36
  Josh: but able to hedge and adjust mid-trade based on its decisions and new information.
• 12:41
  Josh: So DeepSeek and Quen even is kind of similar to this.
• 12:45
  Josh: If you remember from the last episode, Ejaz, Quen was my early favorite.
• 12:48
  Josh: I had hoped that Quen was going to win.
• 12:51
  Josh: Unfortunately, that's not the case at all in season 1.5. Quen has gotten crushed
• 12:55
  Josh: right there with DeepSeek.
• 12:56
  Josh: I can kind of imagine it as like more similar to me, maybe that's why I resonated
• 13:00
  Josh: with it, where it has one big thesis and then it sizes aggressively around that thesis.
• 13:04
  Josh: So if you remember, Quen would only buy Bitcoin or Ether in the last one and
• 13:07
  Josh: it wouldn't buy any other altcoins.
• 13:09
  Josh: It just had a thesis that these major coins were going up, nothing else was.
• 13:12
  Josh: Claude is interesting. It's very
• 13:14
  Josh: reflective of how the actual Claude model works when you engage with it.
• 13:17
  Josh: It's very patient and it's thoughtful, but it occasionally sizes up too much
• 13:21
  Josh: and then it gets crushed by leverage.
• 13:24
  Josh: So, and like, as we go through these, and EJs, I also noticed you assigned a masculine...
• 13:29
  Josh: Personality to gemini you said he when you were talking about google gemini
• 13:32
  Josh: and that's kind of because it's it's daddy right like gemini's been
• 13:35
  Josh: the big boy on top but but in
• 13:39
  Josh: this training competition i don't know if it is i was going through the trades
• 13:42
  Josh: and it very much panic flip flops from shorts to long after losing and it kind
• 13:46
  Josh: of in a way gemini was most reflective of retail behavior because and i'm not
• 13:51
  Josh: sure what we could tie that to but gemini was very reactionary where if it lost
• 13:55
  Josh: money it would flip its position and if it gained money it would it would kind of hedge quickly.
• 13:58
  Josh: So that was interesting. And then we have GPT-5, which is very sophisticated reasoning.
• 14:03
  Josh: But in season one, they over-traded and over-leveraged and got absolutely wiped
• 14:06
  Josh: out. And they were very timid in their way that they went about this.
• 14:09
  Josh: So that's kind of how you can think about these.
• 14:11
  Josh: The final one, which is the secret model, Grok 4.2. If we know anything about
• 14:15
  Josh: Grok, we know that it is a very high risk taker, but a calculated risk taker.
• 14:19
  Josh: And that's probably what put it at the top there. So that's kind of how I would
• 14:22
  Josh: consider all of these models.
• 14:23
  Josh: They're a little different and they are reflective of, if you've used these
• 14:26
  Josh: in person, you could kind of understand the thinking that gets placed behind the trades
• 14:30
  Ejaaz: Yeah i i want to dig into a few
• 14:33
  Ejaaz: things around the the personality or rather the trading styles here josh because
• 14:37
  Ejaaz: um it may not be as explicit as we kind of lay it out like so grok 4 4.20 was
• 14:44
  Ejaaz: the winner right by far and it made money uh it was the top across all of the
• 14:48
  Ejaaz: competitions all four competitions that's great but did you look at the results of grok 4,
• 14:53
  Ejaaz: its predecessor.
• 14:55
  Josh: It was absolutely crushed.
• 14:57
  Ejaaz: It was the worst performing model in this entire competition,
• 15:01
  Ejaaz: which is crazy because in season one, where it was trading crypto,
• 15:06
  Ejaaz: it came in at second or third.
• 15:08
  Ejaaz: And for about 75% of the competition, Josh, it was number one.
• 15:13
  Ejaaz: So it had some kind of an advantage trading kind of very riskily, right?
• 15:19
  Ejaaz: And that might be because of the nature of the instruments that it was trading.
• 15:22
  Ejaaz: Crypto is very volatile and it was kind of going blase.
• 15:24
  Ejaaz: So when it was like 20x bullish Bitcoin, it benefited a lot when Bitcoin price
• 15:29
  Ejaaz: went up, but obviously it like suffered when it went down.
• 15:32
  Ejaaz: It's interesting to see the discourse between these two models and 1.5, right?
• 15:38
  Ejaaz: Grok 4.20, the winner, seems to be a kind of more mature version of Grok 4.
• 15:44
  Ejaaz: It seems to be thinking more about its trades.
• 15:47
  Ejaaz: It has more kind of like risk percentiles and boundaries in place,
• 15:52
  Ejaaz: whereas Grokforce seems to be its kind of usual degenerate self.
• 15:55
  Ejaaz: And I don't know how much of that is reliant on the fact that it's trading stocks,
• 15:59
  Ejaaz: which is generally a less volatile market versus Grok 4.20 being a more thesis
• 16:04
  Ejaaz: driven, sensible trader, as you kind of described.
• 16:07
  Ejaaz: The other one that we have to call out because it's the elephant in the room
• 16:10
  Ejaaz: here, GPT-5 came in at second in season 1.5.
• 16:15
  Josh: Right? 5.1. 5.1.
• 16:17
  Ejaaz: Sorry, 5.1, right? In the previous season, season one, it was the second worst
• 16:24
  Ejaaz: performing. No, sorry, it was the worst performing.
• 16:26
  Josh: It was horrible.
• 16:28
  Ejaaz: It was GPT-5.
• 16:30
  Josh: It was an abomination.
• 16:30
  Ejaaz: And Gemini. So whatever OpenAI has cooked up in the .1, congrats.
• 16:36
  Ejaaz: Because you must have traded on some kind of financial data or you've you've
• 16:40
  Ejaaz: like kind of like implemented a kind of like risk trading strategy that made
• 16:44
  Ejaaz: it a lot more sensible because it made some really great trades on this season
• 16:48
  Ejaaz: so just two different kind of like jumps from season one to 1.5 that i i had to call out.
• 16:53
  Josh: Yeah it makes me excited to see the improvements in these like
• 16:56
  Josh: significant improvements with incremental models because we
• 16:59
  Josh: normally talk about 5 to 5.1 being pretty marginal like
• 17:02
  Josh: there's nothing really noteworthy or exciting and yet the results in the
• 17:05
  Josh: small sample size at least are pretty reassuring that hey there is something
• 17:09
  Josh: new going under the hood and maybe this is an appropriate time to address the
• 17:13
  Josh: i guess the the limitations the kind of bare case of this starting with the
• 17:18
  Josh: sample size um we do have to say i mean this is two weeks ejs this is not a long time um they they
• 17:25
  Josh: placed some trades. Some people maybe got lucky. Some models maybe did not.
• 17:29
  Josh: Is there any real signal here?
• 17:31
  Josh: I'm curious, your take, do you think this is reflective of future performance?
• 17:35
  Josh: Like, is there what is here that's actually valuable versus what is here is actually kind of lucky?
• 17:40
  Ejaaz: I don't think we have enough information to make that call, at least for me.
• 17:45
  Ejaaz: I'll speak for myself personally.
• 17:47
  Ejaaz: The real test is, you know, I asked myself before we recorded this episode,
• 17:51
  Ejaaz: would i give my money to grok 4.2 or the winner that one across all categories
• 17:55
  Ejaaz: and the simple answer is like no like i don't i don't know if it's going to
• 17:59
  Ejaaz: repeat that over week three week four week five it was only two weeks to your
• 18:03
  Ejaaz: point right so i want to see this experiment kind of,
• 18:06
  Ejaaz: rehash like a million times before i'm like okay that's cool um even then it's
• 18:10
  Ejaaz: it's still kind of like risky right it's like i i can justify giving my money
• 18:14
  Ejaaz: to a human that i can kind of relate to that I can call up in speed to,
• 18:18
  Ejaaz: less so when it comes to an AI model, right? But maybe that's my thing,
• 18:22
  Ejaaz: it needs to kind of evolve.
• 18:24
  Ejaaz: The other way I'm thinking about this is there's just a lot of unknowns around this, Josh, right?
• 18:31
  Ejaaz: Like I can see it's thinking, I can see kind of like how the model kind of completes its trades,
• 18:37
  Ejaaz: I don't really know what's going under the hood. Is this just kind of like a
• 18:41
  Ejaaz: pattern matching thing?
• 18:42
  Ejaaz: Does it inherit the risks that a lot of humans have already done?
• 18:45
  Ejaaz: Because it's trained on the same kind of corpus of trading data that we have
• 18:49
  Ejaaz: kind of evaluated on? Or is it kind of net better?
• 18:52
  Ejaaz: Do you feel the same or?
• 18:54
  Josh: Yeah, it's probably, I mean, it's not the new gold standard of AI benchmarks.
• 18:59
  Josh: But it is a standard that I think is interesting. Because this is a benchmark
• 19:03
  Josh: that happens in the real world with real dynamic data that cannot be game.
• 19:07
  Josh: So in that case, I love it.
• 19:08
  Josh: But I saw one writer, they called it Schrodinger's Benchmark,
• 19:12
  Josh: because it's simultaneously serious and degenerate at the same time.
• 19:16
  Josh: And it's like it's entertainment with real money that happens to produce some
• 19:20
  Josh: legitimate insights about AI behavior, but it's not really indicative of future
• 19:24
  Josh: returns at the small of a sample size, at least.
• 19:27
  Josh: And that's kind of where I feel about it. there is one breakthrough that we
• 19:30
  Josh: mentioned earlier that does provide real value, which is the transparency.
• 19:35
  Josh: Every trade being on chain and every step reason being logged is actually really
• 19:39
  Josh: helpful to understanding how these models think and how you can consider thinking.
• 19:43
  Josh: So for example, you could show me every decision Grok 4.20 made on Tesla after
• 19:47
  Josh: the Fed announcement or something like that. And it'll walk you through a chain of thought.
• 19:50
  Josh: And if anything, make you into a better investor.
• 19:54
  Josh: Would I trust the model of my own money? No.
• 19:57
  Josh: Maybe a little bit maybe with a small sample size how
• 20:00
  Josh: much it is that's a
• 20:03
  Josh: good question i'd give it a couple thousand dollars to play around with and see
• 20:06
  Josh: what happens i think that that would be interesting and fun and it's it's
• 20:09
  Josh: low enough stakes but i would trust it enough to not lose it like i'd say i
• 20:14
  Josh: would probably trust grok more with my money than i would the average day trader
• 20:20
  Josh: off the street um which granted they don't have a very good reputation but i
• 20:23
  Josh: think there is some sort of an edge there that doesn't exist in the average person.
• 20:28
  Josh: And if you assume that these models are going to continue to get better and
• 20:31
  Josh: better, well, you have to assume that they're going to form some sort of an
• 20:35
  Josh: edge, but I don't know how much.
• 20:37
  Josh: It's an interesting question because as a quant trading fund too,
• 20:41
  Josh: if your job or as just a trader in general, if your job is to make money off
• 20:44
  Josh: of trading, what are you doing about this information? Are you leaning into AI?
• 20:48
  Josh: Are you trying to get these models to help you with your information flows and make decisions?
• 20:53
  Josh: Are you using them to help you actually transact trades or are you just kind
• 20:56
  Josh: of looking the other way and saying oh this is just a dumb experiment to benchmark
• 21:00
  Josh: models there's no actual signal here and the answer is probably somewhere in the middle right yeah
• 21:04
  Ejaaz: I mean well my initial reaction to that is um,
• 21:07
  Ejaaz: Okay, quant funds already use algorithms. It would make a lot of sense if they
• 21:13
  Ejaaz: started using AI algorithms, right?
• 21:15
  Ejaaz: If you could get a smarter algorithm to trade for your fund, absolutely, right?
• 21:18
  Ejaaz: So it's a no-brainer to me that these hedge funds, quant funds are going to
• 21:21
  Ejaaz: be using AI, probably already using AI.
• 21:24
  Ejaaz: Where I have maybe a hot take is that the transparency is just a nice to have.
• 21:29
  Ejaaz: It is no way going to win in the best of models.
• 21:32
  Ejaaz: Why? Because if you have an AI model that is like better than all the other
• 21:36
  Ejaaz: AI models at trading, why would you make that public?
• 21:39
  Ejaaz: Right. So like, I'm kind of like at ties between this thing,
• 21:43
  Ejaaz: because I think the transparency is a really good thing in kind of like bringing
• 21:46
  Ejaaz: up the floor of trading credibility for people that get access to this type of information.
• 21:52
  Ejaaz: Like I have loved reading through these kind of like trade logs here,
• 21:55
  Ejaaz: seeing how each model thinks and being like, okay, yeah, wow.
• 21:58
  Ejaaz: I actually didn't think about that myself when I was buying that stock.
• 22:01
  Ejaaz: Right. And these are like stocks that I've seen that I, that I can buy,
• 22:04
  Ejaaz: right. The Amazon trade, the NVIDIA trade, I'm just like, oh, okay.
• 22:06
  Ejaaz: I didn't think about that, right, yesterday whenever they made this trade.
• 22:10
  Ejaaz: If I am a hedge fund, I'm like, yeah, if I've fine-tuned a model that is like
• 22:15
  Ejaaz: beating all these models, I don't really want to expose that really.
• 22:18
  Ejaaz: So it's kind of like a push and pull.
• 22:20
  Ejaaz: The other thought I had, Josh, is, and maybe this is kind of like kind of semi-adjacent
• 22:26
  Ejaaz: to what we're discussing here.
• 22:28
  Ejaaz: I couldn't get the thought out of my head that if you could get Grok in X,
• 22:33
  Ejaaz: trading some kind of money for you or guaranteeing you like a 5% to 10% annual
• 22:37
  Ejaaz: return, that is something that I would like if framed correctly,
• 22:41
  Ejaaz: I would put some money into, right?
• 22:43
  Ejaaz: Maybe not over two weeks, but
• 22:45
  Ejaaz: maybe over an adjusted kind of yearly period would be super cool to see.
• 22:49
  Josh: Yeah, that's such a, it's such a fun question to ask is like,
• 22:51
  Josh: what happens when this kind of system runs for two years, but with your,
• 22:56
  Josh: like, let's say it's a large pension management fund and they just want a manager
• 22:59
  Josh: that doesn't take fees and does a pretty good job.
• 23:02
  Josh: Like, is there going to be enough trust in these systems to reliably place money at scale with them?
• 23:07
  Josh: And And you have to assume, given the signal this early on, that the answer will be yes.
• 23:12
  Josh: The question is, how much of a yes will it be?
• 23:15
  Josh: What percentage of management will be AI as it gets better over time?
• 23:20
  Josh: And the sample size sucks. I wish it was more than two weeks. I wish it was two years.
• 23:23
  Josh: In two years from now, think about the progress we're going to see and what
• 23:26
  Josh: type of impact that's going to have on trading models.
• 23:29
  Josh: So this is, it's interesting. It's fascinating.
• 23:32
  Josh: In fact, I'm really curious to actually run this experiment for ourselves.
• 23:35
  Josh: I'd love to try to come up with a little trading model that runs these things
• 23:38
  Josh: and test it out because it's fun and there is some sort of an edge there.
• 23:42
  Ejaaz: I would say, okay, if I were to summarize my lesson from this entire competition
• 23:48
  Ejaaz: or experiment so far, Josh,
• 23:50
  Ejaaz: it is I'm not convinced to give AI models money to trade, but I'm convinced
• 23:56
  Ejaaz: to use AI models to help me trade.
• 23:59
  Ejaaz: So kind of like a human and AI model kind of work together and kind of become
• 24:04
  Ejaaz: a better trader overall, I think is the main takeaway for me here. Do you share the same?
• 24:09
  Josh: It's funny. I mean, this is how agents work today, right?
• 24:12
  Josh: Like if you go on ChatGPT and you say, go book me a reservation,
• 24:16
  Josh: it'll take you to the finish line.
• 24:17
  Josh: And then you as the human provide the final filter and approve or deny.
• 24:20
  Josh: And I think that's probably the happy middle ground while
• 24:23
  Josh: we still don't really trust these models too much is give me
• 24:27
  Josh: the thesis give me the trade i will either approve
• 24:30
  Josh: or deny and that's how the money gets managed so
• 24:33
  Josh: it's cool this is a great experiment i love that we got season 1.5
• 24:36
  Josh: i mean it's fascinating even more fascinating is that we
• 24:38
  Josh: have an early look at grok 4.2 which by all
• 24:41
  Josh: means is the best trading model in the world where will
• 24:44
  Josh: it rank in the other benchmarks we will see we will be covering it as soon as
• 24:48
  Josh: it comes out but i guess that's that's really it for this episode on season
• 24:51
  Josh: 1.5 the question i want to leave everyone else with is i mean would you trust
• 24:55
  Josh: an ai with your part of the portfolio like how much money would you actually
• 24:58
  Josh: give to an ai currently grok 4.2 who just made
• 25:02
  Josh: 60% in two weeks in one of these trading competitions. Is that enough for you to risk your money?
• 25:07
  Josh: Or is it still just this dumb AI system that you don't really trust?
• 25:12
  Ejaaz: Well, if you're interested in this experiment, Josh and I were actually discussing,
• 25:17
  Ejaaz: about potentially giving you guys a tutorial on how to use an AI to trade money
• 25:24
  Ejaaz: for you and kind of like an experiment, this own end of one experiment, but our own.
• 25:29
  Ejaaz: But we want to get a little more signal from you guys. Let us know in the comments
• 25:32
  Ejaaz: whether this is something that you'd be interested in seeing.
• 25:35
  Ejaaz: And I have, Josh, I have a requirement for the listeners.
• 25:40
  Ejaaz: If we do want to put the tutorial out. Our last video that we did on AI trading
• 25:45
  Ejaaz: reached 100,000 views and 3,000 likes.
• 25:49
  Ejaaz: So I'm not going to ask for the 100,000 views, but I will ask for the likes.
• 25:54
  Ejaaz: If this video can get more than 3,000, if it gets 3,000 likes,
• 25:58
  Ejaaz: we will definitely put out that tutorial by the end of the year.
• 26:01
  Ejaaz: And we have a lot of thoughts around this, about how we're going to do it.
• 26:05
  Ejaaz: We're super excited to do it. So help us get there.
• 26:08
  Ejaaz: It is another week of really exciting news. Josh, I don't know if you saw the
• 26:12
  Ejaaz: rumors. Did you see the rumors about OpenAI?
• 26:14
  Josh: Tell me, fill me in.
• 26:15
  Ejaaz: About OpenAI releasing a potential new groundbreaking model?
• 26:18
  Josh: As a matter of fact, the Polymarket is showing that OpenAI is very favored to
• 26:22
  Josh: release the best model of the year.
• 26:25
  Josh: And last I checked, Gemini is the best model of the year. So that implies we're
• 26:28
  Josh: getting something big in the next few weeks.
• 26:30
  Ejaaz: I think we will. and like you said, the Polymarket is kind of like revealing
• 26:35
  Ejaaz: its hands so maybe there's some inside information coming out here.
• 26:37
  Ejaaz: So kind of stay tuned to Limitless. Put the notifications on,
• 26:41
  Ejaaz: guys and also subscribe if you want to get the latest videos.
• 26:44
  Ejaaz: We put out the best content out there.
• 26:47
  Ejaaz: It's unchallenged right now. Josh and I are sitting here unchallenged.
• 26:50
  Ejaaz: You have to like and subscribe if you want to get our content on your feed.
• 26:53
  Ejaaz: Thank you so, so much for listening. Again, let us know what you thought of
• 26:56
  Ejaaz: this episode in the comments. Get that like number up and we will see you on the next one.