Latest / AI Investment Edge: Stocks, Markets & Money / AI Stock Rankings: What Changes the Top Ten? | EP 001
Transcript
- 0:00SPEAKER_00: An AI can give you a beautifully ordered stock list.
- 0:03SPEAKER_00: Change one column and most of that top 10 disappears.
- 0:07SPEAKER_00: Before asking which stock is first, ask with the ranking rewards.
- 0:13SPEAKER_00: Welcome to iPulse AI, that's iPulse AI, research with receipts.
- 0:19SPEAKER_01: Yeah, and we are keeping things incredibly grounded for this deep dive,
- 0:23SPEAKER_01: like no hype, no hot stock tips, just pure calm analysis today.
- 0:28SPEAKER_00: Exactly.
- 0:29SPEAKER_00: I mean, we're looking at this really fascinating experiment by Russlan.
- 0:33SPEAKER_00: He took a historical snapshot of a 383 equity archived September cohort, right?
- 0:40SPEAKER_00: And just tested what happens to an AI's top 10 list when you change
- 0:44SPEAKER_00: a single variable in the math.
- 0:46SPEAKER_01: Right.
- 0:46SPEAKER_01: And to be super precise with our data up front,
- 0:48SPEAKER_01: this snapshot was taken on September 20th and the scoring was calculated
- 0:53SPEAKER_01: the very next day, September 21st.
- 0:54SPEAKER_00: So it's a fixed moment in time.
- 0:57SPEAKER_00: You know,
- 0:57SPEAKER_00: I think human nature assumes that
- 0:59SPEAKER_00: the quote unquote best companies are just universally the best.
- 1:03SPEAKER_00: But this experiment really proves how fragile AI rankings actually are.
- 1:08SPEAKER_01: Oh, absolutely.
- 1:09SPEAKER_01: I mean, first they just tested what happens when you simply change the time frame.
- 1:14SPEAKER_01: So they looked at the AI's top 10 list based on a one year consensus score.
- 1:18SPEAKER_00: Right.
- 1:18SPEAKER_01: And compared it to the five year consensus score.
- 1:21SPEAKER_01: You'd kind of expect a lot of overlap there, wouldn't you?
- 1:23SPEAKER_00: Yeah, you really would.
- 1:25SPEAKER_01: But that gives us our first major metric, which is the six of ten statistic.
- 1:29SPEAKER_01: Only six names managed to stay in the top ten across both of those time horizons.
- 1:34SPEAKER_00: Wow, so only six.
- 1:35SPEAKER_00: Like Intuit was one of the few named examples that actually survived on both lists, I think.
- 1:40SPEAKER_01: Yeah, exactly.
- 1:42SPEAKER_01: Intuit made it.
- 1:43SPEAKER_00: Well, it makes sense if you think about it like packing a bag.
- 1:47SPEAKER_00: Your absolute top ten essential items for a weekend trip are going
- 1:50SPEAKER_00: to look very different from what you'd pack for a month-long expedition.
- 1:54SPEAKER_01: That is a great way to look at it.
- 1:55SPEAKER_00: The destination might be exactly the same,
- 1:58SPEAKER_00: but the longer horizon completely changes your priorities.
- 2:00SPEAKER_01: Right.
- 2:01SPEAKER_01: And the AI does the exact same thing.
- 2:03SPEAKER_01: The underlying business hasn't changed at all,
- 2:06SPEAKER_01: but the mathematical priorities shift just depending on how far out
- 2:09SPEAKER_01: the model is forced to look.
- 2:11SPEAKER_00: So that packing analogy kind of covers time,
- 2:14SPEAKER_00: but time isn't the only thing tweaking the AI's math here.
- 2:17SPEAKER_00: What happens when we completely change the actual goal of the trip?
- 2:21SPEAKER_01: The sort field, yeah.
- 2:23SPEAKER_00: Right.
- 2:23SPEAKER_00: Let's say we stop looking at this balanced consensus score until the AI
- 2:27SPEAKER_00: to sort strictly by raw annualized mean total return scenarios.
- 2:31SPEAKER_00: Basically, just show me the highest potential modeled returns in their native currencies.
- 2:36SPEAKER_01: When you do that, the drop off is just stunning.
- 2:39SPEAKER_01: Comparing the consensus score list to the pure return list at that one year horizon.
- 2:45SPEAKER_01: Well, that gives us the two of 10 statistic.
- 2:47SPEAKER_00: Wait, two of 10?
- 2:49SPEAKER_01: Yeah.
- 2:49SPEAKER_01: Eight out of the 10 stocks are entirely different.
- 2:52SPEAKER_00: Okay, see, this is where I get tripped up looking at the data.
- 2:55SPEAKER_00: If a stock has massive return forecasts,
- 2:57SPEAKER_00: I mean, why wouldn't it be sitting at the very top of the consensus score list?
- 3:01SPEAKER_01: Because the consensus score isn't just this blind chase for yield.
- 3:05SPEAKER_01: It actually penalizes a stock for uncertainty.
- 3:08SPEAKER_00: Oh, okay.
- 3:08SPEAKER_01: Yeah, take a term like dispersion.
- 3:11SPEAKER_01: That just means the underlying data is all over the place
- 3:13SPEAKER_01: because analysts completely disagree on the company's future.
- 3:17SPEAKER_01: The AI sees that disagreement.
- 3:19SPEAKER_00: Or it spots historical volatility, right?
- 3:21SPEAKER_01: Exactly.
- 3:22SPEAKER_01: Volatility, event risk pressure, it flags all of it.
- 3:25SPEAKER_01: So a company might have a massive return forecast,
- 3:28SPEAKER_01: but the AI drops its rank because the path
- 3:30SPEAKER_01: to actually get those returns is just dangerously unstable.
- 3:34SPEAKER_00: Okay, that makes sense for the broad strokes.
- 3:36SPEAKER_00: But let me ask you about the razor thin margins.
- 3:39SPEAKER_00: I mean, looking at the raw scores here, the stock in the 10th spot has a score of, say, 368.
- 3:45SPEAKER_01: Right.
- 3:46SPEAKER_00: And the one right below at an 11th is at 367.
- 3:50SPEAKER_00: My brain immediately says 368 is undeniably better.
- 3:54SPEAKER_00: But practically speaking, is there an actual real world economic difference in a single point?
- 4:00SPEAKER_01: I am so glad you brought that up because the short answer is no.
- 4:03SPEAKER_01: A one point boundary does not establish an economically meaningful advantage.
- 4:07SPEAKER_00: Good to know.
- 4:08SPEAKER_01: Yeah.
- 4:08SPEAKER_01: It's crucial to remember that these AI scores, they're just research signals.
- 4:12SPEAKER_01: They are not calibrated probabilities.
- 4:14SPEAKER_00: Which is exactly why we need to be really clear about the factual limitations
- 4:18SPEAKER_00: of this deep dive.
- 4:20SPEAKER_00: This is one archived cohort.
- 4:23SPEAKER_00: There are absolutely no realized returns in this data,
- 4:25SPEAKER_00: no claims of outperformance,
- 4:27SPEAKER_00: and these are definitely not current topics for today's market.
- 4:31SPEAKER_01: Right.
- 4:32SPEAKER_01: So if you're staring at one of these beautifully ordered AI lists tomorrow,
- 4:37SPEAKER_01: how do you avoid getting tricked by it?
- 4:38SPEAKER_00: We need a checklist.
- 4:40SPEAKER_01: We do.
- 4:41SPEAKER_01: The absolute first rule, document your filters.
- 4:43SPEAKER_01: Save your snapshot date, your horizon, and exactly what sort field you're using.
- 4:48SPEAKER_00: Right.
- 4:49SPEAKER_00: And then your next step should be to actually mess with those filters on purpose, right?
- 4:52SPEAKER_01: Exactly.
- 4:53SPEAKER_00: State your thesis, then change one setting to see how many names actually survive that shift.
- 4:59SPEAKER_00: Oh, and You have to check the math of the universe itself.
- 5:03SPEAKER_01: Yes, the dropouts.
- 5:04SPEAKER_00: Yeah, I noticed the stocks rank can rise simply because other companies fell out of
- 5:08SPEAKER_00: the data set, not because the company actually improved.
- 5:11SPEAKER_01: Always review the raw score and the list membership before you convince yourself
- 5:15SPEAKER_01: a business is doing better.
- 5:17SPEAKER_01: Because rankings just organize your next research question,
- 5:20SPEAKER_01: you know, they don't settle the investment decision.
- 5:23SPEAKER_00: Perfectly said.
- 5:25SPEAKER_00: You can inspect this entire experiment and find the full source evidence
- 5:28SPEAKER_00: for yourself at ipulseai.com.
- 5:31SPEAKER_00: And remember, this is strictly research, not investment advice.
- 5:35SPEAKER_01: Very true.
- 5:36SPEAKER_00: Not investment advice at all.
- 5:38SPEAKER_00: So I'll leave you with this to chew on.
- 5:41SPEAKER_00: If changing a single hidden variable completely rewrites
- 5:44SPEAKER_00: an AI's definitive top 10 list,
- 5:47SPEAKER_00: how much of your own everyday decision making is secretly driven
- 5:50SPEAKER_00: by default sorting filters you've never even thought to question?