Latest / AI Investment Edge: Stocks, Markets & Money / AI Investment Forecasts: Where Are the Receipts? | EP 004
Transcript
- 0:00SPEAKER_01: So have you ever wondered how to actually check an AI prediction?
- 0:04SPEAKER_01: I mean, if an AI makes a forecast today,
- 0:06SPEAKER_01: how do you know it didn't just, you know, rewrite its answer after the fact?
- 0:10SPEAKER_00: Yeah, that is a massive vulnerability, actually.
- 0:12SPEAKER_00: I mean, AI models are these incredibly fluent text generators.
- 0:16SPEAKER_01: Right.
- 0:17SPEAKER_00: So without a paper trail,
- 0:18SPEAKER_00: it's remarkably easy for them to just revise history and look brilliant
- 0:21SPEAKER_00: in hindsight.
- 0:22SPEAKER_01: Which is exactly why we're doing this deep dive today.
- 0:25SPEAKER_01: We're looking at iPulse AI, which is pronounced iPulse AI, by the way.
- 0:29SPEAKER_00: Yeah, the open agentic investment research platform from Future Edge Group.
- 0:33SPEAKER_01: Exactly.
- 0:34SPEAKER_01: And our mission today is just a,
- 0:36SPEAKER_01: well, a calm, precise unpacking of what they call research with receipts.
- 0:40SPEAKER_01: Like why do AI forecasts need immutable records and how do we even verify them?
- 0:44SPEAKER_00: Right.
- 0:44SPEAKER_00: So let's start with what a receipt actually is in this context.
- 0:47SPEAKER_00: I mean, think about the difference between making a prediction and then grading
- 0:51SPEAKER_00: that prediction later.
- 0:52SPEAKER_01: Okay, they're two different steps.
- 0:54SPEAKER_01: Got it.
- 0:55SPEAKER_01: So if the whole theory here is about creating a verifiable track record,
- 0:58SPEAKER_01: has anyone actually stress tested this out in the wild?
- 1:02SPEAKER_00: They have, actually.
- 1:04SPEAKER_00: On October 4th, 2026,
- 1:06SPEAKER_00: there was this specific bounded public metadata audit
- 1:11SPEAKER_00: of their historical consensus snapshot data set.
- 1:14SPEAKER_01: Wow.
- 1:14SPEAKER_01: Okay.
- 1:15SPEAKER_01: How big was it?
- 1:17SPEAKER_00: They audited exactly 746 records.
- 1:20SPEAKER_00: Specifically, that was 373 one-year and 373 five-year asset consensus outputs.
- 1:28SPEAKER_01: Okay, 746 forecasts.
- 1:30SPEAKER_01: But you know,
- 1:30SPEAKER_01: when people hear the word audit,
- 1:32SPEAKER_01: they usually assume it's like this blanket certification that all
- 1:35SPEAKER_01: the underlying math and data are totally flawless.
- 1:38SPEAKER_00: Yeah, we need to be incredibly precise here about what this audit did not verify.
- 1:43SPEAKER_01: Right, what did it actually do?
- 1:45SPEAKER_00: It simply checked that six metadata fields were present.
- 1:48SPEAKER_00: So things like the record ID, the snapshot ID, and that checksum we just talked about.
- 1:53SPEAKER_01: So they just made sure none of those were missing.
- 1:55SPEAKER_00: Exactly.
- 1:56SPEAKER_00: But, and this is crucial, the checksums were not actually recomputed or verified.
- 2:02SPEAKER_01: Wait, really?
- 2:03SPEAKER_01: So they just verified that the digital fingerprint was physically attached to
- 2:07SPEAKER_01: the file,
- 2:08SPEAKER_01: but they didn't run the math to see if the fingerprint actually matched
- 2:11SPEAKER_01: the file holding it.
- 2:12SPEAKER_00: That's right.
- 2:13SPEAKER_00: The scope was strictly limited to presence, not validation.
- 2:17SPEAKER_00: The audit does not validate the source data or the quality of the forecasts
- 2:22SPEAKER_00: or any portfolio performance.
- 2:24SPEAKER_01: Okay, so no claims of trading performance here.
- 2:27SPEAKER_00: Zero.
- 2:28SPEAKER_00: And definitely no instructions to buy or sell.
- 2:31SPEAKER_00: Each row simply represents an asset consensus output for a specific snapshot and horizons.
- 2:37SPEAKER_01: Okay, well,
- 2:37SPEAKER_01: moving past the audit,
- 2:38SPEAKER_01: let's look at how the platform actually generates these forecasts.
- 2:42SPEAKER_01: I know they use multiple advisor personas.
- 2:45SPEAKER_00: Yeah, they do.
- 2:46SPEAKER_01: And my initial thought is,
- 2:48SPEAKER_01: wouldn't using five different AI advisors act sort of like a diversified portfolio?
- 2:54SPEAKER_01: Like if one is wrong,
- 2:55SPEAKER_01: the others catch the mistake and you get this foolproof,
- 2:58SPEAKER_01: statistically independent forecast.
- 3:00SPEAKER_00: I mean, it is really tempting to think so.
- 3:02SPEAKER_00: But even if you have several distinct advisor personas,
- 3:05SPEAKER_00: maybe asking different questions about downside risk or asset quality,
- 3:10SPEAKER_00: they are often sharing the exact same underlying models, prompts and evidence.
- 3:15SPEAKER_01: I see.
- 3:16SPEAKER_01: So it's sort of like five students copying off the exact same wrong cheat sheet during an exam.
- 3:22SPEAKER_01: Like they all have the same answer, but that doesn't make the answer right.
- 3:26SPEAKER_00: That is a perfect analogy.
- 3:28SPEAKER_00: Yeah, agreement among personas does not automatically equal a calibrated probability
- 3:32SPEAKER_00: of correctness.
- 3:34SPEAKER_00: You can't just assume high accuracy because of a consensus.
- 3:37SPEAKER_01: Because they might just be making correlated errors.
- 3:40SPEAKER_00: Exactly.
- 3:40SPEAKER_00: You have to preserve the identity of your experimental unit,
- 3:44SPEAKER_00: which kind of leads to the three distinct layers of a forecast.
- 3:47SPEAKER_01: Right, which are the recorded output,
- 3:50SPEAKER_01: the evaluation inputs, like observed outcomes, and then the final evaluation result.
- 3:56SPEAKER_00: Yes.
- 3:56SPEAKER_00: So the original prediction is kept completely isolated from the grading of that prediction.
- 4:02SPEAKER_01: So how does that isolation handle predictions where the clock is still ticking?
- 4:06SPEAKER_01: Like if you have a one-year forecast, but you're checking it at three months.
- 4:10SPEAKER_00: Well, that is an unresolved outcome.
- 4:12SPEAKER_00: And under their declared policy, it absolutely must remain unresolved.
- 4:16SPEAKER_01: Meaning it can't just silently become a zero or magically disappear.
- 4:21SPEAKER_00: Right.
- 4:21SPEAKER_00: Or be labeled a failed forecast simply because the time horizon isn't finished yet.
- 4:26SPEAKER_01: Yeah, checking a one-year forecast at three months and calling it a failure is like,
- 4:29SPEAKER_01: I don't know,
- 4:29SPEAKER_01: taking a cake out of the oven after 10 minutes and complaining that it's raw.
- 4:33SPEAKER_00: Right.
- 4:34SPEAKER_00: The horizon just hasn't finished.
- 4:36SPEAKER_00: I mean, missing data,
- 4:38SPEAKER_00: inapplicable fields,
- 4:39SPEAKER_00: and a measured zero all mean completely different things mechanically.
- 4:43SPEAKER_01: So basically,
- 4:43SPEAKER_01: receipts make a claim inspectable,
- 4:45SPEAKER_01: but they definitely do not certify investment skill.
- 4:49SPEAKER_00: No, they don't.
- 4:50SPEAKER_00: But you should always demand the receipts anyway.
- 4:54SPEAKER_01: So for you listening,
- 4:56SPEAKER_01: here is a really quick takeaway checklist when you're analyzing this kind
- 4:59SPEAKER_01: of AI data.
- 5:00SPEAKER_01: First, identify exactly what one single row represents.
- 5:05SPEAKER_00: And second, always inspect the definitions and any exclusions.
- 5:08SPEAKER_01: And third, make sure to check if the horizon is actually finished before judging the outcome.
- 5:14SPEAKER_01: If you want to dig deeper,
- 5:15SPEAKER_01: we encourage you to explore the open source evidence links we've put in
- 5:19SPEAKER_01: the deep dive notes.
- 5:20SPEAKER_00: You can also visit their site directly at ipulseai.com.
- 5:23SPEAKER_01: Just to spell that out, that is I-P-U-L-S-E-A-I dot com.
- 5:29SPEAKER_01: And just to reiterate our own disclaimer,
- 5:31SPEAKER_01: this deep dive is strictly covering historical research.
- 5:34SPEAKER_00: Right.
- 5:35SPEAKER_00: This is definitely not investment advice.
- 5:37SPEAKER_01: Not at all.
- 5:38SPEAKER_01: And hey, think about this as you go about your day.
- 5:40SPEAKER_01: Earlier we talked about how shared models create correlated errors.
- 5:44SPEAKER_01: Well, if different AI models ultimately train on the exact same human internet,
- 5:50SPEAKER_01: will we ever truly have a completely independent AI forecast?
- 5:55SPEAKER_01: Something to mull over.