Latest / The Learning Corner by Precursor / Episode #54: Teaching Students to Challenge AI, Is AI The New Shadow Bank?

Transcript

Episode #54: Teaching Students to Challenge AI, Is AI The New Shadow Bank? · The Learning Corner by Precursor

• 0:00
  That'll be the difference between people who are actively using AI and leveraging it to
• 0:03
  create self sustaining viable outputs versus people where their prompts will be a couple of
• 0:08
  chaff back and forth kind of rinse and reuse that because it does pass the job.
• 0:12
  How much free time do people wanna use to basically do the hard work upfront is basically
• 0:17
  gonna be the question.
• 0:18
  It's the week of October 27.
• 0:20
  Welcome to the Learning Corner, a Precursor Ventures podcast.
• 0:28
  Welcome to the Learning Corner by Precursor.
• 0:30
  I'm Mia Farnam, and I'm joined by our GP and managing partner, Charles Hudson, where we will
• 0:35
  be talking through the articles and Goodreads that caught our team's attention over the past
• 0:39
  week.
• 0:40
  Our first topic comes from Nick Potkolitsky, whose piece Beyond True and False, Teaching
• 0:46
  Students to Interrogate AI Unreliability, explores how students can develop a more
• 0:51
  nuanced approach to evaluating AI generated content.
• 0:54
  In his Substack, Nick draws on the three pillars of literary theory created by James
• 0:59
  Phelan, typically used to critique authors and literature, and suggest the same framework can
• 1:05
  be effectively applied to critique AI outputs.
• 1:08
  Nick explains that these framework offer a way forward moving beyond true or false to ask
• 1:14
  richer questions about how AI constructs and presents information.
• 1:18
  So the framework revolves around three axes of unreliability.
• 1:21
  The first is facts and events.
• 1:23
  So what the narrator claims happened, acknowledging the potential for falsehoods or
• 1:27
  omissions.
• 1:28
  Knowledge and perception, the idea that a narrator might misinterpret or overlook
• 1:33
  important nuances and complexities.
• 1:35
  And the third is ethics and values.
• 1:37
  So recognizing that narrators can often misjudge or undervalue aspects of an issue.
• 1:43
  While AI is a narrator in the traditional sense, Nick argues that teaching students to
• 1:48
  apply this kind of critical lens to AI outputs is essential, especially before accepting
• 1:52
  everything at face value.
• 1:54
  He concludes the piece saying, quote, students need to understand that they're co producing AI
• 1:59
  output through their prompts, making them not just consumers of AI unreliability, but
• 2:03
  participants in constructing it as well.
• 2:05
  So Charles, how effective do you think this framework is in helping students and then
• 2:11
  everyday people develop stronger critical thinking and analytical skills when engaging
• 2:15
  with AI?
• 2:16
  I really like this article because I do
• 2:18
  think it gives you a framework for how to think about this.
• 2:21
  I don't know.
• 2:22
  Yeah.
• 2:22
  I feel like it's a really good rigorous way to interrogate outputs, and I fear that I still
• 2:28
  meet a lot of people who type something in, wait for the answer to come back.
• 2:32
  Don't even give it a quick scan and just say, well, that's the answer.
• 2:34
  Even if it has citations.
• 2:36
  And so I think in the same way that people started to learn, hey, some sources on the
• 2:41
  Internet are more trustworthy than others.
• 2:43
  And just because Google presents it as a search result
• 2:45
  doesn't mean that it's true.
• 2:47
  And there are people out here trying to actively influence what Google shows you in
• 2:51
  your search results.
• 2:52
  I think people are going to develop the same kind of skills with chatbots.
• 2:57
  I think the harder thing is it's one thing when you're delivered back a link and you have to
• 3:02
  click on the link and try to assess, does this look real?
• 3:04
  When instead what you get back is a full on answer or a document or thousands of words of
• 3:11
  text.
• 3:11
  I don't know that most people have the patience, rigor, or time to fully interrogate
• 3:17
  this.
• 3:17
  So I think a lot about what's the temptation.
• 3:19
  I think the temptation is you could know that AI outputs are unreliable.
• 3:23
  I really should fact check this thing, but I got a meeting in five minutes.
• 3:27
  I need to know the answer to this thing, or I wanna go in sounding smart.
• 3:30
  I've gotta run the risk that hopefully I won't
• 3:32
  say something that's an obvious hallucination or a silly fact.
• 3:35
  But, you know, I use AI all day, all the time, and sometimes it gives me an up, and I'm like,
• 3:39
  that's not right.
• 3:40
  And I know it's not right because I know intuitively, like, what the answer should look
• 3:43
  like.
• 3:44
  And I'll say, hey.
• 3:44
  That's not right.
• 3:45
  Oh, you know, you're right.
• 3:46
  That's wrong.
• 3:47
  Sorry about that.
• 3:48
  I don't know that we're at the sophistication level yet where everybody's asking the kind of
• 3:52
  questions the headache is, but I think we're gonna have to if we want people to be
• 3:56
  responsible consumers and users of AI output.
• 3:59
  I think we have this weird dichotomy where employers are like, I wanna hire people who are
• 4:03
  skilled with AI, but I also want them to go to college and learn critical thinking skills and
• 4:08
  not use AI.
• 4:08
  I'm like, you're actually putting a lot of pressure on on educational institutions to
• 4:12
  teach people how to use this stuff responsibly.
• 4:15
  And then they're going to a corporate environment where the standards they learned in
• 4:18
  school around rigor and responsibilities may not apply, either because the company has
• 4:22
  different values or everybody else who works at that company didn't have the same trend.
• 4:26
  And, you know, you're seeing these examples of, like, people, you know, lawyers citing cases
• 4:30
  that don't exist, consulting firms publishing reports with the one thing that gives me pause
• 4:35
  is it's not like everybody really got better at using Google.
• 4:39
  It's not like people started using booleans and quotes.
• 4:41
  I think what happened is Google was like, people are bad at searching.
• 4:44
  So on the back end, rather than having people fix their queries, on the front end, we will
• 4:49
  fix whether it's typos, whatever.
• 4:50
  We're do it on the back end, and we're gonna figure out what this person's actually trying
• 4:53
  to solve for.
• 4:54
  And I wonder if LLMs will have the path, which is it'd be nice to think that most people are
• 4:59
  gonna become more critical thinkers and smarter and more willing to interrogate the results.
• 5:04
  The other option is that the LLMs could just decide people are lazy.
• 5:08
  And if we wanna maintain sub supplements of trustworthiness, we have to figure out at the
• 5:12
  back end how to fix some of these things.
• 5:14
  I don't know.
• 5:15
  A few years ago, I read this book called Cointelligence by Ethan Malik, which I read it
• 5:19
  and I kind of got it.
• 5:20
  And I was like, oh, I don't
• 5:21
  work this way.
• 5:22
  A lot of it's like, think about
• 5:23
  AI as a coworker, partner that's gonna work alongside you.
• 5:26
  You guys are gonna co create things.
• 5:28
  And and now that's basically what I do all day, and that makes a lot more sense to me now.
• 5:32
  The one thing I will also call out is that while we're training ourselves to be better and
• 5:36
  more critical users of AI, businesses on the back end who work d to c, they're also
• 5:40
  reconfiguring their entire website, everything they do that interacts, to work better on the
• 5:44
  rails of AI.
• 5:45
  So like there are companies and startups that are created to basically strip your entire web
• 5:49
  site and, like, reconfigure it so that agents will crawl and pick up on things a lot quicker
• 5:54
  so that it surfaces up to you on the foundational elements when you're using chat.
• 5:58
  You're abandoning search.
• 5:59
  So SEO maybe doesn't mean the same thing anymore.
• 6:02
  So basically, how do you reconfigure it?
• 6:03
  It's gonna be two things at once, which will continue to be something we have to watch the
• 6:08
  muscle.
• 6:08
  I mean, I know you just touched upon it, but I know you've been experimenting lot building
• 6:12
  your own AI tools.
• 6:12
  Like, how much deeper do you feel your understanding of AI and underlying models has
• 6:18
  gotten through that process versus just being using it at a a higher level?
• 6:23
  I think the biggest for me has been the more structured you are and your thinking, the
• 6:28
  better the results you get.
• 6:30
  Systems.
• 6:31
  I've built a bunch of tools for my software.
• 6:33
  The original version of the thing I want to build, the first file I wrote.
• 6:37
  I was like, this is pretty detailed.
• 6:39
  I think this covers most of the use cases.
• 6:41
  And then I hand it to an LLM, it's like, well, what about these 10 edge cases that you don't
• 6:44
  explicitly address in this document?
• 6:46
  I'm like, oh, well, you're you're right.
• 6:48
  I didn't It's address those 10 things.
• 6:50
  Yeah.
• 6:50
  I do care about those.
• 6:52
  It's funny.
• 6:52
  If you want a machine to operate on top of this thing,
• 6:55
  you have to give it very specific instructions.
• 6:57
  And I think what I've learned is like, oh, I have a lot of mental shortcuts that I use, that
• 7:02
  as I write them down and make them explicit, they don't make as much sense to me.
• 7:06
  And like part of this has been building these tools has caused me to go back and revisit some
• 7:10
  of my core assumptions or look at gaps and say, well, why do you skip that step?
• 7:15
  I built this reference request generator skill in Clod that, like, can figure out what people
• 7:19
  should I talk to if I'm trying to get build the more complete picture of a founder.
• 7:23
  And there's some things I used to believe.
• 7:25
  Like, for example, in a world where many more people have raised FedRAMP and not all of those
• 7:29
  relationships are going well, does the fact that their previous investors don't love them
• 7:34
  mean as much today as they used to?
• 7:37
  And also the person who wrote the check might not be as experienced as an investor.
• 7:40
  I'm like, oh, maybe it'll change the weight or just the framework I have.
• 7:43
  And that's something that I hadn't really thought until I started writing down all of the
• 7:47
  steps of how I'd pay stuff.
• 7:49
  So I I find that instead of becoming lazier, it's made me be much more explicit.
• 7:54
  And in being explicit, sometimes I'm like, well, that's dumb.
• 7:56
  Or I I I no longer believe that that's the best practice I wanna follow.
• 8:00
  I don't know.
• 8:01
  I feel like
• 8:01
  it's been really powerful.
• 8:02
  You can see here, basically, the impact of when you change one of those levers, how much it
• 8:06
  does affect the output on the other end.
• 8:09
  I mean, other thing I'll say is I've never really had a reason to build a bunch of repos
• 8:13
  in GitHub, now I have a bunch of them and the ability to see what happens to the quality of
• 8:18
  your output as you iterate on something over time with versions.
• 8:22
  It's been astounding to me when I look at the first version of stuff I start with.
• 8:25
  I look at it as now.
• 8:26
  It's like, why did I even think the first version would produce a remotely interesting
• 8:30
  result compared to the thing I have now?
• 8:33
  And that'll be the difference between people who are actively using AI and leveraging it to
• 8:37
  create, like, really self sustaining, long term viable outputs versus, I think, people who are
• 8:43
  their prompts will be maybe, like, a couple of chats back and forth, and they'll just kind of
• 8:47
  rinse and reuse that over and over again because it does pass the job.
• 8:51
  And to your exact point in the beginning, I don't know, like how much free time do people
• 8:55
  wanna use to basically do the hard work upfront is basically gonna be the question.
• 8:58
  Yeah.
• 8:59
  I've like given myself a development budget of like, I won't say.
• 9:02
  It's more time than I it's more time than invited.
• 9:05
  If I didn't have to work, I would probably do it five or six hours a day.
• 9:10
  And that that's not my job, but and my family and and many other things in life that I care
• 9:15
  of.
• 9:15
  But I found that it's been a really interesting educational experience.
• 9:19
  Well, our second piece is from the Substack, but this time different, titled, Is AI the New
• 9:25
  Shadow Bank?
• 9:26
  And then in parentheses, yes.
• 9:28
  The piece references Zoltan Pozar's seminal work on shadow banking, which revealed that
• 9:33
  much modern credit creation happens off official bank balance sheet in these opaque
• 9:39
  shadow markets.
• 9:40
  In this system, collateral, not cash, becomes the true form of money with assets like
• 9:45
  treasury bills or mortgage backed securities serving as the basis for future lending and
• 9:50
  liquidity.
• 9:51
  The piece claimed that a similar shadow credit system has now emerged around AI, but instead
• 9:57
  of mortgages and traditional financial assets, the collateral is AI hardware, compute
• 10:03
  contracts and access models.
• 10:05
  And through this circular self reinforcing system, the same quote unquote stack of GPUs is
• 10:12
  borrowed against, resold, re pledged multiple times, just as mortgaged assets were re
• 10:19
  hypothecated back before 2008.
• 10:22
  The piece closes with a quote, the real miracle isn't the AI.
• 10:26
  It's not we've taught credit to dream again.
• 10:29
  Obviously, wanted your thoughts on this piece.
• 10:33
  Oh, I got this from my friend.
• 10:35
  We're in a GP group chat, and he sent it.
• 10:37
  And my first, was like, this is giving me red string conspiracy theory vibes the first time I
• 10:42
  And read then as I reread it, I was like, this is tying together so many of the things I've
• 10:48
  been struggling to reconcile, which is it's not just a circular, hey.
• 10:51
  I'm gonna I'm gonna put $30,000,000,000 into your company, and you're buy $30 worth of stuff
• 10:56
  for me.
• 10:57
  All these circular transactions is also kind of like, what are all these underlying assets?
• 11:01
  And like, how are they being financialized, for lack of a better word?
• 11:05
  The thing that I've been most worried about, which is there's so much of this stuff that's
• 11:09
  intertwined and interlinked.
• 11:11
  Do we actually understand how it all works?
• 11:13
  Like, we have companies that are signing up for capital expenditures that there's no, as far as
• 11:18
  I can tell, no feasible way for them to finance them under any reasonable set of expectations.
• 11:23
  But they're making them in satin steals, and what does that mean?
• 11:26
  What is the value of a collateralized GPU that's two generations old?
• 11:30
  And I do think that I said this to my friend a while ago.
• 11:33
  This is something that's really been on my mind.
• 11:35
  It is very hard to blow up the economy with an equity.
• 11:39
  You can do it.
• 11:40
  We did it in 2000.
• 11:41
  It is very easy to blow up the economy with debt, and we did that in o eight.
• 11:45
  Then I'm like, if you ask me what's scarier, o one or o eight, o eight, o eight, way scarier.
• 11:50
  When when equity markets seize up, companies have problems, but the rest of the world kind
• 11:55
  of functions.
• 11:56
  When debt markets cease, it's like turning off the oxygen that, like, keeps the economy
• 12:00
  running.
• 12:01
  Because even businesses that have nothing to do with AI will find, hey, I can't borrow money to
• 12:06
  finance this tractor for my farm or to get a mortgage.
• 12:09
  And and I worry that, like, I don't know who's watching this.
• 12:12
  I don't know who's, like, watching the store.
• 12:14
  Last time we learned, people we thought were watching the store were watching the part of
• 12:18
  the store they could see.
• 12:19
  They're right by the banking industry, but didn't really understand the side.
• 12:22
  And I I I'm not gonna say that, like, the AI related investments have reached the scale of
• 12:27
  the mortgage backed securities crisis.
• 12:30
  But I don't know anybody looking at this stuff and saying something here feels off.
• 12:34
  And I mean, I loved reading through, I guess, what Poets are kind of walked through from like
• 12:39
  a shadow banking concept.
• 12:41
  I mean, the whole premise is based on the fact that shadow bank is whatever the financial
• 12:46
  system treats as safe, liquid, and flexible.
• 12:49
  And so I was starting to think of like, at what point did we all decide that all of the
• 12:54
  infrastructure of which AI is built upon is safe, liquid?
• 12:57
  I think it was because that stuff was so scarce.
• 13:00
  Then it felt like, oh, like GPUs, capacity, bottlenecks were scarce.
• 13:05
  So if you had access to it, of course, it's valuable.
• 13:07
  And of course and it's valuable today, and it was hard to conceive of a world where those
• 13:10
  things could become so plentiful that they would decline in value.
• 13:14
  I hope we don't end up in that world.
• 13:16
  But I have, like, so many questions because right now, a lot of this is being propped up by
• 13:21
  fundamental optimists.
• 13:23
  People are like, this is this is all gonna work.
• 13:26
  And what happens if even a modicum of doubt gets peeped into the system?
• 13:31
  Well, that's my last question.
• 13:32
  Kind of like, what does that doubt look like, and what form does it take?
• 13:36
  Because we've been talking about this infrastructure debt issue for months, and I
• 13:41
  think now I'm starting to see more posts pour out about it.
• 13:45
  But the real question remains is, has it ranged investor sentiment?
• 13:49
  And also, does it matter?
• 13:51
  And actions as it pertains to the broader industry overall.
• 13:54
  I read, like, a similar James Day article about how basically there's a bunch of data centers
• 13:59
  being built where the math simply, like, cannot work.
• 14:01
  And I think, to me, that's the thing I'm looking for is, like, right now, data centers
• 14:06
  have been a consensus good investment because of the demands for power, compute capacity.
• 14:12
  How could you bet against data centers?
• 14:13
  I I wouldn't wanna necessarily take the other side of that trade.
• 14:16
  But I think if you were to data centers not pencil as investment scenario, either because
• 14:22
  they're built and not utilized or they're built and utilized, but people can't operate them
• 14:27
  profitably.
• 14:28
  Without word data center and GPU financing, the whole thing grinds to a halt.
• 14:33
  And that those are largely largely the debt market.
• 14:36
  That's financing those projects purchases.
• 14:39
  Look, Mia, if we continue to come up with hundreds of billions of dollars a year, we can
• 14:43
  keep the party going.
• 14:45
  I'm just not sure that can happen.
• 14:46
  Yeah.
• 14:47
  Right now in the middle, it feels like we talked about this internally, I feel like we
• 14:51
  have a pretty grounded sense of, like, optimism and then questions that were made.
• 14:54
  I didn't feel like my peers around me share that same question all the time.
• 15:00
  So I think there's so much more money to be spent here.
• 15:03
  Like, I don't assume that there's gonna be a hold anytime soon.
• 15:05
  Also on the venture side, I think that'll be delayed as well.
• 15:09
  Well, we might have a hold on the credit and then venture will follow later.
• 15:12
  It's not one for the other.
• 15:13
  When it happens, venture will do what we always do, which is we will overreact.
• 15:17
  When it gets bad, everyone will be like, stop investing, turn off the tabs.
• 15:20
  That's what we do.
• 15:21
  Well, we are a lot of time today.
• 15:23
  Thank you, Charles, for stopping through.
• 15:25
  Thank you to our lizard for coming in as well.
• 15:27
  Please like and subscribe, and we will talk to you all next week.
• 15:30
  Bye.