Latest / Masters in Business / Jon McAuliffe on Innovation and Statistical Methods

Transcript

Jon McAuliffe on Innovation and Statistical Methods · Masters in Business

• 0:02
  Speaker 1: This is Master's in Business with Barry Ridholds on Bloomberg Radio.
• 0:08
  Speaker 2: This week on the podcast Strap Yourself In, I have
• 0:12
  Speaker 2: another extra special guest. John mccauliffe is co founder and
• 0:16
  Speaker 2: chief investment officer at the Volleyon Group. They're a five
• 0:19
  Speaker 2: billion dollar hedge funds and one of the earliest shops
• 0:24
  Speaker 2: to ever use machine learning as it applies to training
• 0:27
  Speaker 2: and investment management decisions. It is a full systematic approach
• 0:33
  Speaker 2: to using computer horsepower and database and machine learning and
• 0:39
  Speaker 2: their own predictive engine to make investments and trades, and
• 0:44
  Speaker 2: it's managed to put together quite a track record. Previously,
• 0:48
  Speaker 2: John was at d SHAW where he ran statistical arbitrage.
• 0:53
  Speaker 2: He is one of the people who worked on the
• 0:56
  Speaker 2: Amazon recommendation engine, and he is currently a professor of
• 1:01
  Speaker 2: statistics at Berkeley. I don't even know where to begin,
• 1:06
  Speaker 2: other than say, if you're interested in AI or machine
• 1:10
  Speaker 2: learning or quantitative strategies, this is just a masterclass in
• 1:16
  Speaker 2: how it's done by one of the first people in
• 1:18
  Speaker 2: the space to not only do this sort of machine
• 1:22
  Speaker 2: learning and apply it to investing, but one of the best.
• 1:26
  Speaker 2: I think this is a fascinating conversation and I believe
• 1:29
  Speaker 2: you will find it to be so. Also, with no
• 1:32
  Speaker 2: further ado, my discussion with volleyon groups. John mccauliffe. John mccauliff,
• 1:37
  Speaker 2: Welcome to Bloomberg.
• 1:40
  Speaker 1: Thanks, Barry. I'm really happy to be here.
• 1:41
  Speaker 2: So let's talk a little bit about your academic background. First,
• 1:45
  Speaker 2: you start out undergrad computer science and applied mathematics at
• 1:49
  Speaker 2: Harvard before you go on to get a PhD from
• 1:54
  Speaker 2: California Berkeley. What led to a career in data analysis?
• 1:58
  Speaker 2: How early did you know that's what you wanted to do?
• 2:01
  Speaker 1: Well, it was a winding path. Actually, I was very
• 2:06
  Speaker 1: interested in international relations and foreign languages when I was
• 2:11
  Speaker 1: finishing high school. In fact, I spent the last year
• 2:13
  Speaker 1: of high school as an exchange student in Germany. And
• 2:17
  Speaker 1: so when I got to college, I was expecting to
• 2:20
  Speaker 1: major in government and go on to maybe work in
• 2:23
  Speaker 1: the foreign service something like that.
• 2:25
  Speaker 2: Really, so this is a big shift from your original expectations.
• 2:30
  Speaker 1: Yeah, it took about one semester for me to realize
• 2:33
  Speaker 1: that none of the questions that were being asked in
• 2:36
  Speaker 1: my classes had definitive and correct answers.
• 2:40
  Speaker 2: Did that frustrate you?
• 2:41
  Speaker 1: It did frustrate me. Yeah, And so I stayed home
• 2:45
  Speaker 1: over winter I stayed Excuse me, I didn't go home.
• 2:48
  Speaker 1: I stayed at college over winter break to try to
• 2:51
  Speaker 1: sort out what the heck I was going to do,
• 2:52
  Speaker 1: because I could see that it wasn't My plan was
• 2:54
  Speaker 1: in disarray. And I'd always been interested in computers, had
• 2:58
  Speaker 1: played around with computers, never done anything very serious, but
• 3:02
  Speaker 1: I thought I might as well give it a shot,
• 3:05
  Speaker 1: and so in the spring semester I took my first
• 3:07
  Speaker 1: computer science course. And when you write software, everything has
• 3:13
  Speaker 1: a right answer. It either does what you wanted to
• 3:15
  Speaker 1: do or.
• 3:16
  Speaker 2: It doesn't, does not compile exactly. So that's really quite
• 3:20
  Speaker 2: quite fascinating. So what led you from Berkeley to d
• 3:27
  Speaker 2: sure that they're one of the first quand shops. How
• 3:30
  Speaker 2: did you get there? What sort of research did Yeah?
• 3:31
  Speaker 1: I actually I spent time at d Shot in between
• 3:34
  Speaker 1: my undergrad and my PhD program, So it was after
• 3:39
  Speaker 1: Harvard that I went to that show.
• 3:40
  Speaker 2: Did that light an interest in using machine learning and
• 3:45
  Speaker 2: computers applied to finance or what was that experience like?
• 3:49
  Speaker 1: Yeah, it made me really interested in and excited about
• 3:54
  Speaker 1: using statistical thinking and data analysis to sort of understand
• 3:59
  Speaker 1: then amics of securities prices. Machine learning did not play
• 4:02
  Speaker 1: really a role at that time, I think, not at
• 4:05
  Speaker 1: d SHAW, but you know, probably nowhere it was too
• 4:08
  Speaker 1: immature a feel in the nineties. But I had already
• 4:14
  Speaker 1: been curious and interested in using these kinds of statistical
• 4:20
  Speaker 1: tools in trading and in investing when I was finishing
• 4:23
  Speaker 1: college and then at d SHAW. You know, I had
• 4:25
  Speaker 1: brilliant colleagues and we were working on hard problems. So
• 4:28
  Speaker 1: I really, I really got a lot.
• 4:30
  Speaker 2: Of us still one of the top performing hedge funds,
• 4:32
  Speaker 2: one of the earliest quant hedge funds, A great a
• 4:34
  Speaker 2: great place to absolutely cut your teeth at. So was
• 4:37
  Speaker 2: it Harvard d SHAW and then Berkeley?
• 4:40
  Speaker 1: Yeah, that's right?
• 4:40
  Speaker 2: And then from Berkeley? How did you end up at Amazon?
• 4:44
  Speaker 1: I guess I should correct myself. There was a year
• 4:47
  Speaker 1: at Amazon after d Eshaw, but before Berkeley.
• 4:50
  Speaker 2: And am I reading this correctly? The recommendation engine that
• 4:55
  Speaker 2: Amazon uses you helped develop?
• 4:58
  Speaker 1: I would say I worked on it. I would you
• 5:01
  Speaker 1: know it existed, It was in place when I got there,
• 5:03
  Speaker 1: and sort of the things that are familiar about the
• 5:07
  Speaker 1: recommendation engine had already been built by my manager and
• 5:12
  Speaker 1: his colleagues. But I worked I did research on improvements
• 5:17
  Speaker 1: and different ways of forming recommendations. It was funny because
• 5:21
  Speaker 1: at the time, the entire database of purchase history for
• 5:29
  Speaker 1: all of Amazon fit in one twenty gigabyte file on
• 5:32
  Speaker 1: a disc, so I could just load it on my
• 5:34
  Speaker 1: computer and run.
• 5:35
  Speaker 2: Now I don't think we could do that anymore. We
• 5:37
  Speaker 2: could not, so, thank goodness is Amica Zone cloud services,
• 5:39
  Speaker 2: so you could put what is it, twenty five years
• 5:43
  Speaker 2: and hundreds of billions of dollars of transactions. So my
• 5:50
  Speaker 2: assumption is products like that are highly iterative. The first
• 5:53
  Speaker 2: version is all right, it does a half decent job,
• 5:56
  Speaker 2: and then it gets better, and then it starts to
• 5:58
  Speaker 2: get almost spookily good. It's like, oh, how much of
• 6:01
  Speaker 2: that is just the size of the database, and how
• 6:04
  Speaker 2: much of that is just a clever algorithm.
• 6:08
  Speaker 1: Well, that's a great question, because the two are inextricably linked.
• 6:14
  Speaker 1: The way that you make algorithms great is by making
• 6:18
  Speaker 1: them more powerful, more expressive, able to describe lots of
• 6:21
  Speaker 1: different kinds of patterns and relationships. But those kinds of
• 6:25
  Speaker 1: approaches need huge amounts of data in order to correctly
• 6:29
  Speaker 1: sort out what's signal and what's noise. The more expressive.
• 6:32
  Speaker 1: A tool like that is like a recommender system, the
• 6:35
  Speaker 1: more prone it is to mistake one time noise for
• 6:40
  Speaker 1: persistent signal, and that is a recurring theme in statistical prediction.
• 6:44
  Speaker 1: It is really the central problem in statistical predictions. So
• 6:47
  Speaker 1: you have it in recommender systems, you have it in
• 6:50
  Speaker 1: predicting price action, in the problems that we solve, and elsewhere.
• 6:55
  Speaker 2: There was a pretty infamous New York Times article a
• 6:57
  Speaker 2: couple of years ago about targets using their own recommender
• 7:03
  Speaker 2: system and sending out maternity things to people. A dad
• 7:09
  Speaker 2: gets his young teenage daughters what is this and goes
• 7:13
  Speaker 2: in to yell at them, and turns out she was
• 7:15
  Speaker 2: pregnant and they had pieced it together. How far of
• 7:20
  Speaker 2: a leap is it from these systems to much more
• 7:23
  Speaker 2: sophisticated machine learning and even large language models.
• 7:30
  Speaker 1: It's the answer, it turns out, is that it's a
• 7:32
  Speaker 1: question of scale. That wasn't at all obvious before GPT
• 7:38
  Speaker 1: three and chat GPT, But it just turned out that
• 7:42
  Speaker 1: when you have, for example, GPT is built from a
• 7:46
  Speaker 1: database of sentences in English, it's got a trillion words
• 7:50
  Speaker 1: in it that database, and when you take a trillion
• 7:53
  Speaker 1: words and you use it to fit a model that
• 7:55
  Speaker 1: has one hundred and seventy five billion parameters. There is
• 7:59
  Speaker 1: apparently a kind of transition where things become, you know,
• 8:02
  Speaker 1: frankly astounding. I don't I think, I don't think that
• 8:05
  Speaker 1: anybody who isn't astounded is telling the truth.
• 8:07
  Speaker 2: Right. It's eerie is in terms of how sophisticated it is,
• 8:13
  Speaker 2: but it's also kind of surprising in terms of I
• 8:17
  Speaker 2: guess what the programs like to call hallucinations. I guess
• 8:20
  Speaker 2: if you're using the Internet as your base model, Hey,
• 8:24
  Speaker 2: there's one or two things on the Internet that are wrong,
• 8:27
  Speaker 2: so of course that's going to show up in something
• 8:30
  Speaker 2: like chap GPT.
• 8:31
  Speaker 1: Yeah, you know. Underlyingly, there's this tool GPT three that's
• 8:36
  Speaker 1: really the engine that powers jed GPT, and that tool
• 8:40
  Speaker 1: it has one goal. It's a simple goal. You show
• 8:43
  Speaker 1: at the beginning of a sentence, and it predicts the
• 8:46
  Speaker 1: next word in the sentence, and that's all it is
• 8:48
  Speaker 1: trained to do. I mean, it really is actually that simple.
• 8:51
  Speaker 2: It's a dumb program that looks smart.
• 8:53
  Speaker 1: If you like. But the thing about predicting the next
• 8:58
  Speaker 1: word in a sentence is whether you know the sequence
• 9:01
  Speaker 1: of words that's being output, is leading to something that
• 9:04
  Speaker 1: is true or false, is irrelevant. The only thing that
• 9:07
  Speaker 1: it is trained to do is make highly accurate predictions
• 9:10
  Speaker 1: of next words.
• 9:11
  Speaker 2: So when I said it's really very sophisticated, it just
• 9:16
  Speaker 2: for what we tend to call this artificial intelligence. But
• 9:20
  Speaker 2: I've read a number of people said, hey, this really
• 9:22
  Speaker 2: is an AI. This is something a little more rudimentary.
• 9:26
  Speaker 1: Yeah, I think, you know, a critic would say that
• 9:31
  Speaker 1: artificial intelligence is a complete misnomer. There's sort of nothing
• 9:36
  Speaker 1: remotely intelligent in the colloquial sense about these systems. And
• 9:40
  Speaker 1: then a common defense in AI research is that artificial
• 9:44
  Speaker 1: intelligence is a moving target. As soon as you build
• 9:46
  Speaker 1: a system that does something quasi magical that was the
• 9:50
  Speaker 1: old yardstick of intelligence, then the goalposts get moved by
• 9:53
  Speaker 1: the people who are supplying the evaluations. And I guess
• 9:57
  Speaker 1: I would sit somewhere in between. I think the language
• 9:59
  Speaker 1: is unfortunate because it's so easily misconstrued. I wouldn't call
• 10:03
  Speaker 1: the system dumb, and I wouldn't call it smart. It's
• 10:06
  Speaker 1: you know, those are those are not characteristics of these systems.
• 10:08
  Speaker 2: But it's complex and sophisticated.
• 10:10
  Speaker 1: It certainly is it has one hundred and seventy five
• 10:12
  Speaker 1: billion parameters. That doesn't fit your definition of complex you
• 10:15
  Speaker 1: know what would?
• 10:16
  Speaker 2: Yeah, that works for me. So your in your career line,
• 10:21
  Speaker 2: where is aphametrics and what was that recommendation engine? Like?
• 10:26
  Speaker 1: Yeah, So that was work I did as a summer
• 10:30
  Speaker 1: research intern during my PhD. And that work was about
• 10:35
  Speaker 1: what's called the problem is called genotype calling. So genotype calling,
• 10:40
  Speaker 1: I'll explain, Barry, do you have an identical twin? I
• 10:43
  Speaker 1: do not, Okay, So I can safely say your genome
• 10:46
  Speaker 1: is unique in the world. There's no one else who
• 10:49
  Speaker 1: has exactly your genome. On the other hand, if you
• 10:52
  Speaker 1: were to lay your genome in mind alongside each other
• 10:55
  Speaker 1: lined up, they would be ninety nine point nine percent identical.
• 10:58
  Speaker 1: About one position in a thousand is different. But those
• 11:03
  Speaker 1: differences are what caused you to be you and me
• 11:05
  Speaker 1: to be me. So they're obviously of intense kind of
• 11:08
  Speaker 1: scientific and applied interest. And so it's very important to
• 11:11
  Speaker 1: be able to take a sort of a sample of
• 11:15
  Speaker 1: your DNA and quickly produce a profile of all the
• 11:20
  Speaker 1: places that have variability what your particular values are, Okay,
• 11:24
  Speaker 1: And that problem is the genotyping problem.
• 11:27
  Speaker 2: And this used to be a very expensive, very complex
• 11:31
  Speaker 2: problem to solve that. We've spent billions of dollars figuring
• 11:34
  Speaker 2: out now a lot faster, a lot cheaper.
• 11:37
  Speaker 1: A lot faster. In fact, even the technology I worked
• 11:40
  Speaker 1: on in two thousand and five two thousand and four
• 11:43
  Speaker 1: is multiple generations old and not really what's used anymore.
• 11:47
  Speaker 2: So let's talk about what you did at the efficient frontier.
• 11:51
  Speaker 2: Explain what real time click prediction rules are and how
• 11:56
  Speaker 2: it works for a keyword search.
• 11:59
  Speaker 1: Sure, the revenue engine that drives Google is search keyword ads, right,
• 12:06
  Speaker 1: So every time you do a search, at the top
• 12:07
  Speaker 1: you see ad ad AD, And so how do those
• 12:10
  Speaker 1: ads get there? Well, actually it's surprising maybe if you
• 12:14
  Speaker 1: don't know about it, but every single time you type
• 12:16
  Speaker 1: in a search term on Google and hit return, a
• 12:19
  Speaker 1: very fast auction takes place, and a whole bunch of
• 12:23
  Speaker 1: companies running software bid electronically to place their ads at
• 12:28
  Speaker 1: the top of your search results. And the more or
• 12:33
  Speaker 1: less the results that are shown on the page are
• 12:36
  Speaker 1: in order of how much they bid. It's not quite true,
• 12:38
  Speaker 1: but you could think of it. It's true.
• 12:40
  Speaker 2: A rough outline. So the first three sponsored results on
• 12:44
  Speaker 2: a Google page, go through that auction process, and I
• 12:48
  Speaker 2: think at this point everybody knows what page rank is
• 12:50
  Speaker 2: for for the rest of that that's right, And that
• 12:53
  Speaker 2: seemed to be Google secret sauce early on.
• 12:56
  Speaker 1: Right, Well, you know, to talk about the the ad placement.
• 13:01
  Speaker 1: So the people who are supplying the ad, who are
• 13:04
  Speaker 1: participating in the auctions, they have a problem, which is
• 13:06
  Speaker 1: how much to bid, right, And so how would you
• 13:08
  Speaker 1: decide how much to bid? Well, you want to know
• 13:13
  Speaker 1: basically the probability that somebody is going to click on
• 13:15
  Speaker 1: your ad, and then you would multiply that by how
• 13:19
  Speaker 1: much money you make eventually if they click. And that's
• 13:22
  Speaker 1: kind of an expectation of how much money you'll make.
• 13:25
  Speaker 1: And so then you gear your bid price to make
• 13:30
  Speaker 1: sure that it's going to be profitable for you. And
• 13:32
  Speaker 1: then so really you have to make a decision about
• 13:36
  Speaker 1: what this click through rate is going to be. You
• 13:38
  Speaker 1: have to predict the click through probability.
• 13:40
  Speaker 2: So I was going to say, this sounds like it's
• 13:42
  Speaker 2: a very sophisticated application of computer science probability and statistics.
• 13:48
  Speaker 2: And if you do it right, you make money, and
• 13:50
  Speaker 2: if you do it wrong, your ad budget is a
• 13:54
  Speaker 2: money loser.
• 13:55
  Speaker 1: That's right.
• 13:55
  Speaker 2: Huh. So tell us a little bit about your doctorate,
• 13:58
  Speaker 2: what you wrote about for your PhD at Berkeley.
• 14:02
  Speaker 1: Yeah, so we're back to genomes. Actually, this was around
• 14:06
  Speaker 1: the time when I was in my first year of
• 14:08
  Speaker 1: my PhD program, is when the human genome was published
• 14:11
  Speaker 1: in Nature. So it was kind of really the beginning
• 14:16
  Speaker 1: of the explosion of work on kind of high throughput,
• 14:21
  Speaker 1: large scale genetics research. And one really important question after
• 14:26
  Speaker 1: you've sequenced a genome is well, what are all the
• 14:28
  Speaker 1: bits of it doing. You can look at a string
• 14:30
  Speaker 1: of DNA. It's just made up of these kind of
• 14:33
  Speaker 1: four letters, but you don't want to just know the
• 14:37
  Speaker 1: four letters. They're kind of a code. And some parts
• 14:39
  Speaker 1: of the DNA represent useful stuff that is being turned
• 14:44
  Speaker 1: by your cell into proteins and et cetera, and other
• 14:47
  Speaker 1: parts of the DNA don't appear to have any function
• 14:49
  Speaker 1: at all, and it's really important to know which is
• 14:51
  Speaker 1: which as a biology researcher. And so it's you know,
• 14:56
  Speaker 1: for a long time before high throughput sequencing, biologists would
• 15:01
  Speaker 1: be in the lab and they would very laboriously look
• 15:03
  Speaker 1: at very tiny segments of DNA and establish what their
• 15:05
  Speaker 1: function was. But now we have the whole human genome
• 15:09
  Speaker 1: sitting on disk, and we would like to be able
• 15:10
  Speaker 1: to just run an analysis on it and have the
• 15:13
  Speaker 1: computer spit out everything that is functional and not functional.
• 15:17
  Speaker 1: And so that's the problem I worked on. And a
• 15:21
  Speaker 1: really important insight is that you can take advantage of
• 15:24
  Speaker 1: the idea of natural selection and the idea of evolution
• 15:29
  Speaker 1: to help you. And the way you do that is
• 15:32
  Speaker 1: you have the human genome, you sequence a bunch of
• 15:35
  Speaker 1: primate genomes nearby relatives of the union, and you lay
• 15:38
  Speaker 1: all those genomes on top of each other, and then
• 15:41
  Speaker 1: you look for places where all of the genomes agree. Right,
• 15:45
  Speaker 1: there hasn't been variation that's happening through mutations. And why
• 15:50
  Speaker 1: hasn't there been, Well, the biggest force that throws out
• 15:53
  Speaker 1: variation is natural selection. If you get a mutation in
• 15:56
  Speaker 1: a part of your genome that really matters, then you're
• 15:59
  Speaker 1: kind of on it and you won't have progeny and
• 16:02
  Speaker 1: that'll get stamped out. So natural selection is this very
• 16:06
  Speaker 1: strong force that's causing DNA not to change. And so
• 16:10
  Speaker 1: when you when you make these primate alignments, you can
• 16:13
  Speaker 1: really leverage that fact and look for conservation and use
• 16:18
  Speaker 1: that as a big signal that something is functional.
• 16:20
  Speaker 2: Huh, really really interesting. You mentioned our DNA is ninety
• 16:25
  Speaker 2: nine point ninety nine. I don't know how many places
• 16:28
  Speaker 2: to the right of the decimal point you would want
• 16:30
  Speaker 2: to go, but very similar. How how similar or different
• 16:34
  Speaker 2: are we from let's say, a chimpanzee. I've always questioned,
• 16:38
  Speaker 2: there's an urban legend that they're practically the same. It
• 16:41
  Speaker 2: always seems like it's overstated two percent. So you and
• 16:46
  Speaker 2: I have a point one percent different me and the
• 16:49
  Speaker 2: average chimp. It's two point zero percent.
• 16:52
  Speaker 1: That's exactly right. Yeah, so chimps are essentially our closest
• 16:56
  Speaker 1: non human primate relatives.
• 16:58
  Speaker 2: Really really quite fascinating. So let's talk a little bit
• 17:02
  Speaker 2: about the firm. You guys were one of the earliest
• 17:05
  Speaker 2: pioneers of machine learning research. Explain a little bit what
• 17:09
  Speaker 2: the firm does.
• 17:10
  Speaker 1: Sure, so, we run trading strategies investment strategies that are
• 17:16
  Speaker 1: fully automated, so we call them fully systematic, and that
• 17:20
  Speaker 1: means that we have software systems that run every day
• 17:25
  Speaker 1: during market hours, and they take in information about the
• 17:29
  Speaker 1: characteristics of the securities we're trading. Think of stocks and
• 17:34
  Speaker 1: then they make predictions of how the prices of each
• 17:39
  Speaker 1: security is going to change over time, and then they
• 17:44
  Speaker 1: decide on changes in our inventory, changes in held positions
• 17:48
  Speaker 1: based on those predictions, and then those desired changes are
• 17:53
  Speaker 1: sent into an execution system which automatically carries them out.
• 17:56
  Speaker 2: So fully automated. Is there supervision or it's kind of
• 18:02
  Speaker 2: running on its own with a couple of checks.
• 18:05
  Speaker 1: There's lots of human diagnostic supervision, right, So there are
• 18:09
  Speaker 1: people who are watching screens full of instrumentation and telemetry
• 18:14
  Speaker 1: about what the systems are doing. But those people are
• 18:17
  Speaker 1: not taking any actions, right unless there's a problem, right,
• 18:21
  Speaker 1: and then they do.
• 18:23
  Speaker 2: So let's talk a little bit about how machines learn
• 18:26
  Speaker 2: to identify signals. I'm assuming you start with the giant
• 18:30
  Speaker 2: database that is the history of stock prices, volume movement, etc.
• 18:36
  Speaker 2: And then bring in a lot of additional things to bear.
• 18:39
  Speaker 2: What's the process like developing a particular trading strategy.
• 18:44
  Speaker 1: Yeah, so, as you're saying, we begin with a very
• 18:48
  Speaker 1: large historical data set of prices and volumes, market data
• 18:54
  Speaker 1: that kind, but importantly all kinds of other information about securities,
• 19:00
  Speaker 1: financial statement data, textual data, analyst data.
• 19:05
  Speaker 2: So it's everything from prices fundamental everything from learnings to
• 19:11
  Speaker 2: revenue to sales, etc. I'm assuming the change and the
• 19:14
  Speaker 2: delta of the change is going to be very significant
• 19:17
  Speaker 2: in that. What about macroeconomic what some people call noise,
• 19:22
  Speaker 2: but one would imagine some signal in everything from inflation
• 19:27
  Speaker 2: to interest rates to GDPs firm spending. Are those inputs
• 19:32
  Speaker 2: worthwhile or how do you think about those?
• 19:34
  Speaker 1: So we don't hold portfolios that are exposed to those things.
• 19:38
  Speaker 1: So it's really a business decision on our part. We
• 19:42
  Speaker 1: are working with institutional investors who already have as much
• 19:47
  Speaker 1: exposure as they want to things like the market or
• 19:50
  Speaker 1: to well recognized econometric risk factors like value, and so
• 19:56
  Speaker 1: they don't need our help to be exposed to those things.
• 19:58
  Speaker 1: They are very well equipped to handle that part of
• 20:01
  Speaker 1: their investment process. What we're trying to provide is the
• 20:05
  Speaker 1: most diversification possible. So we want to give them a
• 20:09
  Speaker 1: new return stream which has good and stable returns, but
• 20:14
  Speaker 1: on top of that, importantly, is also not correlated with
• 20:17
  Speaker 1: any of the other return streams that they already that
• 20:19
  Speaker 1: they already have.
• 20:20
  Speaker 2: That's interesting. So can I assume that you're applying your
• 20:25
  Speaker 2: machine learning methodology across different asset classes or is it
• 20:29
  Speaker 2: strictly equities? Oh?
• 20:30
  Speaker 1: No, We apply it to UH to equities, to credit,
• 20:34
  Speaker 1: to corporate bonds, and we trade futures contracts, and in
• 20:39
  Speaker 1: the fullness of time, we hope that we will be
• 20:41
  Speaker 1: trading kind of every security in the world.
• 20:44
  Speaker 2: So, so currently stocks, bonds, When you say futures, I
• 20:47
  Speaker 2: assume commodities, all.
• 20:48
  Speaker 1: Kinds of futures contract.
• 20:49
  Speaker 2: It's really really interesting. So it could be anything from
• 20:52
  Speaker 2: interest rate swaps to commodities to the full gamut. So,
• 20:56
  Speaker 2: so how different is this approach from what other quant
• 21:01
  Speaker 2: shops do that really focus on equities.
• 21:06
  Speaker 1: I think it's kind of the same question as asking, well,
• 21:11
  Speaker 1: what do we mean when we say we use machine
• 21:13
  Speaker 1: learning or that you know we are our principles are
• 21:16
  Speaker 1: our machine learning principles, and so how does that make
• 21:20
  Speaker 1: us different than the kind of standard approach in quantitative trading?
• 21:24
  Speaker 1: And the answer to the question really comes back to
• 21:28
  Speaker 1: this idea we mentioned a little while ago of how
• 21:31
  Speaker 1: powerful the tools are that you're using to form predictions. Right,
• 21:36
  Speaker 1: So in our business, the thing that we build is
• 21:40
  Speaker 1: called a prediction rule. Okay, that's that's our widget and
• 21:44
  Speaker 1: What a prediction rule does is it takes in a
• 21:46
  Speaker 1: bunch of input, a bunch of information about a stock
• 21:49
  Speaker 1: at a moment in time, and it hands you a
• 21:53
  Speaker 1: guess about how that stock's price is going to change
• 21:56
  Speaker 1: over some future period of time. Okay, and so there
• 22:00
  Speaker 1: is one most important question about prediction rules, which is
• 22:05
  Speaker 1: how complex are they? How much complexity do they have?
• 22:08
  Speaker 1: Complexity is a colloquial term. It's unfortunately another example of
• 22:13
  Speaker 1: a place where things can be vague or ambiguous because
• 22:18
  Speaker 1: a general purpose word has been borrowed in a technical setting.
• 22:21
  Speaker 1: But when you use the word complexity in statistical prediction,
• 22:24
  Speaker 1: there's a very specific meaning. It means how much expressive
• 22:28
  Speaker 1: power does this prediction rule have? How good a job
• 22:32
  Speaker 1: can it do of approximating what's going on in the
• 22:35
  Speaker 1: data you show it. Remember, we have these giant historical
• 22:38
  Speaker 1: data sets, and every entry in the data set looks
• 22:41
  Speaker 1: like this. What was going on with the stock at
• 22:44
  Speaker 1: a moment in a certain moment in time, it's price action,
• 22:47
  Speaker 1: it's financials analyst information. And then what did its price
• 22:52
  Speaker 1: do in the subsequent twenty four hours or the subsequent
• 22:55
  Speaker 1: fifteen minutes or whatever. Okay, and so when you talk
• 23:01
  Speaker 1: about the amount of complexity that a prediction rule has,
• 23:04
  Speaker 1: that means how well is it able to capture the
• 23:08
  Speaker 1: relationship between the things that you can show it when
• 23:11
  Speaker 1: you ask it for a prediction, and what actually happens
• 23:14
  Speaker 1: to the price. And naturally you kind of want to
• 23:18
  Speaker 1: use high complexity rules because they have a lot of
• 23:20
  Speaker 1: approximating power. They do a good job of describing anything
• 23:23
  Speaker 1: that's going on. But there are two There are two
• 23:26
  Speaker 1: disadvantages to high complexity. One is it needs a lot
• 23:30
  Speaker 1: of data, otherwise it gets fooled into thinking that randomness
• 23:34
  Speaker 1: is actually signal. And the other is that it's hard
• 23:39
  Speaker 1: to reason about what's going on under the hood. Right,
• 23:41
  Speaker 1: when you have very simple prediction rules, you can sort
• 23:45
  Speaker 1: of summarize everything that's good that they're doing in a sentence. Right,
• 23:48
  Speaker 1: you can look you can look inside them and get
• 23:50
  Speaker 1: a complete understanding of how they behave, and that's not
• 23:53
  Speaker 1: possible with high complexity prediction rules.
• 23:56
  Speaker 2: So I'm glad you brought up the concept of how
• 23:59
  Speaker 2: easy it is or how frequently you can fool an
• 24:04
  Speaker 2: algorithm or a complex rule, because sometimes the results are
• 24:08
  Speaker 2: just random. And it reminds me of the issue of
• 24:11
  Speaker 2: back testing. No one ever shows you a bad back test.
• 24:15
  Speaker 2: How do you deal with the issue of overfitting and
• 24:19
  Speaker 2: back testing that just is geared towards what already happened
• 24:23
  Speaker 2: and not what might happen in the future.
• 24:25
  Speaker 1: Yeah, that is you know, if you like the million
• 24:28
  Speaker 1: dollar question in statistical prediction, Okay, and it might you
• 24:34
  Speaker 1: might find it surprising that relatively straightforward ideas go a
• 24:38
  Speaker 1: long way here. And so let me let me just
• 24:43
  Speaker 1: describe a little scenario of how you deal you can
• 24:45
  Speaker 1: deal with this. All right, we agree, we have this
• 24:47
  Speaker 1: big historical data set, right, One thing you could do
• 24:50
  Speaker 1: is just start analyzing the heck out of that data
• 24:53
  Speaker 1: set and find a complicated prediction rule. But you're you've
• 24:56
  Speaker 1: already started doing it wrong. The first thing you do
• 24:59
  Speaker 1: before or you even look at the data is you
• 25:02
  Speaker 1: randomly pick out half of the data and you lock
• 25:04
  Speaker 1: it in a drawer. Okay, and that leads you with
• 25:07
  Speaker 1: the other half of the data that you haven't locked away.
• 25:09
  Speaker 1: On this half, you get to go hogwild. You build
• 25:12
  Speaker 1: every kind of prediction rule, simple rules, enormously complicated rules,
• 25:16
  Speaker 1: everything in between. Right, and now you can check how
• 25:20
  Speaker 1: accurate all of these prediction rules that you've built are
• 25:24
  Speaker 1: on the data that they have been looking at, and
• 25:26
  Speaker 1: the answer will always be the same. The most complex
• 25:29
  Speaker 1: rules will look the best. Of course, they have the
• 25:32
  Speaker 1: most expressive power, so naturally they do the best job
• 25:35
  Speaker 1: of describing what you showed them. The big problem is
• 25:38
  Speaker 1: that what you showed them is a mix of signal
• 25:41
  Speaker 1: and noise, and there's no way you can tell to
• 25:44
  Speaker 1: what extent a complex rule has found the signal versus
• 25:47
  Speaker 1: the noise. All you know is that it's perfectly described
• 25:49
  Speaker 1: to the data you showed it. You certainly suspect it
• 25:52
  Speaker 1: must be overfitting if it's doing that. Well, okay, so
• 25:55
  Speaker 1: now you freeze all those prediction rules. You're not allowed
• 25:59
  Speaker 1: to change them in any way anymore. And now you
• 26:01
  Speaker 1: unlock the drawer and you pull out all that data
• 26:04
  Speaker 1: that you've never looked at. You can't overfit data that
• 26:06
  Speaker 1: you never fit, and so you take that data and
• 26:09
  Speaker 1: you run it through each of these prediction rules that's frozen,
• 26:12
  Speaker 1: that you built. And now it is not the case
• 26:14
  Speaker 1: at all that the most complex rules look the best. Instead,
• 26:19
  Speaker 1: you'll see a kind of U shaped behavior where the
• 26:23
  Speaker 1: very simple rules are too simple. They've missed signal, they
• 26:25
  Speaker 1: left signal on the table. The two complex rules are
• 26:29
  Speaker 1: also doing badly because they've captured all the signal but
• 26:32
  Speaker 1: also lots of noise. And then somewhere in the middle
• 26:34
  Speaker 1: is a sweet spot where you've struck the right trade
• 26:37
  Speaker 1: off between how much expressive power the prediction rule has
• 26:43
  Speaker 1: and how good a job it is doing of avoiding
• 26:46
  Speaker 1: the mistaking of noise for signal.
• 26:49
  Speaker 2: Really really intriguing. So you guys, have you've built one
• 26:53
  Speaker 2: of the largest specialized machine learning research and development teams
• 26:57
  Speaker 2: in finance. How do you assndle a team like that
• 27:02
  Speaker 2: and how do you get the brain trust to do
• 27:05
  Speaker 2: the sort of work that's applicable to managing assets.
• 27:10
  Speaker 1: Well, the short answer is, we spend a huge amount
• 27:14
  Speaker 1: of energy on recruiting and uh, you know, identifying the
• 27:20
  Speaker 1: sort of premier people in the field of machine learning,
• 27:23
  Speaker 1: kind of both academic and practitioners, and we exhibit a
• 27:28
  Speaker 1: lot of patients. We we wait a really long time
• 27:31
  Speaker 1: to be able to find the people who are kind
• 27:33
  Speaker 1: of really the best, and that that that matters enormously
• 27:37
  Speaker 1: to us, both from the standpoint of the success of
• 27:40
  Speaker 1: the firm and also because it's something that you know,
• 27:43
  Speaker 1: we value extremely highly just having great colleagues, brilliant colleagues
• 27:47
  Speaker 1: that you know, I want to work in a place
• 27:49
  Speaker 1: where I can learn from all the people around me.
• 27:51
  Speaker 1: And you know, when when my co founder Michael Caratanev
• 27:55
  Speaker 1: and I we're talking about starting Bollion, one of the
• 28:01
  Speaker 1: reasons that was on our minds is we wanted to
• 28:04
  Speaker 1: be in control of who we worked with. You know,
• 28:07
  Speaker 1: we really wanted to be able to assemble a group
• 28:10
  Speaker 1: of people who were, you know, as brilliant as we
• 28:14
  Speaker 1: could find, but also you know, good people, people that
• 28:17
  Speaker 1: we liked, people that we were excited to collaborate with.
• 28:20
  Speaker 2: So let's talk about some of the fundamental principles Volnon
• 28:23
  Speaker 2: is built on. You reference a prediction based approach from
• 28:29
  Speaker 2: a paper Leo Briman wrote called two Cultures. Yeah, tell
• 28:33
  Speaker 2: us a little bit about what two cultures actually is.
• 28:37
  Speaker 1: Yeah. So this this paper was written about twenty years ago.
• 28:41
  Speaker 1: Leo Briman was one of the great probabilists and statisticians
• 28:46
  Speaker 1: of his generation. Berkeley professor need I say, and you know,
• 28:53
  Speaker 1: Leo had been a practitioner in statistical consulting actually for
• 28:59
  Speaker 1: quite some time. I'm in between a U. C. L.
• 29:02
  Speaker 1: A tenured job and returning to academia at Berkeley, and
• 29:07
  Speaker 1: he learned a lot in that time about actually solving
• 29:10
  Speaker 1: prediction problems and instead of hypothetically solving them in sort
• 29:15
  Speaker 1: of the academic context. And so all of his insights
• 29:20
  Speaker 1: about the difference really culminated in this paper from two
• 29:25
  Speaker 1: thousand that he wrote.
• 29:26
  Speaker 2: The difference between practical use versus academic theory if you like.
• 29:30
  Speaker 1: Yeah, And so he identified two schools of thought about
• 29:35
  Speaker 1: solving prediction problems, right, and one school is sort of
• 29:40
  Speaker 1: model based. The idea is there's some stuff you're going
• 29:44
  Speaker 1: to get to observe stock characteristics. Let's say there's a
• 29:49
  Speaker 1: thing you wish you knew future price change, let's say,
• 29:51
  Speaker 1: and there's a box in nature that turns those inputs
• 29:55
  Speaker 1: into the output, right. And in the model based school
• 29:59
  Speaker 1: of thought, you try to open that box, reason about
• 30:03
  Speaker 1: how it must work, make theories. In our case, these
• 30:06
  Speaker 1: would be sort of econometric theories, financial economics theories. And
• 30:11
  Speaker 1: then those theories have knobs, not many, and you use
• 30:14
  Speaker 1: data to set the knobs, but otherwise you believe the model.
• 30:19
  Speaker 1: And he contrasts that with the machine learning school of thought,
• 30:22
  Speaker 1: which is also has the idea of Nature's box. The
• 30:27
  Speaker 1: inputs go in, the thing you wish you knew comes out.
• 30:30
  Speaker 1: But in machine learning, you don't try to open the box.
• 30:33
  Speaker 1: You just try to approximate what the box is doing.
• 30:36
  Speaker 1: And your measure of success is predictive accuracy, and is
• 30:39
  Speaker 1: only predictive accuracy. If you build a gadget and that
• 30:43
  Speaker 1: gadget produces predictions that are really accurate they turn out
• 30:47
  Speaker 1: to look like the thing that nature produces, then that
• 30:50
  Speaker 1: is success. And at the time he wrote the paper,
• 30:55
  Speaker 1: his assessment was ninety eight percent of statistics was taking
• 30:58
  Speaker 1: the model based approach, two percent was taking the machine
• 31:01
  Speaker 1: learning approach.
• 31:02
  Speaker 2: And are those statistics still valid today or have we
• 31:05
  Speaker 2: shifted quite a bit?
• 31:06
  Speaker 1: We shifted quite a bit, And different different arenas of
• 31:11
  Speaker 1: prediction problems have different mixes these days. But even in finance,
• 31:16
  Speaker 1: I would say it's it's probably more like fifty to.
• 31:19
  Speaker 2: Fifty really that much?
• 31:21
  Speaker 1: Yeah, I think you know, And if you the logical
• 31:26
  Speaker 1: extreme is natural language modeling, which was done for decades
• 31:31
  Speaker 1: and decades in the model based approach, where you kind
• 31:35
  Speaker 1: of reasoned about linguistic characteristics of how people kind of
• 31:39
  Speaker 1: do dialogue and those models had some parameters and you
• 31:42
  Speaker 1: fit them with data, and then instead you have, as
• 31:47
  Speaker 1: we said, a database of a trillion words and a
• 31:50
  Speaker 1: tool with one hundred and seventy five billion parameters, and
• 31:53
  Speaker 1: you run that and there is no hope of completely
• 31:56
  Speaker 1: understanding what is going on inside of GPD three. But
• 31:59
  Speaker 1: nobody complains about that because the results are astounding. The
• 32:04
  Speaker 1: thing that you get is incredible. And so that is
• 32:08
  Speaker 1: by analogy, the way that we reason about running systematic
• 32:12
  Speaker 1: investment strategies. At the end of the day, predictive accuracy
• 32:16
  Speaker 1: is what creates returns for investors. Being able to give
• 32:21
  Speaker 1: complete descriptions of exactly how the predictions arise does not
• 32:25
  Speaker 1: in itself create returns for investors. Now, I'm not against
• 32:29
  Speaker 1: interpretability and simplicity all equal. I love interpretability and simplicity,
• 32:32
  Speaker 1: but all else is not equal. If you want the
• 32:35
  Speaker 1: most accurate predictions, you are going to have to sacrifice
• 32:39
  Speaker 1: some amount of simplicity. In fact, this truth is so
• 32:43
  Speaker 1: widespread that Leo gave it a name in his paper.
• 32:45
  Speaker 1: He called it Accam's dilemma. So Accam's razor is the
• 32:49
  Speaker 1: philosophical idea that you should choose the simplest explanation that
• 32:52
  Speaker 1: fits the facts. Akam's dilemma is the point that in
• 32:58
  Speaker 1: statistical prediction, simplest approach, even though you wish you could
• 33:02
  Speaker 1: choose it, is not the most accurate approach if you
• 33:04
  Speaker 1: care about predictive accuracy. If you're putting predictive accuracy first,
• 33:09
  Speaker 1: then you have to embrace a certain amount of complexity
• 33:12
  Speaker 1: and lack of interpretability.
• 33:13
  Speaker 2: Huh, that's really quite fascinating. So let's talk a little
• 33:17
  Speaker 2: bit about artificial intelligence and large language models. You follow
• 33:24
  Speaker 2: d Shaw playing in e commerce and biotech. It seems
• 33:29
  Speaker 2: like this approach to using statistics, probability, and computer science
• 33:35
  Speaker 2: is applicable to so many different fields.
• 33:38
  Speaker 1: It is. Yeah, I think you're talking about prediction problems ultimately,
• 33:43
  Speaker 1: So in recommender systems, you can think of the question
• 33:49
  Speaker 1: as being well, if I had to predict what thing
• 33:53
  Speaker 1: I could show a person that would you be most
• 33:56
  Speaker 1: likely to change their behavior and cause them to buy it.
• 34:00
  Speaker 1: It's a kind of prediction problem that motivates recommendations. In biotechnology.
• 34:07
  Speaker 1: Very often we are trying to make predictions about whether someone,
• 34:12
  Speaker 1: let's say, does or doesn't have a condition a disease
• 34:15
  Speaker 1: based on lots of information we can gather from high
• 34:19
  Speaker 1: throughput diagnostic techniques. These days, the keyword in biology and
• 34:25
  Speaker 1: in medicine and biotechnology is high throughput. You're running analyses
• 34:30
  Speaker 1: on an individual that are producing hundreds of thousands of numbers,
• 34:35
  Speaker 1: and you want to be able to take all of
• 34:37
  Speaker 1: that kind of wealth of data and turn it into
• 34:40
  Speaker 1: diagnostic information about.
• 34:42
  Speaker 2: And we've seen AI get applied to pharmaceutical development in
• 34:47
  Speaker 2: ways that people just never really could have imagined just
• 34:52
  Speaker 2: a few short years ago. Is there a field that
• 34:55
  Speaker 2: AI and large language models are not going to touch
• 34:58
  Speaker 2: or is this just the future of everything.
• 35:01
  Speaker 1: The kinds of fields where you would expect uptake to
• 35:04
  Speaker 1: be slow are where it is hard to assemble large
• 35:10
  Speaker 1: data sets of systematically gathered data. And so any field
• 35:15
  Speaker 1: where it's relatively easy to at large scale, let's say,
• 35:20
  Speaker 1: produce the kinds of the same kinds of informations that
• 35:23
  Speaker 1: experts are using to make their decisions, you should expect
• 35:26
  Speaker 1: that field to be impacted by these tools if it
• 35:29
  Speaker 1: hasn't been already.
• 35:30
  Speaker 2: So you're kind of answering my next question, which is
• 35:33
  Speaker 2: what led you back to investment management. But it seems
• 35:37
  Speaker 2: if there's any field that just generates endless amounts of data.
• 35:40
  Speaker 1: It's the markets, that's true. And I had already been
• 35:44
  Speaker 1: really interested in the problems of systematic investment strategies from
• 35:48
  Speaker 1: my time working in d SHAW, and so my co
• 35:52
  Speaker 1: founder Michael Kratanav and I, you know, we were both
• 35:56
  Speaker 1: in the Bay Area in the two thousand and four.
• 36:02
  Speaker 1: He was there because of a firm that he had founded,
• 36:04
  Speaker 1: and I was there finishing my PhD. And we started
• 36:06
  Speaker 1: to talk about the idea of using contemporary machine learning
• 36:10
  Speaker 1: methods to build strategies that would be, you know, really
• 36:14
  Speaker 1: different from strategies that result from classical techniques. And we
• 36:19
  Speaker 1: had met at d SHAW in the nineties and been
• 36:21
  Speaker 1: less excited about this idea because the methods were pretty immature.
• 36:27
  Speaker 1: There wasn't actually a giant diversity of data back in
• 36:29
  Speaker 1: the nineties in financial markets, not like there was in
• 36:33
  Speaker 1: two thousand and five. And compute was really still quite
• 36:36
  Speaker 1: expensive in the nineties, whereas in two thousand and five,
• 36:40
  Speaker 1: you know, it had been dropping in the usual More's
• 36:42
  Speaker 1: Law way. And this was even before GPUs, and so
• 36:45
  Speaker 1: when we looked at the problem in two thousand and five,
• 36:48
  Speaker 1: it felt like there was a very live opportunity to
• 36:53
  Speaker 1: do something with a lot of promise that would be
• 36:56
  Speaker 1: really different. And we had the sense that not a
• 36:59
  Speaker 1: lot of people were of the same opinion, and so
• 37:02
  Speaker 1: it seemed like something that we should try.
• 37:04
  Speaker 2: That there was a void. Nothing nothing in the market
• 37:08
  Speaker 2: hates more than a vacuum and intellectual approach. So so
• 37:12
  Speaker 2: you mentioned the diversity of various data sources. What what
• 37:18
  Speaker 2: don't you consider, like how how far off of price
• 37:21
  Speaker 2: and volume do you go in the net you're casting
• 37:25
  Speaker 2: for inputs into into your systems.
• 37:29
  Speaker 1: Well, I think we're prepared as a you know, as
• 37:33
  Speaker 1: a as a research principle, we're prepared to consider any
• 37:37
  Speaker 1: data that has some bearing on price formation, like some
• 37:41
  Speaker 1: some plausible bearing on how prices are formed. Now, of
• 37:44
  Speaker 1: course we're you know, we're a relatively small group of
• 37:47
  Speaker 1: people with a lot of ideas and uh, and so
• 37:51
  Speaker 1: we have to prioritize so you know, in the event
• 37:55
  Speaker 1: we end up pursuing data that you know makes a
• 37:58
  Speaker 1: lot of sense, you know, we don't we don't try.
• 38:00
  Speaker 2: I mean, can you go as far as politics or
• 38:03
  Speaker 2: the weather, like how far off of prices can you
• 38:06
  Speaker 2: can you look?
• 38:07
  Speaker 1: So, you know, an example would be the weather. You're
• 38:09
  Speaker 1: for most securities, you're not going to be very interested
• 38:12
  Speaker 1: in the weather, but for commodities future as you might be,
• 38:15
  Speaker 1: so that you know, that's the kind of reasoning you
• 38:17
  Speaker 1: would apply.
• 38:18
  Speaker 2: Right, really really interesting. So let's talk about some of
• 38:22
  Speaker 2: the strategies. You guys are running short and mid horizon
• 38:27
  Speaker 2: US equities, European equities, Asian equities, mid horizon US credit,
• 38:33
  Speaker 2: and then cross assets. So I might to assume all
• 38:36
  Speaker 2: of these are machine learning based, and how similar different
• 38:40
  Speaker 2: is each approach to each of those asset classes.
• 38:43
  Speaker 1: Yeah, they're all machine learning based. The kind of principles
• 38:50
  Speaker 1: that I've described of using as much complexity as you
• 38:53
  Speaker 1: need to maximize predictive accuracy, et cetera. Those principles underlie
• 38:57
  Speaker 1: all the systems. But of course it's trading. Trading corporate
• 39:00
  Speaker 1: bonds is very different from trading equities, and so the
• 39:04
  Speaker 1: implementations reflect that reality.
• 39:06
  Speaker 2: Huh. So let's talk a little bit about the four
• 39:09
  Speaker 2: step process that you bring to the systematic approach, and
• 39:15
  Speaker 2: this is off of your site, so it's it's data prediction, engine,
• 39:19
  Speaker 2: portfolio construction, and execution. Yeah, I'm assuming that is heavily
• 39:26
  Speaker 2: computer and machine learning based. At each step along the way.
• 39:30
  Speaker 2: Is that is that fair?
• 39:32
  Speaker 1: I think that's fair. I mean to different degrees. The
• 39:35
  Speaker 1: data gathering that's you know, that's a that's largely a
• 39:41
  Speaker 1: software and kind of operations and infrastructure job.
• 39:46
  Speaker 2: Do you guys have to spend a lot of time
• 39:48
  Speaker 2: cleaning up that data and making sure that because you
• 39:52
  Speaker 2: hear between CRISP and s and P and Bloomberg, sometimes
• 39:56
  Speaker 2: you'll pull something up and they're just all off a
• 39:58
  Speaker 2: little bit from each other because they all bring a
• 40:00
  Speaker 2: very different approach to data assembly. How do you make
• 40:04
  Speaker 2: sure everything is consistent and there's no errors or errants
• 40:09
  Speaker 2: inputs throughout.
• 40:10
  Speaker 1: Yeah, through a lot of effort. Essentially, there there we have.
• 40:15
  Speaker 1: You know, we have an entire group of people who
• 40:17
  Speaker 1: focus on data operations, both for gathering a historical data
• 40:23
  Speaker 1: and for the management of the ongoing live data feeds.
• 40:27
  Speaker 1: There's no way around that. I mean, that's just work
• 40:29
  Speaker 1: that you have to that you have to do.
• 40:31
  Speaker 2: You just have to brute force your way through that.
• 40:33
  Speaker 2: And then the prediction engine. Sounds like that's the single
• 40:36
  Speaker 2: most important part of the machine learning process if I'm
• 40:42
  Speaker 2: understanding you correctly, that that's where all the meat of
• 40:45
  Speaker 2: the technology is.
• 40:47
  Speaker 1: Yeah, I understand the sentiment. I mean, it's worth emphasizing
• 40:50
  Speaker 1: that you do not get to a successful systematic strategy
• 40:54
  Speaker 1: without all the ingredients. You have to have clean data
• 40:57
  Speaker 1: because of the garbage in garbage out. You have to
• 41:01
  Speaker 1: have accurate predictions. But you know, predictions don't automatically translate
• 41:06
  Speaker 1: into returns for investors. Those predictions are kind of the
• 41:10
  Speaker 1: power that drives the portfolio holding part of the system.
• 41:15
  Speaker 2: So let's talk about that portfolio construction. Given that you
• 41:18
  Speaker 2: have a prediction engine that and good data going into it,
• 41:23
  Speaker 2: so you're fairly confident as to the output. How do
• 41:26
  Speaker 2: you then take that output and say, here's how I'm
• 41:28
  Speaker 2: going to build a portfolio based on what this generates.
• 41:32
  Speaker 1: Yeah, so there are three big ingredients in the portfolio construction.
• 41:39
  Speaker 1: The predictions what is usually called a risk model in
• 41:43
  Speaker 1: this business, which means some understanding of how volatile prices
• 41:50
  Speaker 1: are across all the securities you're trading, how correlated they are,
• 41:54
  Speaker 1: how you know if they have a if they have
• 41:57
  Speaker 1: a big movement, how big that movement will be. That's
• 42:00
  Speaker 1: all the risk model. And then the final ingredient is
• 42:04
  Speaker 1: what's usually called a market impact model, and that means
• 42:09
  Speaker 1: an understanding of how much you are going to push
• 42:13
  Speaker 1: prices away from you when you try to trade. This
• 42:15
  Speaker 1: is a reality of all trading. You buy a lot
• 42:18
  Speaker 1: of a security, you push the price up, you push
• 42:21
  Speaker 1: it away from you in the unfavorable direction. And in
• 42:24
  Speaker 1: the systems that we run, the predictions that we're trying
• 42:27
  Speaker 1: to capture are about the same size as the effect
• 42:31
  Speaker 1: that we have on the markets when we trade, and
• 42:34
  Speaker 1: so you cannot neglect that impact effect when you're thinking
• 42:38
  Speaker 1: about what portfolios to hold.
• 42:41
  Speaker 2: So execution becomes really important. If you're not executing well,
• 42:44
  Speaker 2: you are moving prices away from your profit.
• 42:48
  Speaker 1: That's right, and it is you know, probably the single
• 42:53
  Speaker 1: thing that undoes quantitative hedge funds most often is that
• 43:00
  Speaker 1: they misunderstand how much they're moving prices. They get too big,
• 43:04
  Speaker 1: they start trading too much, and they sort of blowed
• 43:08
  Speaker 1: themselves up.
• 43:08
  Speaker 2: It's funny that you say that, because as you were
• 43:11
  Speaker 2: describing that, the first name that popped into my head
• 43:14
  Speaker 2: was long term capital managements trading these really thinly traded
• 43:20
  Speaker 2: obscure fixed income products, and everything they bought they sent
• 43:27
  Speaker 2: higher because there just wasn't any volume in it. And
• 43:30
  Speaker 2: when they needed liquiditly there was none to be had.
• 43:33
  Speaker 2: And you know that plus no risk management one hundred
• 43:36
  Speaker 2: x leverage equals a kaboom.
• 43:39
  Speaker 1: They made a number of mistakes the book. The book
• 43:43
  Speaker 1: is good. So when genius fail in, oh absolutely love
• 43:45
  Speaker 1: that fantastically fascinating.
• 43:46
  Speaker 2: So when you're reading a book like that, somewhere in
• 43:51
  Speaker 2: the back of your head are you thinking, hey, this
• 43:53
  Speaker 2: is like a what not to do when you're setting
• 43:56
  Speaker 2: up a machine learning fund. How influential is something like.
• 44:00
  Speaker 1: Well one hundred percent? I mean, look, I think the
• 44:03
  Speaker 1: most important adage I've ever heard in my professional life
• 44:06
  Speaker 1: is good judgment comes from experience. Experience comes from bad judgment.
• 44:10
  Speaker 1: So the extent to which you can get good judgment
• 44:14
  Speaker 1: from other people's experience, that is that that is like
• 44:17
  Speaker 1: a free tuition. And so we talk a lot about
• 44:22
  Speaker 1: all the all the mistakes that that that other people
• 44:25
  Speaker 1: have made. And you know, we do not congratulate ourselves
• 44:29
  Speaker 1: on having avoided mistakes. We think those people were smart.
• 44:33
  Speaker 1: I mean look that you know, you read about these
• 44:36
  Speaker 1: events and these people. None of these people were dummies.
• 44:38
  Speaker 1: They were sophisticated Nobel laureates.
• 44:40
  Speaker 2: Yeah, right, it's they just didn't have a guide book
• 44:43
  Speaker 2: on what not to do, which which you guys.
• 44:45
  Speaker 1: Do We don't. No, I don't think we do. I
• 44:47
  Speaker 1: mean apart from that, apart from reading about right, But
• 44:50
  Speaker 1: everybody is undone by a failure that they they didn't
• 44:53
  Speaker 1: they did, they didn't think of ever didn't know about yet.
• 44:55
  Speaker 1: And we're extremely cognizant of that.
• 44:57
  Speaker 2: Huh. That has to be somewhat humbling to come being
• 45:01
  Speaker 2: on the lookout for that blind spot that could disrupt everything.
• 45:06
  Speaker 1: Yes, yeah, humility is the key ingredient in running in
• 45:11
  Speaker 1: running these systems.
• 45:13
  Speaker 2: Really quite amazing. So let's talk a little bit about
• 45:18
  Speaker 2: how academically focused volling On is. You guys have a
• 45:23
  Speaker 2: pretty deep R and D team internally, you teach at Berkeley.
• 45:28
  Speaker 2: What does it mean for a Hedge fund to be
• 45:30
  Speaker 2: academically focused?
• 45:32
  Speaker 1: What I would say probably is kind of evidence based
• 45:36
  Speaker 1: rather than academically focused. Saying academically focused gives the impression
• 45:40
  Speaker 1: that kind of papers would be the goal or the
• 45:43
  Speaker 1: desired output, and that's not the case at all. We have,
• 45:46
  Speaker 1: you know, a very specific applied problem that we are
• 45:49
  Speaker 1: trying to solve.
• 45:50
  Speaker 2: Papers are a mean to an end.
• 45:52
  Speaker 1: Papers are you know, we don't write papers for external consumption.
• 45:56
  Speaker 1: We do lots of writing internally, and that's to make
• 45:59
  Speaker 1: sure that that you know, we're keeping track of our
• 46:02
  Speaker 1: own kind of scientific process.
• 46:05
  Speaker 2: But you're fairly widely published in statistics and machine learning. Yes,
• 46:08
  Speaker 2: what purpose does that serve other than a calling card
• 46:13
  Speaker 2: for the fund as well as Hey, I have this
• 46:16
  Speaker 2: idea and I want to see what the rest of
• 46:18
  Speaker 2: my peers think of it. When when you put stuff
• 46:21
  Speaker 2: out into the world, what sort of feedback or pushback
• 46:25
  Speaker 2: do you get?
• 46:27
  Speaker 1: I guess I would have to say, I really I
• 46:29
  Speaker 1: do that as kind of a double life of non
• 46:32
  Speaker 1: financial research. So it's just something that I really enjoy. Principally,
• 46:37
  Speaker 1: what it means is that I get to work with
• 46:39
  Speaker 1: PhD students, and you know, we have really outstanding PhD
• 46:44
  Speaker 1: students at Berkeley in statistics, and so it's an opportunity
• 46:50
  Speaker 1: for me to do a kind of intellectual work that namely,
• 46:58
  Speaker 1: you know, writing a paper laying out an argument for
• 47:01
  Speaker 1: public consumption, et cetera that is kind of closed off
• 47:05
  Speaker 1: as far as so.
• 47:05
  Speaker 2: Not adjacent to what you guys are doing at Volleyon generally. No, No,
• 47:10
  Speaker 2: that's really interesting. So then I always assume that that
• 47:14
  Speaker 2: was part of your process for developing new models to
• 47:17
  Speaker 2: apply machine learning to new assets. Take us through the process.
• 47:22
  Speaker 2: How do you go about saying, Hey, this is an
• 47:25
  Speaker 2: asset class we don't have exposure to. Let's see how
• 47:28
  Speaker 2: to apply what we already know to that specific area.
• 47:32
  Speaker 1: Yeah, we have it's a great question. So we're trying
• 47:35
  Speaker 1: as much as possible to get the problem for a
• 47:39
  Speaker 1: new asset class into a familiar setup, into you know,
• 47:43
  Speaker 1: as standard a setup as we can, and so we
• 47:47
  Speaker 1: know what these systems look like in the world of equity.
• 47:52
  Speaker 1: And so if you're trying to do the same kind,
• 47:55
  Speaker 1: if you're trying to build the same kind of system
• 47:57
  Speaker 1: for corporate bonds, and you start off by saying, well, okay,
• 48:00
  Speaker 1: i'd like I need to know, you know, closing prices
• 48:02
  Speaker 1: or inter day prices for all the bonds. Already, you
• 48:05
  Speaker 1: have a very big problem in corporate bonds because there
• 48:09
  Speaker 1: is no there is no live price feeds that's showing
• 48:15
  Speaker 1: you a bit offer quote in the way that there
• 48:18
  Speaker 1: is inequity. And so before you can even get started
• 48:21
  Speaker 1: thinking about predicting how a price is going to change,
• 48:24
  Speaker 1: it would be nice if you know what the price
• 48:26
  Speaker 1: currently was, and that is already a problem you have
• 48:28
  Speaker 1: to solve in corporate bonds as opposed to being just
• 48:30
  Speaker 1: an input that you have access to.
• 48:32
  Speaker 2: The old joke was trading by appointment only. Yeah, and
• 48:35
  Speaker 2: that seems to be a bit of an issue. And
• 48:37
  Speaker 2: there are so many more bond issues than there are equities. Absolutely,
• 48:42
  Speaker 2: is this just a database challenge or how do you work?
• 48:45
  Speaker 1: No, it's a statistics problem, but it's it's a different
• 48:49
  Speaker 1: kind of statistics problem. We're not in this case. We're
• 48:52
  Speaker 1: not trying to yet. We're not yet trying to predict
• 48:55
  Speaker 1: the future of any quantity. We're trying to say, I
• 48:58
  Speaker 1: wish I knew what the fair value of this of
• 49:00
  Speaker 1: this CSIP was. I can't see that exactly because there's
• 49:05
  Speaker 1: no live order book that with a bid and an
• 49:07
  Speaker 1: offer that's got lots of liquidity that lets me figure
• 49:09
  Speaker 1: out the fair value. But I do know what.
• 49:11
  Speaker 2: At best, you have a recent price, maybe not even
• 49:14
  Speaker 2: so recent.
• 49:14
  Speaker 1: I have lots of related information. I know you know
• 49:18
  Speaker 1: this bond. Maybe this bond didn't trade today, but it
• 49:21
  Speaker 1: traded a few times yesterday. I get to say I
• 49:23
  Speaker 1: know where it traded. I'm in touch with bond dealers,
• 49:26
  Speaker 1: so I know where they've quoted this bond, maybe only
• 49:28
  Speaker 1: on one side over the last few days. I have
• 49:31
  Speaker 1: some information about the company that issued this bond, et cetera.
• 49:35
  Speaker 1: So I have lots of stuff that's related to the
• 49:37
  Speaker 1: number I know that I want to know. I just
• 49:39
  Speaker 1: don't know that number, right, And so what I want
• 49:42
  Speaker 1: to try to do is kind of fill in and
• 49:44
  Speaker 1: do what's what in statistics or in control we would
• 49:46
  Speaker 1: call a now casting problem, huh, And it's an analogy
• 49:50
  Speaker 1: actually is too automatically controlling an airplane. So surprisingly, Oh,
• 49:55
  Speaker 1: the main there there are there are when you're if
• 49:57
  Speaker 1: you're trying if a software is trying to fly in
• 49:59
  Speaker 1: air plane, there are six things that it absolutely has
• 50:03
  Speaker 1: to know. Has to know the x y z of
• 50:05
  Speaker 1: where the plane is and the x y z of
• 50:07
  Speaker 1: its velocity where it's headed. Right, those are the six
• 50:10
  Speaker 1: most important numbers. Now, nature does not just supply those
• 50:14
  Speaker 1: numbers to you. You cannot know those numbers with perfect exactitude.
• 50:17
  Speaker 1: But there's lots of instruments on the plane, and there's
• 50:20
  Speaker 1: GPS and all sorts of information that is very closely
• 50:23
  Speaker 1: related to the numbers You wish you knew, and you
• 50:26
  Speaker 1: can use statistics to go from all that stuff that's
• 50:29
  Speaker 1: adjacent to a guess and infill of the thing you
• 50:32
  Speaker 1: wish you knew, And the same goes with the current
• 50:35
  Speaker 1: price of a corporate bond.
• 50:37
  Speaker 2: Huh. That's really kind of interesting. So I'm curious as
• 50:41
  Speaker 2: to how often you start working your way into one
• 50:45
  Speaker 2: particular asset or a particular strategy for that asset and
• 50:50
  Speaker 2: just suddenly realize, oh, this is wildly different than we
• 50:54
  Speaker 2: previously expected, and suddenly you're down a rabbit hole to
• 50:58
  Speaker 2: just wildly unexpected areas. It sounds like that isn't all
• 51:02
  Speaker 2: then uncommon.
• 51:02
  Speaker 1: It is not uncommon at all.
• 51:04
  Speaker 2: Huh.
• 51:04
  Speaker 1: No, it's a nice you know, there's this kind of
• 51:07
  Speaker 1: wishful thinking that all we have. You know, we figured
• 51:09
  Speaker 1: it out in one asset class in the sense that
• 51:11
  Speaker 1: we have a system that's kind of stable and performing
• 51:14
  Speaker 1: reasonably well that we that we have a feel for,
• 51:17
  Speaker 1: and now we want to take that system and somehow
• 51:20
  Speaker 1: replicate it in a different situation. And while we're going
• 51:23
  Speaker 1: to standardize the new situation to make it look like
• 51:26
  Speaker 1: the old situation. That's the principle. That principle kind of
• 51:29
  Speaker 1: quickly goes out the window when you when you start
• 51:31
  Speaker 1: to make contact with the reality of how the new
• 51:33
  Speaker 1: asset class actually behaves.
• 51:34
  Speaker 2: So stocks are different than credit, are different than bonds,
• 51:37
  Speaker 2: or different than commodities. They're all like starting fresh. Yeah,
• 51:40
  Speaker 2: over what some of the more surprising things you've learned
• 51:42
  Speaker 2: as you've applied machine learning to totally different asset classes.
• 51:47
  Speaker 1: Well, I think, you know, corporate bonds provide a lot
• 51:49
  Speaker 1: of examples of this. I mean, the fact that you
• 51:52
  Speaker 1: don't actually really know a good live price or a
• 51:57
  Speaker 1: good live bid offers it seems seems you know, it's surprising.
• 52:00
  Speaker 1: I mean, this is this fact has started to change.
• 52:03
  Speaker 1: Like over the years, there's been an accelerating electronification of
• 52:07
  Speaker 1: corporate bond treading, and that's you know, that's that's been
• 52:09
  Speaker 1: a big advantage for us actually because we were kind
• 52:11
  Speaker 1: of first movers and so we've really benefited from that.
• 52:14
  Speaker 1: So the problem is diminished relative to how it was,
• 52:17
  Speaker 1: you know, six seven years ago when we started, but
• 52:20
  Speaker 1: it's still relative equities, it's absolutely there.
• 52:23
  Speaker 2: Yeah, So you get so when in other words, if
• 52:25
  Speaker 2: I'm looking at a bond mutual fund or even a
• 52:28
  Speaker 2: bondytf that's trading during the day. That price is somebody's
• 52:33
  Speaker 2: best approximation of the value of all the bonds inside.
• 52:37
  Speaker 2: But really you don't know the nav, do you.
• 52:41
  Speaker 1: You're just kind of guessing, Barry, don't even get me
• 52:43
  Speaker 1: started on bonditfs real because.
• 52:45
  Speaker 2: That it seems like that would be the first place
• 52:48
  Speaker 2: that would show up. Hey, bondytf's sound like throughout the
• 52:52
  Speaker 2: day they're gonna be mispriced a little bit or wildly mispriced.
• 52:57
  Speaker 1: Well, the bond ETF there's a sense if you're a
• 53:00
  Speaker 1: if you're a market purist, in which they can't be
• 53:02
  Speaker 1: mispriced because there's their price is set by supplying demand
• 53:05
  Speaker 1: in the ETF market, and that's a super liquid market,
• 53:08
  Speaker 1: and so there may be a difference between the market
• 53:11
  Speaker 1: price of the ETF and the under the nave of
• 53:14
  Speaker 1: the underlying portfolio, except in many cases with bond ETF
• 53:18
  Speaker 1: there's not even a crisply defined underlying portfolio. It turns
• 53:23
  Speaker 1: out that the authorized participants in those ETF markets can
• 53:27
  Speaker 1: negotiate with the fund manager about exactly what the constituents
• 53:32
  Speaker 1: are of the create redeem baskets, and so it's not
• 53:35
  Speaker 1: even at all clear what you mean when you say
• 53:38
  Speaker 1: that the nav is this or that relative to the
• 53:40
  Speaker 1: price of the ETF.
• 53:41
  Speaker 2: So when I asked about what's surprising when you work
• 53:44
  Speaker 2: you in on a rabbit hole. Hey, we don't know
• 53:46
  Speaker 2: what the hell's in this bond ETF. Trust us, it's
• 53:48
  Speaker 2: all good. That's a pretty surprise. And I'm only exaggerating
• 53:51
  Speaker 2: a little bit, But that seems like that's kind of shocking.
• 53:55
  Speaker 1: It's it is surprising when you find out about it,
• 53:57
  Speaker 1: but you quickly come to understand. If you trade single
• 54:00
  Speaker 1: name bonds, as we do, you quickly come to understand
• 54:03
  Speaker 1: why bond ETFs work that way.
• 54:06
  Speaker 2: I recall a couple of years ago there was a
• 54:08
  Speaker 2: big Wall Street Journal article on the g l d
• 54:13
  Speaker 2: E t F, and from that article I learned that
• 54:18
  Speaker 2: GLD was formed because gold dealers had just excess gold
• 54:22
  Speaker 2: piling up in their warehouses and they needed a way
• 54:25
  Speaker 2: to move it. So that was kind of shocking about
• 54:27
  Speaker 2: that ETF any other space that that led to a
• 54:33
  Speaker 2: sort of big surprise as you worked your way into it.
• 54:37
  Speaker 1: Well, I think ETFs are a kind of a good
• 54:41
  Speaker 1: source of these examples. So the volatility ETFs, the you know,
• 54:45
  Speaker 1: the ETFs that are that are based on the VIX
• 54:47
  Speaker 1: or that are short the vics. You may remember several years.
• 54:50
  Speaker 2: Ago I was gonna say the ones that haven't blown up.
• 54:52
  Speaker 1: Yeah right, there was this event called Valmageddon where.
• 54:56
  Speaker 2: That was ETF notes, wasn't it the Yeah.
• 54:59
  Speaker 1: Right, there are these essentially these investment products that were
• 55:03
  Speaker 1: short VIX, and VIX went through a spike that caused
• 55:07
  Speaker 1: them to have to liquidate, which was part I mean,
• 55:09
  Speaker 1: the people who designed the sixteene traded note. They understood
• 55:13
  Speaker 1: that this was a possibility, so they had a sort
• 55:15
  Speaker 1: of uh descriptions in their in their contract for what
• 55:18
  Speaker 1: it would mean. But yeah, always surprising to watch something
• 55:24
  Speaker 1: suddenly go out of business.
• 55:25
  Speaker 2: We seem to get a thousand year flood every couple
• 55:28
  Speaker 2: of years. Maybe we shouldn't be calling these things thousand
• 55:30
  Speaker 2: year flood. That's right, that's a that's a big misnomer.
• 55:34
  Speaker 1: As statisticians, we tell people, you know, if you if
• 55:36
  Speaker 1: you think that you've experienced a six sigma event, the
• 55:40
  Speaker 1: problem is that you have underestimated sigma.
• 55:43
  Speaker 2: That that's really interesting. So so, given the gap in
• 55:46
  Speaker 2: the world between computer science and an investment management, how
• 55:53
  Speaker 2: long is it going to be before that narrows and
• 55:56
  Speaker 2: we start seeing a whole lot more of the sort
• 55:58
  Speaker 2: of work you're doing applied across the board to to
• 56:02
  Speaker 2: the world of investment.
• 56:04
  Speaker 1: Well, I think it's happening. It's been happening for for
• 56:08
  Speaker 1: quite a long time. I mean, for example, all of
• 56:11
  Speaker 1: modern portfolio theory. Really, it kind of began in the
• 56:15
  Speaker 1: fifties with you know, first of all, Markowitz and other
• 56:18
  Speaker 1: people thinking about, you know, what it means to benefit
• 56:21
  Speaker 1: from diversification, and the idea that you know, diversification is
• 56:24
  Speaker 1: the only free lunch in finance. So I would I
• 56:28
  Speaker 1: would say that, you know, the idea of thinking in
• 56:32
  Speaker 1: a in a systematic and scientific way about how to
• 56:38
  Speaker 1: to manage and grow wealth, not you know, not even
• 56:41
  Speaker 1: just for institutions, but also for individuals. Has is an
• 56:45
  Speaker 1: example of a way that these ideas have kind of
• 56:49
  Speaker 1: had profound effects.
• 56:52
  Speaker 2: I know, I only have you for a little while longer,
• 56:55
  Speaker 2: So let's jump to our favorite questions that we ask
• 56:59
  Speaker 2: all of our guests, starting with tell us what you're
• 57:01
  Speaker 2: streaming these days? What are you either listening to or
• 57:04
  Speaker 2: watching to keep yourself entertained.
• 57:08
  Speaker 1: I A few things I've been watching recently. The Bear,
• 57:11
  Speaker 1: I don't know if you've heard So Great, So great, right,
• 57:14
  Speaker 1: and I'm in Chicago, as I know, we were just
• 57:16
  Speaker 1: from Yeah, so.
• 57:18
  Speaker 2: So and and there are parts of that show that
• 57:21
  Speaker 2: are kind of a love letter to absolutely as you
• 57:23
  Speaker 2: get deeper into the series, because it starts out kind
• 57:26
  Speaker 2: of gritty and you're seeing the underside, and then as
• 57:29
  Speaker 2: we progress, it really becomes like a lovely postcard. Such
• 57:33
  Speaker 2: an amazing show.
• 57:34
  Speaker 1: So really really love that show. Was I was late
• 57:37
  Speaker 1: to better call Saul that I'm finishing up. I think
• 57:41
  Speaker 1: as good as as Breaking Bad, So I maybe when
• 57:45
  Speaker 1: you haven't heard of there's a show called Mister in Between.
• 57:48
  Speaker 2: Which is mister Yeah.
• 57:50
  Speaker 1: It's not Hulu, it's from it's from Australia. It's about
• 57:53
  Speaker 1: a guy who's, you know, a doting father living his life.
• 57:58
  Speaker 1: He's also essentially a muscle man and hitman for for
• 58:04
  Speaker 1: local criminals in his part of Australia. But it's half
• 58:07
  Speaker 1: hour dark comedy.
• 58:09
  Speaker 2: Right, so not quite Barry and not quite Sopranos somewhere.
• 58:12
  Speaker 1: Yeah, that's exactly.
• 58:14
  Speaker 2: Yeah, sounds really interesting. Tell us about your early mentors
• 58:19
  Speaker 2: who helped shape your career.
• 58:21
  Speaker 1: Well, Berry, I'd been lucky to have a lot of
• 58:24
  Speaker 1: people who were you know, both really smart and talented
• 58:28
  Speaker 1: and willing to you know, take the time to help
• 58:32
  Speaker 1: me learn and understand things. So actually, my co founder,
• 58:36
  Speaker 1: Michael Caratanov, he was kind of my first mentor in finance.
• 58:40
  Speaker 1: He he had been a d SHAW for several years
• 58:43
  Speaker 1: when I got there, and he he really taught me
• 58:46
  Speaker 1: kind of the ins and outs of of market micro structure.
• 58:50
  Speaker 1: I worked with a couple of people who managed me
• 58:53
  Speaker 1: at d SHAW yo see Friedman and Kapeel Mature, who
• 58:56
  Speaker 1: have gone on to hugely successful careers in quantitative finance,
• 59:00
  Speaker 1: and they taught me a lot to when I did
• 59:03
  Speaker 1: my PhD. My advisor Mike Jordan, who's a kind of
• 59:06
  Speaker 1: world famous machine learning researcher. You know, I learned enormously
• 59:12
  Speaker 1: from him. And there's another professor of statistics who sadly
• 59:18
  Speaker 1: passed away about fifteen years ago named David Friedman. He
• 59:22
  Speaker 1: was really just an intellectual giant of the twentieth century
• 59:26
  Speaker 1: and probability and statistics. He was both, you know, one
• 59:30
  Speaker 1: of the most brilliant probabilists and also an applied statistician.
• 59:35
  Speaker 1: And this is this is like a pink diamond kind
• 59:38
  Speaker 1: of combination. It's that rare to find someone who has
• 59:42
  Speaker 1: that kind of technical capability but also understands the pragmatics
• 59:46
  Speaker 1: of actually doing data analysis. He spent a lot of
• 59:48
  Speaker 1: time as an expert witness. He was the lead statistical
• 59:53
  Speaker 1: consultant for the case on census adjustment that went to
• 59:56
  Speaker 1: the Supreme Court. In fact, he told me, uh, what
• 1:00:02
  Speaker 1: went that in the end? Uh, you know, the the
• 1:00:05
  Speaker 1: people against adjustment they won in a unanimous Supreme Court decision.
• 1:00:08
  Speaker 1: And David Freeman told me, he said, you know, all
• 1:00:11
  Speaker 1: that work and we only convinced nine people.
• 1:00:15
  Speaker 2: But not nine people that kind of matter, Yeah, exactly.
• 1:00:18
  Speaker 1: So it was just it was a real it was
• 1:00:21
  Speaker 1: kind of a once in a lifetime privilege to get
• 1:00:24
  Speaker 1: to spend time with someone of that intellectual caliber. And
• 1:00:28
  Speaker 1: there were others too. I mean, I've been I've been
• 1:00:30
  Speaker 1: very fortunate that.
• 1:00:31
  Speaker 2: That's quite a list to begin with. Let's talk about books.
• 1:00:35
  Speaker 2: What are some of your favorites and what are you
• 1:00:36
  Speaker 2: reading right now?
• 1:00:38
  Speaker 1: Uh? Well, I'm a I'm a big book reader, so
• 1:00:41
  Speaker 1: I had a long list.
• 1:00:42
  Speaker 2: But probably by the way, this is everybody's favorite section
• 1:00:46
  Speaker 2: of the podcast. People are always looking for good book recommendations,
• 1:00:50
  Speaker 2: and if they like what you said earlier, they're gonna
• 1:00:54
  Speaker 2: love love your book recommendations, so fire away.
• 1:00:57
  Speaker 1: So I'm a big fan of kind of modernist dystopian fiction.
• 1:01:03
  Speaker 1: So a couple of examples of that would be the
• 1:01:05
  Speaker 1: book Infinite Jest by David Foster Wallace, wind Up Bird
• 1:01:10
  Speaker 1: Chronicle by Hirouki Murakami. Those are two of my all
• 1:01:14
  Speaker 1: time favorite books. There's a I think much less well
• 1:01:17
  Speaker 1: known but beautiful novel. It's a kind of academic coming
• 1:01:22
  Speaker 1: of age novel called Stoner by John Williams. A really moving,
• 1:01:28
  Speaker 1: just a tremendous book. Sort of more dystopia would be
• 1:01:33
  Speaker 1: White Noise to Lilo and kind of the classics that
• 1:01:38
  Speaker 1: everybody knows nineteen eighty four and Brave New World. Those
• 1:01:41
  Speaker 1: are two more of my favorite.
• 1:01:42
  Speaker 2: Huh, it's funny when you mentioned The Bear. I'm in
• 1:01:46
  Speaker 2: the middle of reading a book that I would swear
• 1:01:50
  Speaker 2: the writers of The Bear leaned on called Unreasonable Hospitality
• 1:01:55
  Speaker 2: by somebody who worked for the Danny Meyer's Hospitality Group.
• 1:02:00
  Speaker 2: Eleven Madison Park in Ramsey Tavern and all these famous
• 1:02:03
  Speaker 2: New York haunts, and the scene in The Bear where
• 1:02:08
  Speaker 2: they overhear a couple say, oh, we visited Chicago when
• 1:02:12
  Speaker 2: you never had deep dish, So they send the guy
• 1:02:14
  Speaker 2: out to get deep dish. There's part of the book
• 1:02:18
  Speaker 2: where at eleven Medicine Park this people actually showed up
• 1:02:23
  Speaker 2: with suitcases. It was the last thing they would eat
• 1:02:25
  Speaker 2: doing before they heading to the airport. And they said, oh,
• 1:02:28
  Speaker 2: we ate all these great places in New York, but
• 1:02:30
  Speaker 2: we never had a New York hot dog. And what
• 1:02:32
  Speaker 2: do they do. They send them out to get someone
• 1:02:34
  Speaker 2: out to get a hot dog. They played it and
• 1:02:37
  Speaker 2: use all the condiments to make it very special, and
• 1:02:39
  Speaker 2: it looks like it was ripped right out of the Bear,
• 1:02:43
  Speaker 2: or vice versa. But if you're interested in just hey,
• 1:02:47
  Speaker 2: how can we disrupt the restaurant business and make it
• 1:02:51
  Speaker 2: not just about the celebrity chef in the kitchen but
• 1:02:54
  Speaker 2: the whole experience. Fascinating kind of nonfiction book?
• 1:02:58
  Speaker 1: That does sound really interesting.
• 1:02:59
  Speaker 2: Yeah, really, you mentioned the Bear and it just popped
• 1:03:02
  Speaker 2: into my head. Any of the books you want to
• 1:03:04
  Speaker 2: mention that's that's a good list to start with.
• 1:03:06
  Speaker 1: Yeah. My other kind of big interest is science fiction,
• 1:03:10
  Speaker 1: speculative fiction. Unsurprisingly right, Sorry, sorry, but so there are
• 1:03:16
  Speaker 1: some classics that I think everybody should read. Ursula LeGuin
• 1:03:19
  Speaker 1: loves just amazing. So The Dispossessed and The Left Hand
• 1:03:24
  Speaker 1: of Darkness, those are just two of the best books
• 1:03:25
  Speaker 1: I've ever read period.
• 1:03:26
  Speaker 2: Forget Left Handed Darkness stays with you for a long time.
• 1:03:30
  Speaker 1: Yeah right, yeah, really really amazing books. I'm rereading right now,
• 1:03:35
  Speaker 1: Cryptonomicon Neil Stevenson. And one other thing I try to
• 1:03:42
  Speaker 1: do is I have very big gaps in my reading.
• 1:03:45
  Speaker 1: For example, I've never read Updyke, so I started reading
• 1:03:48
  Speaker 1: The Rabbit.
• 1:03:49
  Speaker 2: Serious World of Corn. It's a garb and they're they're
• 1:03:52
  Speaker 2: very much of an era.
• 1:03:54
  Speaker 1: Yeah, that's right.
• 1:03:55
  Speaker 2: What else give us more? Uh?
• 1:03:58
  Speaker 1: Wow? Okay, let's see George so.
• 1:04:01
  Speaker 2: He.
• 1:04:02
  Speaker 1: Oh wow, I think I think you'd love him. So
• 1:04:05
  Speaker 1: He's his real strength is short fiction. He had He's
• 1:04:09
  Speaker 1: written great novels too, but tenth of December this is
• 1:04:13
  Speaker 1: his best collection of of fiction. And that this is
• 1:04:16
  Speaker 1: more kind of modern dystopian, kind of comic dystopian stuff.
• 1:04:23
  Speaker 2: You keep coming back to dystopia, yeasinating.
• 1:04:26
  Speaker 1: I find, you know, it's uh, it's very different from
• 1:04:30
  Speaker 1: my my day to day reality. So I think it's
• 1:04:32
  Speaker 1: a you know, it's a great change of pace for
• 1:04:36
  Speaker 1: me to be able to read this stuff. So, uh,
• 1:04:41
  Speaker 1: some some science writing, I can tell you. Probably the
• 1:04:45
  Speaker 1: best science book I ever read is The Selfish Gene
• 1:04:48
  Speaker 1: by Richard Dawkins, which kind of really you know, you
• 1:04:54
  Speaker 1: have a kind of intuitive understanding of genetics and natural
• 1:04:57
  Speaker 1: selection in Darwin, but the language that Dawkins uses really
• 1:05:01
  Speaker 1: makes you appreciate just how how much the genes are
• 1:05:06
  Speaker 1: in charge and how little we as the as the
• 1:05:09
  Speaker 1: you know he calls he calls organisms survival machines that
• 1:05:13
  Speaker 1: the genes have kind of built and and exist inside
• 1:05:16
  Speaker 1: in order to ensure their propagation. And his whole the
• 1:05:19
  Speaker 1: whole point of view in that book just gives you, Uh,
• 1:05:22
  Speaker 1: it's really eye opening, makes you think about natural selection
• 1:05:25
  Speaker 1: and evolution and genetics in a completely different way, even
• 1:05:28
  Speaker 1: though it's all based on the same kind of facts that.
• 1:05:30
  Speaker 2: You know, it's just framing.
• 1:05:32
  Speaker 1: It's the framing and the perspective that are really that
• 1:05:34
  Speaker 1: really kind of blow your mind. So it's a great
• 1:05:37
  Speaker 1: it's a great book to read.
• 1:05:39
  Speaker 2: Huh, that's a hell of a list. You've given people
• 1:05:41
  Speaker 2: a lot of things to start with, and now down
• 1:05:44
  Speaker 2: to our last two questions, HM, what advice would you
• 1:05:47
  Speaker 2: give to a recent college grad who is interested in
• 1:05:50
  Speaker 2: a career in either investment management or machine learning.
• 1:05:56
  Speaker 1: Yeah? So, I mean I work in a very specialized
• 1:05:59
  Speaker 1: sub domain of finance, So there are a lot of
• 1:06:01
  Speaker 1: people who are going to be interested in investment in
• 1:06:03
  Speaker 1: finance that I that I couldn't give any specific advice to.
• 1:06:06
  Speaker 1: I have kind of general advice that I think is
• 1:06:11
  Speaker 1: useful both for finance and even more broadly. This advice
• 1:06:15
  Speaker 1: is really kind of top of Maslow's pyramid advice. If
• 1:06:19
  Speaker 1: you know, if you're trying to kind of write your
• 1:06:22
  Speaker 1: novel and pay the rent while you get it done,
• 1:06:25
  Speaker 1: this is I can't really help you with that. But
• 1:06:29
  Speaker 1: you know, if what you care about is building this career,
• 1:06:32
  Speaker 1: then I would say number one piece of advice is
• 1:06:34
  Speaker 1: work with incredible people. Like far and away, much more
• 1:06:37
  Speaker 1: important than what the particular field is the details of
• 1:06:40
  Speaker 1: what you're working on, is the caliber of the people
• 1:06:42
  Speaker 1: that you do it with, both in terms of your
• 1:06:45
  Speaker 1: own satisfaction and how much you learn and and and
• 1:06:51
  Speaker 1: all of that. I think you know you'll learn, you'll
• 1:06:55
  Speaker 1: benefit hugely on a personal level from working with incredible
• 1:06:59
  Speaker 1: people and if you don't work with people that are
• 1:07:04
  Speaker 1: like that, then you're probably going to have a lot
• 1:07:06
  Speaker 1: of professional unhappiness. So it's kind of either or.
• 1:07:09
  Speaker 2: That's a really intriguing answer. So final question, what do
• 1:07:14
  Speaker 2: you know about the world of investing, machine learning, large
• 1:07:18
  Speaker 2: language models, just the application of technology to the field
• 1:07:22
  Speaker 2: of investing that you wish you knew twenty five years
• 1:07:25
  Speaker 2: or so ago when you were really first ramping up.
• 1:07:29
  Speaker 1: I think one of the most important lessons that I learned,
• 1:07:33
  Speaker 1: had to learn the hard way kind of going through
• 1:07:35
  Speaker 1: and running these systems, was that it's kind of comes
• 1:07:40
  Speaker 1: back to the point you made earlier about the primacy
• 1:07:43
  Speaker 1: of prediction rules. And it may be true that the
• 1:07:47
  Speaker 1: most important thing is the prediction quality, but there are
• 1:07:51
  Speaker 1: lots of other very necessary, mandatory ingredients, and I would
• 1:07:55
  Speaker 1: put kind of risk management at the top of that list.
• 1:07:57
  Speaker 1: So I think it's easy to to maybe neglect risk
• 1:08:02
  Speaker 1: management to a certain extent and focus all of your
• 1:08:06
  Speaker 1: attention on predictive accuracy. But I think it really does
• 1:08:10
  Speaker 1: turn out that if you don't have high quality risk
• 1:08:13
  Speaker 1: management to go along with that predictive accuracy, you won't succeed.
• 1:08:17
  Speaker 1: And I guess I wish I had appreciated that in
• 1:08:21
  Speaker 1: a really deep way twenty five years ago.
• 1:08:23
  Speaker 2: John, This has been really, absolutely fascinating. I don't even
• 1:08:27
  Speaker 2: know where to begin other than saying thank you for
• 1:08:30
  Speaker 2: being so generous with your time and your expertise. We
• 1:08:34
  Speaker 2: have been speaking with John mccauloff. He is the co
• 1:08:37
  Speaker 2: founder and chief investment officer at the five billion dollar
• 1:08:41
  Speaker 2: hedge fund Volleyon Group. If you enjoy this conversation, well,
• 1:08:46
  Speaker 2: be sure and check out any of the previous five
• 1:08:48
  Speaker 2: hundred we've done over the past nine years. You can
• 1:08:53
  Speaker 2: find those at iTunes, Spotify, YouTube, or wherever you find
• 1:08:57
  Speaker 2: your favorite podcast. Sign up for my daily reading list
• 1:09:01
  Speaker 2: at rid Halts. Follow me on Twitter at Barry Underscore
• 1:09:06
  Speaker 2: Rit Halts until I get my hacked account at rid
• 1:09:10
  Speaker 2: Holt's back. I say that. I say that because the
• 1:09:16
  Speaker 2: process of dealing with the seventeen people left at once
• 1:09:20
  Speaker 2: Twitter now x is unbelievably frustrating and annoying. Follow all
• 1:09:27
  Speaker 2: of the fine family of podcasts on Twitter at podcast
• 1:09:31
  Speaker 2: I would be remiss if I did not thank the
• 1:09:33
  Speaker 2: crack team that helps put these conversations together each week.
• 1:09:38
  Speaker 2: Paris Woald is my producer. Attiko val Bron is my
• 1:09:42
  Speaker 2: project manager. Sean Russo is my director of Research. I'm
• 1:09:47
  Speaker 2: Barry rid Halts. You've been listening to Masters in Business
• 1:09:51
  Speaker 2: on Bloomberg Radio at