Latest / The Information Bottleneck / Daphne Koller - The Future of AI in Biology and Drug Discovery

Transcript

Daphne Koller - The Future of AI in Biology and Drug Discovery · The Information Bottleneck

• Ravid Shwartz-Ziv: Hi, everyone, and welcome back to the Information Battle Mag. And our guest today is Daphne Koller. So, if you learn machine learning from a certain generation of textbooks, you learn part of it from her. She and Nir Friedman wrote their book on probabilistic graphical models. And for a long time, that was how many of us thought about structure, uncertainty, and inference. She co-founded Coursera, and after it she founded Incitro, which is trying to turn drug discovery into machine learning problem. So, our talk today will be both about machine learning and biology. and also about intelligence and what we still do not know how to measure it. So, hi Daphne! Daphne: Hi Rave, it's a pleasure to be here. Thank you for inviting me. Ravid Shwartz-Ziv: Thank you for joining us. So I think we can start with your classical book about machine learning and probabilistic modeling. And I think, at least in my point of view, I read this book many times, and at least my point of view is that structure does matter at the end in machine learning and prediction. But in the recent years, We see some, we can call it like maybe different paradigm, the bitter lesson, if you want, right? Like scaling is all what you need, scale the data, scale the compute. What is your current, like maybe, update view on it? Do you agree with that? Daphne: I think I agree with a lot of elements of the bitter lesson. We have absolutely, I think, seen that if you have a domain where there is a lot of data and where your task is defined as basically pattern recognition, pattern matching. you can find those patterns. The machine can find those patterns in the data like nothing else has ever done, including the human brain, and extract from that incredible capabilities. To you know, generate data, to to generate responses, to generate insights. And I think that has been a transformative technology for the field, and it is absolutely derived from the bitter lesson. But I don't think that that means that structure is irrelevant, and I also don't believe it means that causality is irrelevant, because the other element of probabilistic graphical models is that they do provide specifically the Bayesian network and it's very Various progeny also provide us with a formal framework of causality. And so let me talk a little bit about each of those separately. Starting from structure, I think structure matters in a few different places. One is in domains where no matter how we Think about the world, data will never be quite as much as we would like. In the domain that I'm in, which is biology, the complexity and richness and multifacetedness and multivariedness shows that we're gonna need a tremendous amount of data that is not available to us today. Now, I'm not saying that there is never going to be a world where we have incredible Incredibly sophisticated measurement devices that are not available to us today that can measure biology at orders of magnitude larger than what we have right now, but we're not there yet. And when we don't have that amount of data, then I think you need to combine some of the technologies from the bitter lesson in places where data, sufficient data exists, but also. leverage structure in some ways to sort of create a more compact representation. That's number one. Number two is structure is how we people understand the world. So if you are if what if your goal is not simply to make predictions, but to create mod to create an understanding of the world around us, the world of biology, the world of physics, the world of chemistry, so that we as people can continue to learn An interpretable an uninterpretable black box, you know, trillion parameter neural network is not going to do that for us. And structure, I think, is is an important component of creating an interpretability layer that we people as people can can benefit from. And then coming to the third point, which is that of causality, if you look at the applications that exist today where you know the the various generative AI models are shining, is in places where There's no real intervention in the world. You know, we're we're not w we're helping compose whatever, emails and letters and documents and images and so on. But there's never the question of, and if I were to send that email to someone, how are they going to respond? It's purely like a a pattern recognition and generation problem. There's no question of effect and consequence. And in the domain that I'm in, For example, when you're making medicines, the fundamental question is not an observational question. It's a question of intervention, of an experiment that has not yet been performed, which is I am going to deliver a drug that does not yet exist to a human being who has never received a drug. What is going to happen? That is a causal question. And I don't think you're gonna get that from pattern from algorithms that are 100% about pattern recognition. Allen Roush: Do you know where what that alternative direction might be outside of pattern recognition and function approximation? Daphne: I don't think fatter fatter shock sorry, pattern recognition and function approximation are wrong. I just think it they're not the only thing that we need. So I think the the right future lies in taking those incredible capabilities and incorporating them into models that also include elements of causality and some higher level structure. And if you talk to the people who are the greats in this space who in in the in the jet you know deep deep learning generative AI space, people like you know Yosha Bengio and Jan Lakun, they will tell you that structure and causality are the next front Tier. Ravid Shwartz-Ziv: So like in your, like from your perspective, okay? If now I would just like, I will take like the latest coding agent, yes, and let it run it on developed like drug design. What, why it's like, or maybe it is, but it looks like there are some difficulties, some bottlenecks that like it's not possible. So why you think it is like... What are the current bottlenecks for that? Daphne: So that's a great question. And first of all, let me say that the tools that have been developed, the automatic coding agents, cowork, all of those technologies are incredibly, incredibly valuable, and we use them all the time in our work, including agentic technologies for helping design experiments, monitor the experiments, interpret the data from the experiments, and so on, is absolutely an incredibly powerful capability. I would also say that there are certain subproblems within the space of biology where there is where by a combination of sufficient data collection and the fact that these problems are sufficiently close to physics, we have been able to create Models that are reasonably predictive. Protein folding is a great example of that. Tremendous amount effort on data collection, 40 years of people collecting sequence to structure data, and a domain where the prediction problem is very self-contained. You have a sequence, a protein, you want to predict that singular event of how does the protein fold, or arguably how does a ligand bind to the protein. And it's a problem where the fundamental underlying principles are basically those of physics. The other problems in drug discovery, however, are not about the chemistry or the way in which a protein folds. It's about the way in which human disease is comes to be and the intervention points that allow us to revert, stop or ideally reverse the disease process. That is a question that has to do with measuring. Not individual proteins, but the multiple interconnected layers of biology that go from DNA to RNA to protein to cellular organelles to cells that talk to other cells within organs, circulating blood, you know, comp whatever compounds, molecules that circulate in the blood and are degraded at different rates and go to different places based on signals that we have never measured and do not understand. And so the fundamental problem. In making a drug today isn't when you get to the singular, well-defined place of, okay, I need to make a molecule that does X. The singular problem in drug discovery is figuring out what that X is. What is the X that I need to intervene in in order to revert disease? And the fundamental, even bigger problem than anything that I've articulated is that the places where that question has the biggest unmet need. Is in diseases where the only place we can really measure them is in human beings. Because there are certain processes that are relatively conserved. Cell metabolism, yeah, pretty well conserved, and you can measure it in yeast and you know what and and other and and so on. Cardiovascular function, arguably you can measure it in mammals, and what you measure translates to humans. Neurological diseases do not exist in other Lower order species. Even even non-human primates do not get ALS, do not get schizophrenia, do not get Alzheimer's disease. So if in order to measure those processes, you have to measure them in humans, which makes the whole measurement problem far more difficult, far more expensive, and frankly far more ethically fraught. Allen Roush: So what do we what do we do about some of these ethical problems too? Because we're already seeing, you know, the in in LLM land increasing amounts of focus on human evals. And I think very like a lot of people in non bio related AI research land are not familiar with like IRB or maybe some of the processes that folks that like you go through all the time. so so what do we do to to hopefully prevent kind of ethical catastrophes? Yeah. Daphne: Boy, so I think there are two halves to this question. There is the half that has to do with drug discovery and how do we retain appropriate ethical oversight on the data collection that we need for drug discovery. And then there is the other half, which is how do we make sure that researchers that have never interacted with human subjects research know what is the minimum set of ethics. I mean, honestly, for the latter half, I wish I had a better answer for you. We have been running experiments on human beings at scale in the tech industry for quite a while. I mean some of the experiments that are run by social networks under the guise of A B testing, I mean, we know that those have been very problematic in some cases and the recent spate of lawsuits about did you encourage my children to commit suicide or to, you know, have an eating disorder are the results of algorithmic refinements which are basically Driven from experiments in human beings and no one has exercised ethical oversight over those experiments. I mean, if we were doing experiments like that in a controlled setting within an academic environment and you know, asking whether certain types of intervention encourage teenage girls who are not even adults to have an eating disorder. I mean, there's no way that would pass an IRB. But yet we do it because it's somehow outside the sphere of a regulated environment, which frankly I think is has led us to a lot of the issues that we see in in in how you know, certain tech technologies are deployed and the impact that they have. So I'm gonna put that aside because that is a That is above my pay grade. That is something that, you know, I think governments should be thinking about how to create the appropriate oversight on experiments in human beings at the scale of billions. I think the for the purposes of what we do, I think it's more of a question of how do we create the right resources with the appropriate ethical safeguards to give researchers the appropriate data that they need to measure. diseases that are either intrinsically or predominantly human at sufficient scale so that we can learn what the right intervention points are. And this is a place where I think some really valuable steps has been have been taken. The announcement yesterday on the official launch and enrichment of the All of Us corpus here in the United States, I thought was a huge step forward. we make tremendous use of national biobanks UK Biobank has been our go-to resource for human data. and we've we and many, many other researchers have extracted unbelievable amounts of insight from that resource, not only about medical issues, but also about issues of socioeconomic intervention effects and factors and education and so on and so on and so forth, which have been hugely valuable to understanding how to, you know, make the world a better place for people. I really applaud all of us for following suit and learning from that and creating a resource that has, I think, some limitations relative to UK Biobank, specifically in the richness of the phenotyping, but also some huge advantages, specifically the breadth and diversity of the cohort, because UK Biobank is largely Eurocentric, and all of us was by design intended to capture the much greater diversity of the US population. So I think creating additional resources like that, ones that are ideally even larger, maybe not I mean, you have to kind of always balance the depth and the and the breadth. And so how do we create an appropriately consented, appropriately anonymized, appropriately protected cohort, but one that unlocks the amazing amount of value that exists by studying human beings and extracting the kind of insights that you are never going to extract from any other system, not from animal models and not from single cells, because you can study reductionist pieces of disease in single cells. But if you're studying things that are complex, multicellular, emergent properties of a complex system, we don't have a simulacrum of a human being that we can study in a dish. And I don't know when we will have such a thing. Ravid Shwartz-Ziv: So I want to go a bit more through, again, to technical details. So maybe you can describe, actually, how the pipeline of drug design looks like. What are the different steps that you need from the beginning? Daphne: that's a great question, and I wish me might have started with that because I think it would have allowed me to give a better answer to the to one of the earlier questions. So drug discovery has largely three phases, and and each of them is absolutely critical. The first is what I call disease to mechanism. So it's taking a complex, multifaceted, often often ill-defined disease, such as. ALS or Alzheimer's disease or heart failure, oftentimes these are not one disease. There are multiple biologies that converge to create what we as humans have declared to be the name of the disease. So this is not a biological definition, it's a symptomatic definition. And you need to extract from that an individual biological mechanism. Which is where you need to make an intervention. Like, for example, I want to stop this protein binding to this other protein. That is an example of a mechanism. So going from a disease to a mechanism where you'd like to intervene is step one. Step two is okay, I want to stop the interaction between this protein and that protein. I need to make a molecule that does that. So there's different modalities of small molecules and biologics and RNA drugs and whatever. So you need to pick. The modality and you need to make sure that it actually does the thing that you set out to do. And that furthermore, it does it in the within the construct of a molecule that you can actually deliver to a humanous. It doesn't clump up and turn into a gel. It goes into the right place in your body. It doesn't degrade within 20 minutes. and it doesn't bind to all sorts of other things that you don't want to bind to that might actually have really detrimental effects. So that's what I call mechanism to drug. And then the third piece is drug to patient. So, okay, so I have a molecule, I have a disease, which are the right patients. when do I deliver the drug? How do I measure whether the drug is working? Like, how do I prove the point that the drug is actually having the desired effect? and each of those phases is really important. Each of those, by the way, can hugely benefit from AI technologies, but they are quite different. Now, when I go back to, you know, some of the Like the questions that you asked earlier, a lot of the big successes that we've seen in AI deployed towards biology sits in the middle bucket, which is I want to design a molecule. People like that, they gravitate to that particular question because it's it's self-contained, it's well defined, and you can achieve or not achieve success within, you know, a very limited amount of time, say 18 months. You can look at the molecule and say, I set out to perturb this particular binding property, for example, like the binding of this protein to that protein. Have I successfully done that? And is the molecule a good molecule? That's great. Ravid Shwartz-Ziv: And It's easier to automate it. Daphne: And it's very easy to automate because yes, you could do a lab in the loop, you can measure a lot of those properties, not all of them, and frankly, the ones that are harder to measure, which is what we call in vivo properties, which is, for example, where does the drug go in the body? Those are the ones that we're least good at predicting right now. But a lot of the properties like that that are measurable, like the aggregation and so on, we become really, really good at automating them. And so everybody's like, yay, I solved the drug discovery problem. But you haven't, because the fundamental problem is that you haven't solved the first problem, which is what is the mechanism that I need to intervene in. And so if you look at the drugs that go into the clinic today, 90% of the drugs that go into the clinic fail. The vast majority fail, not because we haven't made a good molecule, but because the molecule doesn't distinguish itself from chalk. It does. What it's supposed to do, it inhibits the mechanism that we wanted to inhibit, and the mechanism does absolutely nothing to address the human disease that it was supposed to address. That is why most drugs fail. And so the problem, the problem in drug discovery today is that we don't have the right understanding of human biology to know what is the appropriate mechanism to intervene in in human disease. Only 22% of diseases today have any FDA-approved drug. It doesn't even have to be a good one, anything. That leaves seventy eight percent of diseases that have nothing. Half of those are common diseases, half of those are rare diseases, and we don't know what is the intervention point in those. Ravid Shwartz-Ziv: But wait, if we have a drug for a disease, it means that we understand this, like the first step, like, Daphne: No. Ravid Shwartz-Ziv: right? We can have a drug that works on it, but we still don't understand it, right? Daphne: You're absolutely right. So to me, 22% is the upper bound of things that we understand. It's not the lower bound. It is many of those drugs were discovered by serendipity. Many of those drugs are actually addressing the symptoms. They're not addressing the root cause of the disease. They're like, you know, pain relief drugs that don't actually address the root cause of the pain. you know, all of these are so yes, the number of drugs where we actually are intervening, what are called disease modifying drugs. Is much, much smaller than the 22% that we talked about. So yes, the vast majority of diseases we simply do not understand. And so if you're asking me what is the core problem that we need to solve in the drug discovery space, it is the problem of biological understanding. It is not the problem of making a molecule. Ravid Shwartz-Ziv: But I will try to push back a bit, because do you think we actually need to understand, because it may be that we can come up with drugs, test them, see if they're safe, and they work. And why we actually need to understand, right? Maybe like, don't know, Ozomafic, right? GLP-1? Yeah, Daphne: For them. Ravid Shwartz-Ziv: yeah, Ozomafic. This is a good example, right? It came up, we know that it works for... one symptom or like one thing and then suddenly we understand it actually can work from many other different things, right? Daphne: No, you're look, you're absolutely right that we don't we as humans do not need to understand, but we do need to be able to predict at least one Of the consequences of the disease. And then you're right. Oftentimes, sometimes a drug surprises us and it does better than we expect. And sometimes, by the way, it surprises us by doing things that are not so good that we also didn't expect. And so the ability to predict would actually be useful, you know, across many things. But in order for you to go through the let's just try it and it works, you need to the only experimental system that we have that allows us to do the right experiments is called a human being. So I mean I'm hoping you're not advocating for putting a whole bunch of random stuff in a human being and seeing what happens. Although the Chinese peptide craze, I guess, is a way of doing that, but no one's actually measuring what that actually does. So let's hope what that is not a good scale. So do you need to be able to predict everything with accuracy? No, I don't think that's a I mean think that's an aspirational goal that we're nowhere close to. But you need when you're coming and saying, I want to find a drug against ALS, putting a whole bunch of random compounds in ALS patients and hoping for the best is not a strategy. Allen Roush: N well, I I I in in the case of GLP ones in particular, it seems that at least some of the effects are explainable as kind of side effects of of ultimately kind of solving metabolic syndrome and people, you know, losing weight. And then it seems to also tell us something about humans, right? That all these other diseases of addiction, right, including like alcoholism, gambling behavior, etcetera, seem to also, in in many humans get get treated quite well. Right. So I'm wondering then, do you believe then that can can we use the few drugs that we do know the effects of as as kind of maybe not I'm not saying I guess like de facto people have been tested for, you know, fifteen years because now we have you know, people have been using this to solve diabetes for so long. So I guess like do you think that this is a mechanism for explaining the body? Daphne: I mean I think that experiments like the GLP ones that have been done at the scale that they have been done are an incredible source of insight. And you know, pharma companies have taken some of those observational learnings, if you will, and turned them into rigorous clinical trials. And sometimes they work out and sometimes they don't, because there's a lot of confounders in human observational data. So for example, a lot of times we see that things work really well in a particular group of patients that are treated with a disease with a drug versus ones that are not treated with that drug because the people who are treated with that drug tend to be of higher socioeconomic status, have access to better health care, and therefore they do better because they have access to a better drug. Not because the drug is working, but because they are from a privileged population. So this coming back to our earlier discussion of association versus causation, there's a lot of these examples that have not panned out because there was a confounder that wasn't appreciated at the time that the observational data was collected, which is why clinical trials, randomized clinical trials, are and should remain the gold standard for understanding whether a drug works. Having said that, I'm not I'm absolutely not in dismissing the potential of the observational found findings in GLP ones. What we've learned from GLP ones is that Absolutely many things are downstream of metabolic syndrome and these drugs not only address metabolic syndrome but many of the downstream consequences. I think that was predictable and it's panned out, which is amazing. We've also seen that even in patients who do not have metabolic syndrome and who do not experience weight loss, we see a diminishment in other things due to, I would say, parallel pathways that don't go through weight loss, that go through things like inflammation goes down in these drugs before you even see the weight loss. And we're I we scientists, not we specifically, but scientists are trying to understand how there is a an effect on inflammation that doesn't go through weight loss. and then to your point, yes, this acts on the brain and acting on the brain Interesting to suggest that there is these common mechanisms of addiction that leads to addiction to food in a similar way that you have addiction to smoking, drinking, gambling, and so on. I think that remains to be proven in rigorous clinical trials, but it's a very exciting mechanism if that turns out to be the case. So coming back to the meta-level point, we should be learning from human Dana as much as we can. Anytime we can because human data is the thing that keeps surprising us and giving us insights on the thing that we care about the most, which is how to cure humans. And so the better we get at collecting, processing, and making fit for purpose for machine learning human acquired data, the faster we will learn about human biology and human disease. Ravid Shwartz-Ziv: So what do you think? OK, so data is a constraint. And now you need to decide which experiments actually to run. So what you're doing in practice, OK? So how much time you spend on active, like, casual experimental design versus modeling? What is your take on it? Daphne: So the RD function of our of it of In Citro is half wet, half dry. So half of our people basically make data, do experiments that make data, and then take the insights that emerge from the data in order to design the next round of experiments. And some of those experiments are at scale data generation that are intended to map. human biology or in this case cell biology at the at frankly enormous scale. We've generated tens of billions of cells so far and continue to generate more. We're, you know, growing I I think we're we've doubled capacity three months ago. We've doubled capacity again this month. We intend to double capacity yet again next year. so I mean in terms of generating cellular data. And then the other Ravid Shwartz-Ziv: And this is because of like, this is because of like just like what, more like robots or like more like experiments, more like people like why, how it's... Daphne: It it's a combination of i it it's a there's a lot of kaizen going on, multiple things that come together to allow us a double capacity. So lab automation, absolutely part of it. better biochemistry so that you can get better yield from each plate of cells, another part of it. the agentic technology that allows us to go faster from a concept of we would like to measure a certain thing to how quickly we have a system in which that thing can be measured in a scalable way. Another huge thing. So a lot of different contributions, some of them from automation, sorry, lab automation, some from agentic automation, some from better biological tools that we develop and constantly improve and enhance. Like we've doubled yield capacity, the the yield of a single experiment by getting better readouts from ourselves. by measuring them in a different way. So it's actually amazing to see how each one contributes 20%, 30%, either to yield or to time or to volume and so on. And it just keeps getting better and better. The other half of the company does the machine learning AI on on what we on what we do. So these are machine learning scientists, software engineers, and so on. And many of the improvements also come from that side. Like for example, If we have AI methods that can, for example, autofocus by looking better at the cells, we again increase by 20 to 30 percent the number of cells that we can read, the number of cells that we need in order to derive insights because we have better embeddings of our cellular images. Again, you get an improvement from that. the ability to automate the process with the scientists of the experimental design using agentic technology. And so all of these actually come from this act actually intertwining of the kind of wet lab components and the computational components, which is why the company's called in citro, because it is the seamless integration of in silico in the computer and in vitro, which means in the lab. Allen Roush: Do you do you think that there's any room for alternative biochemistries to to especially now that AI is assisting in in s all the work that that you're doing? You know, there's all these theories that silicon based biochemistries are possible, right? And and I think other, you know, non-carbon based ones. Do you have any opinion on this? Daphne: I mean only in the small only in the small sense that we've been using things like, for example, alternative nucleic acids, modified nucleic acids, modified amino acids, and various drug design that tends to create molecules that are less susceptible to the standard degradation processes of the body, so they're longer lived, have different efficacy profiles. I think that's amazing. I don't know enough about silicone based. biology to try and hypothesize whether that is a thing. Ravid Shwartz-Ziv: What do you think about auto research? So, there are two... When people are talking about auto research, they talk about many different things, but there are two approaches. One is, let's make fully automated experimental process, that people are designing. The agents are making better and better models on some dancing problem. And the second one is kind of like a co-improvement. It's like you talk with the agent and you discuss the ideas and then it comes up with better ideas, it maybe search the literature and then you can discuss and only then it automated and it gives you the results and you analyze it together and this is kind of like co-improvement with the agent. In your company, do you use or like... Do you think it's possible to use in either of these approaches? Daphne: So we use both. there are certain cases where the problem that the objective that you're trying to achieve is well defin is well defined. So when coming back to our earlier taxonomy, once you've reduced the biology to a mechanism and you're trying to design something that is where you have a very well-defined objective function that you can measure, and a circumscribed search space, like the space of whatever. you know p protein sequences, protein effectors, or a particular set of assay conditions, and you're optimizing within that space, the agents do an amazing job of, you know, we generate a bunch of experiments, you run them through together, you collect the data, you see what the results were, you learn from that. You can think of it as kind of like agent enabled, simultaneous hill climbing and sort of like the the agentic version of what we used to call genetic algorithms, which are obviously long obsolete, but you have a bunch of experiments you simultaneously optimize and get better and better. That's awesome. And we do that all day long. a lot of the problems that we are facing are ones that belong into the ill-defined category where you can't even tell the agent what it is that you're looking to do other than you know, I'm trying to make a drug for Al for Alzheimer's disease or ALS. And we have not as of yet seen a gentic technology that you can just let loose on that problem and it comes out with something meaningful. And that's where I think human taste, human judgment, human critique, human prioritization of a lot I mean, agents are great at coming up with lots and lots of ideas, picking which ones Are actually more likely to get you to a meaningful answer. I mean, you know, that's where a lot of taste and judgment comes in. So I think for a lot of the kind of building block problems, you might call them, we use automated agentic technology and frankly are hoping to automate more and more of that. as as the as the inter integration between the agentic and the lab automation becomes tighter, we can just it, you know, make that loop entirely closed. the higher level thinking pieces, it's very much a a partnership with a human. Ravid Shwartz-Ziv: And what do you think how the future will look like in drug design? Do you think like the scaling will just like because like now I have better methods to decode some information, so it will be much more efficient or do you think like it will be fully automated? Daphne: I think absolutely it will become more efficient. I mean everything is becoming more efficient by leveraging you know, modern day AI, modern day interpretation methods, ag agents that help you think through problems and so on. Everything will become more efficient. I don't think we are at the verge of having AI just go out and solve the problem of curing human disease. I mean, I think there are Multiple challenges, the data question being among them, the slowness of the experimental cycle being another. Because as we discussed, sure, you can experiment on a cellular system or a biochemical assay very effectively and do this closed loop iterative design where the agent gets this feedback. Agents are amazing at learning when you give them immediate quantitative feedback and they can just optimize to that. They will Just bang their head against that feedback and get better and better. The systems that we work with are not the ones where you get that type of feedback. Maybe you can argue cellular systems are somewhere on that spectrum because you get feedback within a matter of weeks, not seconds, weeks, because you know what? The cells need time to grow and respond to the stimulus. And no matter how many GPUs you throw at the cells, they're not gonna grow any faster. So so even that is not your immediate feedback loop. And when your experimental system is a human being, then the feedback loop is not weeks or months, it's years. And so you need to do there's a that's not the problems that AI is just gonna crack instantaneously. Allen Roush: So what do we do about these foundation model labs trying really hard to prevent misuse and inadvertently or maybe advertently making it almost impossible to do any work on healthcare, biotech, etc.? Do you do you like like what's your reaction to this and do you think that if these kind of trends continue it could negatively affect you or the higher industry's work? Daphne: So look, I don't know if the l launch or lack thereof of the fable models from companies like Anthropic is the most rate limiting aspect of what it takes to do biology discovery. I personally continue to think that the most rate limiting thing in our ability to do biology discovery is the lack of data that is available to really understand biologies that we currently don't understand. And so is that a problem for biology research, maybe for certain types of research? I don't know because I haven't tried the fable models compared to the previous to see how big of a delta that is. But I think the fundamental problem is all of those models were trained on the things that we've already measured, on the things that we already know. And there's only so far you can get with the distillation of what has been measured versus the things that we have not even begun to to appropriately interrogate. Ravid Shwartz-Ziv: What do you think about foundation models for biology? Do you think this is something that it's possible to do, or do you think we should have very limited models? Daphne: I mean right now we have built foundation models. By foundation models, I think you mean foundation models for specific biological data modalities versus frontier models for natural Ravid Shwartz-Ziv: Yeah. Daphne: for language, right? we've built so far as a community, I would say a small number of actually useful foundation models and we've built those for data modalities that are relatively abundant. And so, you know, proteins and protein structures were obviously the first and the incredible success of Alpha Fold remains a, you know, kind of a paradigm shifting result and worthy of the accolades that it got. we have built some foundation models for DNA that I think are interesting. I would point out that both of those foundation models have been built in cases where the biology is consistent across all of the st species of life. So proteins fold the same way in bacteria, in yeast, in mice. DNA and the binding of you know factors to DNA. So we had huge amounts of data for those models that were it that enabled these foundation models to be trained. We have not really trained, I would say, meaningful foundation models for most tasks in biology. And it comes back to our ability to measure those biologies at scale. We're orders of magnitude lower on, for example, single cell data. Measuring a single modality like transcriptional profiles relative to what we would need in order to build a foundation model of human trans human single cell transcriptional response. So orders of magnitude. and especially if you want those to also be able to predict causal interventions as opposed to just observational state in kind of wild-type conditions, which is what most most of the data in, say, the human cell atlas is basically in a passive state. So I think I I love foundation models. I think we need more of them. What we need in order to unlock them is measurement at scale. Ravid Shwartz-Ziv: Do you think causality to measure data from causality intervention is important? Or do you think we can... Daphne: No, I mean everything that I have ever seen in cell biology shows that cells behave completely differently when they are perturbed versus when they're just sitting there happily swimming along in whatever medium. i there you know there's and by perturbation then I include environmental perturbations like subjecting them to stress as well as perturbations that are that correspond to What happens when the body dysfunctions and a particular gene isn't performing its function? So the space of perturbations is clearly unboundedly large. I mean it's it's literally infinite. There are st I mean, if you think about combinations of perturbations, like what happens when you shut down two genes, three genes, when you also put that in the context of an egg an environmental perturbation. I mean, at some point we'll start to saturate and we'll be able to sort of interpolate between those different conditions. I don't think you need to measure every single condition in order to get to a model that is predictive. But right now the space of biology that is in distribution is diminishingly small compared to the space of biology that is out of distribution for the data that we've collected to date. So right now we are completely unable to make predictions on the vast majority of conditions that are relevant to disease because they are largely or entirely out of distribution. Allen Roush: Do you do you think that creativity in the space of drug design is is one of the core factors in making better ones, or is it more just like understanding of processes happening in the body? Daphne: Ooh, boy, let me unpack that question. so first of all, I would say that understanding the processes that happen in the body is not a process that lacks creativity. So I don't think it's an either-or. I think there is a lot of creativity in understanding and especially in prioritizing what to measure and what to make of what we measure. And yes, some of that can be done by an AI, but I think there's, you know, whether th that creativity is by an AI or by a person or some combination thereof, I think there's a lot of creativity there. The other thing I would say, and I think it's a really important statement, AI might become really good at, and it maybe is already really good at engineering within the space of things that we understand. But There are some step functions that are taken sometimes that are I haven't seen NAI be able to make. So for example, let's take a technology that many people are familiar with, which is CRISPR. The identification of this weird phenomenon that happens in the immune system of bacteria, and saying, wait, we can turn this into the most powerful gene editing technology that humanity has ever seen and deploy that as a research tool and a therapeutic intervention. So taking something like doing an entirely out-of-the-box step of I'm going to take this from over here and translate this to something that is completely on the surface unrelated, that is a major creativity leap that has given us most of the really big inventions. So is AI gonna get really good at kind of engineering optimizing within the space of things that we've already you know realized? Absolutely. And that's gonna be a huge accelerant. Is it gonna make these out of the box leaps? I don't know. That'll be amazing. Would love to see that. Ravid Shwartz-Ziv: So, if now you need to choose, or like, you need to choose, right? Like, how much time and effort and money I will put on the algorithm versus, like, collecting data versus, like, the experiments themselves, like, the humans? Like, how much you think, like, what is more important, basically? So... And do you think it's like... Even if it's more important, do you think like it's worth the money or like the time or the eff- Daphne: I mean, look, we make these decisions every day. I mean, I allocate capital in the company to generating data, to you know, paying the the salary of our machine learning scientists and engineers and the you know, the fairly large amounts of money that they spend on GPU and tokens, and on the taking some of the insights and developing those and turning them into drugs that actually get delivered to people. And so If you're asking me which is more important, I invest capital in all of those because they are all critical. I mean, generating data on its own is not useful if you are unable to interpret what you see. Developing better and better methods that operate on the same diminishingly small subset of biology without giving it the right fodder to learn some of those foundation models and to expand the scope of what is in distribution is also not very useful. And all of that is not very useful at all unless you turn it into value. And value is you know, people don't pay you for insights. people pay you for things that make their lives better, which in the industry that we're in is people pay you for medicines. And so you have to also solve the how do I take those insights and turn them into value and deliver them to people. And so we make investments in in all of those. I mean, I'm not gonna break down our internal budget for you, and there's also you know, different cost differentials, experiments in humans, also known as clinical trials, just tend to be expensive for reasons that are, you know, unfortunately you know, hard constraints on on that. But I mean we make significant investments in all three. Ravid Shwartz-Ziv: And what do you think about simulations? Do you think this is like... Are there like any simulators that are close to be valuable? Daphne: I think in the f areas that are close to physics, the simulations are getting to the point where they are not bad. So in silico protein folding, not bad. In silico binding models, not bad. we use those. We have virtual screening that is AI enabled, to understand when a small molecule might bind and where it binds to a protein. Absolutely. Those are the areas that are that are either or both. close to physics because physics is a place where actually are at this point good at simulating. So chemistry is reasonably close to physics, protein folding is reasonably close to physics, or we've collected enough data. Unfortunately, those two spaces, the things that are physics adjacent, I mean physics is just basically math, math with some phys principles that we've learned from the world, right? So so there is physics and physics adjacent. There's places that we've collected enough data, and then there's the huge space of biology that is neither of the two. Because you see, biology is not an engineered system that works on rational principles. Biology is the result of a stochastic evolutionary process where things happen to come into being that may or may not quote make sense, that may or may not be grounded in the principles of good design. and that's the system we need to deal with, is this like mess. It's a mess. Biology is a mess. And we need and the only way to get at what biology does is frankly to look at it and say, Wow, I would never have expected that. And then you learn from that and you move on. Ravid Shwartz-Ziv: But okay, so do you think like learning from I know classical theories about like random Metrics is or like whatever right stochastic processes or things like that. Do you think this is helpful in like today? analyze and and design and Daphne: I mean I I look, I don't want to speak to there are certainly mathematical frameworks that have been proven to be useful in certain places in biology. I mean, look, this statistics of natural recombination have been the foundation for statistical genetics. And so absolutely when you look at how do we understand human genetics in the at the population level. That mathematical foundation is invaluable. And similarly, we talked about the use of mathematical models for the, you know, physics and protein folding and so on. So there are absolutely places, subfields where a mathematical foundation has proved to be hugely enabling. And then there's the rest of the mess, and we're nowhere close to having a mathematical foundation for most of that. So I I think the answer is. We use whatever tools we have that make sense. And in some cases, those are mathematical models. And whenever we find one of those, everybody celebrates because everybody would love to have a mathematical model that was predictive of what the experiment actually shows you. I think for a lot of the what happens in biology, it's so complex and so kind of random. in some ways in terms of how it evolved to doing what it does, that the only principles that we will ever be able to get are taking enough data and fitting a very complex function to it and and having that become more and more predictive. It's not like there's a clear mathematical principl first principles framework. Allen Roush: Do you do w d do you think that there's significantly more of in in biochemistry to discover in terms of like fundamental mechanics of how it operates? And if so, do you believe that AI will significantly assist in that? Daphne: I don't know that I can really speak to biochemistry. It's not it's not an area where I'm particularly e an expert in. Well yeah, no Allen Roush: Or or tr dr drug design, sorry, it's just one of the main things. Daphne: no. So but when I think about drug design, drug design really is a as as I said, as much about or more about biology today than it is about chemistry. It's the understanding of the human biology and the And consequently, the intervention points that we need to have. And absolutely, biology continues to surprise us with new, completely new mechanisms that we had never previously anticipated. I will give you an example from, you know, relatively recently, the discovery of these so-called membraneless organelles, which is a discovery from about the last five to seven years, I forget the exact time, which are these things that form These organelles that form in a cell that are just like a bunch of things coming together to create a functioning entity that didn't exist before. And it doesn't have a membrane. It's not something that was there to begin with. It just kind of auto-creates, auto-generates. And it turns out that that so-called condensate plays a huge role in multiple diseases, including, for example, many neurodegenerative diseases, where proteins misfold and they call on other proteins. They kind of pull them in and they create a little community of proteins that never existed before. What is that? What is the origins of that? I mean, that's a new type of entity that we didn't realize existed. So, and that was recently, right? So are there more like that? Probably. Ravid Shwartz-Ziv: Okay, so maybe let's go to the earlier days. like, what actually, like what you wanted to build there, and like why you decided to build Corsair? Daphne: So we originally built Coursera, I would say, via serendipitous process. I had been at Stanford working on you know, technology-assisted education and how do we leverage the tools that we had at the time to give a better educational experience to the students that we had at Stanford. And then we realized that if we were doing that, then we could also take that. at least the technological component and offer it to a much much broader community of of people and we put a few courses out there almost as a kind of like a one-time experiment and were just awestruck by the uptake and and the demand for high quality education from among people who never had access to that because this was you know pre-LLMs you didn't have all of the ability to ask whatever chat or claude or Gemini about you know something that you wanted to learn about. And so I think that was I think that was we were just awestruck by the how impactful this was, which is what led me to basically leave Stanford and leave behind a you know flourishing lab of students who were like, wait, you said you would be our advisor and you're not there anymore. What happened? so yeah I can't say I feel great about that. but yeah, but the impact that we were able to have by creating that educational experience was I think un was phenomenal. I mean if I were building it again today, I would absolutely be building it differently. I would be building it based on, you know, the latest LLMs and really leveraging the ability of LLMs to personalize. the experience with the kind of insight that a you know a top instructor slash researcher has about kind of creating guideposts along the way of an educational journey. Because I think we did that part really well at the time, which is, you know, what why do some why do people go to a top academic institutions versus just read a textbook? It's because the ability of a professor, a really great professor, to inspire a particular, really unique and impactful learning journey is something that can be transformative to the lives of to the life of a student. So how do you combine that kind of like signposting guiding with the personalization of an LM would be how I would build the company if I were doing it today. Ravid Shwartz-Ziv: And do think today universities are still important or necessary for people, for the labor market? Daphne: so I think these are two different questions. I have two daughters, both of whom recently finished university. My youngest graduated from college earlier in the spring. And I think the experience that they had of, you know, their first chapter as adults in a setting that had A lot of like-minded, you know, people to socialize with, learn from, form lifelong friendships as the first chapter of their adult life was absolutely transformative. do I think it's better to just send them directly into the workforce and learn on the job? I'm not sure. I'm not sure that's a better experience for for people as kind of like their sort of first step into their adult life. So I think that is something that I would hate to take away from, you know, kids as they become adults. whether you need it for on-the-job training, I think what you really need for on-the-job training is, you know, something that is much more rapidly forming and evolving. I mean, a lot of the things that kids learn in college isn't about. The specifics of what you learn in college. Like, do I use today the calculus that I learned back when I was an undergrad? Absolutely not. But it gave me a lot of thinking skills, learning skills, social skills, a lot of like core capabilities that helped me kind of become the adult that I am. Writing skills, thinking skills, presentation skills. I think a lot of the Kind of like, I need to learn this thing that frankly didn't in most cases exist when I did my university or high school education. It's a brand new thing that that that is that that wasn't even on the horizon. I don't think that is a thing for which you need to go back to universities. I think what we need to create is a much better framework for lifelong learning. And where that belongs, is that something that universities can create? Is I think a very interesting question for universities to be asking themselves. We tried to ask them that in the Coursera days of how do you create lifelong learning experiences that are more adaptive to people's needs? I think we got them part way on the journey, but not all the way, unfortunately. But if I were a university, At this point, that would be the existential question that I would be asking myself. Am I solely for that, you know, four years of formative experience before people go off? and you know, whatever, plus teaching researchers to become researchers as an apprenticeship system and the PhD program, or is there a way for me to become relevant to the much larger number of individuals that are not in a sort of you know, well-defined, curated educational experience? Ravid Shwartz-Ziv: So, I Daphne: Okay. Ravid Shwartz-Ziv: have some personal questions. So, you did a lot of things, right? In a lot of different... kind of related, but different fields, what was the most fun part? Daphne: I think the fun part is the variety. I think I personally, I mean, some people love the I do one thing, I do it really well, I do it better and better all the time. You know, Wiley, who proved Fermat's last theorem, spent 20 years doing that and loved doing that. And I admire that. I have a lot of respect for it. I like variety. And that is because when people ask me like what is my superpower, I say that my superpower is connecting dots across things that people often don't view as being related. and that was certainly what I did in my in my life as an academic, and even more so now that I'm in industries, connecting dots between disciplines that generally don't talk to each other, pattern matching something that you see in one field, translating it onto another field. So that is the fun part. Ravid Shwartz-Ziv: And do you optimistic for the future of drug design? you think like we... So you described a lot of challenges. Do you think like the future will see more and more drugs come out of it? Daphne: I am very optimistic about the future of of drug discovery. I prefer to use the word drug discovery because drug design often relates that middle bucket that we talked about, which is I know what I need to design, now I need to design it, versus drug discovery, which is the process of understanding first of all what it is that I need to make. I am very optimistic about that because I think we are now at the point where we are seeing these two tidal waves of innovation that are coming together. One is the ability to measure biology at unprecedented fidelity and scale, whether that is measuring human cohorts like UK Biobank, All of Us, and various resources that I hope will come, or measuring cell biology and over time, not just individual cells, but organoids and multicellular systems and so on, that we are getting better and better at engineering. So that is one revolution. Which gives us enormous amounts of data, which is then the fodder for the ability to use the second revolution, which is our ability to use AI and machine learning to interrogate and see patterns in these data that a human eye will never be able to see. And Getting us to insights that we would never otherwise have had. And that has been the thing that we're missing. All of that sort of emergent complexity that we talked about in biological systems, combined with the inability to measure enough examples, have really made it hard for us as people to understand what we are, understand what we are as a biological system. And I think we're at the cusp of bringing together the data and the capability to interrogate that data. And that will teach us who we are and how we. Yeah. Ravid Shwartz-Ziv: Sounds good. Do you have anything else that you want to add? Daphne: yeah, I would like maybe to talk for just a moment about Culture and the importance of bridging within a culture of an organization. So I alluded earlier to the name in Citro, which is the integration of InSilico and Invitro, which really was about bringing together two not only two types of technology, which is the biology and the computation, but also two different groups of people that normally don't communicate very well with each other, not because they're bad people and don't Want to talk, but because they each have their own jargon, their own way of thinking, their own notion of timelines of progress. I mean, this discrepancy that we talked about earlier between, you know, computational people are used to getting immediate feedback and having the algorithm just continue to, you know, optimize in a feedback loop that has become Instant almost instantaneously, instantaneously fast. And biologists think in the scale of years to months. And so you have one group that thinks in everything from, you know, two-week sprints to one-second iteration loops, and you have or microsecond or whatever, millisecond. And and on the other side, you have people whose experiments take months or years. And so, how do you bridge between those two communities is really important. and I think we have done a particularly good job of creating like a single company out of these very disparate types of individuals. I think we are now in the midst of another bridging experiment, which is bridging between people and agents, which is another very different kind of bridge. but you have to adjust the People to the agents and the agents to the people and get them to collaborate effectively. And that is a different but related type of challenge. And I'm hoping that we're doing that again better than most. But I would say that organizations that do not solve this problem are going to be considerably less successful than ones that do. And by that I mean both of those examples. Like how do you bridge between Entities, people with very different skill sets, people and agents that have very different skill sets and how do you get the best of both is going those organizations that solve that are gonna be the ones that win. Ravid Shwartz-Ziv: I agree. Daphne, thank you so much for joining us. It was a real pleasure to have you. Allen Roush: Yeah, it was it was great. Daphne: Thank you. Ravid Shwartz-Ziv: Thank you. Daphne: thank you for having me and I we covered a lot of ground, so this was fun. Ravid Shwartz-Ziv: Thanks.