Latest / Code & Cure / #23 - Designing Antivenom With Diffusion Models

Transcript

#23 - Designing Antivenom With Diffusion Models · Code & Cure

0:00
SPEAKER_00: Snakes have had millions of years to perfect venom.
0:04
SPEAKER_00: AI needed about five minutes and a graphics card to help undo it.
0:19
SPEAKER_01: Hello and welcome to Code and Cure, where we discuss decoding health in the age of AI.
0:23
SPEAKER_01: My name is Vasan Sadathi, and I'm a cognitive scientist and AI researcher, and I'm with Warra Hagopian.
0:29
SPEAKER_00: I'm an emergency medicine physician and I work in digital health.
0:32
SPEAKER_00: And uh especially relevant to this episode is I am fellowship trained in wilderness medicine.
0:39
SPEAKER_00: Oh, that's right.
0:40
SPEAKER_01: I think I knew that about you.
0:42
SPEAKER_01: What um what does that mean?
0:44
SPEAKER_00: I mean, it means I got to go camping for weeks and learn how to carry someone down a mountain with a broken foot.
0:53
SPEAKER_00: It was pretty fun.
0:54
SPEAKER_00: And I also got to learn about um things that can happen in the wild, like snake bites, bee stings, all that good stuff.
1:04
SPEAKER_01: Yeah, yeah.
1:05
SPEAKER_01: And I think that's I'm very excited about today's topic, to be fully honest, about learning about because it's just completely different.
1:11
SPEAKER_01: Um we talk about venom and snake bites and anti-venom and so on.
1:15
SPEAKER_01: But I'm really excited about it because it's both different medically from what we've been talking about, but also technically, there's some really cool stuff and technologies that we really haven't discussed.
1:23
SPEAKER_01: We've talked endlessly about LLMs, but this is different.
1:26
SPEAKER_01: And so I'm really this is not an LLM.
1:29
SPEAKER_01: No, it's not.
1:29
SPEAKER_01: And I'm very excited about that.
1:31
SPEAKER_01: So, but before we get there, before we get there, um, let's just talk a little bit about like venom and what it means um to get what is what is anti-venom and so on.
1:43
SPEAKER_00: Yeah, so anti-venom is used for a variety of things if you have a severe snake bite, for example, or a spider bite, um, etc.
1:53
SPEAKER_00: And the way that it's usually made to neutralize, it's used to neutralize the venom, right?
2:00
SPEAKER_00: So that you don't get sick and die, right?
2:01
SPEAKER_00: Some of these venoms can do all they can do all sorts of things.
2:04
SPEAKER_00: Um in the paper that we're talking about today, they were looking at venom that paralyzes your muscles um because it blocks like the acetylcholine receptors.
2:14
SPEAKER_00: Got it.
2:16
SPEAKER_00: Um and so what happens with anti-venom is that it binds to the venom and prevents it from working.
2:25
SPEAKER_00: Ah, got it.
2:26
SPEAKER_00: Okay.
2:26
SPEAKER_00: But it's like a big process to make it because what they do is that they they collect venom from a snake.
2:34
SPEAKER_00: They like milk the snake, and then they like inject the venom into an animal.
2:39
SPEAKER_01: Well, I think I've seen the milking part.
2:40
SPEAKER_01: They like they like stick the fangs into the into a like a jar.
2:42
SPEAKER_01: I've seen that before, but they open the fangs up and stick it into a jar and it and it collects the venom, right?
2:47
SPEAKER_01: I don't I don't think I've actually seen how they milk a snake.
2:51
SPEAKER_01: Well, I've seen I've seen those um videos on on like like Jack Hannah or um, you know, other wildlife experts talk about these things.
3:01
SPEAKER_00: So anyway, so anyways, once they have the venom, they'll inject it into an animal, like not a not a human, but uh like a horse.
3:10
SPEAKER_00: And then what they do is they the horse makes antibodies to the venom, and we collect those antibodies and like purify them out.
3:21
SPEAKER_00: Got it.
3:21
SPEAKER_00: And so now you have an anti-venom.
3:24
SPEAKER_00: So if a human gets bitten by a snake that's like poisonous and they're gonna die or have some severe reaction, you can inject them with this purified horse serum anti-venom to bind to the venom and prevent it from getting into the cells and causing all sorts of disasters.
3:41
SPEAKER_01: Ah, okay.
3:42
SPEAKER_01: And so the idea is that the essentially the horse or whatever animal is used is the one that's generating the anti-venom and generating the structure for the anti-venom sort of naturally.
3:53
SPEAKER_01: And we can we can now get that because we've injected the animal with the venom.
3:57
SPEAKER_01: So it creates generates that when we need it.
3:59
SPEAKER_01: As long as it doesn't die, you know.
4:01
SPEAKER_01: Well, yes, but but yeah, outside of the the the animal ethics issues, that that's how anti-venom has been typically made, right?
4:09
SPEAKER_01: In the past.
4:10
SPEAKER_00: But if you think about this, you're like, well, that's gotta be, it's like a whole process, right?
4:14
SPEAKER_00: It's like very expensive to do.
4:17
SPEAKER_00: Um and these these anti-venoms, they they don't last on the shelf for a long time, especially if they're not refrigerated.
4:26
SPEAKER_00: Um, and in addition to that, you have to give them in healthcare facilities because the side effects can be like very significant.
4:34
SPEAKER_00: You can get um allergic reactions to them, you can have trouble breathing.
4:39
SPEAKER_00: They're not like benign things to give someone.
4:41
SPEAKER_01: Oh, okay.
4:42
SPEAKER_00: So there's a lot of problems with them.
4:44
SPEAKER_00: And you're relying on the animal to be able to make the antibody.
4:50
SPEAKER_01: So that's the part that was super interesting to me here, which I didn't know before.
4:54
SPEAKER_01: And I'm gonna mispronounce this word, but do it anyway.
4:57
SPEAKER_01: Immunogenicity.
4:59
SPEAKER_01: Genicity?
5:00
SPEAKER_01: You were you're close there.
5:01
SPEAKER_00: Immunogenicity.
5:02
SPEAKER_01: Immunogenicity.
5:03
SPEAKER_01: Okay.
5:04
SPEAKER_00: So this is the ability of a substance like a protein to trigger an immune response.
5:11
SPEAKER_00: And some toxins, like like uh toxins that go after your nerves or go after the cells, some of them are like not immunogenic, meaning it's hard for us for a horse or for a sheep or whatever to make an antibody against it.
5:30
SPEAKER_00: It's like it doesn't, it doesn't recognize it as foreign.
5:34
SPEAKER_00: Right, it's just like under the radar in some sense.
5:36
SPEAKER_00: Exactly.
5:36
SPEAKER_00: And so if this horse can't make an antibody or doesn't make an antibody because, you know, the the venom has poor immunogenicity, then you're like, well, what do we do?
5:51
SPEAKER_01: Yeah, and these are the and and it turns out that I think what I also found interesting was it turns out that some of these low immunogenic um venom um were like what I think they call it three-finger uh proteins or whatever, three-finger um substances.
6:07
SPEAKER_01: But anyways, they're some of the most toxic ones, right?
6:10
SPEAKER_01: They're found in creatures like um the island snake, the mambas, and so on.
6:15
SPEAKER_01: These are some of the most poisonous snakes in the world.
6:17
SPEAKER_01: And that's the kind of pro that's the kind of venom that they have, which is you know, all the more reason to me, it seems like that, you know, we need to figure out how we can actually produce anti-venom for those cases.
6:30
SPEAKER_00: Yeah, and I actually had to look up, I was like, why are they called three-finger toxins?
6:34
SPEAKER_00: But if you look at the chemical structure of them, it's like three loops.
6:38
SPEAKER_00: That's it.
6:38
SPEAKER_00: So they like they're like a Y shape or something, or they're no, they're it's like three separate loops, and each of the loops is like a finger.
6:45
SPEAKER_01: Oh, I get it.
6:46
SPEAKER_01: Okay, okay.
6:46
SPEAKER_01: So it looks like a three-fingered hand in some sense.
6:49
SPEAKER_01: And you look at the at least in the picture, in the it looks like a blob with sort of three protruding fingers, right?
6:55
SPEAKER_00: I think that's that's the idea.
6:57
SPEAKER_00: Yeah, yeah, yeah.
6:58
SPEAKER_00: Um and so, like say there was a cobra bite, and the toxin gets into your cells.
7:05
SPEAKER_00: What the toxin does is it um it blocks the nicotinic cholinergic receptor, and then that makes it so that your muscles don't work and you need your muscles to breathe.
7:19
SPEAKER_00: And so if you don't breathe, then you die, right?
7:21
SPEAKER_00: And so the whole idea here is hey, can we get something to bind on to that three-finger toxin so that it doesn't bind on to the acetylcholine receptor in the cell.
7:36
SPEAKER_00: So that if it doesn't get into the cell, then the muscles can contract, you keep breathing, you don't die.
7:43
SPEAKER_00: Right.
7:44
SPEAKER_00: That's it.
7:44
SPEAKER_00: Yeah, yeah.
7:45
SPEAKER_01: So that's this sets this sets it up nicely for an AI system because there's been some recent progress about developing and synthesizing proteins.
7:55
SPEAKER_01: Um, and I want to talk a little bit about that and talk a little bit about that in the context of other AI systems that you might know about, which is like ChatGPT and so on.
8:05
SPEAKER_01: And uh here they use an AI system called um diffusion model, which we'll talk about in just a second here.
8:12
SPEAKER_01: Um, but broadly, you know, so far, um, we've been talking about LLMs, right?
8:18
SPEAKER_01: Uh and LLMs take in a word and they produce the next word in the sequence.
8:25
SPEAKER_01: And what they're in on So they can't take in a protein and spit out an anti-venom.
8:30
SPEAKER_01: That's right.
8:31
SPEAKER_01: Well, they could, you could have it, but what what it's doing actually, an LLM, what it's doing behind the scenes, is that it works because it has learned um what makes a valid sentence in human communication.
8:43
SPEAKER_01: And so it's able to produce more valid sentences in human communication.
8:48
SPEAKER_00: That's not quite what we're looking for here.
8:50
SPEAKER_01: Right, because we're not looking for human sentences, we're looking for specific proteins, right?
8:54
SPEAKER_01: So, but but but the point is that that sort of sequen uh LLMs do sequence predictions, those are actually useful in figuring out sequences.
9:03
SPEAKER_01: But and and some might some might say that proteins are essentially sequences of amino acids.
9:08
SPEAKER_00: Yeah, but the structure is what's really important that matters, right?
9:13
SPEAKER_00: The three fingers.
9:14
SPEAKER_00: We're coming back to it.
9:15
SPEAKER_00: I didn't know we would, but here we are.
9:17
SPEAKER_01: Yes, so the 3D shape of the structure makes determines function.
9:21
SPEAKER_01: And I think an analogy that really helped me understand this was the notion of a key to a door.
9:27
SPEAKER_01: Your key can be made of metal, it can be made of any material, it doesn't matter, but it's that shape of the pointy parts that makes all the difference in its function, determines how well it works for a particular lock.
9:37
SPEAKER_00: I don't want to keep made of plastic, just for the record, but I still like your analogy.
9:42
SPEAKER_01: Yeah, yeah.
9:43
SPEAKER_01: And so I think that that's that's the point I want to drive home, which is that the shape and the 3D structure really matter.
9:49
SPEAKER_01: And so uh there was a lot of work, um, I want to say a few years ago, where uh people developed a new AI model called Alpha Fold, which was a really big, really big discovery, really big um yeah, discovery.
10:03
SPEAKER_01: And in that setting, they were able to synthesize these geometric shapes, these 3D shapes from just sequences.
10:10
SPEAKER_01: So given a protein sequence, they were able to generate 3D shapes of the protein.
10:14
SPEAKER_01: That is very powerful already.
10:16
SPEAKER_01: That is something that didn't exist before, it was really helpful in a lot of different settings.
10:19
SPEAKER_01: And that you can imagine in drug discovery settings and all kinds of other settings, right?
10:24
SPEAKER_01: Having a sequence isn't enough.
10:25
SPEAKER_01: You you actually have now the 3D structure that goes with it.
10:29
SPEAKER_01: But in that case, you still had the sequence to start with, right?
10:33
SPEAKER_01: Right.
10:34
SPEAKER_00: And here we don't.
10:35
SPEAKER_01: Here you're designing brand new novel proteins.
10:38
SPEAKER_01: You have no idea.
10:39
SPEAKER_00: Or you like have the, I don't know if this is a good analogy, I'll have to tell you, but like we have the key, but we need to design like some piece of the lock that will fit on to like the edge of the key.
10:50
SPEAKER_01: I'm not sure that no.
10:51
SPEAKER_01: I I think you're you're right on the analogy piece in general using the key, but I think the key here is the is the protein we're designing.
10:58
SPEAKER_01: So we have a lock, but we we keep if we already knew that we needed certain number of pointy parts, then we could come up with the key structure for it, but maybe not.
11:07
SPEAKER_01: That's not enough.
11:08
SPEAKER_01: I think this this key metaphor is a good thing.
11:09
SPEAKER_01: I've lost it.
11:10
SPEAKER_00: I've lost it.
11:10
SPEAKER_00: Okay, we're falling apart here.
11:12
SPEAKER_01: We might be losing you listeners as well in the process.
11:14
SPEAKER_01: But come back if you're we're done with the key analogies for today, I think.
11:18
SPEAKER_01: Um but um yeah, so I the the key the key aspect here is that um we're designing brand new proteins, and what you end up wanting to provide as input is not a sequence, but instead you might have a set of constraints.
11:36
SPEAKER_01: You might say that you need a protein that grabs onto this virus or catalyzes this reaction.
11:42
SPEAKER_01: You might have a set of things, requirements for that protein.
11:45
SPEAKER_00: Or neutralizes this venom toxin.
11:47
SPEAKER_01: Yeah, yeah, exactly.
11:48
SPEAKER_01: Exactly.
11:49
SPEAKER_00: And it's the shape, like you have this goal, but you you also need the shape of the protein to fit on to the venom toxin.
11:58
SPEAKER_00: Yes.
11:58
SPEAKER_00: To bind it.
11:59
SPEAKER_00: Exactly.
11:59
SPEAKER_00: It's the structure that's the most important.
12:02
SPEAKER_00: Yeah, yeah, yeah.
12:02
SPEAKER_00: Exactly.
12:03
SPEAKER_00: You need to understand what the shape is for it to bind onto that protein the best.
12:09
SPEAKER_00: Yes.
12:09
SPEAKER_00: It can.
12:10
SPEAKER_01: No, exactly, exactly.
12:11
SPEAKER_01: And I think that that's the key piece here.
12:13
SPEAKER_01: And so researchers came up with this um a new model called um uh Rosetta Fold diffusion.
12:20
SPEAKER_01: And I'll explain diffusion models in a second.
12:22
SPEAKER_01: But Rosetta, it was called Rosetta Fold because Rosetta was a well-known uh sort of modeling, protein modeling, protein shaping tool that people have been using for a long time.
12:32
SPEAKER_01: So instead of honoring that and honoring Alpha Fold that we just talked about, um, created what's called Rosetta Fold, which was able to generate uh new shapes from scratch.
12:41
SPEAKER_01: Um and diffusion models are what's being used if you think about image generation right now and video generation and all these tools that are quote unquote being used for creating AI art and so on.
12:53
SPEAKER_01: Um stable diffusion, mid-journey, if you know those names.
12:56
SPEAKER_01: Um but basically what these models do, which is works completely differently from an LLM, is that they start with random noise.
13:02
SPEAKER_01: They start with a random, so if you look at the images, they start with a random image, random noise image, and and gradually through iterations of processing that image, they come up with something final.
13:12
SPEAKER_01: And so that's a diffusion model.
13:14
SPEAKER_01: It's uh it's a new type of approach that has been around for a little bit now, but it's um used heavily for image generation tools.
13:21
SPEAKER_01: And RF diffusion, Rosetta Fold Diffusion, used it for synthesizing proteins, right?
13:27
SPEAKER_01: So like now you can use the same tool to generate new proteins.
13:32
SPEAKER_01: So what you do is you start with a random set of points in space, just like you have a random image.
13:37
SPEAKER_01: And gradually you build out the shape of the protein.
13:41
SPEAKER_01: And uh the system ensures much like we had um LLMs in some sense learning the patterns in human language, uh these models learn patterns in good, valid protein shapes.
13:55
SPEAKER_01: And as such, they what ends up happening is you start with these random points in space and then you gradually build up the protein shape as long as it and ensure that it obeys physics and biology constraints and any other constraints you might have.
14:09
SPEAKER_01: So it unlike an LLM, it actually generates the 3D structure at the end of this whole process, starting from a random set of points.
14:17
SPEAKER_01: Now, you can imagine in this particular instance, now imagine you had like this blob with the three fingers that we just talked about, right?
14:25
SPEAKER_01: And you threw a bunch of random points near the three fingers, roughly.
14:29
SPEAKER_01: You have no idea what the points are.
14:30
SPEAKER_01: One of the fingers, whatever.
14:31
SPEAKER_01: Yeah.
14:31
SPEAKER_01: And then you let the diffusion model gradually um uh keep adjusting those points till it arrives at something that binds with the three fingers.
14:40
SPEAKER_01: Now, what's cool about this is that we can enforce physics and biology constraints, and now we have those three finger constraints as well.
14:46
SPEAKER_01: And that's sort of the quote unquote prompt of these models.
14:50
SPEAKER_01: So you're prompting it with the venom and you're telling it bind to it, right?
14:54
SPEAKER_01: And you're essentially creating these um novel structures just through a gradual process of the technical term is denoising.
15:04
SPEAKER_01: And that's why they're called um, you know, the diffusion models, because what they do is they they start from noise and they denoise through the process.
15:12
SPEAKER_01: And what they've learned to do is to learn how to do that.
15:15
SPEAKER_01: And actually during the training process, uh, for at least for images, what happens is you give them a good image and then you keep adding a little bit more noise to that image.
15:24
SPEAKER_01: And for a human eye, if I add a little bit of noise to an image, you still know that it's an image of something.
15:28
SPEAKER_01: If I showed you an image of a chair, you would, even if I made it a little grainy, you still would know that it's a chair.
15:33
SPEAKER_01: Yeah, of course.
15:34
SPEAKER_01: So that's the key piece.
15:35
SPEAKER_01: And you keep adding noise, keep adding noise till you get to random noise, and then you add back in the pieces to create the original image.
15:41
SPEAKER_01: And that's how these systems are trained.
15:43
SPEAKER_01: And how well it re-re reconstructs the image from noise is kind of how, and then you do that a million times or whatever.
15:49
SPEAKER_00: So you feed in that protein and figure out, or maybe the system figures out like what structural features it wants to go after.
15:57
SPEAKER_00: Like you don't need to bind all three fingers, right?
15:59
SPEAKER_00: You need to figure out, hey, like I want it to bind here, or maybe the system figures out it here are three spots we could try to bind the easiest.
16:10
SPEAKER_00: And then that's when it does this like denoising thing.
16:16
SPEAKER_00: And if you have something that can bind on to the venom toxin, that basically neutralizes it because it can't, if it's binding into the right spot where it would have gotten into the cell, it can't bind on to the cell now.
16:30
SPEAKER_01: Yes, and now you have protein folds and structures that now are candidates for actually synthesizing them, actually making them.
16:40
SPEAKER_00: Which is cool because now you have something that is probably more thermostable, right?
16:46
SPEAKER_00: It's gotta it's gonna have a longer shelf life.
16:48
SPEAKER_00: It doesn't need to be refrigerated.
16:50
SPEAKER_00: Um, and you're not like doing this on animals and getting limited quantities of anti-venom.
16:56
SPEAKER_00: You could you could just like make it, make a bunch of it, make a ton of it.
17:00
SPEAKER_00: And it could be available in more places.
17:03
SPEAKER_00: Like, you know, it's not just gonna be necessarily in in hospitals.
17:07
SPEAKER_01: Yes, and all you need is samples of venom, right?
17:10
SPEAKER_00: Right.
17:10
SPEAKER_00: So you can still you can still watch your videos about milking snakes.
17:14
SPEAKER_00: Thanks.
17:16
SPEAKER_00: Um, and so what they did in this study was they took the ones that they said, okay, we think these have the greatest binding affinity for the toxins, and they tested them on mice to see if the mice who got the the toxin and who also the neurotoxin and who also got the anti-venom that was created in this new way survived and if they kept breathing, because that's the the type of venom they used.
17:48
SPEAKER_00: Drumroll.
17:49
SPEAKER_00: And it worked, it worked, which is so cool.
17:53
SPEAKER_00: Amazing.
17:54
SPEAKER_00: Because as we talked about at the beginning, there were so many problems with the way that we produce anti-venom now, and this is a novel way to do it, and to be able to find something with this type of binding affinity so quickly.
18:11
SPEAKER_00: Yeah, it's like, you know, it used to be when we talked about things like drug discovery, you're like throwing darts at a dartboard, and now it's like, hey, we can zoom in on the structure of the neurotoxin and figure out how to neutralize it.
18:28
SPEAKER_01: Yeah, and that's incredible.
18:29
SPEAKER_01: And and and the uh, you know, the the mice that they studied, you know, not only did well immediately after receiving it, but also like several minutes later too, right?
18:39
SPEAKER_01: That the the the talk the effect of the the new drug, uh the new sorry, the new anti-venom held on for a little bit longer, right?
18:47
SPEAKER_01: Or something like that in the in the study I remember reading about that.
18:51
SPEAKER_00: Yeah, I mean, you want these things to have a pretty long half-life, right?
18:56
SPEAKER_00: So that any toxin that gets released later on.
18:59
SPEAKER_00: Oh, I gotcha, or that gets into the body a little later on, that reaches the circulatory system later, it also gets neutralized.
19:05
SPEAKER_00: So you want these to be like very stable molecules that get all over your body.
19:10
SPEAKER_01: Ah, I gotcha.
19:11
SPEAKER_00: Right.
19:12
SPEAKER_00: In order to work.
19:12
SPEAKER_00: And so that's there's like some conceptual stuff that goes into like what makes a good venom inhibitor, essentially.
19:19
SPEAKER_01: Yeah.
19:19
SPEAKER_00: And and they were able to prove this in mice.
19:22
SPEAKER_00: Now it's like early times, it's early stages, right?
19:24
SPEAKER_00: But it's overarchingly a promising idea in this situation.
19:29
SPEAKER_00: And for drug discovery in general, if we know what we're going after and we know the shape of it and the structure of it, and we know what we want to do with it, yeah.
19:38
SPEAKER_00: This is where these machine learning techniques can come in and help us figure out what's gonna have that binding affinity.
19:48
SPEAKER_01: Yeah, yeah, yeah.
19:49
SPEAKER_01: Yeah, that's kind of what I wanted to talk about with this with this paper.
19:53
SPEAKER_01: It's very exciting, very exciting stuff.
19:55
SPEAKER_00: Yeah.
19:56
SPEAKER_00: I I think this is really cool, especially because the one.
20:00
SPEAKER_00: They tested it on was something a protein that really didn't trigger a great immune response.
20:05
SPEAKER_00: Um, the three-finger toxin family, which is in like like you said, cobras, mambas, sneeze, sea snakes, among others, it doesn't generate a great immune response in horses, for example.
20:19
SPEAKER_00: And so this is something that solves that problem, and it does so in a way that is scalable and reproducible.
20:30
SPEAKER_00: So I'm excited to see, you know, human studies on this and um and where this can go beyond neutralizing venom.
20:38
SPEAKER_01: Yeah, yeah.
20:40
SPEAKER_00: And uh that's it for this week.
20:42
SPEAKER_00: We will see you next time on Code and Cure.
20:44
SPEAKER_00: Thank you for joining us.