Latest / The Adults In The AI Room / Earley AI Podcast - Episode #59 Exploring AI Limitations and Future Potential with Ron Green

Transcript

Earley AI Podcast - Episode #59 Exploring AI Limitations and Future Potential with Ron Green · The Adults In The AI Room

• 0:00
  SPEAKER_02: So welcome to the Early AI podcast.
• 0:02
  SPEAKER_02: I'm Chris Featherstone, and unfortunately Seth can't join me today, but it'll be it'll be really fun to dive into our guests today in terms of talking about AI misconceptions and the emergence of AI capabilities, the role of knowledge and architecture and AI systems.
• 0:21
  SPEAKER_02: Let me get to our guests, because that's the probably the most important thing that all of you uh listeners want to hear about anyway, instead of me rambling on.
• 0:28
  SPEAKER_02: So our guest today is a digital guru with a background in computer science and a master's degree in artificial intelligence from Sussex.
• 0:35
  SPEAKER_02: He's the founder of Kung Fu AI and brings over 25 years of experience working in artificial intelligence and machine learning technologies across industries like biotech, security, and telecommunications.
• 0:47
  SPEAKER_02: So without further ado, Ron Green, welcome to the show, my friend.
• 0:50
  SPEAKER_02: Hey, thanks for having me, Chris.
• 0:52
  SPEAKER_02: Listen, it's it's been, as you know, an extremely crazy ride across AI and ML in general, especially within the last we'll say 18 months.
• 1:04
  SPEAKER_02: And it drives me crazy and yet it's super exciting at the same time that people believe that this is now AI.
• 1:12
  SPEAKER_02: And I would love to know from your experience, because being in it 25 years, what did you work on back then?
• 1:20
  SPEAKER_02: Like what were the types of models and what were you trying to solve and things like that?
• 1:24
  SPEAKER_02: I'd love to get, you know, not to go back to your history, really.
• 1:27
  SPEAKER_02: Like it may bring up facial ticks, but I'd love to know some of the things that you were working on back then.
• 1:32
  SPEAKER_01: Yeah, yeah, absolutely.
• 1:34
  SPEAKER_01: So back in the 90s, when we were building neural network-based systems, they were just so much smaller than they were today.
• 1:41
  SPEAKER_01: So the types of problems you could solve would be viewed as beyond toy problems now.
• 1:46
  SPEAKER_01: So we were doing things like using neural networks to try to adjust um the um output of like DNA sequencing hardware where you would you would fluoresce the nucleotides and you would run them through the DNA sequencer.
• 2:04
  SPEAKER_01: But as those uh nucleotides sort of elongated through that process, the waves would get all mixed up, and it was literally difficult to tell whether it was A, C, G, T or was it was that C before the G, because the waves would kind of just wash over each other.
• 2:20
  SPEAKER_01: So we were using um neural networks to solve problems like that.
• 2:24
  SPEAKER_01: We were using them for rudimentary vision problems.
• 2:29
  SPEAKER_01: So within simulated environments, like imagine autonomous robots, little sort of car-like figures, giving them just enough of a visual signal that they could move left or move right in and these sort of uh dynamic evolving systems interacting with each other, wayfinding things like that.
• 2:50
  SPEAKER_01: But the the biggest problem about AI back then was it just was so hit or miss.
• 2:56
  SPEAKER_01: We're we're definitely still in the realm of sort of it's more art than science.
• 3:00
  SPEAKER_01: We don't have a super deep understanding of what's going on.
• 3:03
  SPEAKER_01: But back in the 90s, it was not even clear if we were on the right path.
• 3:07
  SPEAKER_01: I've told this story many times, which is the folks working in symbolic AI, you know, one of the main, you know, sort of alternatives to connectionist and neural-based AI, they would joke us with all the time.
• 3:20
  SPEAKER_01: They would say, when your AI systems don't work, you don't know why.
• 3:24
  SPEAKER_01: And when they do work, you don't know why.
• 3:27
  SPEAKER_01: And so there was this sense that it was pretty far afield of science and more uh just just throwing stuff against the wall.
• 3:34
  SPEAKER_01: Um and today we're we're solving problems that would have been a dream, you know, back 25 years ago.
• 3:40
  SPEAKER_02: So you you said something that was interesting in terms of that even today it's still more art and science.
• 3:46
  SPEAKER_02: Double click on that for many, because I want to understand, and then we'll get to like misconceptions and stuff, but I'd love to double-click what you mean by that.
• 3:55
  SPEAKER_01: Yeah, absolutely.
• 3:56
  SPEAKER_01: So it's it's still more art than science in many ways, not the least of which is, you know, we don't understand at a deep level how our own thought processes work, how our own consciousness works, what what it even means to know something at the philosophical level.
• 4:14
  SPEAKER_01: And we're seeing that, we're seeing that right now with, you know, these neural networks, especially these large, large, highly parameterized systems with billions or trillions of parameters.
• 4:24
  SPEAKER_01: So you can introspect them to an extent and you can like let's let's use a concrete example.
• 4:30
  SPEAKER_01: We built a system that can accurately predict breast cancer risk using just mammograms five years out.
• 4:40
  SPEAKER_01: It's it's at the FDA right now for approval.
• 4:43
  SPEAKER_01: What's amazing about this is it's not detecting cancer.
• 4:47
  SPEAKER_01: There is no cancer in these mammograms, but it is picking up on biological signals of cancer, sort of whether it's whether it's biological visual signals of cancer correlation or early stage signs of cancer, we don't know.
• 5:05
  SPEAKER_01: We flat out don't know.
• 5:06
  SPEAKER_01: And if you were to show radiologists these images, they would give these women clean bill of health.
• 5:12
  SPEAKER_01: And so even when we've done like ablation studies, even when we've done things like rad cam, where you can go in and you can look and say, show me exactly what the model's looking at and and where is it focusing and and look at look at that attention across thousands and thousands of different mammograms.
• 5:31
  SPEAKER_01: It's just unclear at a deep level what it's doing.
• 5:36
  SPEAKER_01: And this this flows all the way to language models where you can you can ask a language model a question and it's not clear exactly how or where that information is bedded, where that knowledge is embedded in it, because it's spread across you know so many of these sort of higher dimensions.
• 5:54
  SPEAKER_02: So with those with those those vision services and stuff and looking at the I don't even know if you call it precursors to cancer, right?
• 6:01
  SPEAKER_02: Because they're, I mean, you're really just the pattern recognition of these.
• 6:04
  SPEAKER_02: How do you get the clinicians on board with when you're showing them something, you're like, hey, listen, by the way, you look at this image and they're like, yeah, that looks completely clear, because I do radiology all day long.
• 6:17
  SPEAKER_02: This looks completely clear.
• 6:18
  SPEAKER_02: And you're like, no, if you actually put this in a million of these images, you know that this is uh five years out.
• 6:25
  SPEAKER_02: How do you get them to like agree?
• 6:27
  SPEAKER_01: Yeah, that that's a really that's a really good point because you're you're essentially asking these professionals who have years of experience to trust some sort of black box system, right?
• 6:37
  SPEAKER_01: What we what I always advise is as much as possible, try to avoid the scenario where you have some black box AI making a determination and then you blindly accept it, whether it's whether you're talking about loan approvals, cancer prediction, or even something as maybe as prosaic as like summarizing content now.
• 6:59
  SPEAKER_01: It's always good to have some types of checks and balances and human in the loop.
• 7:03
  SPEAKER_01: And we're at it comes with AI, we're really big fans of sort of AI as an augmentator to humans, not as a replacer of humans.
• 7:11
  SPEAKER_01: But within the specific breast cancer example we're talking about there, it's literally just statistical.
• 7:18
  SPEAKER_01: We, you know, to get an AI model like this to the FDA for approval, it is an astoundingly expensive, time-consuming process because you essentially have to convince the FDA that you have no bias within the model.
• 7:35
  SPEAKER_01: And then the amount of bias, it would just blow your mind.
• 7:38
  SPEAKER_01: For example, one of the things you'll run into is as patients become older and the chance of cancer increases, or maybe they even were screened and there were some warning signs, they will go have follow-on scans at better machinery.
• 7:53
  SPEAKER_01: Well, it's very easy for these AO models just to note that the type of mammogram correlates with cancer.
• 8:02
  SPEAKER_01: So we had to do all kinds of work to make sure there was none of that sort of bias within the model.
• 8:07
  SPEAKER_01: And then it's simply a matter of overwhelming statistical evidence.
• 8:11
  SPEAKER_01: And so we uh on this particular example, we had uh terabytes of mammograms longitudinally spread out across over a decade from three different continents across multiple manufacturers.
• 8:24
  SPEAKER_01: So the point is that at that point you have you have such broad statistical evidence of efficacy that even you know, even clinical professionals can't help but look at the evidence and be convinced.
• 8:39
  SPEAKER_02: I mean, I can see this across definitely across healthcare, because the outcomes is driving pay like life and death.
• 8:46
  SPEAKER_02: Financial services, same thing, right?
• 8:48
  SPEAKER_02: In terms of trust.
• 8:50
  SPEAKER_02: What other misconceptions do you run into just in terms of like areas?
• 8:55
  SPEAKER_02: Just besides trust, like and and accuracy, but what other misconceptions run into?
• 9:00
  SPEAKER_01: Probably the biggest conception, misconception that I see these days is since chat GPT came out almost two years ago.
• 9:08
  SPEAKER_01: That the average person, if you stopped on the street, they would uh equate that with AI full stop, right?
• 9:15
  SPEAKER_01: It's just you know, chat models, large language models, that is AI.
• 9:21
  SPEAKER_01: Um, and um that's that is amazing.
• 9:26
  SPEAKER_01: And in some ways it's understandable because Chat GPT, in my opinion, is is a really impressive accomplishment.
• 9:34
  SPEAKER_01: The fact that scaling up language models to the trillion parameter level is is uh um allowing us to build systems with these broad capabilities is amazing.
• 9:45
  SPEAKER_01: But most of the AI that is being used in the real world right now to drive business ROI is what I call domain specific.
• 9:54
  SPEAKER_01: So it's it's a it's a type of AI that was trained to solve one problem very specifically and to be very, very good at that, but that's all it can do.
• 10:04
  SPEAKER_01: Where you can talk to Chat GPT and you ask it to write a poem or summarize or classify.
• 10:10
  SPEAKER_01: Um, and it has sort of broad um capabilities.
• 10:14
  SPEAKER_01: Most of the ROI out there that that businesses can reach for now using AI is something like, uh like we talked about, um, automating certain processes, maybe loan decision or document processing or sales prediction or product recommendations, whatever it may be.
• 10:34
  SPEAKER_01: And the AIs are incredibly good and powerful at those specific narrow tasks.
• 10:40
  SPEAKER_01: Um, and so the idea that generative AI is synonymous with all AI is probably one of the biggest misconceptions I see out there.
• 10:49
  SPEAKER_02: Oh, yeah, there's there's so many times we have to help people unlearn and then relearn to open up, you know, the pathways of, oh, there's more enlightenment, right?
• 10:58
  SPEAKER_02: That comes from this.
• 10:59
  SPEAKER_02: I'd love to get your take.
• 11:00
  SPEAKER_02: I don't know if you saw in the uh interwebs this uh this week about they're trying trying to map it to IQ, right?
• 11:06
  SPEAKER_02: And they've been doing these IQ tests, right, with these large language models.
• 11:11
  SPEAKER_02: And to me, um it's you know, IQ is I think a little bit contextual, of course, as well as semantic.
• 11:17
  SPEAKER_02: Yep.
• 11:18
  SPEAKER_02: What are we talking about IQ level, just in terms of the vernacular of reading and comprehension or recall or what?
• 11:25
  SPEAKER_02: That's right.
• 11:26
  SPEAKER_01: Yeah, that's exactly right.
• 11:28
  SPEAKER_01: We're actually working right now um uh on a problem that's kind of related to that, which is um trying to um ascertain and uh basically provide a metric for how much incremental knowledge any one single piece of training data may provide to a language model.
• 11:50
  SPEAKER_01: Um and it's really, really interesting because it goes back to what we were talking about at the beginning of the conversation, which is, you know, what does it even mean to know something?
• 11:59
  SPEAKER_01: And what does it even mean to trust these black box models where the information and the knowledge is so spread out across all these sort of like high-dimensional embedding spaces?
• 12:11
  SPEAKER_01: And so I think it's very closely tied to some of our own lack of understanding about what it even means for us to know or or understand anything, because we we still don't have a strong understanding or it really any capability to introspect our own you know consciousness yet.
• 12:29
  SPEAKER_02: Yeah, that's crazy.
• 12:30
  SPEAKER_02: Um, I know in the pre-call, you guys um there's some discussion around just the rapid advancement and merging capabilities, especially in text-to-video systems.
• 12:40
  SPEAKER_02: Yes, which is borderline, you know, creepy in a lot of ways, right?
• 12:46
  SPEAKER_02: Just in terms of what we see out there in terms of, hey, listen, I'm gonna create this, and now all of a sudden I've got this this um rendition of just some text in a really, really succinct, crisp way, right?
• 12:58
  SPEAKER_02: But I'd love to get you know get your take on you know on what you think, you know, and their ability to you know simulate maybe complex phenomena without a lot of let's say instruction, right?
• 13:11
  SPEAKER_02: Teaching.
• 13:12
  SPEAKER_02: Yeah, what uh give me your take on that.
• 13:15
  SPEAKER_01: I so this is an area that I am deeply fascinated with because if you uh uh like to maybe kind of set the table for this.
• 13:25
  SPEAKER_01: Um when text of video first started being pioneered, which was just a few years ago, wasn't that long, the videos were really blurry and um uh they would really quickly sort of lose context.
• 13:42
  SPEAKER_01: And so you if you're familiar with this, I'm sure everybody's listening to this is hands would come and go, and new arms would come and go, and um you know, maybe a person would be one moment they're on the beach and the next moment they're in the office.
• 13:55
  SPEAKER_01: So continuity was just really a problem at that point.
• 13:59
  SPEAKER_01: And now the new, if you look at like runway ML and some of these other um text-to-video um creators out there, it is astounding how consistent the context is and how realistic.
• 14:14
  SPEAKER_01: But even somebody like me who's living and breathing in this every day kind of missed something that I I think in hindsight was staggering.
• 14:22
  SPEAKER_01: If you look at those videos, they're like um one of my favorite examples is the open AI Sora example of two miniature pirate ships having a battle inside a coffee cup.
• 14:35
  SPEAKER_01: And it's photorealistic.
• 14:37
  SPEAKER_01: And if you look at it, it you you just immediately you're impressed, but it takes a moment to realize it is simulating fluid dynamics, it is simulating ray tracing as they move around and the light is bouncing off.
• 14:51
  SPEAKER_01: Um, it is uh simulating buoyancy and smooth movements of these ships, and none of that was programmed.
• 15:00
  SPEAKER_01: None of that was programmed.
• 15:01
  SPEAKER_01: That all of that was sort of emergent capabilities from just being trained on you know petabytes of video data.
• 15:08
  SPEAKER_01: And so that means that not only are these sort of text to video systems capable of generating accurate lifelike renderings of whatever you type in, but they have embedded in them entire physics engines that are compressed in some really, really deep latent space.
• 15:28
  SPEAKER_01: And that just kind of fell out of the training emergently.
• 15:31
  SPEAKER_01: And to me, that is fascinating.
• 15:33
  SPEAKER_01: And nobody understands what's going on in a deep way there yet.
• 15:37
  SPEAKER_02: See, that's that's the scary part, right?
• 15:40
  SPEAKER_02: I mean, I love the fact it's it's so refreshing to see a uh a you know a tried and truth practitioner like yourself get excited about what's next and all these different aspects and it kind of blows your mind, right?
• 15:54
  SPEAKER_02: With the foundation and stuff.
• 15:55
  SPEAKER_02: And at the same time, there's some scary notions out there, right?
• 15:58
  SPEAKER_02: And you talked a little bit about bias, right?
• 16:00
  SPEAKER_02: With how and part of that bias is as you can imagine, walking into the um the different governing bodies and stuff that are like, did you take all these edge cases into consideration too, right?
• 16:11
  SPEAKER_02: Like right.
• 16:12
  SPEAKER_02: So listen, you're talking about mammograms, but you're male, right?
• 16:15
  SPEAKER_02: Do you have you know even those types of things?
• 16:18
  SPEAKER_02: So how many you know of these types of characters?
• 16:20
  SPEAKER_02: What about ethical considerations, right?
• 16:22
  SPEAKER_02: Because part of this is, you know, in the beginning you were talking about there was no explainability, and yet we explainability is a big, you know, audacious thing now that everybody pivots on, and yet you still can't explain a lot of these kind of things, like you're saying, without divulging maybe IP, without divulging across all that kind of stuff.
• 16:44
  SPEAKER_02: So let's go down the ethical route rabbit hole a little bit, and yeah, you know, and I mean explainability is different from ethics, but but there's still a lot of it because one begets the other, that kind of thing.
• 16:54
  SPEAKER_02: But that's totally right.
• 16:56
  SPEAKER_02: Let's let's what do you what do you think about that?
• 16:58
  SPEAKER_02: What do you think about the risks are with some of this?
• 17:01
  SPEAKER_01: I um I it's it's a really, really important area, and uh I'm always candid about this.
• 17:06
  SPEAKER_01: You know, I've been doing this so long that I remember when ethics would come up, let's say in the mid-90s, and I, you know, my reaction was ethics, this stuff doesn't even work.
• 17:19
  SPEAKER_01: Like who you know, who cares about that, right?
• 17:21
  SPEAKER_01: Yeah.
• 17:21
  SPEAKER_01: And it's and I use this analogy.
• 17:23
  SPEAKER_01: It's it's really analogous to like the internet.
• 17:25
  SPEAKER_01: When the internet was first built, coming out of the you know, the late 60s, early 70s, they weren't focused on security.
• 17:32
  SPEAKER_01: They were just focusing on reliable network connections.
• 17:36
  SPEAKER_01: And then once the internet took off and they they'd really sort of ironed out most of the that networking stack, it was time to take security seriously.
• 17:47
  SPEAKER_01: And it kind of had to be layered on top.
• 17:49
  SPEAKER_01: It kind of had to be like wedged into the networking stack because it really wasn't designed for that from the beginning.
• 17:58
  SPEAKER_01: AI is really seeing a lot of that as well.
• 18:00
  SPEAKER_01: We went from systems that were a toy to systems that could kind of work, but were absolutely not production ready, to just almost overnight, it feels like these systems that are superhuman capable can even be autonomous.
• 18:16
  SPEAKER_01: And now we are having to rush to kind of get our act together on sort of the bias ethical front.
• 18:23
  SPEAKER_01: And a really great example of that is if you think about like loan decisioning.
• 18:28
  SPEAKER_01: We have real firsthand experience with this.
• 18:31
  SPEAKER_01: We were building a system to do loan decisioning.
• 18:34
  SPEAKER_01: And one of the beautiful things about something like that is you could say that this system was um unbiased, it doesn't see race, it doesn't care about any of those classic ethical bias issues that plague humanity.
• 18:52
  SPEAKER_01: But the data set it was trained on actually had all of that historically embedded in it.
• 18:58
  SPEAKER_02: Yeah.
• 18:58
  SPEAKER_01: Okay.
• 18:59
  SPEAKER_01: And so when we trained that system, and we anticipated this, when we trained that system, we then did a bunch of studies to figure out what were the racial or ethnic biases based into that loan decisioning model.
• 19:13
  SPEAKER_01: And then we were able to remove it.
• 19:15
  SPEAKER_01: We were basically able to suppress those.
• 19:18
  SPEAKER_01: But too many, too many of the um of the customers that we work with day to day aren't really aware of that issue.
• 19:27
  SPEAKER_01: And if you train AI systems on historical data and you are just blind to the bias baked within it, that AI model will replicate all of that bias, and you you will not be in a position to say that it's making independent objective decisions because you've basically taught it to be bias.
• 19:47
  SPEAKER_02: So then where do you because it seems to me like the inverse of that is profiling.
• 19:53
  SPEAKER_02: Profiling is what we look at for classification and for recommendation engines and stuff, and which alone type of approval decision making process is based off of a profile and a recommendation of variance, right?
• 20:08
  SPEAKER_02: Because I look at my what are all the factors that go into shoot?
• 20:13
  SPEAKER_02: I don't even know, because like credit, for instance, credit scores, like it seems like it's talking about a black box.
• 20:18
  SPEAKER_02: The same like the same algorithm that's deciding that is the one that's deciding airplane ticket prices and Vegas hotel prices.
• 20:25
  SPEAKER_02: Like who the hell is running that?
• 20:27
  SPEAKER_02: You know what I mean?
• 20:28
  SPEAKER_02: Right.
• 20:28
  SPEAKER_02: So so where does the profiling then come in?
• 20:31
  SPEAKER_02: Because you just, you know, we talk about you remove all the bias, but then you remove all of the ability to make a sound decision, too.
• 20:39
  SPEAKER_01: It's a really good point.
• 20:40
  SPEAKER_01: The way I think about it is every model is going to be biased in the same way that in the same way that you there's no way you could grab all of humanity and get them a degree 100% on anything.
• 20:56
  SPEAKER_01: Everybody, every human being has some type of bias.
• 21:01
  SPEAKER_01: And and much of that bias um that we all have, and I'm I guarantee you I'm in this category, I'm not even aware of, right?
• 21:09
  SPEAKER_01: I have some just built-in bias that that is you know sort of below my consciousness.
• 21:16
  SPEAKER_01: So it's not really a question ever saying, hey, we're gonna build models that have no bias.
• 21:20
  SPEAKER_01: It's just getting comfortable with what type of bias it's gonna be.
• 21:24
  SPEAKER_01: And and from my mind, what you want to do is you want to focus on the bias that is unfair.
• 21:31
  SPEAKER_01: So if you're making, let's say, loan decisioning, you probably really do have to look at somebody's income as a part of that decision process.
• 21:40
  SPEAKER_01: And you can make an argument that that's a bias, but you could also make the argument it's an appropriate bias for this particular scenario.
• 21:48
  SPEAKER_01: But looking at their age, maybe less justification.
• 21:53
  SPEAKER_01: Looking at their race, absolutely no justification.
• 21:56
  SPEAKER_01: And so it's really kind of a spectrum, and there's no one size fits all.
• 22:00
  SPEAKER_01: You just the the most important thing about bias is to make sure that the people building the AI systems and the the whoever is collab, whatever the domain is, that they collectively are working in surfacing these issues early into the process and addressing them.
• 22:18
  SPEAKER_01: Because if you wait till the end, it is very, very difficult to kind of retroactively remove the bias in a systematic way.
• 22:25
  SPEAKER_02: I think to me that is one of the better definitions of explainability, knowing you know, actively what you're doing, as opposed to, well, here's where we're getting the data from and here's the path to the algorithm and stuff.
• 22:37
  SPEAKER_02: No, no, no, no.
• 22:38
  SPEAKER_02: Here's the explicit things that we are looking for in a pattern, the explicit things we're excluding.
• 22:46
  SPEAKER_02: And then you get into how to explain it.
• 22:49
  SPEAKER_01: And another thing, too, you know, going back a little bit, I guess, to like common misconceptions is we run into this all the time where we'll work with companies and they think that if there is some type of AI model um that will be deployed, that it's kind of all or nothing.
• 23:05
  SPEAKER_01: It's like all black box, you can't control it.
• 23:08
  SPEAKER_01: You are essentially letting go of the decision making around whatever that model is touching.
• 23:16
  SPEAKER_01: And we actually um advocate really strongly against that, which you can put in all kinds of oversight, like you mentioned, around explainability.
• 23:26
  SPEAKER_01: You can have thresholding, you can have policies that override.
• 23:31
  SPEAKER_01: So, for example, with loan decisioning, you might have models that are making recommendations, and they be they could be scoring along many dimensions, you know, anticipating fraud, anticipating um, you know, likelihood of repayment, all the normal dimensions that you might think about around loan decisioning.
• 23:51
  SPEAKER_01: But you can have separate thresholds that you tune and that you change over time around what confidence level on those different dimensions you live with.
• 23:59
  SPEAKER_01: And completely independent of that, you can put in policies.
• 24:02
  SPEAKER_01: They can be just old school traditional software policies that say, hey, if these things, if these conditions are not met, I don't care what the AI model says.
• 24:13
  SPEAKER_01: We are going to maybe reject the decision or we're going to put this through some type of human review.
• 24:18
  SPEAKER_01: So that's another really common misconception is the lack of explainability, therefore forces these models to be entirely autonomous.
• 24:27
  SPEAKER_02: Do you find, too, that there's also a misconception on what data is needed?
• 24:31
  SPEAKER_02: I mean, I mean, we work with data scientists all the time, and yet, and I feel like, by the way, just my opinion, is that we put either a lot of pressure and or a lot of gratis on them that they know all and all data sources and everything else, which to me there's they're like another model.
• 24:50
  SPEAKER_02: They're only as good as the data they you know are provided, right?
• 24:53
  SPEAKER_02: To look at.
• 24:54
  SPEAKER_02: How do you, you know, when you talk to your customers and stuff, what does the data picture and and perspective look like?
• 25:01
  SPEAKER_01: That is a really important area.
• 25:03
  SPEAKER_01: I'm I'm actually I'm actually really glad you brought that up.
• 25:05
  SPEAKER_01: And there's like sort of three things that come to my mind is one, we we we've been doing this now, we've been building AI production systems for almost seven years.
• 25:16
  SPEAKER_01: This has only happened a handful of times, but we've had some clients where we are, you know, early stages of the product development, and it becomes clear that the data they thought they had was actually not there.
• 25:28
  SPEAKER_01: And it was because either IT was deleting it or they were compressing it in some way to save money and basically destroyed all the signal.
• 25:36
  SPEAKER_01: So I always tell clients, look, the, you know, it's like the old proverb the best time to plant a tree was 20 years ago, the second best time is today.
• 25:44
  SPEAKER_01: Same thing with your data.
• 25:45
  SPEAKER_01: It's it's a it's a it's a very, very valuable you know, source of intellectual property.
• 25:51
  SPEAKER_01: So make sure you're not throwing it away or uh compressing it too much.
• 25:55
  SPEAKER_01: The other thing we'll run into is is is uh um you know, companies underestimating how much data they need.
• 26:04
  SPEAKER_01: Depending upon what type of model you're building, it can vary.
• 26:07
  SPEAKER_01: So if you're building, let's say a language model and you're able to work in a space that is uh, let's say somewhat generic, like you're just trying to let's say you're a language model that you want to fine-tune on your own proprietary documents, but you're within some vertical and there's a little customization.
• 26:27
  SPEAKER_01: But for the most part, the for the most part, the language and the semantics are you know common English or whatever your language may be.
• 26:34
  SPEAKER_01: That's one thing.
• 26:35
  SPEAKER_01: You can use off-the-shelf open source models and and really move pretty quickly.
• 26:39
  SPEAKER_01: If you're working in sort of specialized domains, legal, medical, the bar is higher on the data acquisition.
• 26:48
  SPEAKER_01: And then the last one, and we do see this every now and then, is some people just say, hey, I have this data lake, and there are terabytes of data there.
• 26:56
  SPEAKER_01: Can we just go release an AI model?
• 26:58
  SPEAKER_01: Just like let it go, look at it all, and come back and be smart.
• 27:02
  SPEAKER_01: And it doesn't work like that at all yet, um, obviously, because most AI systems today are supervised learning based.
• 27:11
  SPEAKER_01: You have to have labels, meaning you have to have what the input to the model would be and what the correct output of the model should be.
• 27:19
  SPEAKER_01: And very often, the sort of the worst case scenario is that labeling is an incredibly you know intensive human process.
• 27:30
  SPEAKER_01: Maybe it's 30 minutes, maybe sometimes we've literally had instances where they say it could be a week of discovery that has to go into something to just come up with one label for one example of uh training data.
• 27:45
  SPEAKER_01: And those instances really um you're you're really um can you know almost conflating sort of like data mining with AI.
• 27:54
  SPEAKER_01: And so those are again are some of the issues we run into to data on the data side.
• 27:58
  SPEAKER_01: I'll finish by saying the good news is most companies fall into that middle category where you don't have to start from scratch.
• 28:07
  SPEAKER_01: You can leverage open source, and the lift on getting additional labeled data is not that high.
• 28:13
  SPEAKER_02: So whether it was implicit or or something, I love what you said around that the data that you're you're deleting your signals, right?
• 28:22
  SPEAKER_02: And that data are that's exactly what they are, is your signals coming in to know exactly what to do.
• 28:28
  SPEAKER_02: What are your thoughts too then?
• 28:30
  SPEAKER_02: Because you know, we also I'll run into quite a few customers that we'll we'll talk about them believing that generative is everything and they forget that there's classic recognition models that they need or pattern recognition models that they absolutely have to have, find it and then explain it, kind of a thing, as opposed to no, the generative model is going to do it all, right?
• 28:50
  SPEAKER_02: Where do you see like a good graph environments coming into play for these?
• 28:55
  SPEAKER_02: Because I feel like there's a lot of customers who miss the whole boat on the ability to use a graph model, especially with beta lake and things.
• 29:02
  SPEAKER_02: Where what are your thoughts there?
• 29:04
  SPEAKER_01: Yeah, we're we're seeing more and more utilization of graph, especially things like sort of graph-based rag, meaning, you know, graph-based retrieval augmentation generation.
• 29:16
  SPEAKER_01: It it's really, really especially handy in situations where you have relationships that need to be a part of the modeling process.
• 29:26
  SPEAKER_01: And so the way I always think about it like this um, you know, there's the the famous no-free lunch there, and that there is there is no one algorithm that will always be optimal for solving any problem.
• 29:38
  SPEAKER_01: And if you think about computer vision and the use of like uh convolutional neural networks, and just you know, uh from sort of a non-technical perspective, the way to describe that is convolutional neural networks allow the model to kind of not really care about uh where something is in an image.
• 29:59
  SPEAKER_01: It's sort of translation invariant.
• 30:01
  SPEAKER_01: That's it's it's it's more than that, but that's one of the main ideas.
• 30:05
  SPEAKER_01: Well, convolutions sort of bring that inductive bias to the model.
• 30:11
  SPEAKER_01: It it forces it right out of the gate to not care if the cat is in the top left or the bottom left.
• 30:16
  SPEAKER_01: It's still a cat.
• 30:18
  SPEAKER_01: With graphs, graph neural networks, the beautiful part about that is you're bringing this inductive bias to the model where you're saying relationships really, really matter.
• 30:28
  SPEAKER_01: And that you you're forcing the model to learn through sort of that lens.
• 30:34
  SPEAKER_01: And I think that's incredibly important.
• 30:35
  SPEAKER_01: The interesting thing, well, maybe one, you know, kind of one more note on that, is that's also the main reason we're seeing such amazing results right now with the other um AIs we've been talking around, language larger large language models and text to video and things like that, is because they're based on the transform architecture, which also allows that sort of contextual weighted attention to be applied in the model.
• 31:02
  SPEAKER_01: And so one of the the big takeaway, the the I guess the point I'm trying to make is anytime you can align inductive bias, meaning forcing your model to kind of think or operate in some certain way that aligns with the problem, you're gonna get you're gonna get better results and you're actually gonna streamline the path to explainability.
• 31:22
  SPEAKER_02: Yeah, I think of this as um it's in any AI models, like even when I'm talking to my family or folks that aren't in tech, right?
• 31:31
  SPEAKER_02: The goal is always accuracy, right?
• 31:33
  SPEAKER_02: Accuracy builds trust.
• 31:34
  SPEAKER_02: And then you can actually understand how to use it in in various ways.
• 31:38
  SPEAKER_02: Part of that for me, especially graph was always, like you said, super interesting around the metadata about all these different things.
• 31:45
  SPEAKER_02: I worked early on Sparkle Base with one of my good friends, Barry Zane.
• 31:50
  SPEAKER_02: And it was, it was, it seems like to me, in fact, I I saw something on on uh the uh the web that was basically is this is 2024, the year of the graph, right?
• 32:00
  SPEAKER_02: And I think part of it is what you're teasing out is the fact that when we talk about generative and or pattern recognition models, the goal is always accuracy, of course.
• 32:11
  SPEAKER_02: Part of that accuracy comes from hey, if I want to take a just a base foundation model or model off the off the cart, I've got to do some really neat and interesting things around the guardrails of what I can and can't do with it, meaning ask it questions or what I can't ask it and things like that.
• 32:26
  SPEAKER_02: And that comes in the form of that accuracy.
• 32:28
  SPEAKER_02: The second part of that is I don't know all the edge questions that I could or couldn't ask it to come back with maybe some false positives or false negatives.
• 32:38
  SPEAKER_02: So let me put a graph environment in there, create all those associations, and at least what I'm doing is I'm increasing exponentially the ability to gather way more accurate result sets because I'm gonna take those, put them into an embeddings environment, vectorize it.
• 32:54
  SPEAKER_02: Now I've got a really great representation as you know, in addition to all this not structured data to actually deliver my results.
• 33:02
  SPEAKER_02: I feel like it's like almost essential, especially when talking about customer experience related items and fraud-based items, and you know, anyway, all those kind of things.
• 33:13
  SPEAKER_01: Yeah, I could I couldn't agree more, 100%.
• 33:15
  SPEAKER_02: So it's yeah, it's I'm always especially with somebody gets so excited about some of these types of topics, which I appreciate all the energy you bring too, because it's it's palpable, just in terms of like, hey, there's something new out there.
• 33:28
  SPEAKER_02: Well, we're gonna embrace it, figure it out, and then this is how we do things.
• 33:32
  SPEAKER_02: So let me let me switch gears a little bit because it's always interesting to to know the the person behind all of the the great work that you do.
• 33:41
  SPEAKER_02: So tell me a little bit about yourself personally.
• 33:42
  SPEAKER_02: Like, what do you hobbies, interests, all those kinds, and please don't say that I spent all my extra time you know training AI and ML models.
• 33:51
  SPEAKER_01: Yeah, yeah.
• 33:52
  SPEAKER_01: Absolutely.
• 33:53
  SPEAKER_01: Yeah, I I do, you know, it is funny to stay up to date.
• 33:56
  SPEAKER_01: It is kind of shocking how much time you do actually have to spend uh reading these days just to stay uh stay on the uh the cutting edge.
• 34:04
  SPEAKER_01: But um uh I'm you know, probably you know, some of my favorite pastimes are uh playing the piano.
• 34:11
  SPEAKER_01: I'm uh I'm pretty passionate about that.
• 34:12
  SPEAKER_01: I've been playing for for decades.
• 34:14
  SPEAKER_01: I would say I spend a good chunk of my Sundays just um dedicated to that.
• 34:21
  SPEAKER_01: Uh big reader, loved to read, loved to travel.
• 34:24
  SPEAKER_01: And um, you know, when it comes to AI, I just kind of feel like, you know, for for somebody who who who lived through the AI winter, that that AI winter from the 90s, early 2000s, and to be um have the entire field not only resuscitate and now going through this sort of absolute renaissance, this revolution, um, it is it's actually a joy when I spend my weekends reading white papers or prototyping with new models or trying out new packages.
• 34:57
  SPEAKER_01: I mean, I just I spend most of my uh most of my week with a smile on my face because I just am so excited about where we're at right now and where we're going in AI.
• 35:08
  SPEAKER_01: I think, I think, I think the vast majority of people out there fail to recognize just how dramatic the changes are gonna be in the near future in the most positive possible ways.
• 35:20
  SPEAKER_01: Because we're gonna we're gonna start moving up sort of that exponential curve.
• 35:24
  SPEAKER_01: And the best example of that is just the fact that there's more and more evidence this year.
• 35:29
  SPEAKER_01: I feel like every major model I see, um, if you look, if you read the white paper, they say it was trained on a considerable amount of data that was output by the previous model, right?
• 35:44
  SPEAKER_01: So we're we're you know the models are starting to stand on the shoulders of each other.
• 35:48
  SPEAKER_01: And it's just astounding.
• 35:50
  SPEAKER_01: And so I'm, you know, that is actually, you know, uh one of my favorite hobbies is to just stay up to date and catch up on AI on the weekend.
• 35:59
  SPEAKER_02: Do you think do you think that you ever seen the movie Multiplicity, right?
• 36:02
  SPEAKER_02: If you multiply something of the multiply, you know, multiplies and starts to diminish.
• 36:07
  SPEAKER_02: Do you think we're going to reach those types of scenarios as well where the output of a model, as fictitious as it may be, may actually really you know poison the well of the next model.
• 36:18
  SPEAKER_02: Right, right, right.
• 36:19
  SPEAKER_01: Yeah, that's a that's a really good question.
• 36:22
  SPEAKER_01: I I have a couple of thoughts on that.
• 36:24
  SPEAKER_01: One is I think that if you I think that if you naively take the approach that the let me back up and say this.
• 36:33
  SPEAKER_01: The the classic example would be, hey, we've chained, we've trained this particular large language model, let's say it's Chat GPT-4, on all of the publicly accessible information on the internet.
• 36:45
  SPEAKER_01: And so there's no more training data.
• 36:47
  SPEAKER_01: Well, what do we do?
• 36:48
  SPEAKER_01: Oh, well, we'll have it generate data.
• 36:50
  SPEAKER_01: And then now you've got this sort of feedback loop where it's training on its own output, and you just kind of you end up with this sort of like uh uh, you know, averaging out effect, right?
• 37:02
  SPEAKER_01: And you end up with actually a loss in abilities.
• 37:06
  SPEAKER_01: I think that sort of naive approach is definitely not the way to go.
• 37:10
  SPEAKER_01: But what we're seeing are more sort of teacher-student approaches where you can have multiple separate models making determinations and then grading the output of models.
• 37:22
  SPEAKER_01: And so there's this, there's this recent move towards, I think it was Microsoft's paper FI3 on FI3, where they where they were sort of um, Sebastian was joking about like textbooks are all you need.
• 37:35
  SPEAKER_01: They found that if you that if you concentrate on really, really high quality training data, you can build small models that outperform larger models because you're just basically concentrating that signal.
• 37:47
  SPEAKER_01: And I think that that's we're seeing that all over the place.
• 37:50
  SPEAKER_01: In fact, last week, Alpha Proteo, which is a sort of a protein uh fold prediction model from DeepMind, they said it was trained on 100 million uh predicted proteins from alpha fold.
• 38:05
  SPEAKER_01: So that means that this problem that was unsolved as of like five years ago, and then which is that is how would an amino acid sequence fold into a protein?
• 38:15
  SPEAKER_01: We now have models that are so good at solving that problem that you can generate 100 million training examples and then use that to train a downstream model that has a new capability beyond just protein fold prediction, but it can actually have the ability to predict how proteins will bind with other proteins.
• 38:34
  SPEAKER_01: And so I'm I I that's a long way of saying I am deeply not worried about that problem right now.
• 38:40
  SPEAKER_01: I think there's a lot of room at the top to grow still.
• 38:44
  SPEAKER_02: That's awesome.
• 38:45
  SPEAKER_02: So do you have a favorite uh proprietary LLM?
• 38:50
  SPEAKER_01: Oh, good question.
• 38:51
  SPEAKER_01: I'm I'm pretty, I'm pretty um, I'm I'm pretty excited about Lama 3.170B.
• 38:59
  SPEAKER_01: I can literally run it on my laptop on like literally an Apple laptop.
• 39:05
  SPEAKER_01: Um, and then Fi3, which I actually mentioned a second ago, because it's such a small model.
• 39:10
  SPEAKER_01: Anything I can run locally on my laptop, I'm I'm a big fan of.
• 39:15
  SPEAKER_02: Yeah, I'm I'm impressed with the speed and and actually the llama models too.
• 39:18
  SPEAKER_02: You know, I we've I work with a number of a lot smarter folks than than I am, and and we're we're we're doing use cases where we need instant response on simple stream it, of course, and things like that.
• 39:31
  SPEAKER_02: And it's amazing.
• 39:32
  SPEAKER_02: It's coming back.
• 39:34
  SPEAKER_02: But it really is.
• 39:35
  SPEAKER_02: Well, listen, Ron, thanks so much for uh for the time.
• 39:38
  SPEAKER_02: Is there anything that um that I should have asked you that I didn't?
• 39:42
  SPEAKER_01: No, no, this was a a really fun, wide-ranging conversation.
• 39:45
  SPEAKER_01: I would say feel free to reach out to me, anybody listening.
• 39:48
  SPEAKER_01: If you have any questions, Ron at Kung Fu.ai.
• 39:52
  SPEAKER_01: I'm happy to connect and uh and uh if I have a podcast called The Hidden Layers, if you want to dig in on uh different AI technical topics, check it out.
• 40:02
  SPEAKER_02: Awesome.
• 40:02
  SPEAKER_02: Yeah, we'll we'll make sure that you know way to get a hold of you and and to you know see where you're at on the uh on the web is in the show notes and stuff.
• 40:10
  SPEAKER_02: But listen, you know, thank you so much for spending the time and uh taking time out of your busy schedule to to do this and wealth of information and and really, really appreciate it.
• 40:20
  SPEAKER_01: I really appreciate it too, Chris.
• 40:22
  SPEAKER_01: Thanks for having me.
• 40:23
  SPEAKER_01: Thanks, brother.
• 40:24
  SPEAKER_00: Take care.
• 40:25
  SPEAKER_00: Take care.
• 40:26
  SPEAKER_00: Visit early.com to find links to the full podcast on all audio platforms and to listen on the go.
• 40:32
  SPEAKER_00: Thank you for watching.
• 40:34
  SPEAKER_00: Contact us today at info at early.com.