Latest / Machine Dreams / Cameras Used to Just Record Us. AI Is Changing That
Transcript
- speaker-0: For most of their history, cameras have had a pretty simple job. And that's to record what happened. But it seems now that AI is beginning to change that as there are now more and more cameras that can make sense of what they're saying while it's happening and and perhaps eventually understand what might happen next. And so the big question today on the show is what happens when cameras can actually think about what They see. Welcome again to Machine Dreams, where we talk honestly to the people building AI about what works, what doesn't, and what's next. My name is Kalawale Samuel Adibaya. I'm a technology analyst working at the intersection of AI, cybersecurity, business, and society. And usually I would have my co-host, ⁓ the remarkable, adorable Lea Stan, ⁓ on the show with me, but unfortunately it's gonna be a Solo ride ⁓ today. But it's it's gonna be fun. So sit back and relax and enjoy. And thankfully, she's also gonna be back with us soon. So without further ado, let's get into today's conversation. Today we're joined by Professor Ron Kimmel, who is the chief scientific officer at Lumana and a professor of computer science at the Technion. Ron has spent decades working across computer vision, image processing, and 3D analysis. And more recently, medical imaging is also the founder of the Technion's Geometric Image Processing Lab. So there are actually probably few people better placed to help us understand how far machines have come in learning to see, and more importantly, understanding what they see. Ron, welcome to Machine Dreams. speaker-2: Thank you for hosting me. speaker-0: It's great to have you. ⁓ you know, as you know, we've actually had Ofre Mulat, who is the CDO CTO ⁓ Lumano before on the show. We spoke to him about ⁓ physical AI and what it takes for machines to make sense of the physical world. But today I wanna take that conversation somewhere different. Recognizing a person or an object is one thing. But you know, understanding what's actually happening seems much harder. You know, if a camera, for instance, seems to be running, for example, how does he know whether I'm exercising, you know, or how does he know whether I'm late for a flight? Or how does he know whether I'm running away from danger? Right? So the big question, and this is really where I want us to begin, is how do you make sense of this? What does it mean for a camera to actually move beyond recording or seeing what's happening? to understanding and I would say even predicting what might happen next. speaker-2: Okay, so before ⁓ this lec this talk, ⁓ this podcast, I consulted with Jiminin, ⁓ which suggested that humans still hold the edge in distinguishing between ⁓ apparent fall and a genuine one. I'm no longer certain that ⁓ this is the truth. Just as in traditional language, ⁓ the Turing test became obsolete by raw compute and massive ⁓ data sets about a decade ago, ⁓ the visual Turing test is also ⁓ falling apart next. ⁓ large AI models can now or will very shortly interpret complex human behavior in video and sequences ⁓ with almost ⁓ human level ⁓ accuracy. Now we lived in ⁓ this exact paragraph paradigm shift about 15 years ago, ⁓ when computer vision was fundamentally made ⁓ went through a tr a fundamental revolution. we transitioned from what we called axiomatic discipline. which is seeking for elegant and compact ⁓ equations that to describe how light and objects interact, to engineer solution into what is known as data driven film, where the primary challenge is simply becoming ⁓ sourcing enough data to train massive massimodels. ⁓ I can talk a little bit more about ⁓ what happens today in pure mathematics. ⁓ pure mathematics, for example, is facing a very similar ⁓ disruption. AI models have recently tackled fundamental challenges like the aspects of the Navier Stokes equations, almost ⁓ a casual exercise ⁓ to boost corporate ⁓ evaluations, specifically open AI. They are also solving district geometry puzzles like the earth unit ⁓ distance problem. while later might seems that the last ⁓ problem ⁓ seems may seem like an esoteric one, the navio stocks is a really fundamental one. It's filler in physics. so the prominent mathematicians recently, if if you look at ⁓ the expert in in number theory, turn stuff recently said that ⁓ AI problem solving is like a helicopter dropping you directly into the mountain summit, entirely ⁓ bypassing the the climb itself. The danger, of course, ⁓ is that walking in the past is how you naturally spot the next distance summit. So Or which summits to con to conquer. In fields like number theory, the question is who will define those important summits is probably the most important one ⁓ when the journey is skipped. In computer vision, we are somewhat ⁓ isolated from this philosophical crisis because our customers define our summits. Our industry primary ⁓ debate is simply ⁓ the the goals that they put into ⁓ to our path. Is they define the summits that we need to conquer. And luckily the summits are a little bit lower than in ⁓ in pure mathematics. So maximizing the accuracy while driving down ⁓ compute costs is what leads us to to what we are doing today. The bottom line, the bottom line is slightly ⁓ bittersweet, if you like. AI has largely overcomes the the classical open problems in computer vision. So ⁓ there are still physical glitches in how we deploy these systems, ⁓ limitations that commercial ⁓ evolution will inevitably have to correct, ⁓ like ⁓ stereoscopic vision, okay? ⁓ which biological ⁓ evolution perfected ⁓ to assess world geometry. ⁓ if you try pouring water into a glass with only one eye open, the loss of depth perception is immediate and and ⁓ disorienting. despise this. ⁓ for example, the automotive industry probably decided, broadly decided that cars only need a single forward looking camera. There are very few exceptions like the Subaru dual camera eyesight system. Most modern vehicles, including my Tesla, really use on molecular setup. We have actively chosen to build mechanical cyclops, if you will, ignoring hundreds of millions of years of revolutionary design. So ⁓ in in in many ways it is our own h ⁓ human hardware ⁓ choices that currently bottleneck what AI could truly achieve for us. However, this is just ⁓ as a biological evolution optimizes ⁓ for survival, commercial evolution will probably ultimately force correcting these glitches. speaker-0: ⁓ thank you for such a very robust response, Ron. ⁓ and I think it's actually fantastic that you're open about talking ⁓ about some of the limitations of this computer vision systems, ⁓ great and you know, amazing as they may be. ⁓ and I think my next question really butthers around I don't know if to call it limitation now, but the question is, ⁓ it I think that it might be scary. ⁓ because you know, there are concerns around bias and hallucinations for AI system. It's still a major problem today, even though there've been a lot of advancements with AI systems and AI models, right? And so I'm I'm I'm thinking about a potential situation where maybe a camera misreads what I'm doing, you know, maybe I'm just do going about my my normal day, I'm going about my day-to-day activity, and I'm just doing my stuff. And then the camera misreads what I'm doing as being suspicious or dangerous when in fact it isn't. ⁓ so I'm curious about your take on hallucinations and more importantly, how you've built a system that is tackling or addressing that challenge. speaker-2: Okay, so my next answers would be a little bit shorter. So if you think about ⁓ I think who was it? Was it Niels Bohr who said that ⁓ prediction is very difficult, especially about ⁓ especially about the future. If someone believes that AI will solve the deep causality problem and literally predict the future as it is, they're probably wrong. ⁓ any machine makes mistakes and AI is not exception. On the other hand, identifying the signs of something that is going wrong in in is i is highly ⁓ practical today. If there is ⁓ a correlation between an unplaced wet floor sign and subsequent fall or or a worker enters a construction site without mandatory helmet, the current systems can our current systems can actually reliably identify and report these ⁓ these events. If I go back to your first question, ⁓ I worked with Domana CEO, Sagib and Moshev for almost two decades now. Our first company ⁓ was called Invision, and it was acquired by Intel and became ⁓ the foundation of something which is known as RealSense, the RealSense Linework product. Now it's a separate company. And this technology is now enabling most of the commercial robot ⁓ robots today. They enable them to assess with the geometry ⁓ of the environment. Interestingly, hardware evolution has been nonlinear. Apple, for example, made amazing technological leap. By introducing 3D sensors into iPhones. But some recent phone factor evaluations are going backwards. For example, now with their what I call iFold or the iPhone Duo, they are dropping the 3D sensor in favor of just fingerprint videos. Meanwhile, if we go into hallucination, then Professor Milanfau, who runs the Google Pixel camera RD team, demonstrated how diffusion models, which are, if you like, realistic hallucination engines act as at as the core of their ⁓ behind their ⁓ camera, maybe not bypassing the need to geometry sensors but providing amazingly looking pictures. Ultimately as long as we do not ⁓ completely degrade ⁓ the raw sensing quality and quantity like expecting AI to recognize moving cars license plate ⁓ captured ⁓ in a single pixel, these models can do an incredible job job for us. So for now I would like to ignore these biases, ⁓ and focus on on what we can do, ⁓ so that we do good. speaker-0: ⁓ run most businesses today have cameras and most of them bought those cameras mainly for security purposes. ⁓ but now a as you're saying, if AI can actually understand what's happening in in you know, in the f footage that the camera records, I'm curious to know what else a business can learn from cameras that it already has. And I think this is an important ⁓ question for many of the businesses that ⁓ business leaders to the watching the show. speaker-2: Yeah, so so from a single sensor, ⁓ probably you cannot do much, but if if you integrate the entire network of cameras to extract global behavior, you can significantly improve the environment your of your employees, if your customers, vehicles that are moving through, ⁓ by connecting the the sort of dots, ⁓ you can link events that occur in entirely different locations and different times, allowing the the ⁓ we don't have a smart camera, we have a system. That integrates all this information and automatically alerts you when ⁓ the moment ⁓ an exception happens. So so this is how you can actually benefit from an intellectual system that ⁓ thinks of all these cameras sensors, and these sensors could be any, and and that then ⁓ gives you something that you can benefit from. speaker-0: Hmm. I think that's interesting. You know, w a a major thing, as I've said, is the conversation around bias and hallucinations. And I think you've done a great job to to talk about what should be pri prioritized right now. ⁓ but I'm wondering, and I'm thinking to myself, we now have these cameras that are able to understand people's behaviors and the things they're doing and set the context in a way that most cameras can't do. But I'm wondering to myself, is there a line to be drawn about what cameras can actually see about a person? speaker-2: ⁓ well, let me focus on Lomana. ⁓ what we do is provide tools that ⁓ to do good, to monitor and make process more efficient, ⁓ to alert us when specific events ⁓ pose threat. Can someone take a tool essentially aimed at benefiting society and turn it into something that causes damage? I would like to think that ⁓ this wouldn't be the case. But if our tools are are ever misused, we will stop it immediately. So would AI enable if you do things that are ⁓ that you cannot that would somehow ⁓ hide evil ⁓ behavior? Not as I see it right now, but again, I cannot predict the future. I we will always go back to Bohr's ⁓ statement. speaker-0: Tell me tell me more about the great stuff that you know, you're you're building a system that is helping people to do really great stuff. So but can you tell us more about this good things that, you know, these cameras are helping business leaders to do? speaker-2: Yeah, so what we do is we we have a distributed system where we have the brain sitting on each and every site. ⁓ there is a small brain that captures, I don't know, ten or one hundred cameras and doing ⁓ all the detection and and let's call it ⁓ low level tasks locally. And whenever there is a need for acceleration, for for ⁓ understanding more complex events, it consults with the cloud. So our Whole idea is optimizing the performance per cost for the for the client. ⁓ in some cases, clients are coming with ⁓ very specific requirements. I mean, they want to do something like one person is monitoring another, and it's very important to them to to to see that s somebody is looking at someone else, ⁓ counting money or doing a specific operation. And we do this ⁓ very fast, very efficient. ⁓ a very short round ⁓ of development ⁓ models for them and this is part of what we are doing. I mean we are ⁓ a distributed system that is optimizing the cost that you would ⁓ need in order to have an ⁓ supervising ⁓ alert system that does whatever you like to ⁓ do from images and and the ⁓ performance are amazing the ⁓ the input that we get back from our cust customers is amazing, especially if you if they compare us to our competitors. speaker-0: Great. Y you know, you've said already that prediction is difficult, right? It's very difficult thing to do. But I wanna p push a little bit, ⁓ I I think it's a very interesting question. We're we're totally we're mostly used to understanding what a camera sees after the fact, right? ⁓ but I'm wondering about how close we might be to cameras that could give warning signs ⁓ before something happens. speaker-2: So again, as I mentioned before, ⁓ if there is ⁓ an event, if there is some correlation between a behavior and outcome, a com a a camera would ⁓ would be able to do it as well as people if they are trying to do that. ⁓ but you cannot expect ⁓ AI system to predict the future. I mean there is no way of ⁓ of ⁓ translating ⁓ of ⁓ finding the the causality and predicting the future. When the number of unknowns are actually ⁓ well, if if if let me give you an example ⁓ about learning from the physical world. ⁓ we are ⁓ when we travel today, for example, most of the cars and planes already operate by ⁓ drive by wire or fly by wire systems. ⁓ there is an understanding and an underlying algorithm ⁓ translates our physical actions into velocity, altitude, the trajectory. speaker-0: What speaker-2: AI simply extends the horizon that ⁓ of what happens. So AI is the is a very similar paradigm of making decisions which are in a sense less trivial. ⁓ you can use it to for driving cars and avoiding obstacles, and you can also use it to alert people that something is going to happen when similar events happened before. It will not be able to predict something new for you, and you shouldn't expect it to do something like that. speaker-0: Professor Ron, there is a part of your career that I find really interesting. You've worked with medical images, right? ⁓ and you've done a lot of work in that space. My question is, I and I I just really want to know, has trying to teach AI to see things in the human body changed the the way you think about what machines can see that humans sometimes miss? speaker-2: So again, learning from the physical world. ⁓ the the whole notion of what constitutes learning, or probably perhaps you mean understanding, is not fully baked yet. ⁓ I d I I work today in computational pathology, which is a field I I focus on, and we have been able to show that AI inspection of a tumor, of an image of a tumor, ⁓ can fine tune personalized medicine for the better. I mean, tell somebody that he or she could avoid ⁓ radiation or or ⁓ Taking some chemicals. I would be happy to elaborate if we have the time, ⁓ or better to direct it to Dr. Gil Shamai, who has made an amazing discovery ⁓ in this domain. And in fact, he was able to show that ⁓ you can use these images in order to ⁓ fine-tune something that was done until today by by ⁓ analyzing the DNA of the tumor. So by looking at an image, you can decipher even larger parts because there is evidence ⁓ in the image of the tumor that. ⁓ people have been avoiding today. By the way, in an isotarian ⁓ story, the pathologist that we have been working with, we have been collaborating with he's the head of the department of a very famous hostotel, ⁓ told us that if we find a mistake in Islam, he would jump out of a window. So after we have been collaborating for a while, his intern started closing the windows. Now it was not his it was not actually his mistakes. In fact what happens ⁓ Analyzing these these ⁓ tumors is a complicated story. You slice them in various slides, and then you look at each and every slide and you decide whether there is tumor or not, and then you s slice parallel slides in order to do something which is called immuno osteochemistry, which is going deeper into the problem. But you make much fewer immunoosteochemistry and from there you decide how to treat. Now you may take a slice ⁓ where the immuno osteochemistry is telling you something which is completely different than what what you could have ⁓ deciphered. When you looked at the original slides, which is what we are doing. So by training an AI to look at various clues, we have been able to to do something which I think is amazing. It's it was published in the Lancet ⁓ onko ⁓ oncology and and I think it's ⁓ it's an amazing ⁓ discovery. AI can actually have help us to better treat diseases. ⁓ I'm not discovering something new, but AI with images. by looking at images can do amazing things. speaker-0: So machines that can understand the world, and understand context around the physical world, cameras that can see and process the things that they're recording and maybe even warn before an action occurs. We're seeing all of these interesting things. My one big final question to you is what could the future of physical AI look like? speaker-2: So again, predicting the future is something that I would really hesitate ⁓ to to to to do. But I think that with AI we can solve and decipher many problems that we have ⁓ just imagined a couple of ⁓ a decade ago. In my pro specific field, in order to design an AI system which involves cameras, it took us about a year to define the problem and then a couple of more years to to put everything together. About a decade to have ⁓ a system working and then and even then it was not working so great. Now, with ⁓ deep learning ⁓ as the backbone, ⁓ you can do amazing things by just feeding a lot of data and training these backbones without understanding. So there is no understanding inside. There is just this methodology of feeding a lot of data and how it is annotated. And and ⁓ the important question is: what is the important question? I mean, you need to define the l loss or the penalty. By which you train these networks and if you do it properly, you can do amazing things. And and this is what we are doing here in the company as well. I mean, ⁓ it it actually blows my mind every time a customer is is defining a difficult problem and we sit in it for a while and after about a month there is a solution. It is mind blowing. I mean it is it is ⁓ like seeing a miracle in happening. And I couldn't imagine that happening ⁓ twenty years ago or even fifteen years ago. speaker-0: Well that's it from Professor Ron Kimmel, Chief Scientific Officer at Lumana. Thank you so much for joining me on Machine Dreams today. speaker-2: Thank you for hosting me. Thank you very much. speaker-0: And thank you to everyone watching and listening. ⁓ Leia will be back with me soon and we have many more conversations coming with the people building and shaping the future of AI. Until the next time. Bye.