Latest / EDGE AI POD / When Edge AI Meets Hearing Loss, Access Gets Real

Transcript

When Edge AI Meets Hearing Loss, Access Gets Real · EDGE AI POD

• 0:15
  SPEAKER_00: Okay, awesome, great.
• 0:17
  SPEAKER_00: So I will be talking about squeezing intelligence into the ear, a practical deployment of AI speech enhancement and hearing aids.
• 0:26
  SPEAKER_00: Before I get started, I'm just gonna introduce Femto AI a little bit.
• 0:31
  SPEAKER_00: So hopefully the video works.
• 0:33
  SPEAKER_00: There we go.
• 0:34
  SPEAKER_00: We are really proud of this animation.
• 0:36
  SPEAKER_00: Uh so FemtoAI, we were founded in 2018.
• 0:38
  SPEAKER_00: We used to be called FemtoSense up until last Thursday.
• 0:41
  SPEAKER_00: So if you've you know not heard this name before, don't worry, um, it's new.
• 0:46
  SPEAKER_00: And so what we do is we're all about empowering empowering intelligence and everyday devices.
• 0:51
  SPEAKER_00: That means you know bringing this AI accelerated chip into really small uh edge applications.
• 0:58
  SPEAKER_00: And a bit about me.
• 1:00
  SPEAKER_00: So my name is Pong.
• 1:01
  SPEAKER_00: I'm the director of strategy and chief of staff at FETO AI.
• 1:04
  SPEAKER_00: Um I'm actually substituting for our senior deep learning engineer, Raphael, who could not join us um at the conference this week.
• 1:11
  SPEAKER_00: And you may notice, Raphael, he's very technical.
• 1:15
  SPEAKER_00: You look at me, I'm not technical.
• 1:17
  SPEAKER_00: So you're welcome to ask any of the questions afterwards.
• 1:20
  SPEAKER_00: Uh I can forward it to the engineers if I cannot answer them.
• 1:24
  SPEAKER_00: Alright, so now let's get into the talk.
• 1:28
  SPEAKER_00: Hearing aids.
• 1:29
  SPEAKER_00: Um according to WHO, it turns out that 17% of people who who could benefit from hearing aids don't actually end up using them.
• 1:39
  SPEAKER_00: I'm looking around and I see a lot of people with glasses.
• 1:41
  SPEAKER_00: Now imagine if 17% of people who can use glasses don't use glasses.
• 1:46
  SPEAKER_00: A lot of people that you know won't be able to see.
• 1:49
  SPEAKER_00: With hearing aids, a lot of people aren't able to hear clearly.
• 1:52
  SPEAKER_00: And we wonder why is that?
• 1:56
  SPEAKER_00: The top reason for that is that you know people realize that you know there are poor benefits from hearing aids.
• 2:03
  SPEAKER_00: Even if you wear them, it doesn't actually help with your hearing.
• 2:07
  SPEAKER_00: And a second reason uh that people stop using hearing aids is that they don't perform well in noisy situations.
• 2:14
  SPEAKER_00: Um I have a little demo downstairs that can demonstrate what this means.
• 2:17
  SPEAKER_00: But essentially, what happens with hearing aids now is that there's a microphone, it takes a signal, it does maybe some filtering, uh, and then it boosts everything.
• 2:26
  SPEAKER_00: So noise is amplified, your speech is amplified, um, everything's amplified.
• 2:31
  SPEAKER_00: So, how can we solve it?
• 2:34
  SPEAKER_00: Well, we're in an AI conference, so let's throw some deep learning at it.
• 2:39
  SPEAKER_00: Great, problem solved.
• 2:41
  SPEAKER_00: So deep learning, it can separate the speech from noise, check.
• 2:45
  SPEAKER_00: Deep learning, it can also preserve speech quality, check.
• 2:49
  SPEAKER_00: And then one of the biggest problems with um hearing aids right now is if you have a sharp impulse, let's say cutlery, for example, when you're out having dinner, it's like super high frequency, super high amplitude, and it's really, really painful.
• 3:02
  SPEAKER_00: So deep learning, great.
• 3:04
  SPEAKER_00: You can also detect for that and then solve that.
• 3:07
  SPEAKER_00: And in fact, you can experience this sort of um deep learning approach to um noise reduction in Google Meets, Zoom, um Teams right now.
• 3:17
  SPEAKER_00: All right, so thank you for attending my talk.
• 3:19
  SPEAKER_00: Um, we're done, we solved the problem.
• 3:21
  SPEAKER_00: Goodbye.
• 3:22
  SPEAKER_00: No, we're actually at the edge AI talk.
• 3:25
  SPEAKER_00: We're about connecting AI to the real world.
• 3:28
  SPEAKER_00: So let's think about deployment.
• 3:30
  SPEAKER_00: Okay, let's strap a GPU onto a hearing aid.
• 3:34
  SPEAKER_00: Okay, could that work?
• 3:36
  SPEAKER_00: Um maybe not.
• 3:38
  SPEAKER_00: Because right now we see that okay, with power of typical um noise production algorithms, it takes about like one to ten watts.
• 3:45
  SPEAKER_00: For latency, if you're doing something like Google Meets call, um, you know, like you have the latency budget.
• 3:52
  SPEAKER_00: It doesn't matter if you're hearing real time or not, like 50 milliseconds is totally fine.
• 3:56
  SPEAKER_00: And the size, I won't go into details, but it's big.
• 4:01
  SPEAKER_00: So in a hearing aid deployment, um power, we want to make sure that it actually lasts in a hearing aid form factor for a whole day.
• 4:09
  SPEAKER_00: So 20 hours, assuming you sleep for at least four hours a day.
• 4:12
  SPEAKER_00: Um and in a hearing aid size battery, that translates to about one milliwatt of power that can be allocated to the AI part.
• 4:22
  SPEAKER_00: For latency, um, you want to make sure it's perceived as real time.
• 4:26
  SPEAKER_00: So that translates to about less than 10 milliseconds uh of latency.
• 4:32
  SPEAKER_00: And then for a size to fit in a form factor, it has to be tiny.
• 4:37
  SPEAKER_00: So we have a solution, we call it the Clara AI speech enhancement, and this is the flow.
• 4:42
  SPEAKER_00: Uh once again, I want to remind you that I am not a technical person, so I will just read my notes here of what the components mean.
• 4:49
  SPEAKER_00: So we start on the top left hand side with a noisy audio input coming from the microphone.
• 4:54
  SPEAKER_00: And then we bring it into our chip, the SPU001, and we do an uh forage transform on this to turn from the time domain to frequency domain.
• 5:04
  SPEAKER_00: Followed by that, we do log mill transform, um, and that basically puts you know the signal into like frequency bins uh and you know that kind of maps to what the hearing, the human hearing experience is like.
• 5:16
  SPEAKER_00: Then we do some uh gate invariants like the delta MFE um just to make sure that, okay, it's gate invariant.
• 5:24
  SPEAKER_00: Here comes the fun part.
• 5:26
  SPEAKER_00: Then we feed it into a DNN.
• 5:29
  SPEAKER_00: I will go into more details later.
• 5:32
  SPEAKER_00: After that, you know, DNN predicts a mask um that can be used to, you know, multiply like with the um signal to then generate this enhanced um STFT that you know has the speech but does not have the noise.
• 5:47
  SPEAKER_00: And then we inject that back to the top signal flow.
• 5:50
  SPEAKER_00: I realize I have a laser.
• 5:52
  SPEAKER_00: Ta-da! Um and then here we can do some remixing where if you want to preserve some of the original noisy signal just for like to make it sound natural, you can maybe inject maybe like 20% of this, and then you have like you know the rest, 80%, be this cleanup signal, uh, and finally have your uh clean output.
• 6:14
  SPEAKER_00: So earlier I mentioned the DNN part.
• 6:17
  SPEAKER_00: Let's dig a little bit deeper into that.
• 6:21
  SPEAKER_00: Oh yeah, before I get into that, uh, in addition to all this wonderful stuff we just talked about, uh, we can also add a lot of other uh signal processing to this as well.
• 6:31
  SPEAKER_00: Uh, one of the fun ones is the wide dynamic range compression, which is um really helpful in hearing aids.
• 6:37
  SPEAKER_00: But okay, back to the DNN.
• 6:39
  SPEAKER_00: What is in here?
• 6:41
  SPEAKER_00: Uh we have a 40 layer uh temporal uh convolutional recurrent neural network.
• 6:46
  SPEAKER_00: Uh so the temporal convolution part is just a sliding window um over time, and then uh yeah, the recurrent neural network, yeah, it is able to do the wonderful predictions.
• 6:56
  SPEAKER_00: Um I don't know anything further than that, not a technical person.
• 7:01
  SPEAKER_00: All right, so how are we able to fit that 40 layer uh network onto a chip?
• 7:07
  SPEAKER_00: So at FEMTOAI, uh we're all about you know using the technique called sparsity.
• 7:13
  SPEAKER_00: It was inspired by like neuromorphic research that the founders uh did at Stanford University, um, but then pivoted over to purely digital design.
• 7:21
  SPEAKER_00: So a bit about sparsity.
• 7:23
  SPEAKER_00: I think a lot of people in the audience probably knows about it more than I do, but I'm just gonna do a high-level overview.
• 7:29
  SPEAKER_00: So neural networks is just a bunch of matrix operations, mostly multiplication.
• 7:34
  SPEAKER_00: It turns out that you know in these matrices there are some values that have really low um magnitudes, so like they're really close to zero.
• 7:42
  SPEAKER_00: And if you prune them down to zero, it doesn't really affect the accuracy of the model uh significantly.
• 7:49
  SPEAKER_00: So let's say we go ahead and prune it down to zero.
• 7:53
  SPEAKER_00: If you multiply anything by zero, it's zero.
• 7:56
  SPEAKER_00: And so, you know, conventional approaches might just do the operation anyway.
• 8:01
  SPEAKER_00: But what we have uh with our chip is custom instruction set that once it detects the zero in like the unstructured sparsity, we're able to skip the calculations.
• 8:09
  SPEAKER_00: And so the result of that, let's say we do 90% pruning, uh, then we're able to you know save the memory footprint by 10x, and then the power consumption actually by 100x, because if we have double dual sparsity of both the weights and the activations, it multiplies.
• 8:25
  SPEAKER_00: And you know, it looks great in theory.
• 8:28
  SPEAKER_00: And it's also reality.
• 8:30
  SPEAKER_00: So we have this tiny little chip called the SPU001, which I'm gonna hold right here, but I don't know if you're able to see it because it is absolutely tiny.
• 8:40
  SPEAKER_00: Um you can also just yeah, okay, I see you're all squinting your eyes um at this.
• 8:45
  SPEAKER_00: Uh that's a tweezer.
• 8:47
  SPEAKER_00: Those are fingers.
• 8:49
  SPEAKER_00: Small.
• 8:50
  SPEAKER_00: Uh it is about 3.5 um millimeter squared.
• 8:54
  SPEAKER_00: And you know, in it, we say it has 10 megabytes of effective memory.
• 9:00
  SPEAKER_00: So what does effective mean here?
• 9:02
  SPEAKER_00: So, okay, so it has one megabyte of raw SRAM.
• 9:05
  SPEAKER_00: And then let's say if you do let's say 90% pruning, then you can fit, in theory, um a 10 megabyte model in the one megabyte footprint.
• 9:14
  SPEAKER_00: And then in terms of efficiency running like the algorithm I showed you earlier, we're talking about the microwatt scale, maybe up to like one 1.2 milliwatts.
• 9:26
  SPEAKER_00: So let's see, like going back to the earlier part about noise reduction, okay, the experience of the hearing aid user, they care about these three main things.
• 9:36
  SPEAKER_00: And then for deployment, these are the things that are the constraints that we have to work with.
• 9:42
  SPEAKER_00: Going into this first part, um we have a few results.
• 9:49
  SPEAKER_00: So on the y-axis here we have, let's say, the signal signal-to-noise ratio, um, where the higher the number, uh, you know, the more signal, like let's say clean speech there is to the noise that we don't want to supp um like that we want to suppress.
• 10:08
  SPEAKER_00: The blue line is the enhanced signal, uh, whereas the orange line down here is the noisy signal.
• 10:16
  SPEAKER_00: So what is SISDR?
• 10:18
  SPEAKER_00: It is a scale invariant signal to distortion ratio.
• 10:21
  SPEAKER_00: The higher um the score here, it means that it is a clearer signal.
• 10:27
  SPEAKER_00: So you see that you know across different SNRs, once we enhance it, no surprise, it has um greater like uh signal-to-noise ratio.
• 10:38
  SPEAKER_00: Not signal to noise ratio, uh great clearer signal.
• 10:42
  SPEAKER_00: The next metric that we have here is what's called the HASCI metric.
• 10:47
  SPEAKER_00: So it's you know a really common metric in the hearing aid development.
• 10:50
  SPEAKER_00: Uh it stands for the hearing aid speech quality index.
• 10:54
  SPEAKER_00: So the higher the number, the greater the the greater the um speech quality.
• 11:01
  SPEAKER_00: So great.
• 11:02
  SPEAKER_00: We're able to have clearer signal, greater speech quality.
• 11:07
  SPEAKER_00: And then finally, the last part, um, after all, like if you're wearing hearing aids, you know, how does it actually feel?
• 11:13
  SPEAKER_00: So here we have some metrics about um the actual experience that users have.
• 11:19
  SPEAKER_00: And no surprise here, when we enhance a signal, it's a greater it's a better experience.
• 11:26
  SPEAKER_00: So you know, all of these things are now we're able to do.
• 11:32
  SPEAKER_00: And now moving on to the deployment part.
• 11:36
  SPEAKER_00: So over here in my hands, I have hearing aid with the chips actually in here, and then also the algorithm running on it.
• 11:44
  SPEAKER_00: And running the algorithm AI for the whole time, it is able, the battery is able to last for 20 hours.
• 11:52
  SPEAKER_00: The latency that's in here, it's for all for the algorithm itself, it's about I think eight milliseconds.
• 11:58
  SPEAKER_00: And so overall, the signal from input coming in to hearing, uh, it's about less than 10 milliseconds.
• 12:05
  SPEAKER_00: And then finally, well, the chip is in here, it's working, it does fit in the form factor.
• 12:13
  SPEAKER_00: Alright, so checked all the boxes.
• 12:16
  SPEAKER_00: Uh and I just want to talk a little bit about um the you know this hearing aid.
• 12:22
  SPEAKER_00: So it's made by a company called New Sound, um, based in China, and uh uh they launched it early this year in March at the AAA conference.
• 12:31
  SPEAKER_00: It's the American Audiology Association, if I remember correctly.
• 12:35
  SPEAKER_00: Um and you know, there were a lot of audiologists who came to try this out.
• 12:40
  SPEAKER_00: And this is the part where I think is pretty interesting.
• 12:43
  SPEAKER_00: Uh they walked around you know the show floor, they tried out different AI hearing aids, and then it came to you know our booth, tried this.
• 12:52
  SPEAKER_00: And then one of the quotes that they said was I walked around, tried a bunch of AI hearing aids, I thought AI was a buzzword.
• 12:59
  SPEAKER_00: Then it tried this, and they're like, oh, okay, now I finally hear the difference.
• 13:02
  SPEAKER_00: So with that, I actually want to invite you to come to a demo table downstairs and actually hear this.
• 13:08
  SPEAKER_00: Another thing I want to note about New Sound is that they got in touch with us like a few months ago, and then they were like, all right, okay, we're interested in your chip, let's try it out, let's build it in.
• 13:18
  SPEAKER_00: And then they were able to just build everything in in three months, and we're like, whoa, okay, like that's pretty fast.
• 13:25
  SPEAKER_00: Because if you're familiar with hardware development cycles, it can be really long.
• 13:29
  SPEAKER_00: So huge kudos to our friends at New Sound.
• 13:33
  SPEAKER_00: There's gonna be a lot more hearing aids uh with our chips building coming soon, so there's a little sales pitch.
• 13:38
  SPEAKER_00: Um, yeah, it'll be fun.
• 13:41
  SPEAKER_00: And also other devices too.
• 13:43
  SPEAKER_00: Um all right, and you know, with that, come to the end of the presentation.
• 13:48
  SPEAKER_00: Thank you for being here, thank you for listening.
• 13:50
  SPEAKER_00: Uh just wanna you know share a few other things.
• 13:54
  SPEAKER_00: So, yeah, please come try out the demo at our demo table.
• 14:00
  SPEAKER_00: If you know someone who uses hearing aids um or who could benefit from a hearing aid, let me know.
• 14:06
  SPEAKER_00: We can put you in touch to get one of these, or maybe any other hearing aids and like share more.
• 14:12
  SPEAKER_00: Uh if you're interested in deploying your own algorithms onto a chip, uh, we do have a developer portal uh that you can go in and then try that.
• 14:20
  SPEAKER_00: And then finally, uh next week, my colleague John will be giving a talk in Geneva at the AI for good global summit, where he'll be actually talking about the hearing aid and how like edge AI can actually help um you know more people benefit from this technology.
• 14:34
  SPEAKER_00: So thank you, and now open for questions.