Latest / EDGE AI POD / AI solutions for embedded systems and challenges for GenAI implementations

Transcript

AI solutions for embedded systems and challenges for GenAI implementations · EDGE AI POD

• 0:09
  SPEAKER_00: This is Seven Chan.
• 0:10
  SPEAKER_00: I'm running for HR marketing for Aleph Semiconductor.
• 0:15
  SPEAKER_00: And thanks for HAI Foundation.
• 0:17
  SPEAKER_00: Get a chance for me to introduce the evaluation of HAI and also the products.
• 0:24
  SPEAKER_00: So let me start with my presentation.
• 0:26
  SPEAKER_00: And uh Aleph is a company based in California, San Jose.
• 0:32
  SPEAKER_00: So this is a product we are going to promote in the future.
• 0:37
  SPEAKER_00: So this is the picture I always use to start the presentation slide.
• 0:43
  SPEAKER_00: I think whether you knew this person, you have the same age as I do.
• 0:48
  SPEAKER_00: Sorry for that.
• 0:50
  SPEAKER_00: But this is the perfect example for HGI from my personal perspective.
• 0:57
  SPEAKER_00: Because it received information.
• 1:01
  SPEAKER_00: It made judgment by itself and take the action.
• 1:15
  SPEAKER_00: And now we are designed, we're trying to implement the real product in the world.
• 1:22
  SPEAKER_00: So take example of the uh very emerging case like air glass.
• 1:29
  SPEAKER_00: To design the air gas for the AI product, especially age air product, there's a lot of a challenge and the limitation for the product.
• 1:42
  SPEAKER_00: For example, the most critical thing is the weight for the air gas.
• 1:49
  SPEAKER_00: People cannot wear the glass more than 40 grand of the weight for the whole day.
• 1:58
  SPEAKER_00: And for the battery, it must be supporting at least for the whole day usage, like eight to ten hours.
• 2:09
  SPEAKER_00: To run for one to two hours is not the practical product for the AR class at all.
• 2:16
  SPEAKER_00: And also you need module, you know, cameras for the pictures, for video streaming, even for inferencing, right?
• 2:28
  SPEAKER_00: And also for the control, for the user interface.
• 2:34
  SPEAKER_00: For the class, you don't have the hand to touch the display, right?
• 2:39
  SPEAKER_00: So like the guest, the eyes tracking for the UI interaction is also another requirement.
• 2:48
  SPEAKER_00: And you would like to have the very high definition of the resolution of the display on the glasses.
• 2:56
  SPEAKER_00: So a lot of the requirements to really design the L class for the HEI product.
• 3:04
  SPEAKER_00: So how we can make the solution to really fit in the new requirement and the need for the HEI, starting from the next generation, the SLC product, like you need to really design the high performance of the inferencing processor accelerator.
• 3:28
  SPEAKER_00: Like from the result, you may need more than 10 tops computing power by 2030.
• 3:37
  SPEAKER_00: And for running the uh I would say maybe language model, small language model, you need higher memory uh footprint like the RAM or flash.
• 3:52
  SPEAKER_00: So for the tech current technology we use so far is up to 64 megabytes.
• 3:59
  SPEAKER_00: So you need higher and higher.
• 4:01
  SPEAKER_00: So it needs the help from the technology from the second vendor to make the solution better.
• 4:08
  SPEAKER_00: You also need the longer battery time, so the less power consumed by the solution, by the processor, by the highway, your glass can last longer.
• 4:23
  SPEAKER_00: And lastly, uh they are much more sensors, cameras, everything will be connected to the uh controller or the silicon.
• 4:34
  SPEAKER_00: So how to really design the silicon can fit in with the HEI-based solution.
• 4:42
  SPEAKER_00: So these are all the challenges.
• 4:44
  SPEAKER_00: And one thing more we believe the NPU, the neural processing unit, it will be also impact the uh software incosistent.
• 4:56
  SPEAKER_00: And from a left perspective, uh we think the open source structure is much better than the proprietary one because there's a limitation and advantage, disadvantage in the slide here.
• 5:13
  SPEAKER_00: And from day one, uh left is licensed from ARMS MPU solution.
• 5:19
  SPEAKER_00: That's why we believed with more and more uh users develop, they are devoted, implementing the algorithm, and people are uh developing without limitation to get the system and the MPU more performance can be run, more solution can be implemented through the models.
• 5:43
  SPEAKER_00: That would be something we think in the uh MPU and the Solvey ecosystem would be helping the system get more uh possibility.
• 5:56
  SPEAKER_00: And lastly, uh we working with um from day one through U85 to U from U55 to 85.
• 6:06
  SPEAKER_00: So from our first generation product, from the E1 to E7, this is the family we actually focus on the performance and power efficiency, and we demonstrate multiple uh algorithm and uh inference model can be run into a single solution.
• 6:26
  SPEAKER_00: And then we we announced the new ensemble family this June with the U85.
• 6:35
  SPEAKER_00: Uh let me check.
• 6:37
  SPEAKER_00: Uh this U85 is building with several improvements.
• 6:42
  SPEAKER_00: The first one is U85 MPU.
• 6:46
  SPEAKER_00: This one actually supports the Gen Native AI and transform transformer operator.
• 6:53
  SPEAKER_00: I think uh maybe this is the the first MCU solution with the uh Gen AI be supported in the age without internet, without uh cloud connection.
• 7:08
  SPEAKER_00: So with this kind of uh new hardware from on, uh we can really support the generative and small language model with the limited the database size.
• 7:21
  SPEAKER_00: The secondly, we enhance the image pipeline capability.
• 7:25
  SPEAKER_00: We in we embedded the ISP inside, support higher uh performance and the resolution of the image capability.
• 7:36
  SPEAKER_00: So by the NH it can be reduced like 30% and performance higher.
• 7:43
  SPEAKER_00: Lastly, uh with the external memory, maybe you need to access the data sets.
• 7:51
  SPEAKER_00: So based on the memory interface, we increase the uh interface from lower uh octo SPI into the hex SPI.
• 8:02
  SPEAKER_00: So the data rate is increased like four times better than before.
• 8:07
  SPEAKER_00: So with all this kind of a product and integrate with the new technology, we believe this is something more thorough and closing to the uh beta and H AI solutions.
• 8:21
  SPEAKER_00: And we do have the demonstration with our EA silicon in the booth.
• 8:25
  SPEAKER_00: So uh if you guys are interested to take a look, uh you're welcome to uh walk to our booth to see the actual the real Gen AI demonstration on the MCU platform.
• 8:39
  SPEAKER_00: And uh uh our left is based on the Silicon Valley I just introduced, and we have several uh office worldwide from Sicken Valley, Irvine, uh Ulu, Finland, and Bangaro, India, India, also operation office in uh Singapore.
• 8:59
  SPEAKER_00: So uh please talk to us if you are interested for any of our products or technology.
• 9:06
  SPEAKER_00: Uh I'll be in the booth for the further conversation.
• 9:09
  SPEAKER_00: And thank you for the time today.