Latest / EDGE AI POD / Smarter AI, Faster Hardware

Transcript

Smarter AI, Faster Hardware · EDGE AI POD

• 0:09
  SPEAKER_00: Hello, I'm here representing Siemens EDA and Catapult High Level Synthesis, and I'm super excited to talk to you about our efficient inferencing hackathon.
• 0:19
  SPEAKER_00: So also immediately following this, we are going to be having a presentation workshop about high-level synthesis.
• 0:26
  SPEAKER_00: So this is going to be a little sneak peek into our presentation as well as the introduction to the hackathon.
• 0:34
  SPEAKER_00: Talk a little bit about introduction, some background, the challenge itself, details, and the prizes.
• 0:44
  SPEAKER_00: Now, you can't have a good hackathon without a problem.
• 0:48
  SPEAKER_00: But what exactly is the problem that we're facing?
• 0:51
  SPEAKER_00: Devices are getting smarter, and a lot smarter.
• 0:55
  SPEAKER_00: You have smartphones, smart glasses, smart homes, smart watches, smart speakers, smart fridges, smart toilets, you get the idea.
• 1:04
  SPEAKER_00: And as we get these more devices and we start implementing these AI inferencing algorithms in them, we can see that there will be more and more different types of embedded devices.
• 1:15
  SPEAKER_00: As of today, there's about 50 embedded devices for every personal computer.
• 1:21
  SPEAKER_00: But you may ask, how do we actually go about creating these embedded devices?
• 1:27
  SPEAKER_00: And that's what I'm trying to be here to show you today.
• 1:31
  SPEAKER_00: Now, this graph is the computational load in gigaflops for the ImageNet algorithm.
• 1:37
  SPEAKER_00: It is an increase by a factor of a hundred in the past five years.
• 1:41
  SPEAKER_00: Back in 2017, there were 30 billion floating point operations, and today there's been over 3 trillion.
• 1:49
  SPEAKER_00: And this is only one model.
• 1:51
  SPEAKER_00: The image ImageNet, which I mentioned.
• 1:54
  SPEAKER_00: But there's also a variety of others of a lot of different implementations.
• 2:00
  SPEAKER_00: So not only is there a large variety of models, but these models themselves are getting larger and larger.
• 2:05
  SPEAKER_00: And you may ask, where exactly do these models run?
• 2:09
  SPEAKER_00: So everyone is familiar with the building box of a computer, so you can start on a CPU.
• 2:16
  SPEAKER_00: If your device, if your inferencing algorithm works with an off-the-shelf CPU, you're golden.
• 2:22
  SPEAKER_00: They're cheap, they're easy to use, they're reprogrammable, but it doesn't work for every application.
• 2:28
  SPEAKER_00: The performance and the energy may not be there.
• 2:31
  SPEAKER_00: That's where you start looking at alternatives, one of which is starting to target your GPU for your algorithms.
• 2:38
  SPEAKER_00: That may work, but that may also not be enough.
• 2:41
  SPEAKER_00: And you start targeting more and more specific pieces of hardware, one of which would be the TPUs or MPUs.
• 2:48
  SPEAKER_00: And then lastly, will be these custom pieces of hardware, whether it be an FPGA or an ASIC processor, this is where you get that highest performance and lowest energy.
• 2:59
  SPEAKER_00: And that is what we focus on at Siemens ZDA with our catapult high-level synthesis tool is being able to bring to you an easier development solution to get those custom pieces of hardware.
• 3:10
  SPEAKER_00: And what exactly is that solution?
• 3:12
  SPEAKER_00: That solution is high-level synthesis.
• 3:16
  SPEAKER_00: So instead of designing hardware by hand using Verilog VHDL, we are able to use a higher extrapolation language, which in this case will be C or System C.
• 3:28
  SPEAKER_00: Now, instead of detailing every single register, every single operator, every single wire, we have some level of automation that helps handle these details.
• 3:37
  SPEAKER_00: And not only that, we also allow for design space exploration by letting you try out a variety of implementations for you to pick the best solution.
• 3:46
  SPEAKER_00: So this lowers your development time and what I like to call intelligent implementations, where you're able to try a variety of different implementations to try to figure out what works the best for you.
• 4:00
  SPEAKER_00: And there are other features that are available with this, like parallelism or implementation for precision, which is going to be finding your right bits for your operations.
• 4:12
  SPEAKER_00: Something that I'd like to mention is that Python for AI development tends to work in Flow32.
• 4:18
  SPEAKER_00: Not every single algorithm actually needs that full bit precision.
• 4:22
  SPEAKER_00: Some can work on much smaller values.
• 4:24
  SPEAKER_00: And Catapult allows you to modulate those values in hardware to allow you to run inferencing algorithms with the least needed amount of hardware.
• 4:34
  SPEAKER_00: And that's exactly where we bring us to our hackathon, where we are trying to help you design your hardware using high-level synthesis and show you all the different ways that you can optimize.
• 4:46
  SPEAKER_00: You can start in software by making sure that your architecture of your model, your layers and channels are at the correct level for what you need for your accuracy.
• 4:56
  SPEAKER_00: We can bring to your that quantization, which is that precision for the number of bits for your algorithms, which lets you try a variety of different ones both while training the neural net and also in the hardware itself.
• 5:09
  SPEAKER_00: You can get parallelization to try a variety of different reuse factors to measure your throughput and your latency.
• 5:17
  SPEAKER_00: And all of this can be done in our tool to compare this hardware and software.
• 5:22
  SPEAKER_00: And it's a much easier device to be able to do than handwritten RTL.
• 5:27
  SPEAKER_00: I'm not sure how many of you in this room has worked with handwritten RTL before.
• 5:33
  SPEAKER_00: One, two, three, four.
• 5:35
  SPEAKER_00: Oh, perfect.
• 5:36
  SPEAKER_00: We got like a handful here.
• 5:37
  SPEAKER_00: That's great.
• 5:38
  SPEAKER_00: So you guys know that if someone were to come up to you and say, hey, I have this design uh for you, and you give them a lead time about two years, or not even that, you give them a lead time of a year or the six months.
• 5:50
  SPEAKER_00: And then like a month or two before you're done, someone comes up and they're like, hey, actually, I think I want to change the precision on this.
• 5:56
  SPEAKER_00: Do you mind going back through your entire design and changing all of this?
• 6:00
  SPEAKER_00: You would probably throw it in their face, right?
• 6:02
  SPEAKER_00: You'd be like, I don't want to work here anymore.
• 6:05
  SPEAKER_00: So that is what our tool is able to do is bring that development time down from that year or six months, down to a few weeks, down to a month, and it lets you try the variety.
• 6:15
  SPEAKER_00: So instead of making that one design in that six months, you're able to make 50 or 100 and determine what makes the most sense for you.
• 6:23
  SPEAKER_00: Now, introducing the hackathon itself.
• 6:27
  SPEAKER_00: So everyone in this room and everyone who attends our uh presentation today is going to get a head start and ability to join our hackathon.
• 6:36
  SPEAKER_00: We're going to be using the MNIST algorithm, which I'm going to show on the next slide for those who don't know, and we are going to be judging you off your power, performance, area, and accuracy.
• 6:46
  SPEAKER_00: We are also going to be providing a full on-demand training that will take you start to finish on how to use the tool and how to give you some tips on how to make the most efficient solution.
• 6:58
  SPEAKER_00: Now, for those who are not aware, the MNIST algorithm is a very common uh AI inferencing algorithm, very much the hello world of the AI world.
• 7:07
  SPEAKER_00: So we recognize handwritten digits from 0 to 9, and we start with a very small neural net, and it's able to be retrained very well, well characterized, and we are going to be using the base data set that's in the TensorFlow library for both training and accuracy validation.
• 7:26
  SPEAKER_00: Now, your mission, if you choose to accept it, will be to combine hardware and software to create the fastest running implementation that fits accuracy, area, and energy efficient requirements.
• 7:40
  SPEAKER_00: You will be provided a bare metal software app with MNIST already on that's running on a rocket core with over one millisecond current latency.
• 7:50
  SPEAKER_00: Now that can be optimized down much, much farther, down to around 10 milliseconds or even faster.
• 7:56
  SPEAKER_00: And this is something that we want you to try is to bring a solution that's already built for you and optimize it using our tool.
• 8:07
  SPEAKER_00: So after signing up with the virtual machine, you'll be able to you'll be provided the CNN for Keras for MNIST, the RISC-V Rocket Core design, the entire suite of Siemens EDA tools, catapult high-level synthesis, quest of logic simulation, power pro power analysis, and all the variety of other products needed to perform these operations.
• 8:28
  SPEAKER_00: You'll also be provided a C algorithms for the convolutional and dense layer, and you'll be provided the entire thing coded in high-level synthesis.
• 8:39
  SPEAKER_00: Now, for this process of optimization, we want you to optimize your neural net, quantize your numbers, introduce parallelism, and optimize the data flow.
• 8:49
  SPEAKER_00: All of these are going to help you bring the most efficient solution to our project and hopefully other projects that you work on.
• 8:58
  SPEAKER_00: Like I mentioned, we're gonna step you through the bases and we'll take you from Python, Keras, TensorFlow, C to Verilog.
• 9:05
  SPEAKER_00: We are also going to allow you to have full access to all of the metrics that are available in the tool, and you'll be able to compile your hardware, your customized hardware, and see a variety of different uh critiques about it.
• 9:19
  SPEAKER_00: And we'll make these steps easy for you.
• 9:21
  SPEAKER_00: Like I mentioned, we're gonna be providing an entire intellectual suite for you to understand from start to finish.
• 9:28
  SPEAKER_00: Now, the hackathon is gonna run starting today to October 31st.
• 9:34
  SPEAKER_00: We're gonna be providing a full featured virtual machine, all the tools, licenses, and IP needed.
• 9:39
  SPEAKER_00: Your virtual machine will be eligible for 30 days.
• 9:42
  SPEAKER_00: So the entire hackathon runs for about 30 uh three months, but you will get 30 days of the actual tools for you to determine what works.
• 9:50
  SPEAKER_00: And this will all be running through our brand new academy, which I'm excited to announce, which is hls.academy.
• 9:57
  SPEAKER_00: This is an online uh forum that allows you to learn about high-level synthesis and talk to other people in industry.
• 10:04
  SPEAKER_00: You can talk to people like myself and other experts in the field, and we are looking to expand this widely and you'll be able to talk to other people working on the hackathon about this project.
• 10:17
  SPEAKER_00: The winners are going to be announced in November at the Edge AI Foundation event in Taipei.
• 10:23
  SPEAKER_00: Now, what exactly do you win?
• 10:25
  SPEAKER_00: That's why we're all here, right?
• 10:26
  SPEAKER_00: So, first place we'll get the LEGU Neptune 4 Max 3D printer, second place we'll get an FPGA development board, third place we'll get Bose Noise Canceling Earbuds, and everyone who participates will get a LinkedIn badge that is provided through the Siemens EDA training academy.
• 10:45
  SPEAKER_00: Now, here is a QR code to sign up now.
• 10:48
  SPEAKER_00: This will link you to our uh HLS Academy where you can create an account and you'll be able to sign up for on-demand training.
• 10:54
  SPEAKER_00: Online, there's going to be a uh promo code that's in the instructions, just saying it out loud here.
• 11:01
  SPEAKER_00: It is two underscores, not one underscore.
• 11:05
  SPEAKER_00: So you can get the full free discount to make sure that you can uh participate.
• 11:11
  SPEAKER_00: This is the same QR code, don't worry.
• 11:13
  SPEAKER_00: So I would like you to join us for the workshop that we'll be having over the next three days.
• 11:19
  SPEAKER_00: It will be the same workshop for the three days, so if you can't make it today or tomorrow, you can make it on Friday and all of the above.
• 11:26
  SPEAKER_00: Once again, I'd like to thank you for listening to me talk today, and I hope to see you uh for my presentation, and also I'll be available at the demo tube of Zelda.