Latest / EDGE AI POD / Edge AI, Energy, And Autonomy

Transcript

Edge AI, Energy, And Autonomy · EDGE AI POD

• 0:05
  SPEAKER_00: Is that we bring together not only folks from industry to talk about what's happening, but also academia to talk about their research.
• 0:14
  SPEAKER_00: And I was talking to someone in the hallway, Sam in the hallway, and we were saying, like, in a very nascent market like this, the blend of academia and industry is really important because it really gives us an insight as to what's happening in this space kind of around the corner.
• 0:28
  SPEAKER_00: Without further ado, Winston.
• 0:33
  SPEAKER_01: Thank you.
• 0:34
  SPEAKER_01: Welcome to Taipei.
• 0:35
  SPEAKER_01: It's really my uh great honor and pleasure to uh share the AI transformation um here in Taipei 2025.
• 0:46
  SPEAKER_01: And I think most exciting that this is my first time joining this community, and I'd like to thank Pete and many community members' uh endorsement invitation.
• 0:55
  SPEAKER_01: And these two days and beyond, I'd like to learn more from you folks.
• 0:59
  SPEAKER_01: I think this is a community effort.
• 1:02
  SPEAKER_01: Today's topics, I even though I'm in academic, I'd like to share uh uh roughly what the technology breakthrough in the past uh 12 or 20 years.
• 1:12
  SPEAKER_01: And I think most important is what the technology we learned in which verticals and likely what kind of commercial opportunity we can bring in in the uh coming years.
• 1:25
  SPEAKER_01: So I give some context, and I'm in the academic and working on some um national key AI initiatives in the past few years, but also Pete just mentioned I also have the automotive role, and I learned how this uh the supply chain and how you make hardware software assistance and even meet the requirement and safety requirements, functional safety requirements in the uh regulation in the industry in different uh regions, and also how energies actually uh bring a huge impact in the past and even the coming year, and how AGAI really also plays a critical role in the um energy uh sectors.
• 2:10
  SPEAKER_01: So so that I would uh get my academia and industry experience and share what was the opportunity ahead and especially for HAI.
• 2:20
  SPEAKER_01: And actually, the opportunity is not does not come from a single pillar or single uh vertical that would bring the impact.
• 2:29
  SPEAKER_01: From my experience in the past few years, I think that would be a critical but uh very intertwined uh pillars ahead for H AI as AI energy and autonomy as as well.
• 2:49
  SPEAKER_01: Why?
• 2:49
  SPEAKER_01: I think uh the AI, everybody understands that it's it's quite uh important to bring the intelligence to the device level, but the AI requires tons of energies.
• 3:01
  SPEAKER_01: So energy defines the boundary, defined the limitations of AI capabilities, more tokens, more energy, you have more tokens, you got to bring more intelligence.
• 3:12
  SPEAKER_01: But the other hand is AI actually have energy a lot from our experience that the AI on the age side and cross side can give better forecast, predictive maintenance for the grid, for the power generations, even bring the energy uh efficiency in the home or in the factory or in the uh data centers.
• 3:34
  SPEAKER_01: So AI have a lot for energies.
• 3:37
  SPEAKER_01: And also autonomy that no matter its agent, physical agent doing the control or autonomous driving and all the coming opportunity for the robot also require the energy to drive the motor, drive the application, also require the AI intelligence that can have autonomy really operate uh well smoothly in such dynamic and challenge change uh environment environment.
• 4:01
  SPEAKER_01: So I would say that AI, energy, autonomy actually intertwine together, as you you will see that in the uh coming tech.
• 4:10
  SPEAKER_01: It's really hard to predict the future because I always have uh lots of questions from the government, from the industry partner.
• 4:18
  SPEAKER_01: Could you say what should we do?
• 4:20
  SPEAKER_01: What will be the focus, what will happen in the technology sector?
• 4:24
  SPEAKER_01: So that's why I found that the best answers go to the history.
• 4:28
  SPEAKER_01: Here is the curve showing the GDPs of the world in the past uh 200 years.
• 4:34
  SPEAKER_01: This is log scale, even though it's all they linearly growth, but actually it's actually exponential growth here.
• 4:42
  SPEAKER_01: And it's you see that, yeah, this can need growth, this is good for the society, but uh GDP also uh can reflect the economic conditions and also the wealth and also the productivity for the nation or for the world.
• 4:56
  SPEAKER_01: But is really GDP continued to growth?
• 4:58
  SPEAKER_01: And this is one of the questions many people ask, even productivity, you can need to grow.
• 5:03
  SPEAKER_01: But we saw that actually the driving the for the growth for the GDP aligned with the technology uh revolution.
• 5:11
  SPEAKER_01: So you starting for the first industry revolution, starting from uh 1770, is the introduction of water powers that drive the machines that have more efficiently uh threading the uh the cut threading and spinning the cast threads, and also following with in the 80s, in 1800s, there for the introduction of electricity, heavy machine, oil, etc.
• 5:36
  SPEAKER_01: So you can see that the industry revolutions, they bring the energy first, water, electricity, oil, and then drive the automation.
• 5:47
  SPEAKER_01: Even though it's not fully autonomy yet, but I think this intention that they try to uh drive the the machine uh automation that reduces the human power, improve the productivity.
• 5:58
  SPEAKER_01: That's the same as what we want to expect for the autonomy.
• 6:01
  SPEAKER_01: So you can see that in the past, the energy autonomy determined the productivity, determined the competition, determined the GDPs for the country, for the enterprise.
• 6:12
  SPEAKER_01: And then uh following the the last uh innovation or technology uh uh improvement in 1971, the ID technology we generate more data, more machines, more competition, it gets into a digital war, and then following the last uh revolution in AI revolution.
• 6:29
  SPEAKER_01: So we can see that energy automation and then following the AI will be the next right for the coming uh war.
• 6:37
  SPEAKER_01: So we need to pay more attention on that.
• 6:40
  SPEAKER_01: And then look at first why AI would be a good fit for HAI.
• 6:45
  SPEAKER_01: But I want to emphasize that the automation, autonomy, energy, AI intertwine all together.
• 6:52
  SPEAKER_01: You should look then in the holistic way.
• 6:56
  SPEAKER_01: So these curves just summarize the AI technology uh development in the past uh 12 uh years.
• 7:04
  SPEAKER_01: We have been working on these AI technologies since 2002.
• 7:08
  SPEAKER_01: It's 23 years.
• 7:10
  SPEAKER_01: So we can easily sense the dramatic change in the past 23 years.
• 7:16
  SPEAKER_01: So the curve showing that is the growth of the number of parameters in different uh major parameters.
• 7:24
  SPEAKER_01: So you can see the curve starting from 2013 and continued growth, the number size of parameters starting from a median scale to billion scale, even trillion scale right now.
• 7:35
  SPEAKER_01: But we can say that the more uh parameters, the power the more powerful the model.
• 7:42
  SPEAKER_01: But generally, generally we can categorize that into four stages.
• 7:46
  SPEAKER_01: The first stage is regarding the procession, it means that at the time the neon world, uh convolutional neuronal world, redominate the uh the the academic or industry, so all the convolution technology you need every developed uh at that stage, and then introduction of transformer, then the natural language processing community understands how to use the technology to uh improve the uh Q and A translation, etc.
• 8:15
  SPEAKER_01: And later people find that transformer is easy to scale to billion scale pair merger, then the capability emerge and and the introduction of uh GPT-3 in uh 2020 and also the end of 2022 for the uh GPT is really uh bring quite impact to the uh industry and also the academia as well.
• 8:39
  SPEAKER_01: Why did the impact?
• 8:41
  SPEAKER_01: Because it carries very, very important uh uh property co-generation.
• 8:46
  SPEAKER_01: It means that one was the model trained in certain conditions, they can generate well too many occasions.
• 8:51
  SPEAKER_01: This kind of generation capability, and also we found that the model also carries some uh uh task or capability that within the single model.
• 9:01
  SPEAKER_01: This has never happened uh uh before.
• 9:03
  SPEAKER_01: So, generation, my tasking is bringing lots of imagination for the industry, seeing that this model can do lots of things.
• 9:12
  SPEAKER_01: That's why we saw the huge growth in this AI ecosystem.
• 9:16
  SPEAKER_01: Normally it's the OE model, even how system age as well, we impact a lot.
• 9:21
  SPEAKER_01: But we said that the the most thing we should uh care about is the four-stage reasoning uh capability that just emerged from last year.
• 9:32
  SPEAKER_01: When you use more token, more time to do in the inference, you can try to deal with very challenged problems, like system two reasoning and quitting thinking, science, math, physical problem, uh, and AI for science as well.
• 9:46
  SPEAKER_01: You think about that, it's not only the chipbot, but also this kind of quitting thinking, training, can lending from the crowd from the crowd to the edge.
• 9:56
  SPEAKER_01: What other application you can do you can bring in?
• 9:59
  SPEAKER_01: You can think that you have some assistant along with you at home, at very private sections, and even in lots of a critical area, you have a thinker go with you.
• 10:10
  SPEAKER_01: It will enable lots of exciting applications.
• 10:14
  SPEAKER_01: But one question people have is this is a building scale.
• 10:18
  SPEAKER_01: Is even trillion scale model, we're resource for that.
• 10:22
  SPEAKER_01: Is it really possible that we can bring that into the SI?
• 10:26
  SPEAKER_01: One year before I say, I don't know, but when I would say, yes.
• 10:31
  SPEAKER_01: I think you already experienced, I call the deep sick incident, uh, I think in early January 2025, uh this year.
• 10:40
  SPEAKER_01: So this is uh uh a company that brings lots of uh uh noise or brings lots of tensions in early 25.
• 10:49
  SPEAKER_01: But I think this is not a technology breakthrough, but uh we said that from the academic view, it's a great engineer implementation.
• 10:57
  SPEAKER_01: From their proposal, they bring the uh training and inference efficiency into the large energy model.
• 11:04
  SPEAKER_01: And it also intensifies the competition in the large learning model company.
• 11:09
  SPEAKER_01: You see more capacks and efficiency, but also reduces the price for the uh tokens.
• 11:16
  SPEAKER_01: But actually, I would say that the AGI community actually benefits a lot from this uh computation.
• 11:23
  SPEAKER_01: And it's through this competition that you the community, the large learning model community actually accelerates the development of some of small but very powerful models such as large language model and also even the VLM model that is multimodal combined with language, with vision, with speech, speech as as well.
• 11:44
  SPEAKER_01: And also, the model also embeds some, I said, very important quick thinking, planning, reasoning capabilities.
• 11:52
  SPEAKER_01: Most important, this model, we see some small model, a billion and two dozen billion, even a hundred billion model, and actually through many, many experiments show that this billion, hundred billion scale of model is a good fit for the edge device.
• 12:09
  SPEAKER_01: Even three billion model is a good edge device as well as well.
• 12:13
  SPEAKER_01: We don't have time to go through that, but actually lots of powerful model we can work on that.
• 12:18
  SPEAKER_01: But it also starts with uh trigger the decentralization, uh they migrate the technology from the car to edge.
• 12:26
  SPEAKER_01: Actually, this kind of decentralization happened before.
• 12:30
  SPEAKER_01: Uh internet age, that when the packet gets cheaper, it drives the decentralization into the the age.
• 12:37
  SPEAKER_01: It's also right now, token got cheaper.
• 12:40
  SPEAKER_01: I think it's also an opportunity for the age.
• 12:42
  SPEAKER_01: As Pete just mentioned that uh some the application require real-time uh privacy, even low latency, they require uh this kind of technology, and so high growth super role.
• 12:54
  SPEAKER_01: But I see that uh uh from the market public report, we saw that uh, at least from the SOC, uh their uh quoting report, we saw that the smartphone, PC, PC benefit a lot from the HGAI.
• 13:07
  SPEAKER_01: I would say this is just a starting point, it's just a starting point from that, even some ARGS, iPhone, etc.
• 13:13
  SPEAKER_01: And this potentially we see the home appliance.
• 13:17
  SPEAKER_01: Uh, I think uh Peter just mentioned the air conditioned cases as is also in this category, toys, education, office, knowledge management would benefit a lot.
• 13:27
  SPEAKER_01: And right now, lots of people pay attention really.
• 13:30
  SPEAKER_01: Is this possible that this kind of AI can migrate from the digital world into real world and really doing some actions that fall into robot and autonomy?
• 13:40
  SPEAKER_01: And this is with the other pillar we mentioned is autonomy and pillars.
• 13:45
  SPEAKER_01: In this pillar, most people care about the robot taxi and like require a lot of sensors, a lot of computing, and also huge capex investment.
• 13:54
  SPEAKER_01: But honestly, from my experience in the supply chain, in the automobile industry, and also workers with those uh OEMs and global OEMs, we know that the also many reports showing that the volume of Robot Taxi will be few, even in the coming 10 to 2035.
• 14:14
  SPEAKER_01: It's only a small volume.
• 14:16
  SPEAKER_01: Every year there's around 80 million new vehicle manufacturers in the year, but the majority for the autonomy will be L2 Plus.
• 14:25
  SPEAKER_01: It's I think many people already experienced that.
• 14:28
  SPEAKER_01: It's it's partial autonomy means that sometimes you can hand uh you can hands off, but you still have some attention for that.
• 14:35
  SPEAKER_01: But still, this L2 autonomy, plus autonomy already bring lots of convenience for many users along with the uh software define vehicle.
• 14:45
  SPEAKER_01: We need to bring better experience onto the diesel carpet and also some safety requirements.
• 14:51
  SPEAKER_01: So we can say that LO1, LO2, even L2 Plus will occupy huge potentials.
• 14:57
  SPEAKER_01: You so you see that it's kind of transition from the distributed uh issues into centralized SPC, it requires more connection, more memory, more uh uh requirement on that.
• 15:09
  SPEAKER_01: And the automotive experience actually can help a lot for the robot industry as well, because it's timing that how we have this bot can go into the the real world.
• 15:23
  SPEAKER_01: It's not only uh in the digital world for the reasoning, for the perception only.
• 15:27
  SPEAKER_01: It needs to turn to the action, you need to do grasping, it needs to the planning, you need to do navigation, you need to uh um uh interact with the real world.
• 15:36
  SPEAKER_01: But the real world is so dynamic, it's so challenging.
• 15:39
  SPEAKER_01: And there's really quite strong needs, even I I go I involve some national program that because of the aging of people and shortage of labors, the service healthcare logists have strong needs for this kind of robot.
• 15:55
  SPEAKER_01: And the same thing is that if we really can really scale out the deployment for the robot, we need the certain type of generation capability, multitasking capability, as we see from the large language model in the digital world.
• 16:11
  SPEAKER_01: Can we really reach that in the coming years?
• 16:14
  SPEAKER_01: I think they are really quite attempt.
• 16:16
  SPEAKER_01: They try to bring the large language models, VLM model, and to help the robot world.
• 16:21
  SPEAKER_01: And I would say that's the key to unlock these critical capabilities.
• 16:29
  SPEAKER_01: I just showed some few work from my lab.
• 16:32
  SPEAKER_01: So we are working on that uh in the past 10 years and try to uh bring the uh have a smart brand for robots, so we saw that the Last Rail model can help even have prone to uh guide the robots to do some specific tasks.
• 16:48
  SPEAKER_01: And also the VLO model can help not only build a map for the robots, it's helped to navigate the world, even doing the search, but also the current technology can help uh real time build up very, very dense map and keep in mind that these things all can be done, have to be done in the edge side.
• 17:10
  SPEAKER_01: And because this we need to do that in real time.
• 17:12
  SPEAKER_01: And also the large learning models, LLM can also help bring the common sense to the robot, train the robot, and they have some common sense as they navigate in the uh real world.
• 17:22
  SPEAKER_01: And also data is always a big issue.
• 17:25
  SPEAKER_01: How you much data you need to train the robot.
• 17:29
  SPEAKER_01: I think the LM model benefits from using the tons of free data, conducting unsupervised uh trainings.
• 17:36
  SPEAKER_01: Is it possible that in robot world we can also have this kind of unsupervised learning?
• 17:42
  SPEAKER_01: Just take the uh video data and put it in the training space and robot can learn from that.
• 17:48
  SPEAKER_01: Um so I show you uh quite quick demos.
• 17:52
  SPEAKER_01: This is we did it five years before.
• 17:54
  SPEAKER_01: So you can see we use prompt to drive the machine to finish the task we tend to have.
• 18:00
  SPEAKER_01: So they can understand the spatial relationship, um, they can uh grasp any kind of object, uh, even though they don't recognize the object.
• 18:08
  SPEAKER_01: But they already have this kind of capability.
• 18:12
  SPEAKER_01: So this is technology have been, we did it five years before.
• 18:15
  SPEAKER_01: You can imagine that right now, the five years, the large unit model evolve a lot.
• 18:19
  SPEAKER_01: They will bring more impact.
• 18:21
  SPEAKER_01: The final demo I will show you that is it possible to run through this regular video to the YouTube video and try to find the locomotion navigation and try to teach the robot for that.
• 18:34
  SPEAKER_01: And actually, this through two or three videos, the robot can easily learn how to navigate in the real environment, even the robot in different forms.
• 18:43
  SPEAKER_01: So this is called cross embodiment learning.
• 18:46
  SPEAKER_01: So I will show you, yes, that's really quite invest improvement in this technology, help the robot to learn the real world.
• 18:54
  SPEAKER_01: But people will ask questions is really we can bring the large language model, fraudulent model into the robot, and then they can then mitigate in the real world, and also have this kind of generation multitasking capability as the ChatGPT, if that's really great, they will be imposed lots of opportunity for that.
• 19:15
  SPEAKER_01: I would say yes, the community already working a lot for that.
• 19:20
  SPEAKER_01: I think this since the end of uh of 2020, I think also motivated by the ChatGPT that uh started introducing the RT1, RT2, RTX, and also the open community, uh open source community introduced the open VOA.
• 19:36
  SPEAKER_01: Recently, the uh staff called uh Physics AI, they introduced Pi Zero and Pi O5.
• 19:43
  SPEAKER_01: I would say they're really uh uh quite inspiring.
• 19:46
  SPEAKER_01: You see that, you can try to see the Pi O5 demo.
• 19:50
  SPEAKER_01: They try to train the model in the virtual world and bring the the robot into different homes they rented from Airbnb.
• 19:59
  SPEAKER_01: It can follow the comment to clean the kitchen and clean the bedroom, etc.
• 20:05
  SPEAKER_01: So we saw is apparently some generation uh multitasking can be already go into these kind of solutions.
• 20:15
  SPEAKER_01: Even though it's, I think this is still a little distance from the real application, but I think current technology can already help a lot in the closed environment, in the hospital, in the industry, in many side, really bring some really flexibility, intelligence, and efficiency to the robot already.
• 20:34
  SPEAKER_01: We can use that already.
• 20:35
  SPEAKER_01: But the other question is but is it's really a training scale model or even hundreds of something billion a model for that?
• 20:44
  SPEAKER_01: No, this model you can get from open source is three to fifty uh five billion.
• 20:49
  SPEAKER_01: Even the model I mentioned, Pi 05, is three billion only.
• 20:52
  SPEAKER_01: So easily you can take that to uh implement in the HSI or radio.
• 21:00
  SPEAKER_01: But very different from the uh digital world, the ChatGPT sticker they have, uh they will make mistakes.
• 21:07
  SPEAKER_01: They will do hallucinations.
• 21:09
  SPEAKER_01: But the hallucinations in the real world would cause lots and lots of risk.
• 21:14
  SPEAKER_01: You can imagine that if the autonomous driving robot taxi or the robot hallucinate in the real world, what might happen?
• 21:24
  SPEAKER_01: But so that's why in autonomic industry, automatic industry, they have very strong regulation.
• 21:30
  SPEAKER_01: They have a lot of standards, compliance.
• 21:33
  SPEAKER_01: For example, how you make sure the functional safety or the quality in the long term, and also safety in lots of scenarios, in design, and use of cybersecurity and for that.
• 21:44
  SPEAKER_01: And actually, those experiences going through in the automatic industry, standards in the automotive industry can also help a lot for the robot development as well.
• 21:55
  SPEAKER_01: I would say that this is quite early days, it's it's really coming for this uh foundation model into the robot worlds.
• 22:04
  SPEAKER_01: But I would say that the robot will be in many, many kinds of forms that it will be out of our imaginations.
• 22:11
  SPEAKER_01: The next pillar I would talk about is the energies parts.
• 22:17
  SPEAKER_01: Why energies?
• 22:19
  SPEAKER_01: That we have uh quite uh uh surging energy investment because the AI requires energy and the the energy uh sectors uh uh is quite uh uh developing with introduction of renewable energy, the grid needs to be uh uh upgraded and the generation from centralization into uh uh decentralization and also the sustainability needs for IE100 or 24-7 CFE.
• 22:48
  SPEAKER_01: But I wanted to emphasize that the HGAI actually played a very, very critical role to bring the efficiency and also the resiliency for the energy uh network, even energy at home at a uh factory.
• 23:02
  SPEAKER_01: This is the best story that we we all know that in April it happened in Spain and Portugal because of some issue that uh made the uh power outage.
• 23:13
  SPEAKER_01: It caused lots of problems.
• 23:14
  SPEAKER_01: But actually, through many, many analyses, they found that if you bring additional uh battery and also some smart control, intelligent control here, actually we can avoid even mitigate these uh outages.
• 23:27
  SPEAKER_01: That's why uh we also uh we have uh this uh company really right now operate uh management uh 5.5 gigawatt over the world, and we daily we process 1.3 billion data points in the age, also in the cloud.
• 23:42
  SPEAKER_01: We operate uh more than uh, I think it's it's 13 countries uh right now and manage more than 10,000 uh data centers.
• 23:49
  SPEAKER_01: Uh one of that, like this, this is the largest data center, uh the solar panel power plant in Taiwan.
• 23:55
  SPEAKER_01: This is one of the sites we management uh uh right now.
• 23:58
  SPEAKER_01: You can imagine lots of age activities.
• 24:00
  SPEAKER_01: Some it's not only monitoring, sometimes we need to control as well.
• 24:04
  SPEAKER_01: But this control also incurs lots of risk.
• 24:07
  SPEAKER_01: This is a final example I want to show you that that should actually, one of the key uh critical applications either we can use in the in the data centers, use in the factory or use it at home is the so-called battery storages that can try to bring the resilience, even the cost uh reduction uh in the in the in at home or in the factory or in in the uh utility scale data center.
• 24:34
  SPEAKER_01: Then require to integrate the energy system storage with the grid, with the power generation, uh solar, uh wind, or EV charging, battery storage.
• 24:43
  SPEAKER_01: So we need require dynamic inter control, try to optimize, release, optimize the cost.
• 24:49
  SPEAKER_01: And we found that we just pilot a project that this kind of technology requires reinforcement, the one that FIGO use in the HSI.
• 24:59
  SPEAKER_01: So we already deployed that in some of our customers in Japan, that we try to use this AI technology, try to bring the efficiency and resilience for the energy sector.
• 25:11
  SPEAKER_01: This is my last slide.
• 25:12
  SPEAKER_01: So what we I want to highlight that it's just the start of HGAI community, and we found the critical pillar we can uh contribute the most for AGI community is energy, AI and autonomy.
• 25:26
  SPEAKER_01: We need to think that all consider all together.
• 25:29
  SPEAKER_01: You can see there are lots of opportunity.
• 25:31
  SPEAKER_01: We don't have time to cover that, you see lots of opportunity within this one.
• 25:35
  SPEAKER_01: And right now, the technology is already moving from the centralizations, official AI to the distribution.
• 25:42
  SPEAKER_01: And and the I assure you that the current model is already powerful enough and it's is it's engineer-friendly for HAI already.
• 25:55
  SPEAKER_01: But for a company that's designed a product, yeah, we don't know if it really could fit post-customers.
• 26:01
  SPEAKER_01: So my experience is also showing that we need to design lots of POCs and try and error with the system company.
• 26:08
  SPEAKER_01: And but we need to keep in mind that this, when the HDI go into real world, conduct lots of control, lots of action, it's not only a digital world.
• 26:16
  SPEAKER_01: So we need to avoid the hallucination in the real world.
• 26:19
  SPEAKER_01: So the action might turn to some mistakes and risk, and how to mitigate the risk is a huge concern.
• 26:25
  SPEAKER_01: Safety, cybersecurity, and also the other consideration need to be required for SOC or citizen company as well.
• 26:32
  SPEAKER_01: So that's why we have this HGI foundation and we can work together for this opportunity and changes.
• 26:39
  SPEAKER_01: Thank you.