Latest / Black Holes and Bubblegum / Voice Commands to Robotic Fingers
Transcript
- 0:00Okay, so, um, picture this.
- 0:03You want a robot hand to pick up this paper cup on my desk,
- 0:06right?
- 0:07But instead of flexing your arm muscles, you just say it out loud.
- 0:11Wait, so you just speak to it?
- 0:13Because I thought real bionic arms, like, like the Open Bionics Hero Arm,
- 0:17didn't they rely on muscle sensors?
- 0:19Yeah!
- 0:20Exactly!
- 0:21Real prostheses like the Hero Arm use electromyography, or EMG sensors.
- 0:27They, uh, they stick to your residual limb and detect tiny electrical signals,
- 0:33like micro-volts from your muscles tensing, and then turn that into movement.
- 0:38Micro-volts, wow.
- 0:39But for your invention fair tabletop model, you are replacing those muscle signals
- 0:42with spoken words.
- 0:44Right!
- 0:44It is like a relay race with four runners passing a baton.
- 0:48If one runner drops it, the fingers do nothing.
- 0:52A relay race, okay walk me through runner number one.
- 0:54Runner one is voice input!
- 0:58Sound waves enter speech recognition software on the computer and turn into text.
- 1:04Then runner two, the AI agent, looks at the text and picks a command,
- 1:09like GRASP.
- 1:11Got it.
- 1:11So text to command.
- 1:12Who gets the baton next?
- 1:13Runner three is the microcontroller, like an Arduino board!
- 1:18The computer sends that command over a serial connection to the Arduino.
- 1:23And runner four, the Arduino tells the servo motors in my uHand UNO kit to
- 1:30rotate!
- 1:31And those motors pull strings or tendons that curl the plastic fingers around the
- 1:35paper cup.
- 1:36Yes!
- 1:36But, er, watch what happens when runner three stumbles.
- 1:39Wait, what is it doing right now?
- 1:41It is just, uh, hovering right over the cup!
- 1:43See?
- 1:44That was a signal lag between the computer and the controller!
- 1:48The computer sent GRASP, but the serial message got delayed,
- 1:52so the servos never turned, and the hand was just frozen in mid-air!
- 1:57It looks like it is contemplating the cup instead of grabbing it!
- 2:01I know!
- 2:01That is why testing every handoff in the relay is so important.
- 2:06So if the computer is translating what you say, why not just let an advanced AI
- 2:12model figure out how to move the fingers directly?
- 2:15Oh, no, no, no, that would be dangerous!
- 2:17In AI research, like the work from Anthropic, an agent is defined as a system where
- 2:24a model chooses steps or tools to reach a goal.
- 2:27But you do not want an AI inventing raw finger movements on the fly.
- 2:32Right, because if it invents a move, it might crush the cup or snap a mechanical
- 2:36tendon.
- 2:37Exactly!
- 2:38So my AI agent works like a dispatcher.
- 2:41It can only pick from a strict, approved action menu that I coded into the
- 2:46microcontroller.
- 2:47So what is on this dispatcher menu?
- 2:49Just four safe choices: OPEN, GRASP, RELEASE, and ASK.
- 2:55If a person says hold the cup, or grab this, or do not drop it,
- 3:00the agent translates all those different human phrases into GRASP.
- 3:05Ah, clever!
- 3:06What if someone says something confusing, like, do the thing?
- 3:08Then it triggers ASK, and asks which action you mean!
- 3:13And if there is background noise, it chooses to do nothing at all and stays
- 3:18completely still.
- 3:19That makes total sense for safety.
- 3:22Now, how are you testing all these spoken commands for the science fair?
- 3:26Are you getting your friends to talk to it?
- 3:28I, I wanted to!
- 3:30But the school packet has a strict rule: no testing on human subjects without prior
- 3:35approval!
- 3:36So I had to pivot.
- 3:38Oh, right!
- 3:39Science fair rules can be tricky.
- 3:41So how do you test speech recognition without human test subjects?
- 3:45I am using computer generated text to speech audio files!
- 3:49I play pre-recorded synthetic voices into the microphone to run standardized tests.
- 3:55That is so smart, Santi!
- 3:57It keeps the test consistent every single time.
- 3:59Yeah!
- 4:00I log every trial in my inventor notebook.
- 4:04If an action fails, I document why, fix the code, and try again.
- 4:09Science fair judges love seeing how you learn from failure!
- 4:14Well, you have built a clear, safe design plan from voice command to moving fingers.
- 4:19Good luck at the fair, inventor!