Latest / AI Fire Daily / #127 Robin: This 1.7B Local AI Runs Agents, Vision, Voice - Even on an $80 Arduino

Transcript

#127 Robin: This 1.7B Local AI Runs Agents, Vision, Voice - Even on an $80 Arduino · AI Fire Daily

• 0:00
  A tiny AI was asked to give a browser OS more
• 0:03
  pizzazz, but it completely misunderstood the
• 0:06
  assignment. It decided to build a functional
• 0:09
  pizza browser OS instead. Seriously. It is a
• 0:13
  genuinely hilarious mistake. But here is the
• 0:16
  truly crazy part. That tiny 1 .7 billion parameter
• 0:20
  model did actual real work. Right. It built working
• 0:24
  interfaces. It edited system files autonomously.
• 0:27
  And it did all of this on a standard everyday
• 0:30
  laptop. Welcome to today's deep dive. We are
• 0:33
  exploring Liquid AI's new LFM3 family. Specifically,
• 0:37
  we are looking at their 1 .7 billion parameter
• 0:40
  models. Which are incredibly small, you know,
• 0:43
  by today's standards. Exactly. We want to know
• 0:45
  what happens when you ditch expensive GPUs. We
• 0:48
  are running AI entirely locally, completely offline.
• 0:52
  We even test it on an $80 Arduino board. It is
• 0:56
  absolutely wild to think about. we are putting
• 0:58
  highly capable AI on budget hardware. That is
• 1:02
  computing power cheaper than a good pair of sneakers.
• 1:05
  Yeah, it really is. The accessibility implications
• 1:07
  here are truly massive for everyone. But before
• 1:10
  we give this AI eyes and ears, we need to test
• 1:12
  its raw brain power on a normal laptop. Right.
• 1:15
  So the LFM3 family includes five distinct models.
• 1:20
  They handle text, vision, audio, Omni, and decision
• 1:24
  tasks. each model sits right around that 1 .7
• 1:28
  billion parameter mark. That is a tiny footprint.
• 1:31
  The text model handles the heavy logical lifting,
• 1:34
  but a few specific specs really caught my eye
• 1:36
  here. It was trained on 22 trillion tokens? Which
• 1:40
  is just a staggering number. It really is. Let's
• 1:43
  define a token quickly for you. A token is basically
• 1:46
  a tiny piece of a word. That is a massive amount
• 1:49
  of data for something this small. It completely
• 1:51
  flips traditional machine learning training on
• 1:54
  its head. Usually developers train small models
• 1:56
  on much smaller data sets. Right, they scale
• 1:59
  the data to the model size. Exactly. But Liquid
• 2:02
  AI basically overtrained this tiny neural network
• 2:05
  on purpose. Why would they force so much data
• 2:07
  into a tiny model? Well, it forces the model
• 2:10
  to compress the underlying logic severely. It
• 2:14
  is like Boiling down billions of books into a
• 2:17
  thick dense syrup. Oh, I like that analogy. Yeah,
• 2:20
  you keep the pure concepts but you lose the exact
• 2:24
  verbatim text. That makes a lot more sense. It
• 2:27
  also has a 256k context window, which is the
• 2:30
  AI's short -term memory limit during a single
• 2:32
  chat session. And handling that much memory usually
• 2:35
  requires giant corporate servers. Right, and
• 2:37
  seeing that on a local model is staggering. You
• 2:41
  can feed it entire software code bases at once.
• 2:44
  Yeah, or it can read massive instruction manuals
• 2:46
  in a single prompt. They also provide official
• 2:49
  gguf quantizations for developers. And gguf is
• 2:53
  just a file format. It makes running local models
• 2:55
  much easier. But we should mention one major
• 2:57
  catch here. The licensing, right? Yeah. The LFM3
• 3:01
  uses a proprietary license structure. It is not
• 3:04
  open source like MIT or Apache. Commercial developers
• 3:08
  definitely need to check those terms carefully.
• 3:10
  But let's look at the actual testing environment.
• 3:13
  The test used a standard Microsoft Surface laptop.
• 3:16
  Just a normal, everyday work laptop. Exactly.
• 3:19
  It had 64 gigabytes of unified memory running
• 3:22
  on the CPU. Unified memory just means the CPU
• 3:25
  and graphics share the same RAM pool. But this
• 3:28
  was running strictly on the CPU, right? Yep.
• 3:30
  There was no heavy graphics card involved at
• 3:32
  all. Right. And CPU processing is typically much
• 3:36
  slower for artificial intelligence, but memory
• 3:38
  bandwidth matters much less for these tiny models.
• 3:42
  We ran the text model at Q8 quantization. Let
• 3:45
  me jump in there. Quantization means shrinking
• 3:47
  the model's file size to run on slower, cheaper
• 3:49
  hardware. Exactly. And Q8 is a high -quality
• 3:53
  compression level. It shrinks the model without
• 3:55
  making the AI too stupid. Right. So basic chat
• 3:59
  responses were fast and surprisingly fluid, but
• 4:02
  we wanted to push it much harder. Yeah, we really
• 4:04
  wanted to break it. The first coding test was
• 4:07
  building a basic browser OS. The final interface
• 4:11
  was pretty simple, but the buttons worked correctly.
• 4:14
  That is impressive for a CPU -only local model.
• 4:17
  It really is. But then things got significantly
• 4:19
  more complex. We tested it using the Hermes agent
• 4:23
  framework. This tool lets AI interact directly
• 4:26
  with your computer's files. Right, and this is
• 4:28
  where it gets really interesting. It doesn't
• 4:31
  just output text onto a screen anymore. It actually
• 4:34
  executes commands in your terminal directly.
• 4:37
  It becomes an active participant in your operating
• 4:39
  system. So first, the LFM3 successfully hunted
• 4:43
  down specific hidden files. It found a file after
• 4:46
  it was intentionally moved to the desktop. Which
• 4:48
  is huge. That requires the AI to actively search
• 4:51
  the directory structure. Yeah, it is not just
• 4:53
  guessing. Right. Next, it started editing the
• 4:56
  code it had just written. It read its own browser
• 4:59
  OS codebase independently. And it successfully
• 5:02
  added a functional taskbar and a settings menu.
• 5:05
  It even built a working calculator interface
• 5:07
  from scratch. Though it built the calculator
• 5:10
  completely without a number pad, which is hilarious,
• 5:13
  but it highlights a specific AI weakness. Spatial
• 5:16
  reasoning in code is historically very hard for
• 5:19
  text -based models. Yeah, they struggle to visualize
• 5:22
  the layout. And, of course, it confused pizzazz
• 5:26
  with pizza. I still wrestle with prompt drift
• 5:30
  myself, so seeing this tiny model handle autonomous
• 5:34
  agent tasks is wild. The autonomous looping is
• 5:38
  the actual magic part here. The agent architecture
• 5:40
  forces the AI to observe, act, and evaluate.
• 5:44
  It tries something, checks the result, and adjusts
• 5:47
  its approach. Exactly. Let's look at the system
• 5:50
  permission test as an example. It tried to write
• 5:53
  a file outside its designated workspace. The
• 5:56
  operating system naturally threw a permission
• 5:58
  denied error. A normal script would just crash
• 6:01
  and give up right there. But this agent read
• 6:03
  the error. It understood the problem. Yeah. It
• 6:06
  smartly saved the updated file to a permitted
• 6:09
  scratch location instead. That shows a level
• 6:12
  of dynamic problem solving you rarely see. It
• 6:15
  also conducted independent online research perfectly.
• 6:19
  It used web searches to gather external information.
• 6:22
  Right. It created a Wikipedia -style personal
• 6:24
  profile with references. Some of the personality
• 6:27
  analysis was definitely a bit questionable, but
• 6:30
  the agent workflow functioned exactly as intended.
• 6:34
  So does a Does 256K context window even matter
• 6:38
  if the model isn't smart enough to use it? Well,
• 6:41
  having massive memory doesn't guarantee deep
• 6:44
  comprehension. It will recall every detail you
• 6:46
  provide flawlessly. But if the instruction is
• 6:49
  too nuanced, it will execute the wrong logic
• 6:52
  entirely. It remembers everything perfectly,
• 6:55
  but might completely misunderstand your actual
• 6:57
  complex instructions. Got it. Exactly. It has
• 7:01
  the data, just not always the wisdom. Since we
• 7:05
  know the text model struggles with nuanced instructions,
• 7:07
  let's change gears. What happens when we force
• 7:09
  it to look at the real world? Multimodal capabilities
• 7:13
  make local LLMs vastly more useful. We tested
• 7:16
  the vision model with four real -world images.
• 7:19
  We deliberately chose tricky visual scenarios
• 7:22
  to test its limits. Right. The first test was
• 7:25
  a photo of a Westie dog. The AI correctly identified
• 7:28
  the specific dog breed, but it completely missed
• 7:31
  the emotion of the animal. The dog was... clearly
• 7:33
  upset in the photo. Yeah, but the AI thought
• 7:35
  the dog looked completely relaxed. It failed
• 7:38
  to read the nuanced emotional context. Why does
• 7:41
  that happen? I mean, it can identify the breed
• 7:43
  perfectly. Well, vision models map geometric
• 7:46
  patterns and textures. They learn the pixel arrangement
• 7:49
  of a west -east fur. Right, the physical shape.
• 7:52
  Exactly. But they do not possess actual empathetic
• 7:55
  context. Recognizing a shape is easy. Interpreting
• 7:59
  a feeling requires complex human intuition. That
• 8:03
  makes perfect sense. It sees the pixels, not
• 8:05
  the mood. Yep. The second image was a Subaru
• 8:08
  WRX meme. The model understood part of the visual
• 8:12
  joke. But it completely missed the head gasket
• 8:15
  humor. Right. That requires highly specific automotive
• 8:19
  cultural knowledge. Yeah, multimodal models struggled
• 8:22
  to cross -reference image recognition with niche
• 8:25
  cultural text data. It recognized the car, just
• 8:28
  not the internet joke associated with it. Next,
• 8:31
  we showed it a Toshiba Cosmio laptop. It recognized
• 8:33
  the laptop by reading its label. But it actually
• 8:36
  misspelled the name in its final output. It also
• 8:39
  struggled heavily with the tiny chassis stickers.
• 8:41
  It could not decipher the small logos accurately.
• 8:44
  Small text and blurry images is really tough
• 8:47
  for a tiny parameter model. But the final image
• 8:50
  was a carfuse manual. This result was actually
• 8:53
  incredibly useful and practical. Yeah, this was
• 8:55
  a huge win. It converted a blurry photo into
• 8:59
  a clean, readable table. Imagine taking a photo
• 9:02
  of a dense manual with your phone. Exactly. You
• 9:05
  can have an offline AI formatted instantly. Optical
• 9:09
  character recognition is a major strong point
• 9:11
  for this model. Reading a fuse box perfectly
• 9:16
  but missing a dog's emotion is functionally weird.
• 9:19
  Right, and it shows how these models actually
• 9:21
  see the world. They are excellent at finding
• 9:24
  rigid patterns, like formatted text, but they
• 9:27
  struggle with fluid concepts. like animal emotions
• 9:30
  or humor. You still need to double -check important
• 9:32
  details constantly. Since it struggles with visual
• 9:36
  subtext, I'm curious about audio. Does that lack
• 9:39
  of nuance bleed into how it processes human voices?
• 9:42
  Well, the audio model functions as a voice assistant
• 9:45
  without internet. It features built -in voice
• 9:47
  activity detection technology. This means it
• 9:50
  listens and responds automatically to your voice.
• 9:52
  You do not have to press a button each time you
• 9:54
  speak. Right. The conversation flow felt incredibly
• 9:57
  quick and snappy. But accuracy was an entirely
• 10:01
  different story. Yeah, it suffered from some
• 10:03
  very confident hallucinations. It completely
• 10:06
  invented specifications for a MacBook Pro. It
• 10:09
  claimed the laptop had Apple Pencil support.
• 10:11
  It even confidently made up a fake phone number.
• 10:14
  It presents false information with absolute unwavering
• 10:17
  certainty. That happens a lot with these small
• 10:19
  models. But why does an offline model just invent
• 10:22
  a phone number? When an offline model lacks internet
• 10:25
  grounding, it defaults to statistical probability.
• 10:28
  Okay, what does that mean in practice? It knows
• 10:31
  what a phone number looks like structurally.
• 10:33
  So it just generates a sequence of numbers that
• 10:35
  fits the pattern. Oh, I see. It does not know
• 10:38
  the number is fake. It just knows the format
• 10:39
  is right. That is slightly terrifying, honestly.
• 10:43
  It also severely misunderstood human speech during
• 10:46
  testing. Yeah, when asked to draft a simple message,
• 10:48
  it failed. It completely reversed the meaning
• 10:51
  of a key sentence. It also lacked a vital conversational
• 10:54
  feature. It could not be interrupted while it
• 10:57
  was speaking. Right. You literally have to wait
• 11:00
  for it to finish its entire thought completely.
• 11:02
  That makes the interaction feel incredibly rigid
• 11:05
  and unnatural. Humans interrupt each other constantly
• 11:08
  for clarification. We really do. Still, having
• 11:11
  a working offline voice assistant is an impressive
• 11:13
  baseline. We also tested the OmniModel capabilities.
• 11:17
  The OmniModel processes text, images, video,
• 11:20
  and audio simultaneously. Right. And it can respond
• 11:23
  using either text or synthesized speech output.
• 11:27
  We tested it on a short promotional video clip.
• 11:30
  It was an advertisement for the open source Perry
• 11:32
  chatbot. The model responded surprisingly fast
• 11:35
  to the video feed. It understood most of the
• 11:38
  visual content perfectly, but it completely misread
• 11:41
  the text saying to join the wait list. Also,
• 11:44
  the audio understanding completely failed in
• 11:47
  this specific test. Yeah, but the audio failure
• 11:49
  was likely a web UI issue, though. The custom
• 11:52
  interface simply did not pass the audio correctly
• 11:54
  to the model. But the core video understanding
• 11:57
  was highly solid for its tiny size. So, is an
• 12:02
  offline voice assistant actually useful if it
• 12:05
  confidently invents fake phone numbers? Well,
• 12:08
  it depends entirely on what you ask it to do.
• 12:10
  It is fantastic for controlling local devices
• 12:13
  where facts don't matter. But you shouldn't use
• 12:16
  it for research or data retrieval. Great for
• 12:19
  offline smart home commands, but terrible for
• 12:21
  factual research or heavy tasks. That is a very
• 12:24
  fair assessment. Okay, so if LFM3 runs decently
• 12:29
  on a normal laptop, how far down the hardware
• 12:31
  ladder can we push it? Right. Let's completely
• 12:34
  ditch the PC. Let's see where this technology
• 12:37
  actually breaks. This is where things get truly
• 12:40
  fascinating. Can a 1 .7 billion parameter AI
• 12:44
  run on an $80 microcomputer? Can it do this entirely
• 12:47
  without internet access or massive servers? The
• 12:50
  answer is, surprisingly, yes. We tested it on
• 12:53
  an Arduino Uno Q setup. This tiny board uses
• 12:56
  a Qualcomm Dragonwing chip? Let's explain what
• 12:58
  that hardware actually is. The Qualcomm Dragonwing
• 13:01
  is a highly efficient mobile processor. It is
• 13:04
  designed specifically for low -power robotics
• 13:06
  and edge devices. And it is paired with just
• 13:08
  four gigabytes of LPDDR4 RAM. That is significantly
• 13:13
  less memory than a cheap smartphone. Oh, way
• 13:15
  less. Let's break down the exact hardware costs
• 13:17
  involved here. The Arduino board itself costs
• 13:19
  $80. You also need a $10 USB -C adapter. And
• 13:23
  you need a $20 power brick to run it. The total
• 13:26
  cost comes out to $110. You will need a standard
• 13:30
  monitor and a keyboard, too, of course. Right.
• 13:32
  But you can reuse old peripherals you already
• 13:34
  have sitting around. To make the model actually
• 13:36
  fit, we use quantization again. We used an extreme
• 13:40
  four -bit quantization for this specific test.
• 13:42
  This level of compression is absolutely essential
• 13:45
  for devices with limited RAM. Yeah, it drastically
• 13:48
  reduces the mathematical precision of the model's
• 13:51
  internal weights. It essentially chops off the
• 13:53
  decimal points to save space. Right. But how
• 13:56
  fast does it actually run on the board? The Arduino
• 13:59
  manages around three tokens per second. A token
• 14:02
  is roughly three quarters of a standard English
• 14:04
  word. That is undeniably slow for a long, complex
• 14:08
  conversation. It feels like watching someone
• 14:11
  type very, very methodically. It is painfully
• 14:14
  slow for real -time human chat, but the incredibly
• 14:17
  important part is that it actually works. Well,
• 14:21
  imagine scaling this to a billion cheap household
• 14:24
  edge devices. That is a staggering thought. You
• 14:27
  are putting localized brains into everyday dumb
• 14:30
  appliances. Six years ago, this concept would
• 14:33
  have sounded completely crazy. Running functional
• 14:35
  AI on an $80 computer was pure science fiction.
• 14:39
  And now it is literally sitting on a desk working.
• 14:42
  Yes, you sacrifice raw speed. And you lose some
• 14:46
  intelligence. But you gain a private, entirely
• 14:50
  offline AI assistant. Exactly. It never sends
• 14:53
  your data to a corporate server. You avoid expensive
• 14:56
  hardware purchases entirely. You also eliminate
• 14:59
  all ongoing cloud API costs forever. But at a
• 15:05
  painfully slow three tokens a second, isn't this
• 15:08
  just a frustrating tech novelty? Well, when you
• 15:11
  are building a smart home system, latency is
• 15:14
  secondary. If your goal is zero data leakage,
• 15:17
  you gladly accept the slower speed. Speed doesn't
• 15:20
  matter when your priority is absolute privacy
• 15:22
  and zero ongoing cloud costs. Exactly. That makes
• 15:26
  perfect sense for many users. The latency is
• 15:28
  a fair trade for complete digital sovereignty.
• 15:31
  Let's synthesize the main takeaway for you today.
• 15:33
  Liquid AI's LFM3 isn't trying to dethrone massive
• 15:36
  cloud models. It is not competing with GPT -4
• 15:39
  or Claude directly. It serves an entirely different
• 15:42
  purpose in the AI ecosystem. Right. It is proving
• 15:44
  that highly capable, private AI is now accessible.
• 15:48
  Everyday people can run it on standard, affordable
• 15:50
  hardware. We must acknowledge the current warts
• 15:53
  and flaws, obviously. Yeah, we saw the confident
• 15:56
  hallucinations with the fake phone numbers. And
• 15:58
  the optical character recognition is still imperfect
• 16:01
  on tiny stickers. The coding results are sometimes
• 16:03
  clumsy, like the missing numpad. Right, and the
• 16:06
  proprietary licensing is a drawback for commercial
• 16:08
  developers. Sure. But we absolutely cannot let
• 16:12
  those flaws obscure the bigger picture here.
• 16:15
  We are witnessing a monumental shift in computing
• 16:18
  infrastructure right now. Running a functional
• 16:20
  AI chatbot on an $80 offline board changes everything.
• 16:25
  It completely decentralizes where artificial
• 16:27
  intelligence can live and operate. It removes
• 16:30
  the massive financial barrier to entry for experimentation.
• 16:33
  You do not need thousands of dollars in server
• 16:35
  equipment anymore. You just need curiosity and
• 16:38
  a basic laptop computer. Or just a hundred bucks
• 16:40
  and a tiny Arduino board. The democratization
• 16:43
  of this technology is happening incredibly fast.
• 16:46
  These small, highly efficient models are going
• 16:48
  to become ubiquitous very soon. Absolutely. We
• 16:52
  want to leave you with a final, unmentioned thread
• 16:55
  to mull over. Six years ago, this level of local
• 16:58
  AI was unthinkable. Right. If an $80 board can
• 17:01
  act as an autonomous offline agent today, What
• 17:06
  happens to our privacy and daily routines when
• 17:08
  every appliance in your house has its own localized
• 17:12
  1 .7 billion parameter brain quietly making decisions
• 17:16
  in the background? Thank you for joining us on
• 17:20
  this deep dive. We appreciate your time and your
• 17:23
  curiosity today. Take care and keep asking the
• 17:25
  big questions.