Latest / The Windows Podcast with Fexingo: Microsoft, PC, and Enterprise Windows Conversations / How Windows 11 Is Changing Enterprise Keyboard Firmware
Transcript
- Lucas: So here's a scenario that landed in my inbox last week from an IT director at a mid-sized financial services firm. They rolled out Windows 11 to their trading floor — maybe 400 seats — and within 48 hours, every single keyboard on that floor stopped working. Not the OS ignoring the keyboard. The keyboards themselves. Bricked. Luna: Wait — bricked from a Windows update? I've heard of driver issues, but the actual keyboard firmware? Lucas: Exactly. And that's the story I want to unpack today. Because what happened here isn't a one-off glitch. It's a sign of a much bigger shift in how Windows 11 handles input devices — specifically, how keyboard firmware updates are now routed through the operating system in a way that Windows 10 never did. Luna: If today's conversation saves one IT admin from that kind of Monday morning, it's worth it. And it's a good reminder that this show exists because listeners like you help keep it going — no ads, just a small group of folks chipping in at buy me a coffee dot com slash fexingo. That's what lets us dig into stories like this. Lucas: Yeah, and we're grateful for that. So — back to those keyboards. The core issue is that Windows 11, starting with version 22H2 and continuing through 24H2, has been phasing out the legacy PS/2 driver stack in favor of a pure USB HID — Human Interface Device — path. That sounds like a boring technical detail, but it has real consequences. Luna: Because USB HID allows the operating system to talk directly to the keyboard's firmware, not just the keys. Lucas: Right. On Windows 10, if you plugged in a USB keyboard, the OS saw it as a generic HID device. But any firmware updates — say, to fix a key chatter issue or add a new shortcut — were handled by a vendor-specific utility that ran outside of Windows. It might flash the firmware over a separate USB endpoint, but it wasn't managed by the OS. Luna: So Windows 11 is now inserting itself into that process. Lucas: Yes. Microsoft introduced something called Windows Update for Firmware, which lets device manufacturers push firmware updates through Windows Update just like a driver. For keyboards, that means the firmware payload now travels through the OS, through the HID stack, and into the keyboard's microcontroller. It's elegant in theory — centralized updates, no third-party updaters. Luna: But in practice, the trading floor story suggests it's fragile. Lucas: Fragile is one word. In that firm's case, the keyboard vendor — I'll call them Vendor X — had signed a firmware update that was supposed to fix a USB wake-on-key issue. The update was delivered through Windows Update as a critical patch. But the firmware binary had a checksum error that wasn't caught during validation. When it flashed, it corrupted the keyboard's bootloader. Luna: So the keyboard became a paperweight. And you can't fix that remotely — you have to physically replace it or reflash the chip with a programmer. Lucas: Exactly. And on a trading floor, that's catastrophic. They had to pull 400 keyboards over the weekend, source replacements, and send the old ones back to the vendor for reflashing. Cost them something like 80 grand in expedited shipping and labor alone. Luna: What's the root cause here? Is this a Microsoft problem, a vendor problem, or both? Lucas: I think it's a system problem. Microsoft provides the pipeline — the Windows Update for Firmware infrastructure. But the firmware itself is authored and signed by the vendor. Microsoft doesn't functionally validate it beyond checking the digital signature and the capsule header. If the vendor ships a bad binary, Windows pushes it out confidently. Luna: So the IT department is the last line of defense. But they're not testing each firmware update on every model of keyboard. Lucas: They can't. A typical enterprise might have twenty different keyboard models in the field — from generic office keyboards to specialized mechanical ones for developers. Testing each firmware update on every variant before deployment is not practical. So the update goes out to a pilot group, but even that didn't catch this because the bug was intermittent — it only triggered under certain USB host controller conditions. Luna: Which are different on the trading floor's Dell OptiPlex versus the Lenovo laptops in the pilot. Lucas: Bingo. So the question becomes: what can IT do to prevent this? There are a few strategies emerging. One is to disable Windows Update for Firmware for peripherals entirely via Group Policy. You can set the 'Disable automatic device firmware updates' policy under Computer Configuration > Administrative Templates > System > Device Installation. That blocks any firmware update from coming through Windows Update. Luna: But then you lose the convenience of centralized updates. You'd have to go back to vendor-specific tools. Lucas: Which is what many enterprises are doing. Another approach is to use Windows Update for Business with a deferral policy. You set a 30-day deferral for firmware updates, giving you time to test them in a lab with a representative hardware sample. But that only works if you have a lab with the exact same keyboards and USB controllers. Luna: And that's expensive. So the third option? Lucas: Vendor qualification. Some large enterprises are now requiring keyboard vendors to provide a firmware update history and a signed attestation that the firmware has been tested against the specific USB host controllers used in the enterprise's fleet. That's a contractual shift — it puts the onus on the vendor to certify compatibility. Luna: Interesting. So the IT department becomes more of a procurement gatekeeper than a hands-on tester. Lucas: Exactly. And I think that's the direction Microsoft wants to push. They've been clear that Windows 11 is a 'modern' OS that expects modern hardware and modern firmware workflows. PS/2 is gone, and with it, the era of the keyboard as a dumb device. Now your keyboard is a miniature computer with firmware that can be updated — and bricked — over the wire. Luna: What about the keyboards that come built into laptops? Same risk? Lucas: Even more so, because you can't just unplug a laptop keyboard. The same Windows Update for Firmware mechanism applies to embedded keyboards. And there have been reports of laptop keyboard firmware updates failing during a BIOS update, leaving the keyboard non-functional until the next BIOS update — if the vendor provides one. For a remote worker, that's a support call and a laptop replacement. Luna: So the lesson for IT admins is: treat keyboard firmware updates like you treat BIOS updates. They carry the same risk. Lucas: Absolutely. And it's not just keyboards. Mice, webcams, even usb c docking stations are now getting firmware through this pipeline. So the risk surface is widening. The trading floor story is just the canary in the coal mine. Luna: I'm curious — does Microsoft acknowledge this as a risk? Have they published any guidance? Lucas: They have a support document — KB5005468 — that covers Windows Update for Firmware, but it's very high-level. It basically says, 'Vendors are responsible for their firmware.' There's no mention of validation testing or recommended IT policies for peripherals. So the burden is entirely on the enterprise. Luna: Which is frustrating, because this is a direct consequence of Microsoft's architectural change. Lucas: It is. And I think we'll start seeing more enterprises pushing back. Some are already requiring that keyboard firmware updates be delivered via vendor-specific tools for at least the next two years, until the ecosystem matures. Others are building custom Windows images that strip out the firmware update capability for USB devices. Luna: Is there a middle ground? Maybe a whitelist of approved firmware versions? Lucas: That's actually a smart idea. You can use Windows Update for Business to create a 'feature update deferral' for firmware, and then selectively approve updates via a management tool like Microsoft Intune. But that requires the firmware to be in the Windows Update catalog, which not all vendors support yet. So you end up with a hybrid approach — some keyboards updated via Windows, some via vendor tools. Luna: Which is exactly the kind of complexity IT hates. Lucas: Right. And that's the real story here. Windows 11 is pushing the industry toward a more automated, os managed firmware lifecycle. But the safety net isn't there yet. Until vendors and Microsoft agree on a validation standard, IT is left holding the bag. Luna: So for now, the best advice is to treat every keyboard firmware update as a potential brick event. Test aggressively, defer if you can, and have a rollback plan. Lucas: And maybe keep a box of PS/2 adapters in the closet. Just in case. Luna: That's oddly nostalgic. But I think we've given listeners a concrete takeaway. If you're managing Windows 11 devices, check your firmware update policies today. Lucas: Agreed. And if you've had your own keyboard firmware disaster — or success story — we'd love to hear it. You can find us on the show's page. For now, thanks for listening.