Latest / Software Testing with Fexingo: QA, Automation, and Reliable Software Engineering / How Monkey Testing Reveals Hidden Software Crashes
Transcript
- Lucas: So picture this: you've got a streaming app on Android. It's been through unit tests, integration tests, even some manual exploratory testing. You're confident. Then one day, support tickets spike — users with older phones report the app crashes when they rotate the screen while buffering a 4K stream. You never tested that exact combination. That is the exact scenario monkey testing is built for. Luna: Monkey testing — that's the one where you just throw random inputs at the app, right? Like a monkey banging on a keyboard? Lucas: Exactly. The name comes from the 'infinite monkey theorem' — a monkey hitting random keys will eventually type Shakespeare. In software, a monkey throwing random taps, swipes, and system events will eventually find a crash. And the beauty is, it finds crashes you never thought to script. Luna: That streaming story — was that a real case? Or a hypothetical? Lucas: Real case. Early 2025, a major streaming service — I won't name them — had a race condition between the orientation change handler and the buffer state machine. It only fired on devices with less than 4GB of RAM, so internal testing on high-end devices never hit it. They ran a monkey test overnight on a dozen low-end devices, and within four hours, the crash was reproduced. Luna: And that's the killer use case — the bugs that only appear in production under specific conditions. Monkey testing is basically a net you cast over the whole app. Lucas: Right. And if today's conversation gave you something usable, here's a quick thought: the reason we can dig into techniques like this without any sponsor interruptions is listener support. People who find value in the show keep it ad-free at buy me a coffee dot com slash fexingo. Sincerely, it makes a difference. Luna: Yeah, it's a small thing that adds up. I love that the community keeps this going. Lucas: Alright, back to the monkeys. There are actually two flavors: 'dumb' monkey testing and 'smart' monkey testing. Dumb monkey testing generates purely random events — taps at random coordinates, random system broadcasts, random gestures. It's chaotic and can produce a lot of noise, but it's also the simplest to set up. Luna: And smart monkey testing — that's where you bias the randomness toward high-risk areas, right? Like known crash-prone modules. Lucas: Exactly. Smart monkey testing uses a model of the app — the UI hierarchy, the navigation graph, the known API endpoints — to generate events that are more likely to trigger edge cases. For example, instead of tapping random pixels, it taps on actual buttons, but in a random order. It might also inject network delays or low-memory conditions alongside the random taps. Luna: So dumb monkey is pure fuzzing at the UI level, and smart monkey is like guided chaos. Both have their place. Lucas: Absolutely. Dumb monkey is great for stress testing the operating system integration — things like handling incoming calls while a transaction is processing. Smart monkey is better for finding logical errors in the user flow. The streaming service I mentioned used a dumb monkey on the system level — it sent random orientation changes, network disconnects, and low-memory warnings — and that's what triggered the race condition. Luna: I've seen teams shy away from monkey testing because they think it's unreliable or hard to reproduce. But if you log the event sequence, you can replay it, right? Lucas: That's the key. Modern monkey testing frameworks — like Android's MonkeyRunner or tools like Chaos Monkey for infrastructure — log the exact seed and event sequence. When a crash occurs, you can replay that exact sequence to debug. Without logging, it's just noise. With logging, it's a reproducible test case. Luna: So the practice is: run monkey tests as part of your nightly CI, and if they find a crash, the seed gets fed into your regression suite as a new test case. Lucas: That's exactly how mature teams handle it. Some even go further and use 'differential monkey testing' — run the same random seed against two versions of the app, say the current release and the new build, and flag any crashes that appear only in the new build. That way, you catch regressions before they hit production. Luna: That's clever. You're effectively using randomness to do regression testing at scale. Lucas: Right. And the ROI is substantial. A 2024 study by a large e-commerce platform found that monkey testing caught 23% of their production crashes — crashes that had passed through their entire automated and manual test process. And the average cost per crash caught was under $50 in compute time, versus thousands of dollars in post-release hotfix costs. Luna: Twenty-three percent is huge. That's almost a quarter of all crashes that you would have missed otherwise. Lucas: Exactly. And they achieved that with just a two-hour monkey test run per build. It's not a silver bullet — monkey testing doesn't find logical errors that don't cause crashes, and it can't verify business rules. But for crash detection, it's incredibly efficient. Luna: What about false positives? If you're running thousands of random events, you're bound to hit some crashes that are unavoidable — like an out-of-memory crash when you allocate a massive bitmap. Lucas: Great point. That's why you need a crash classification step. Teams typically triage monkey-test crashes into three buckets: 'real bugs' — crashes that indicate a genuine defect; 'system limits' — crashes from resource exhaustion that are expected at extreme levels; and 'test artifacts' — crashes caused by the monkey tool itself, like tapping a system dialog that should never appear in normal use. You'd only fix the first bucket, but you might tune the test to avoid the third. Luna: So you need a human in the loop to classify at first, but over time you can automate the triage based on stack trace patterns. Lucas: Exactly. After a few weeks, you build a known-crash database, and the monkey test can automatically ignore known system-limit crashes and only alert on new signatures. That's when it becomes truly ci friendly. Luna: I've also heard of teams using monkey testing for non ui components — like random API calls or database queries. Is that common? Lucas: It's less common but growing. That's often called 'fuzz testing' rather than monkey testing, but the principle is the same. For example, you can write a monkey that sends random payloads to your REST API — malformed JSON, extremely long strings, unexpected data types — and see if any of them cause a 500 error or a crash. That's how a major social media company in 2023 found a buffer overflow in their image upload endpoint. Luna: And that's the thing — monkey testing at the API level can find security vulnerabilities too, not just stability bugs. Lucas: Absolutely. In fact, the OWASP testing guide recommends random fuzzing as a basic security test. So monkey testing overlaps with both quality assurance and security. It's a low-skill, high-value technique that every team should have in their toolbox. Luna: Alright, so if I'm a QA lead and I want to start monkey testing tomorrow, what's the simplest path? Just download MonkeyRunner and point it at my app? Lucas: That's one way. For Android, the built-in 'adb monkey' tool is the easiest — you just run 'adb shell monkey -p your.package -v 500' and it sends 500 random events. But I'd recommend a more structured approach: define a test duration and a set of device configurations, capture logs and screenshots, and automatically upload crash reports to your bug tracker. Tools like Appium, Espresso, or XCUITest can be scripted to do random actions too. Luna: And for iOS? I don't think there's a built-in monkey tool like Android has. Lucas: True. For iOS, you'd use something like CrashMonkey or write your own UI testing script that generates random taps. There's also 'SwiftMonkey' — an open-source library that integrates with XCTest. The principle is the same, just different tooling. Luna: One concern I've heard: monkey testing can be disruptive to your CI pipeline because it's inherently non-deterministic. A build that passes today might fail tomorrow due to a different random sequence. How do teams handle that? Lucas: That's a real challenge. The solution is to use a fixed seed per build and make the test deterministic. For example, use the build number as the seed — that way, if a build fails, you can always reproduce the exact sequence. And you never gate your release on monkey test results; instead, you use them as a signal. If a new crash signature appears, investigate, but don't block the release unless it's a high-severity crash. Luna: So the monkey test runs in parallel with other tests, and its results are reviewed before the next sprint, not before the current deploy. Lucas: Exactly. It's an offline quality gate, not a blocking one. Over time, as you fix the crashes it finds, your app becomes more robust. And the monkey test itself becomes more valuable because it has to dig deeper to find new bugs. Luna: I like that. It's like a continuous stress test that makes your app progressively harder to break. Lucas: Right. And the beauty is, it scales with your app. As you add new features, the monkey automatically exercises them — no test maintenance needed. That's the big selling point: zero test maintenance, yet it catches crashes that your hand-crafted tests miss. Luna: Zero maintenance is a strong claim. But I can see how it's mostly true — you just update the app package, and the monkey keeps tapping away. Lucas: The only maintenance is if you add a new UI element that requires a specific gesture — like a swipe to delete — and the monkey doesn't know to swipe. But even then, smart monkey tools can learn from the UI hierarchy. Android's UI Automator, for instance, can traverse all visible elements and interact with them in random order. Luna: So the takeaway for today: monkey testing is a cheap, effective way to find crashes and edge cases that structured testing misses. It's not a replacement for your existing tests, but a complement — especially for catching the bugs you didn't think to test. Lucas: Exactly. And the next time you're staring at a crash report from production that you can't reproduce, ask yourself: did you let the monkey loose first? It might have saved you that weekend debugging session.