Latest / Marketplace Businesses with Fexingo: Two-Sided Networks, Liquidity, and Take Rates / How Marketplaces Use A B Testing to Optimize Take Rate
Transcript
- Lucas: If these conversations are useful for what you're building or running, we keep them ad-free because listeners like you chip in. It's a tiny ask: if today's episode was worth a coffee to you, that's the link — buy me a coffee dot com slash fexingo. No pressure, just gratitude. Luna: And honestly, knowing that the show helps people think about their own marketplace or business — that's the whole point. So whether you chip in or not, we're glad you're here. Lucas: Alright. Let's talk about the moment a marketplace decides to raise its fee. It's one of the most nerve-wracking decisions a platform makes, because you're essentially testing how much loyalty you actually have. Luna: And most marketplaces don't just flip a switch. They test it, right? On a subset of users. Lucas: Exactly. A/B testing is how they do it. And the example I keep coming back to is Etsy's 2023 fee increase. They raised the transaction fee from 5 percent to 6.5 percent — but not for everyone at the same time. They ran a test on a small percentage of sellers first, measured churn, buyer conversion, and overall gross merchandise volume before rolling it out broadly. Luna: I remember that caused a lot of controversy. Sellers were furious. But from Etsy's perspective, they needed to see if the platform would break. Lucas: Right. And the data from that test was critical. They saw that seller churn was within acceptable bounds — some sellers left, but not enough to hurt the buyer experience. And buyer conversion barely budged. So they went ahead. The take rate went from about 19 percent to over 21 percent when you factor in payment processing and other fees. Luna: Was there any backlash from sellers who were in the test group? I imagine feeling like a guinea pig doesn't sit well. Lucas: Absolutely. But Etsy framed it as a way to reinvest in the platform — marketing, seller tools, buyer acquisition. And the test gave them confidence that the increase wouldn't kill liquidity. That's the key insight: you don't want to learn that your take rate is too high by watching your marketplace collapse. Luna: So A/B testing lets you find the 'sweet spot' — the highest take rate that doesn't trigger a death spiral. Lucas: Precisely. And different marketplaces have different elasticities. Uber, for example, tests surge pricing thresholds constantly. They'll show one version of the app with a 1.5x multiplier and another with 2.0x for the same route, and see how many riders still book. If demand drops too much, they dial it back. Luna: But Uber's a bit different because the supply side — drivers — also sees the surge. So it's a two-sided test. Higher surge attracts more drivers, which reduces wait times, which makes riders more willing to pay. Lucas: That's a great point. The test has to account for both sides. If you raise the take rate on sellers, you might lose inventory. If you raise it on buyers, you might lose demand. The best tests measure the net effect on total transactions, not just one side. Luna: Airbnb is another interesting case. They A/B tested shifting more of the service fee from guests to hosts. They found that guests are more price-sensitive than hosts, so moving the fee to hosts actually increased bookings — even though hosts grumbled. Lucas: Right. The take rate stayed the same, but the allocation changed. That's a subtler kind of test. And it shows that take rate isn't just about the percentage — it's about who feels it. Luna: So the methodology matters a lot. How long do you run the test? How do you decide what's a statistically significant change? Lucas: Great question. For a marketplace, you need at least a few weeks — ideally a full month — because buyer and seller behavior has weekly cycles. And you need to measure not just the primary metric but also secondary ones: repeat purchase rate, customer support tickets, seller satisfaction scores. Luna: I've heard that some marketplaces run tests where they don't even tell the internal team which group is the control. That way, no one can bias the results. Lucas: That's called a blind test. Very common at companies like Amazon. They've run tests where they raised the referral fee for certain categories without telling the category managers. The fear is that managers might overcompensate with promotions or manual interventions. Luna: Which brings up an ethical question. Is it okay to test on your users without their knowledge? Most terms of service allow it, but some sellers feel manipulated. Lucas: It's a tension that every platform lives with. The counterargument is that without testing, you'd make decisions based on gut feelings or loud voices — which could be worse for everyone. A/B testing is, in theory, more democratic because it relies on actual behavior, not just complaints. Luna: But still, the power imbalance is real. A seller on Etsy can't opt out of the test. They just wake up one day with higher fees. Lucas: True. And that's why transparency matters. Etsy did eventually announce the fee increase broadly, but during the test period, only the test group knew. Some of them found out because they did the math on their own transactions. Luna: So what's the best practice if you're a marketplace operator listening to this? How should you approach an A/B test on take rate? Lucas: First, start small — test on less than 5 percent of your user base. Second, define your success metrics upfront: you want to see no significant drop in gross merchandise value, no spike in churn. Third, run the test long enough to capture a full cycle — at least two weeks, ideally a month. And fourth, have a plan for communicating the results, whether you roll out the change or not. Luna: And if the test fails — meaning you see a big drop in transactions — you have to be willing to walk away. Not every take rate increase is a good idea. Lucas: Exactly. And sometimes the test tells you that you can raise the take rate, but only if you add value. For example, if you offer better seller tools or buyer guarantees, users are more tolerant. That's what we talked about in episode 104 about on-time guarantees. Luna: So the takeaway is: take rate is not a number you set and forget. It's a variable you optimize through experimentation. And the marketplaces that do it well — like Etsy, Uber, Airbnb — treat it as a continuous process, not a one-time decision. Lucas: And that's what separates platforms that grow sustainably from those that squeeze too hard and break. The data lets you find the edge without falling off. Luna: I think that's a good place to leave it. Next time we might talk about what happens when a marketplace decides to lower its take rate — because that's a strategy too. Lucas: Absolutely. There are cases where cutting fees actually grows revenue. But that's for another episode.