Latest / Deep Dive / The Shadow Housing Market: Why Capping Airbnbs Can Backfire
Transcript
- 0:00Imagine an apartment building like to a city planner looking
- 0:03at a blueprint. It just houses 20 families.
- 0:06Right, right. It's just standard residential.
- 0:08Exactly. It's zoned residential, it's
- 0:10color-coded on a map, it's neat, and it's totally predictable.
- 0:13But to the algorithm running on a server thousands of miles
- 0:17away, it's. Completely different, yeah.
- 0:19That exact same building houses 80 transient workers sleeping in
- 0:24bunk beds, right alongside half a dozen unlisted, unregulated
- 0:29hotels. Today we are exploring the
- 0:32Shadow City. It really is this completely
- 0:34invisible layer of geography. I mean, the physical buildings
- 0:37haven't changed at all, but the way human beings are actually
- 0:40utilizing the space inside them is shifting by the minute, and
- 0:44it's dictated entirely by these peer-to-peer networks.
- 0:47Welcome to the Deep Dive. Our mission today is to explore
- 0:50this shared economy of housing, and we're going to look at the
- 0:53entire spectrum. From the hyperlocal micro level
- 0:56of finding a single room on a site like flatmates dot dot com
- 0:59all the way up to the massive macro level of booking an
- 1:02Airbnb. Right.
- 1:03We want to understand how these digital platforms are secretly
- 1:07and in some cases drastically rewiring our neighborhoods.
- 1:11And to do this, we're looking at a stack of incredibly detailed
- 1:14research from Australian universities.
- 1:16Yeah, we've got studies published by AUCRI.
- 1:19That's the Australian Housing and Urban Research Institute, as
- 1:22well as papers from journals like Urban Analytics and City
- 1:25Science. And you know, the researchers
- 1:27behind these papers aren't just looking at standard census data,
- 1:31they are using massive data scraping techniques.
- 1:34Plus, incredibly complex economic models to figure out
- 1:38how this peer-to-peer sharing is actually functioning in the real
- 1:41world. Exactly.
- 1:43So whether you are a renter handing over like half your
- 1:46paycheck every month, or a homeowner wondering what on
- 1:49earth is happening next door, or just someone fascinated by how
- 1:52modern cities evolve, you are going to want to hear what this
- 1:55data reveals. Because it's wild, we are going
- 1:57to look at the hidden geography of share houses, the very real
- 2:00financial incentives driving severe overcrowding.
- 2:03Oh, the overcrowding data is intense.
- 2:06And we're also going to look at a completely fascinating case
- 2:08study where a government policy that was explicitly designed to
- 2:12lower rents by restricting Airbnb's, well, it actually
- 2:16caused those rents to increase. It's the ultimate example of
- 2:19unintended consequences, you know, trying to regulate a
- 2:22digital market with analog laws. It's just a profound study in
- 2:26human behavior and economics. Let's jump into the first major
- 2:29study we have. This is from Sokar and Gurren.
- 2:32Back in 2017. They were investigating what
- 2:35they called shared urbanism. Right.
- 2:37They wanted to look at the informal sharing market, so
- 2:39platforms like flatmates.com and Gumtree.
- 2:42But right out of the gate, they hit this massive wall, didn't
- 2:44they? Like there's basically no
- 2:46official data on this stuff. Not at all, because traditional
- 2:49data collection is just agonizingly slow.
- 2:52I mean, the Australian Bureau of Statistics relies on a census
- 2:55every few years or periodic highly structured surveys.
- 2:59And the shared economy definitely does not operate on a
- 3:02five year cycle. Exactly.
- 3:04It's incredibly ephemeral. A room is listed on a Tuesday
- 3:07afternoon, rented to a backpacker on a Thursday
- 3:10morning, and the listing completely vanishes from the
- 3:13Internet by Thursday night. Poof.
- 3:15It's gone. Right.
- 3:16So it's this massive glaring blind spot for urban planners.
- 3:20They literally cannot see it happening.
- 3:22So if the government can't see it, how did these researchers
- 3:24actually measure it? They had to build their own
- 3:26tools to capture the shadow city.
- 3:28The researchers wrote custom Python scripts.
- 3:31So basically computer code designed to mimic a human
- 3:35browsing the web. Yeah, but doing it thousands of
- 3:38times a second, they deployed these scripts to just scrape
- 3:41data from these sharing websites weekly over an 8 week period.
- 3:45That is such a fascinating way to do research.
- 3:48They essentially built a digital trawling net to catch these
- 3:50listings before they evaporated. Yeah, but why 8 weeks?
- 3:54Is that really enough time to get a real picture of a massive
- 3:57cities housing market? It's actually a sweet spot for
- 3:59this specific type of highly volatile data.
- 4:02If you only scrape for one week, you might just catch a weird
- 4:05anomaly. Like a university just starting
- 4:08its semester, so there's a flood of students looking for rooms.
- 4:11Exactly. But by stretching it across two
- 4:13months, you capture the churn. You see what appears, what
- 4:17disappears, and, crucially, what stays.
- 4:19And they caught a lot. Right, a staggering amount.
- 4:22They captured 26,864 unique listings across Australia's five
- 4:28major cities, so Sydney, Melbourne, Brisbane, Perth and
- 4:32Adelaide. Wow, and they didn't just count
- 4:35the listings, did they? Like they pulled everything.
- 4:37Everything the algorithm could grab.
- 4:38Location, asking rent, the type of dwelling, the number of
- 4:42existing occupants. They even scrape the raw text
- 4:45descriptions written by the people posting the ads.
- 4:47And because they track it temporally over those eight
- 4:49weeks, they discovered something that really challenged the
- 4:52assumption that this market is just, you know, pure chaos.
- 4:55Right. They found that about 69% of
- 4:58listings from week one were actually still active in Week 2,
- 5:01and nearly 1/4 of them were still hanging around by week 8.
- 5:04So it wasn't just a frantic overnight churn.
- 5:07It's a stable stationary market. It's just operating completely
- 5:11under the radar of the official zoning boards.
- 5:13Precisely. It has its own steady heartbeat.
- 5:17The thing that really stood out to me in this data was where
- 5:20these listings were actually physically located.
- 5:22The geography of it. Yeah, because when I hear the
- 5:25phrase share house, I have a very specific image in my head.
- 5:30I immediately picture like 20 something university students
- 5:33living off cheap noodles in a rundown inner city terrace.
- 5:36Or maybe a bunch of backpackers crammed into an apartment right
- 5:39by the beach in Bondi. Exactly, but the data maps tell
- 5:44a totally different story. Yeah, the stereotype isn't
- 5:46necessarily wrong, it's just woefully incomplete.
- 5:50When the researchers took those 26,000 listings and mapped their
- 5:54coordinates, the spatial spread was eye opening.
- 5:56Because they didn't just cluster tightly around universities and
- 6:00central business districts. No, not at all.
- 6:02The hotspots bled deep into the middle and outer suburban rings.
- 6:06Like Parramatta in Sydney. For those who aren't familiar
- 6:09with Sydney's geography, Parramatta is way out in the
- 6:12geographic center of the sprawling metropolis.
- 6:15It's miles and miles away from the traditional downtown CBD or
- 6:19the famous beaches. And yet, according to this
- 6:21study, Parramatta actually cracked the top 10 suburbs for
- 6:25the sheer volume of shared listings.
- 6:27Which completely upends our traditional model of suburban
- 6:30housing. And it went even further out
- 6:32than that, didn't it? Oh yeah, they found significant
- 6:35clusters of listings out in regional areas place like
- 6:38Gosford, King Cumber and Woy Woy.
- 6:41I'm really trying to wrap my head around this.
- 6:43If it's not university students throwing parties in the inner
- 6:46city, who on earth is out in Gosford or Parramatta renting a
- 6:50single room in a shared house? Well, the researchers overlaid
- 6:53transport maps onto the listing maps, and this massive pattern
- 6:56emerged. Transport accessibility,
- 6:58specifically being along major train routes, was a huge driver.
- 7:01So the listings effectively trace the transit arteries of
- 7:04the city. Exactly.
- 7:06And what this tells us is that we are looking at 2 entirely
- 7:10different demographic groups using the exact same digital
- 7:13platforms for completely different reasons.
- 7:15OK, lay those two groups out for me.
- 7:17The first group is the one you already picture, the young
- 7:20person, student, the backpacker, transitioning through a
- 7:24temporary, highly mobile phase of life.
- 7:26They want to be near the action, near the universities, near the
- 7:29beaches. Makes sense.
- 7:31And the second group. The second group is much more
- 7:33systemic and frankly a symptom of a broken in broader market.
- 7:37We are seeing young working families or a newly arrived
- 7:41migrants engaging in long term sharing in these middle ring and
- 7:45outer suburbs. Because they just can't afford a
- 7:47whole house. Because single family homes in
- 7:49those areas have become entirely unaffordable.
- 7:52I mean, they aren't sharing a house out in Parramatta because
- 7:54it's a fun social rite of passage, they are doing it out
- 7:57of absolute economic necessity for them.
- 8:00Logging on to flatmates.com isn't about finding a cool
- 8:03roommate, it's about finding a semi permanent way to keep a
- 8:07roof over their family's head while still having access to a
- 8:11train line that can get them to their jobs.
- 8:13That just completely shatters the illusion of the suburbs.
- 8:16The whole cultural myth of the suburb is the nuclear family,
- 8:19right? One house, one family, a
- 8:20backyard, right? But the data shows that the
- 8:24existing physical structure of the suburbs is secretly being
- 8:27hacked. The houses are being expanded
- 8:29from the inside out to accommodate multiple disparate
- 8:32family units or individuals. Yeah, In a formal housing
- 8:35market, if a city's population grows, you theoretically build
- 8:38more houses. In the informal shared market,
- 8:41you just cram more people into the houses that are already
- 8:43standing. And the dwelling typologies in
- 8:45the study prove this. They really do.
- 8:48In Sydney, flat shares were the dominant mode, making up 51% of
- 8:51the listings, with house shares at 48%.
- 8:54But in cities with a more sprawling suburban foot print,
- 8:57it was different, right? Yeah.
- 8:58Like Perth and Adelaide, house sharing was overwhelmingly the
- 9:01main category there, 80% and 82% respectively.
- 9:05And the vast majority of these properties were standard 2 and 3
- 9:08bedroom homes. Which means standard family
- 9:09homes are being carved up into multi tenant boarding houses.
- 9:12Exactly. And if families and migrants are
- 9:15turning to this shadow market just to survive the cost of
- 9:18living, we have to talk about the money.
- 9:21Because wherever you find economic desperation, you were
- 9:24almost guaranteed to find someone figuring out how to turn
- 9:27a massive profit from it. Oh absolutely.
- 9:29The economics of this are where the research gets incredibly
- 9:33granular and quite confronting. To really understand the
- 9:38financial engine driving this, the researchers isolated a
- 9:42specific case study. Parmont.
- 9:44OK Parmont. Yeah.
- 9:45It's an inner city suburb of Sydney, right on the water,
- 9:48heavily dense, and it had the highest absolute number of
- 9:51listings in their 2017 scrape. I actually want to read directly
- 9:55from the study here because they included the actual text of one
- 9:58of the listings they scrape from Parmont.
- 10:00This is a real advertisement on flatmates.com.
- 10:02Go for it. OK, it reads.
- 10:04Only female unit, No minimum stay.
- 10:08Master bedroom 4 persons room $135 per week.
- 10:12Second room 4 persons room $125 per week.
- 10:15People are from Australia, German, French, Korea, China,
- 10:20Japan. Very clean and spacious.
- 10:22Very clean and spacious is doing a lot of heavy lifting in that
- 10:25sentence. Right?
- 10:25Let's actually do the math on that listing because it lays
- 10:28bare the entire business model of the Shadow City.
- 10:31We are looking at a standard 2 bedroom apartment and the
- 10:35advertisement is offering four beds in the master bedroom and
- 10:38four beds in the second bedroom. That is 8 adult human beings
- 10:42living in a single two-bedroom apartment.
- 10:44Think about the logistics of that for a second.
- 10:46That's eight people sharing one, maybe 2 bathroom.
- 10:48You get. They're lucky.
- 10:49Eight people trying to store food in one standard sized
- 10:52refrigerator. Eight people trying to cook
- 10:54dinner on A4 burner stove. It's intense physical
- 10:57overcrowding, but look at it from the perspective of the
- 10:59person holding the lease. Four people paying $135 a week
- 11:03in the master bedroom equals $540.
- 11:06Another four people paying $125.00 a week in the second
- 11:09room equals $500. Combine those and that single
- 11:13two-bedroom apartment is generating $1040 in rent every
- 11:17single week. Wow, and what was the actual
- 11:20market rate for a normal 2 bedroom apartment in Parmont at
- 11:24that time? Like if a single family walked
- 11:26into a real estate agency and signed a standard lease, what
- 11:28would they pay? Well, based on formal real
- 11:31estate data from that exact same period, the median rent for a
- 11:35standard 2 bedroom dwelling in Parmont was $775 per week.
- 11:40It's a massive arbitrage opportunity.
- 11:41It's literally like buying a standard chocolate bar for
- 11:44$2.00, breaking it into 8 tiny squares, and selling each square
- 11:47for $0.50. That's a perfect way to picture.
- 11:49It suddenly the total is worth way more than the original bar.
- 11:52You take a $775 apartment, break it down into 8 individual beds,
- 11:56and suddenly you are pulling in over $1000 a week.
- 11:59You're creating a $265 weekly premium out of thin air just by
- 12:04maximizing the density. The chocolate bar analogy is
- 12:08spot on. It's all about maximizing yield,
- 12:11and it raises a really vital question.
- 12:13You know, who is actually running these listings?
- 12:16Is it just a broke student subletting their living room to
- 12:19make rent? That's what you'd assume.
- 12:22But the data indicates otherwise.
- 12:24For a huge proportion of these high density listings, the owner
- 12:27category on the platform was explicitly listed as false.
- 12:31Meaning the person living there and acting as the host doesn't
- 12:34actually own the property. Or more likely, the person
- 12:37managing the listing doesn't live there at all.
- 12:40These properties are frequently managed by non resident
- 12:43investors or specialized agents whose entire business model is
- 12:46leasing standard apartments and converting them into unregulated
- 12:49micro hostels. Unmanned, it creates A
- 12:52terrifying structural incentive. If you are an investor looking
- 12:55purely at a spreadsheet, why on earth would you rent your
- 12:58apartment to a local family for $775 when you could throw four
- 13:03sets of bunk beds in there and rent it to 8 desperate migrants
- 13:06or international students for $1040?
- 13:09Exactly. You wouldn't.
- 13:11The informal market generates a premium yield that the formal
- 13:14market simply cannot match. And the perman example wasn't an
- 13:17isolated rogue listing. The researchers ran a broad
- 13:21comparison across their entire data set.
- 13:24What did they find? They compared the stated number
- 13:26of bedrooms in a property against the stated occupant
- 13:29count in the listing. Let me guess, Sydney took the
- 13:32crown for stuffing the most people into the small spaces.
- 13:35You'd be right. Sydney exhibited the highest
- 13:37rate of potential overcrowding in the entire country.
- 13:41A full 13% of all listings scraped in Sydney had an
- 13:45occupant count that was higher than the physical number of
- 13:47bedrooms. 13%, that's huge. They found widespread examples
- 13:52of up to six or seven people routinely crammed into standard
- 13:552 bedroom places. It completely flips the
- 13:58narrative of the sharing economy.
- 14:00Isn't it? Totally.
- 14:01The original utopian pitch of these platforms was all about
- 14:04community building, sharing excess resources, helping people
- 14:08connect. But in this context, it has
- 14:10nothing to do with friendly community living.
- 14:12No, it is a hyper capitalist mechanism for maximizing the
- 14:16financial yield on a physical asset by actively exploiting the
- 14:19lack of formal regulation. It's the Shadow City operating
- 14:23in plain sight, completely superseding the intent of the
- 14:26city planners. It certainly does free up
- 14:29dormant space, like if there is an empty bed, the algorithm will
- 14:32fill it. But it's simultaneously creates
- 14:34profound unmonitored risks regarding fire safety,
- 14:38sanitation, mental health, and basic human living standards.
- 14:42Absolutely. So we have this informal share
- 14:44house market allowing investors to quietly hack the long term
- 14:47rental market by cramming beds into a room.
- 14:50But what happens when a tech platform comes along that lets
- 14:53them hack the market by the day instead of by the month?
- 14:55Airbnb. Yeah, well, all this was
- 14:57happening on sites like Flatmates and Gumtree.
- 14:59Airbnb was exploding and that completely changed the equation
- 15:02for property investors. Which brings us to the second
- 15:05major study in our stack, the 20252026 research by Rudnitsky
- 15:09and Sarkar, which looks specifically at the short term
- 15:12rental market and how governments try and often fail
- 15:15to control it. But before we get into the
- 15:17policy, we need to set the stage.
- 15:20We really need to understand the sheer scale of the housing
- 15:23crisis that Sydney was and honestly still is facing,
- 15:27because you can't understand the government's reaction without
- 15:30understanding the sheer panic they were dealing with.
- 15:33The numbers are frankly terrifying.
- 15:35The backdrop to the study is a city under immense structural
- 15:38stress. According to the research,
- 15:41Sydney is facing a projected deficit of 40,000 dwellings by
- 15:452029. 40,000. They are simply not building
- 15:49enough places for people to live.
- 15:50And the places that do exist are fiercely guarded.
- 15:53The vacancy rate, which is the percentage of all rental
- 15:56properties that are currently empty and available for a tenant
- 15:58to move into, plummeted to 1.8% by the end of 2022.
- 16:03And just for context, anything under 3% is generally considered
- 16:06a landlords market where tenants have absolutely 0 bargaining
- 16:09power. Exactly.
- 16:10Yeah, at 1.8% you literally have queues of 50 people wrapping
- 16:14around the block just to inspect a damp studio apartment.
- 16:18The financial toll of that extreme scarcity is staggering.
- 16:22Between 2010 and 2022, long term rents in Sydney jumped by 41%.
- 16:27House prices exploded by 114%. It's just wild.
- 16:31We are talking about an entire generation experiencing severe
- 16:34unaffordability and housing stress.
- 16:36This isn't just an inconvenience, it's devastating
- 16:39lower and moderate income earners because wage growth
- 16:42simply hasn't come anywhere close to keeping up with
- 16:45household debt and housing costs.
- 16:47It got so bad that it actually triggered a mass exodus.
- 16:50The data shows that in 2023 alone, over 31,000 more people
- 16:54left NSW for other states than arrived.
- 16:58People are literally fleeing the state because they have been
- 17:00priced out of existence. And at the exact same time, this
- 17:02housing shortage was becoming a full blown crisis.
- 17:05Airbnb was absolutely booming since it really took off in
- 17:08Australia around 2012. It grew exponentially.
- 17:11By 2023, guests on the platform were spending billions of
- 17:14dollars supporting jobs and massively besting the tourism
- 17:17sector. But there is a direct,
- 17:19undeniable physical friction here.
- 17:21Yeah, because the city has a finite amount of space.
- 17:24It's a 0 sum game when it comes to the physical structure of a
- 17:27building. You either sleep a tourist who
- 17:29is visiting for three days, or you sleep a local resident who
- 17:33works at the local hospital or school.
- 17:35You literally cannot do both in the same bed at the same time.
- 17:39And the profitability of short term rentals was vastly
- 17:42outpacing long term rentals. Property owners were financially
- 17:46incentivized to prioritize tourists over local tenants.
- 17:49Of course the government of NSW looked at the 1.8% vacancy rate.
- 17:54They looked at the thousands of entire homes listed full time on
- 17:57Airbnb and they realized they had to intervene.
- 18:00They decided to target what they officially classified as Short
- 18:03Term Rental Accommodation, or STRA.
- 18:06I always find it fascinating how governments actually go about
- 18:09trying to put a leash on technology.
- 18:10They rarely just drop a banhammer overnight, right?
- 18:13It's usually a slow creep of bureaucracy.
- 18:15Very true. It was a phased multi year
- 18:18approach. They started back in June 2018
- 18:21by announcing an initial framework.
- 18:23They tinkered with Strada legislation giving apartment
- 18:26buildings a bit more power to govern themselves.
- 18:29They introduced A mandatory code of conduct, and they eventually
- 18:32opened a statewide registry. So all short term hosts were
- 18:36legally required to register their properties.
- 18:38Yes, exactly. Which is step one of any
- 18:40government intervention, right? Figure out who is actually doing
- 18:43the thing you want to regulate. They also made a crucial legal
- 18:46distinction that set the stage for everything else.
- 18:49They defined two types of estray.
- 18:51There is Hosted SGRA which is the original Airbnb ideal.
- 18:57The owner actually lives in the house and is just renting out
- 19:00their spare bedroom. OK, that seems.
- 19:01Fine, and then there is non hosted STRA where the owner does
- 19:05not live on the premises at all. It is a dedicated empty
- 19:08investment property used exclusively for short term
- 19:12guests. That's the classic lock box on
- 19:14the fence setup. You book the place, punch in a
- 19:16code, get the key, and you never actually lay eyes on the owner.
- 19:19Exactly. The government recognized that
- 19:21non hosted STRA was the real threat to the long term housing
- 19:25supply, so after setting up the registry they brought out the
- 19:28big guns. What do they do?
- 19:30On April 9th 2021 the government announced a major new policy
- 19:35intervention, A strict 180 day cap.
- 19:38They mandated that any non hosted STRA in Greater Sydney
- 19:42could only be rented out for a maximum of 180 days per calendar
- 19:45year. Essentially half the year.
- 19:47Yep, this policy was then officially implemented a few
- 19:50months later, on November 1st, 2021.
- 19:53I want to pause and really think about the logic here because if
- 19:56you put yourself in the shoes of a policy maker it seems
- 19:59incredibly sound. You tell a property investor,
- 20:02hey you can only run your mini hotel for half the year, the
- 20:05other 185 days it has to sit completely empty generating
- 20:08absolutely zero revenue. Right?
- 20:10The government is hoping the landlord runs the math, realizes
- 20:13they are losing money by leaving an empty half the year, throws
- 20:16their hands up in frustration, deletes their Airbnb account,
- 20:19and puts the property back onto the normal long term rental
- 20:22market for a local family to sign a 12 month lease.
- 20:24That was the core underlying hypothesis of the SDRA laws.
- 20:28The goal was to artificially cap the profitability of the short
- 20:32term market to force supply back into the long term market,
- 20:35thereby increasing supply and lowering rents for locals.
- 20:39Make Airbnb less attractive and the long term market heals.
- 20:43It's an elegant logical theory, but the real world is rarely
- 20:47elegant. To see if this actually worked,
- 20:50the researchers Roninski and Sarkar didn't just look at a few
- 20:53random listings or interview a couple of angry landlords.
- 20:56They built a massive, highly sophisticated statistical model.
- 21:00And this is where the methodology of this research
- 21:02gets incredibly impressive. It is a master class in urban
- 21:05analytics. How so?
- 21:06Well, the problem with measuring the effect of a policy like 180
- 21:10day cap is that you can't just look at Sydney's rent prices
- 21:13before the law, look at them after the law and say aha, the
- 21:16law did this. Right, because rents in Sydney
- 21:18could would have been going up anyway.
- 21:20You've got inflation, population growth, interest rate hikes,
- 21:23building material shortages. How do you isolate the effect of
- 21:26this one specific law from the noise of the entire global
- 21:29economy? You need a control group.
- 21:31It's like testing a new car engine.
- 21:33Like if you want to know if a new spark plug makes a car
- 21:36faster, you don't just drive one car on a Tuesday and then drive
- 21:39it again on a Thursday and compare the times.
- 21:42The wind might be different, The track temperature might be
- 21:44different. Right, there are too many
- 21:45variables. You take two identical cars, put
- 21:48the new spark plug in one, leave the standard plug in the other,
- 21:51and race them on parallel tracks at the exact same time.
- 21:55The difference in their finish time is your answer.
- 21:57That is a brilliant way to conceptualize it.
- 22:00In economics, this methodology is called a Difference in
- 22:03Differences regression model, or did Z for short.
- 22:07Difference in differences. Got it.
- 22:08Sydney is the car that gets the experimental spark plug, the
- 22:11treatment group that received the 180 day cap policy.
- 22:16Melbourne was chosen as the other car, the control group,
- 22:18because it did not implement a similar 180 day cap during this
- 22:22period. But it faces very similar
- 22:23macroeconomic pressures as Sydney.
- 22:26OK, that makes sense. The researchers tracked massive
- 22:29amounts of data across both cities from the first quarter of
- 22:332019 all the way to the third quarter of 2023.
- 22:36But wait. How do you prove the two cars
- 22:38are actually identical? Sydney and Melbourne have
- 22:41different geographies, different populations, different local
- 22:45economies. That is the most crucial
- 22:47question in any Diddy study, and it's why the researchers had to
- 22:51meticulously test for what economists call parallel trends.
- 22:55Parallel friends? Yes, they used an event study
- 22:57approach to analyze the rental markets in both cities in the
- 23:01quarters before the policy was ever announced.
- 23:04They needed to mathematically prove that without any
- 23:07government intervention, rental prices in Sydney and Melbourne
- 23:10were tracking on very similar trajectories.
- 23:13And did. They they graphed it out and the
- 23:15statistical tests confirmed it. The pretreatment trends were
- 23:18parallel. The two cars were driving at the
- 23:20exact same speed before the experiment started.
- 23:23OK, so the foundation is rock solid, but cities are messy.
- 23:27What if one specific neighborhood in Sydney suddenly
- 23:30got a new train line or a new shopping center and rents spike
- 23:33there for a completely unrelated reason?
- 23:36They accounted for that using a technique called fixed effects.
- 23:40This is a statistical way to control for variables that don't
- 23:43change over time within a specific location.
- 23:46It essentially neutralizes the inherent vibe or static
- 23:50geography of a neighborhood, like its proximity to the beach
- 23:53or the fact that it has nice heritage homes.
- 23:55Oh, that's Clover. They also controlled for time
- 23:58specific shocks that hit both cities simultaneously, like the
- 24:01COVID-19 lockdowns. I read they also use something
- 24:04called a Hausman test. What on earth is that?
- 24:07A Hausmann test is basically a mathematical stress test for a
- 24:10statistical model. When you are building these
- 24:13models, you have to choose between a random effects model,
- 24:17which assumes the unobserved variables in a neighborhood are
- 24:19just random noise, or a fixed effects model which assumes
- 24:23those unobserved variables are correlated with the things you
- 24:25were actually trying to measure. OK, I'm fed.
- 24:27The Hausmann test compares the estimates of both models.
- 24:31If they diverge significantly, it mathematically proves you
- 24:34must use the fixed effects model to avoid biased results.
- 24:38They ran the test, it pointed to fixed effects, and that's what
- 24:42they used. It proves their methodology
- 24:44wasn't just a guess, it was mathematically mandated.
- 24:47They really bulletproofed this. They did, and they fed this
- 24:50model an incredible, dizzying array of variables to make
- 24:54absolutely sure nothing else was secretly driving the rental
- 24:57prices. They didn't just look at Airbnb
- 24:59data. Scrape from inside Airbnb dash
- 25:01dot dot com. What else did they look at?
- 25:03They layered in the Index of Community Socio Educational
- 25:06advantage, the ICSCA, to control for the quality of local schools
- 25:10in every specific neighborhood. Because obviously people pay a
- 25:14premium to live in a good school zone.
- 25:16Exactly. They also pull data from the
- 25:19Bureau of Crime Statistics and Research tracking the highly
- 25:22localized rates of assault, theft and property damage per
- 25:25100,000 people. Wow, they really got granular.
- 25:28Oh, they went further. They used the Index of Economic
- 25:31Resources, the IER, to control for the baseline wealth and
- 25:35infrastructure of different suburbs, and they tracked the
- 25:38total number of rental bonds held by the government.
- 25:41Let's explain that one. Because a rental bond is just
- 25:43the security deposit you pay when you sign a new lease,
- 25:46right? Yeah.
- 25:47Why is that important to a statistical model?
- 25:49Because it acts as a perfect proxy for the overall volume of
- 25:53activity and turnover in the rental market.
- 25:56If the government is suddenly holding more bonds in a specific
- 25:59suburb, it means more properties are actively turning over and
- 26:03new leases are being signed. It measures the velocity of the
- 26:06market. I love how deep this goes.
- 26:08Yeah. So the tension is building.
- 26:10We had the perfect experimental set up, the housing crisis is
- 26:13raging, the government steps in with a 180 day cap, the racing
- 26:17cars are on the track and all the vital signs are being
- 26:19monitored by this hyper sophisticated houseman tested
- 26:23fixed effect statistical model. They crunch the numbers.
- 26:27What actually happened to the rents?
- 26:28Did the policy work? It didn't just fail to work, it
- 26:32actively made the situation worse.
- 26:35The shock finding of this entire deep dive, The thing that should
- 26:39make every city planner sit up in their chair, is that the
- 26:41policy completely backfired. Wait, the DD model proved with
- 26:46absolute statistical significance that the stra laws
- 26:50actually cause long term rental prices to increase.
- 26:52I. Want to make sure I'm grasping
- 26:53the magnitude of this. Yeah, the government recognizes
- 26:56that Airbnb is cannibalizing the long term rental market.
- 27:00They write a law explicitly designed to stop that
- 27:02cannibalization and bring rents down for local families.
- 27:05And the direct, mathematically proven result of that law is
- 27:09that rents for local families went up.
- 27:10Yes. Give me the numbers.
- 27:12How much did they go up? The model isolated the specific
- 27:15effect of the policy intervention from all the other
- 27:17noise. It found that long term rents
- 27:19went up by 3.55% as a direct causal result of the policy
- 27:23merely being announced in April 2021.
- 27:25Just from the announcement. Yes.
- 27:27And then when the policy was actually implemented in November
- 27:302021, rents went up by further 2.73%.
- 27:35To put that in terms, the average weekly rent in Sydney
- 27:38across this entire study period went from $497.00 a week before
- 27:42the announcement to $664 a week by the end of 2023.
- 27:46That's a massive jump. And the model proves that a
- 27:49distinct, roughly 3 to 3.5% chunk of that massive increase
- 27:54was caused specifically by the government's attempts to help.
- 27:57I'm genuinely struggling to understand the mechanism here.
- 28:00How is that economically possible?
- 28:03If you restrict the supply of Airbnb's by capping the days
- 28:05they can operate, that should logically push properties back
- 28:08into the long term pool. More long term supply means less
- 28:11competition, which means prices should drop.
- 28:13What are the model get wrong? Or, more to the point, what did
- 28:16the policymakers completely misunderstand about human
- 28:18behavior? The policymakers fundamentally
- 28:21misunderstood the behavioral economics of the investor class.
- 28:24The core assumption was that landlords would rationally look
- 28:26at a 180 day limit, decide short term letting was no longer
- 28:30viable and switch to long term leasing.
- 28:32That makes sense on paper. But the data reveals that the
- 28:35short term rental market is incredibly sticky.
- 28:38Sticky meaning the investors really, really didn't want to
- 28:40leave it. Right investors who had become
- 28:42accustomed to the highly lucrative, highly flexible
- 28:45returns of Airbnb flat out refused to return to the stable
- 28:49but heavily regulated and lower yielding long term market.
- 28:52They didn't want to deal with 12 month leases and tenant rights.
- 28:56So if they refused to leave Airbnb, but they were legally
- 28:59only allowed to rent for a 180 days, what did they do to make
- 29:03up for the lost revenue? The researchers hypothesized
- 29:07they did 2 distinct things to compensate.
- 29:09First, they simply raised their nightly asking prices on Airbnb.
- 29:13If they could only operate for half the year, they decided to
- 29:16squeeze a full year's worth of profit out of those 180 days.
- 29:19And people paid it. The data from inside Airbnb
- 29:22backs this up. Florida State.
- 29:24The average nightly price of an Airbnb in Sydney climbed
- 29:27steadily from $201.00 back in early 2019 to nearly $249 a
- 29:32night by late 2023. They passed the regulatory cost
- 29:36directly onto the tourists. OK.
- 29:37But what about the ones who did switch back to the long term
- 29:39market? Did they lower the rents?
- 29:41Quite the opposite. If an investor grudgingly
- 29:44decided to move their property back to the long term rental
- 29:47market, they set the asking rent significantly higher than the
- 29:51historical average for that property or neighborhood.
- 29:53Why would they do that? They were actively attempting to
- 29:56recover the perceived financial losses they experienced from
- 29:59being forced out of the short term.
- 30:01Oh wow, so the landlord's internal monologue is basically
- 30:05fine government. You win.
- 30:07I will rent my apartment to a local family instead of
- 30:09tourists, but I was pulling in $60,000 a year on Airbnb so I'm
- 30:14going to charge this family a massive premium to try and match
- 30:17what I used to make. Exactly.
- 30:19And because the vacancy rate across the entire city was
- 30:22hovering at 1.8%, the local families were so desperate they
- 30:25had absolutely no choice but to pay that premium.
- 30:28That is exactly the dynamic. The extreme scarcity in the
- 30:31broader long term market acted as a force multiplier.
- 30:35It meant landlords could successfully pass those
- 30:38artificially inflated yield expectations directly onto
- 30:41desperate tenants. That is just brutal.
- 30:44But there is an even more glaring, almost comedic flaw in
- 30:48the policy design that the researchers highlight, which
- 30:51really sealed the fate of the long term market I'm.
- 30:53Trying to imagine how this gets worse.
- 30:54What's the flaw? Look at the exact wording of the
- 30:57law. The policy restricted the number
- 31:00of days a specific property could be rented on the short
- 31:03term market. It did absolutely nothing to
- 31:05restrict the total number of dwellings that could be
- 31:07converted into Airbnb's in the first place.
- 31:10Oh no, you got to be kidding. Because of this massive
- 31:13loophole, new Airbnb's just kept opening.
- 31:16The policy did not stem the bleeding of properties out of
- 31:19the long term market. In fact it seems to have
- 31:22accelerated it. Sydney saw a massive surge in
- 31:25new short term listings. How massive?
- 31:27Well, in the quarter immediately preceding the policy
- 31:29announcement, there were only 219 newly established Airbnb's
- 31:33in Sydney. By the end of the third quarter
- 31:36of 2023, there were 11,188 newly established Airbnb's opening up.
- 31:42So let me summarize this spectacular failure.
- 31:45The government says you can only run your unregulated hotel for
- 31:48half the year and the markets collective response was no
- 31:52problem. We will just open five times as
- 31:54many unregulated hotels and we will charge the guests
- 31:57significantly more per night. Yep, that's what happened.
- 32:00That is the absolute definition of a regulatory backfire.
- 32:04The self organizing, profit driven nature of the digital
- 32:07market completely outmaneuvered the top down analog regulation
- 32:11of the government. It did, and the data gets even
- 32:14more nuanced and frankly more tragic when we look at exactly
- 32:17who bore the brunt of this backfire.
- 32:20Because, you know, a city is not a monolith.
- 32:22A 3.5% rent increase hits a wealthy family in a waterfront
- 32:26mansion very differently than it has a single parent in a western
- 32:29suburb. Absolutely.
- 32:30This brings us to a fascinating part of the methodology, the
- 32:33heterogeneous treatment effects or HTE analysis.
- 32:36That's a mouthful. Heterogeneous treatment effects
- 32:38break that down for me. It's statistical jargon for
- 32:41acknowledging that a single medicine effects different
- 32:43patients differently. To measure this, the researchers
- 32:46split all the suburbs in Sydney into two distinct groups based
- 32:50on that index of economic resources, the IUR we talked
- 32:53about earlier. OK, so dividing by wealth.
- 32:56Right. They effectively drew line down
- 32:58the middle of the city's wealth distribution.
- 33:00They separated the economically advantage areas from the
- 33:03economically disadvantaged areas.
- 33:06Then they reran the model to see how long term rents reacted in
- 33:10those two distinct groups when the 180 day cap policy hit.
- 33:14So we are looking at the different panic responses of
- 33:16wealthy landlords versus lower income landlords.
- 33:19What happened in April 2021 when the government went on TV and
- 33:221st announced this looming 180 day cap?
- 33:25The disparity was massive when the policy was first announced.
- 33:28Economically disadvantaged areas saw an immediate severe 4.99%
- 33:32spike in long term rents. Meanwhile, the wealthy advantage
- 33:36areas only saw 1.95% increase. The poorer areas got hit more
- 33:40than twice as hard right out of the gate.
- 33:42Why? I mean I would have assumed the
- 33:44wealthy areas with all the premium beach side Airbnb's
- 33:47would have reacted more violently.
- 33:49It comes down to financial resilience, the researchers
- 33:51explain. This is likely because landlords
- 33:53operating in lower income areas rely much more heavily on their
- 33:56short term rentals as a crucial structural secondary income
- 34:00source. If you are a wealthy investor
- 34:03with a portfolio of premium properties in Bondi, a 180 day
- 34:07cap is an annoyance. You adjust your spreadsheet and
- 34:09move on. But if you are an investor in a
- 34:11less wealthy area who is actively piercing together a
- 34:14livelihood leveraging a granny flat or a single environment
- 34:18property on Airbnb just to pay your own mortgage, the
- 34:21announcement of a 180 cap is an existential threat to your cash
- 34:24flow. Right, it induces immediate
- 34:26panic. Exactly.
- 34:27You saw a much sharper, more aggressive, panic driven market
- 34:31reaction in those disadvantaged areas.
- 34:33Landlords immediately hiked their long term rents or
- 34:36aggressively jacked up their Airbnb prices to build a
- 34:39financial buffer against the incoming regulation and protect
- 34:42their livelihoods. It's the classic tragedy of
- 34:44economics. The people with the least
- 34:46financial buffer absorb the hardest shock.
- 34:49The turbulence always hits the back of the plane first.
- 34:52Now, interestingly, by the time the policy was actually
- 34:54implemented in November, that disparity had flattened out.
- 34:58The initial localized panic subsided, but it was replaced by
- 35:01a broad systemic inflation. The lower IUR areas saw a 3.09%
- 35:08increase upon implementation and the higher IER areas saw a 2.67%
- 35:13increase. So the entire market eventually
- 35:15reacted, raising rents across the board regardless of wealth.
- 35:19Yes, that is just incredible. The sheer gravity of how an
- 35:22algorithm can dictate the reality of a physical
- 35:24neighborhood is staggering. But I want to pivot slightly
- 35:27because earlier you mentioned the model tracked all these
- 35:29other fascinating variables like crime rates and local schools,
- 35:32just to isolate the Airbnb effect.
- 35:34Yeah, the control variables. Those other variables actually
- 35:37affect the rent in this giant mathematical model, like what
- 35:39else decides what we pay for a roof over our heads?
- 35:42They absolutely did, and the coefficients from the regression
- 35:44model give us a really cold, clear picture of how a city
- 35:48values itself. Let's start with the sheer
- 35:50density of Airbnb's. The model proved mathematically
- 35:54that for every single additional Airbnb that opened up within a
- 35:57one square kilometer area, long term rents in that specific
- 36:01neighborhood rose by 0002 percent. .0002% sounds like a
- 36:07rounding error, but in reality Airbnb's don't open one by one
- 36:11in isolation, do they? No, they cluster.
- 36:13Right, So if you have a dense cluster of 100 Airbnb's in a
- 36:16square kilometer, which is very common in tourist hotspots, that
- 36:20translates to a .2% permanent premium slapped out of the local
- 36:24rent just for the privilege of living near tourists.
- 36:27Which compounds over time. Then there's the crime data.
- 36:30The model confirmed what we intuitively know, but detached A
- 36:33precise decimal point to it. Higher crime rates provide a
- 36:36discount on rent. How much of a discount?
- 36:37Specifically, a measurable increase.
- 36:39It's in the crime rate, which they defined as assault, thefts
- 36:42and property damage in a statistical area reduced rents
- 36:45by .033% to .042%. So if your car gets broken into
- 36:50on your street, at least your landlord might hold off on a
- 36:53rent hike. I guess that's the grim silver
- 36:55lining of urban living. It's a very grim calculus, but
- 36:58the market prices it in meticulously.
- 37:01They also looked at the rental bonds data.
- 37:04A 1% increase in the total number of rental bonds, meaning
- 37:07a higher volume of properties actively hitting the rental
- 37:10market and turning over, raised average rents by roughly 0.18%
- 37:15to .21%. That feels counterintuitive.
- 37:18More properties on the market means higher rents.
- 37:20It does in a hyper competitive market with a 1.8% vacancy rate.
- 37:25In that environment, velocity usually means prices are
- 37:27escalating. When a lease turns over, the
- 37:29landlord almost never drops price.
- 37:32I see. The new lease is almost always
- 37:34aggressively marked up to match the current desperation of the
- 37:37market. High turnover in a constrained
- 37:39supply environment just means more opportunities for landlords
- 37:43to reset the baseline higher. And what about the schools?
- 37:46You hear parents obsessing over getting into the right public
- 37:49school zone. They move houses, they afford
- 37:51dresses. It's a blood sport.
- 37:53This was one of the more surprising findings of the
- 37:55study. In this specific model, which
- 37:58was highly focused on tracking short term quarter by quarter
- 38:01order price fluctuations over a few years, the quality of local
- 38:04schools measured by the ICSCA score did not show a
- 38:08statistically significant effect on the immediate rental price
- 38:11jumps. Really, school zones didn't
- 38:13matter. It's not that they don't matter.
- 38:15School quality absolutely effects baseline property values
- 38:19and long term established rents over a decade.
- 38:22But it wasn't the immediate volatile driver of these acute
- 38:25quarterly rent spikes. So what was the main driver?
- 38:28The driver of the acute chaos wasn't local amenities, it was
- 38:32the distortion caused by the shared economy platforms and the
- 38:35government's failed attempt to regulate them.
- 38:37Man, what an incredible journey through the data.
- 38:39Let's try to synthesize all of this.
- 38:41Yeah, because we have covered a massive amount of ground.
- 38:44We started this deep dive looking at the shadow market of
- 38:47overcrowded share houses. We saw how the sheer lack of
- 38:49affordable housing is driving young families in migrants to
- 38:53essentially rent a quarter of a bedroom in places like Permont
- 38:56for extortionate premiums. It's generating massive yields
- 39:00for unseen investors, expanding the capacity of the suburbs from
- 39:03the inside out while risking basic health and safety.
- 39:06And then we transitioned into the formal digital market of
- 39:09Airbnb's. We saw a government staring down
- 39:11a 40,000 home deficit and a crippling 1.8% vacancy rate,
- 39:17desperately trying to pull a regulatory lever to fix it.
- 39:20But the machine didn't respond the way the blueprint and said
- 39:22it would. The government tried to fix a
- 39:25massive, complex housing crisis by tweaking a single analog Dow,
- 39:29capping Airbnb days to 180. But the self organizing nature
- 39:33of the digital market push back instantly.
- 39:35It really did. The investors didn't retreat to
- 39:37the long term market, They doubled down.
- 39:40They raised nightly rates, they opened thousands of new Airbnb's
- 39:43through a legislative loophole, and ultimately their collective
- 39:45behavior resulted in a 3.5% increase in long term rents for
- 39:49everyday people. The medicine they prescribed.
- 39:51Literally made the patient sicker.
- 39:53It is a profound and humbling lesson in urban economics.
- 39:57You simply cannot regulate a complex, adaptive,
- 40:00algorithmically driven system with a simple static
- 40:03one-dimensional rule without expecting massive unintended
- 40:07consequences. I keep thinking back to that
- 40:09blueprint metaphor we started with at the top of the show, the
- 40:12idea that a city is just a physical map of color-coded
- 40:15buildings. But this data proves
- 40:18definitively that a city is no longer defined by its physical
- 40:21zoning. It's really not.
- 40:23It is actually defined by the invisible software running
- 40:25inside those buildings. Yeah, the flatmates listings,
- 40:27the Airbnb algorithms, they're actively overriding the physical
- 40:31reality of the city. And that leads directly to the
- 40:33core tension we have to face moving forward.
- 40:36These platforms were originally born from the utopian idea of a
- 40:39sharing economy, a decentralized peer-to-peer network where you
- 40:42let a backpacker crash on your couch to make a few extra bucks
- 40:46and meet someone new. But as these networks scale to
- 40:49the size of millions of users and billions of dollars in
- 40:51revenue, they cease to be just apps.
- 40:55They become massive, self organizing financial systems
- 40:58that entirely outsmart traditional city planning and
- 41:01slow moving government regulation.
- 41:03They're just too fast, too liquid and too adaptable for an
- 41:06old school government registry to ever catch.
- 41:07Precisely. And this raises an entirely new,
- 41:10deeply provocative question for us to consider as we look to the
- 41:13future. We've seen how algorithms can
- 41:16optimize space to cram 8 people into an apartment, and we've
- 41:19seen how they can optimize pricing to evade 100 any day
- 41:22cap. But AI is advancing
- 41:24exponentially. What happens when artificial
- 41:26intelligence starts dynamically pricing and leasing these shadow
- 41:29rooms by the minute? Imagine a future where an AI
- 41:33landlord algorithm doesn't just manage nightly Airbnb rates, but
- 41:37dynamically shifts a property between a share house, a short
- 41:40term rental and a Co working space hour by hour based on
- 41:44micro fluctuations and local demand.
- 41:46That's intense. Are we moving toward a future
- 41:48where the very concept of a long term lease doesn't even exist
- 41:51anymore, replaced entirely by algorithmic minute by minute
- 41:55subscriptions to living spaces? As our cities continue to grow
- 41:59and face critical housing shortages, we have to ask, can
- 42:02we still afford to treat these platforms as tech novelties to
- 42:05be managed with a few tweaks to local laws?
- 42:08Or do we need to completely redesign the concept of property
- 42:11rights and urban zoning from the ground up for an AI age?
- 42:15That is the perfect, slightly terrifying question to leave
- 42:19hanging in the air because the algorithm certainly aren't
- 42:22slowing down, and unfortunately, neither is the rent.
- 42:25Thank you so much for joining us on this deep dive.
- 42:27I hope it fundamentally changed the way you look at the
- 42:29buildings on your street and the invisible digital markets
- 42:32operating inside them. Keep questioning the world
- 42:35around you, keep looking past the blueprints, and we will
- 42:37catch you next time.