Latest / Now Shipping: A Mind the Product podcast / How Anthropic’s agents work 6× more than humans

Transcript

How Anthropic’s agents work 6× more than humans · Now Shipping: A Mind the Product podcast

• 0:00
  SPEAKER_00: Six to one.
• 0:02
  SPEAKER_00: That's the ratio.
• 0:04
  SPEAKER_00: Right now at Anthropic, the AI agents that employees kick off themselves are working six hours for every one hour their human handlers puts in.
• 0:14
  SPEAKER_00: Six to one.
• 0:16
  SPEAKER_00: I'm Mike Belsito, and this is Now Shipping, the weekly AI news show for product people.
• 0:22
  SPEAKER_00: Now, Anthropic just published a report on Friday, October 16th.
• 0:27
  SPEAKER_00: It's called How Employment is Changing at Anthropic.
• 0:30
  SPEAKER_00: Now, I do want to start with a caveat.
• 0:33
  SPEAKER_00: This is Anthropic studying Anthropic.
• 0:36
  SPEAKER_00: It's fair to point out that they sell the thing that they say is changing their work.
• 0:40
  SPEAKER_00: So there is that.
• 0:41
  SPEAKER_00: But that caveat aside, their findings are really interesting.
• 0:45
  SPEAKER_00: And I'm going to hold it up against some more research that came out the very same week that Anthropic released this report.
• 0:51
  SPEAKER_00: And the final result might surprise you a little bit.
• 0:54
  SPEAKER_00: They pulled internal data on employment, output, hiring decisions, internal communications, employee calendars.
• 1:02
  SPEAKER_00: They combined that all with employee surveys.
• 1:04
  SPEAKER_00: And then they sat down for 17 interviews with people about how their jobs have changed.
• 1:10
  SPEAKER_00: Now, Anthropic is five years old and it takes in two to four times as much revenue per employee as Meta and Google did at the very same age.
• 1:18
  SPEAKER_00: Those are companies that we used to point to as the picture of rapid, efficient growth, and Anthropic has blown past them very quickly.
• 1:27
  SPEAKER_00: But how?
• 1:28
  SPEAKER_00: Well, part of it is probably product, yeah, but part of it is who they hired and who they didn't hire.
• 1:35
  SPEAKER_00: Now the report says that back in January, the teams that leaned hardest on Claude started noticing something.
• 1:41
  SPEAKER_00: Writing code, finding bugs.
• 1:44
  SPEAKER_00: These are things that could increasingly be handed off to Claude.
• 1:48
  SPEAKER_00: And then new models landed.
• 1:49
  SPEAKER_00: And one person described it like this: They said, quote, the thing that raised eyebrows was the uplift for performance engineers in particular.
• 1:57
  SPEAKER_00: And it basically overnight changed what we were looking for.
• 2:01
  SPEAKER_00: Now they said that.
• 2:02
  SPEAKER_00: They said overnight.
• 2:03
  SPEAKER_00: Now, by April, engineering and research teams started pulling back on hiring for the lower rungs.
• 2:10
  SPEAKER_00: When managers were surveyed that month, a majority said that they were hiring for a different set of skills because of AI.
• 2:17
  SPEAKER_00: About a third of machine learning managers and a fifth of software engineering managers said that they were overall hiring less.
• 2:24
  SPEAKER_00: Now, since early 2025, the share of junior people in Anthropics technical departments, it fell from 21% to 14%.
• 2:34
  SPEAKER_00: But look at hires specifically.
• 2:36
  SPEAKER_00: Junior share of technical hires were 27%.
• 2:41
  SPEAKER_00: Then they went to just 9%.
• 2:44
  SPEAKER_00: That's a basically a 67% cut.
• 2:47
  SPEAKER_00: And it didn't come out of nowhere.
• 2:49
  SPEAKER_00: Back in January at Davos, Dario Amodai, Anthropics CEO, said that he could already see inside of his own company.
• 2:56
  SPEAKER_00: On the more junior end and the more intermediate end, he said that they actually needed less and not more people.
• 3:03
  SPEAKER_00: And around that same time, Anthropics chief product officer Mike Krieger said that they'd be hiring fewer fresh college grads and looking for experienced people that he called orchestrators of Claude.
• 3:15
  SPEAKER_00: But what do those orchestrators actually produce?
• 3:18
  SPEAKER_00: Apparently a lot.
• 3:19
  SPEAKER_00: I mean, if you fast forward to this past December from where we currently were, commits per engineered Anthropic, they went up about five times.
• 3:28
  SPEAKER_00: Queries per data scientist?
• 3:30
  SPEAKER_00: Up three times.
• 3:32
  SPEAKER_00: Contract per legal employee, up three times.
• 3:35
  SPEAKER_00: A sales rep at Anthropic with no engineering background.
• 3:38
  SPEAKER_00: He talked about how he built a custom dashboard and pipeline tool for their own work.
• 3:43
  SPEAKER_00: They built the dashboard tool in just three to four hours.
• 3:46
  SPEAKER_00: I mean, it's a clear example of how they started seeing the builder mentality live on in real life in all parts of the business.
• 3:54
  SPEAKER_00: Now the report tracked non-technical employees, people with no code commits at baseline, running coding agent sessions.
• 4:01
  SPEAKER_00: In August 2025, there's only about two a month, but by May of this year, there were 90 per month from that group.
• 4:09
  SPEAKER_00: And the org chart is seeing a lot of changes due to all this, too.
• 4:13
  SPEAKER_00: The ratio of engineers to product managers at Anthropic, it now sits around 17 to 1, much higher than maybe what you might be used to.
• 4:22
  SPEAKER_00: Now, the supervising and coordinating roles on technical teams, they've increasingly involved building and shipping code.
• 4:29
  SPEAKER_00: And I'm hearing this from product people outside of Anthropic too.
• 4:33
  SPEAKER_00: Um, a lot, actually, where product teams are now shipping codes.
• 4:37
  SPEAKER_00: One executive assistant credits Claude with augmenting away the really entry-level admin type of work.
• 4:43
  SPEAKER_00: And now the people that were doing that kind of work before are now doing more elevated executive assistant work, almost acting as like chiefs of staff for the executives that they're teamed up with.
• 4:55
  SPEAKER_00: Now, there was a product design team that wrote the greetings on Claude.ai, you know, the coffee and Claude time, uh, you know, those types of things.
• 5:04
  SPEAKER_00: Anyway, in the spring, they built a context graph that taught agents how to write content as well, and it really scaled the team out.
• 5:12
  SPEAKER_00: But then that came the question well, what's left for the human writers?
• 5:15
  SPEAKER_00: So the team started rotating into writing evals and prompts for Claude.
• 5:20
  SPEAKER_00: In the report, it talked about writers who use the language to now drive Claude behavior.
• 5:26
  SPEAKER_00: But is all this actually Claude behind the major shift and how everybody worked?
• 5:31
  SPEAKER_00: Well, Anthropic wanted to know that, so they ran a little test.
• 5:34
  SPEAKER_00: They compared people in the same month at the same company under the same business conditions, some using coding agents, some not.
• 5:42
  SPEAKER_00: 192 engineers who adopted agents against 74 who didn't.
• 5:47
  SPEAKER_00: After adoption, the amount of code they added jumped.
• 5:50
  SPEAKER_00: And on quality, they compared code the same person wrote by hand against code that the person's agent wrote.
• 5:57
  SPEAKER_00: Same month, same directory, they found no difference in quality.
• 6:01
  SPEAKER_00: Now, to be fair, the people who adopted agentic AI, they were already on that steep climb before they adopted it.
• 6:08
  SPEAKER_00: So, you know, tool adoption isn't random.
• 6:11
  SPEAKER_00: Um, the people that tend to tool up are people that are really leaning in.
• 6:16
  SPEAKER_00: So I'd say like this is all evidence.
• 6:18
  SPEAKER_00: It's not really proof, though, right?
• 6:20
  SPEAKER_00: And the report itself says more code doesn't necessarily mean better code or business impact.
• 6:26
  SPEAKER_00: Now, I want to set that aside and come back to it in a minute.
• 6:30
  SPEAKER_00: Um, because here's the part of the report that I might like put on some big poster.
• 6:34
  SPEAKER_00: Human attention is the bottleneck.
• 6:37
  SPEAKER_00: Those interactive clawed sessions, they spent around 80% of their time idle, waiting for a person.
• 6:44
  SPEAKER_00: One engineer made a comment in the report that the agents would be even more productive if they weren't waiting on the people.
• 6:50
  SPEAKER_00: And they did the math on what it would take to actually read everything the agents produced.
• 6:55
  SPEAKER_00: They converted output to words at average reading speed.
• 6:59
  SPEAKER_00: And even for non-technical teams, reading all of it would take up more than half of a work week.
• 7:04
  SPEAKER_00: Now, for engineering and research, the chart tops out at about 400 hours per person per week.
• 7:12
  SPEAKER_00: You could do the math here.
• 7:13
  SPEAKER_00: Now, this is all an estimate.
• 7:15
  SPEAKER_00: It assumes you read every word of code of every output, whatever it is that Claude is producing.
• 7:21
  SPEAKER_00: And the reality is you're not going to do that in the cases where Claude is producing things like, you know, a massive amount of code.
• 7:29
  SPEAKER_00: But one engineer compared overseeing their Claude agents to almost like babysitting.
• 7:34
  SPEAKER_00: And I'm gonna read this because you'll never forget it.
• 7:37
  SPEAKER_00: They're like, quote, I need my juice box open, and then it's emergency that my juice box is opened.
• 7:42
  SPEAKER_00: And that's voiced with the same level of concern as the one who sliced their finger and is bleeding.
• 7:48
  SPEAKER_00: So what do people do?
• 7:50
  SPEAKER_00: They build tools to manage the agents.
• 7:52
  SPEAKER_00: Now, the report called it metawork, the share of a typical employees' agent session spent building skills, plans, reviews for other agents, dashboards.
• 8:02
  SPEAKER_00: All of that work back in fall 2025 took about 10% of a person's time.
• 8:08
  SPEAKER_00: Well, that shot up to 22% by this past spring.
• 8:12
  SPEAKER_00: In infrastructure teams, it shot up to 29%.
• 8:15
  SPEAKER_00: Written standing instructions for agents grew from a few hundred in July 2025 to 8,500 a year later.
• 8:24
  SPEAKER_00: One engineer built an agent chief of staff just to triage requests from other agents.
• 8:30
  SPEAKER_00: Another wrote a performance review for their clod.
• 8:32
  SPEAKER_00: A lawyer went from reviewing legal questions to what they called SEO for legal advice.
• 8:38
  SPEAKER_00: April survey, one in five employees said that their role had shifted from doing the work to managing clods.
• 8:45
  SPEAKER_00: But the part that they were automating away by managing these clods, it was the 80% of their work that they felt like could be automated away, the mechanics of their work.
• 8:55
  SPEAKER_00: The 20% that was left, this was the stuff they could now focus on.
• 8:59
  SPEAKER_00: Now, in this AI everything world, what happens to the team?
• 9:03
  SPEAKER_00: Well, at Anthropic, they started seeing new team members speaking up.
• 9:07
  SPEAKER_00: Agent team members.
• 9:09
  SPEAKER_00: The agents were showing up in Slack.
• 9:11
  SPEAKER_00: They now appear in nearly half of active public channels.
• 9:14
  SPEAKER_00: The report maps the communication network of one big department.
• 9:18
  SPEAKER_00: Blue dots are people, orange dots are the agent threads.
• 9:22
  SPEAKER_00: In the picture before, it was mostly blue.
• 9:24
  SPEAKER_00: In the after, orange swallows the whole thing.
• 9:27
  SPEAKER_00: Now, agents didn't replace the human conversation.
• 9:31
  SPEAKER_00: Human threads in that department got the same number of human responses.
• 9:35
  SPEAKER_00: The agents just happen to layer on top of things and they cluster in dedicated clawed and human channels.
• 9:41
  SPEAKER_00: They aren't hubs linked to lots of people together.
• 9:44
  SPEAKER_00: They became a team for a single employee.
• 9:47
  SPEAKER_00: So think about this.
• 9:48
  SPEAKER_00: Every person effectively got their own little team, their own agent team.
• 9:53
  SPEAKER_00: Now, if agents work off context, the context has to be visible.
• 9:58
  SPEAKER_00: A decision made at an unrecorded meeting is invisible to the agents.
• 10:02
  SPEAKER_00: One employee had their name on an old doc about Claude Swag.
• 10:06
  SPEAKER_00: Well, long after their job changed, agents kept bringing them requests about t-shirts and mugs.
• 10:12
  SPEAKER_00: Private work in unlogged meetings and direct messages becomes confusing.
• 10:16
  SPEAKER_00: Discouraging even.
• 10:18
  SPEAKER_00: And most organizations don't necessarily run this way.
• 10:20
  SPEAKER_00: So it's a big shift to start to essentially run your organization completely out in the open when anybody has access to what's happened in any meeting.
• 10:29
  SPEAKER_00: Now, speaking of meetings, the median employee spent 20% more time in meetings than a year ago.
• 10:35
  SPEAKER_00: Even technical individual contributors saw their meeting time go up 16%.
• 10:40
  SPEAKER_00: I was honestly pretty surprised by this.
• 10:42
  SPEAKER_00: I thought maybe more agents meant less meetings, but that's not necessarily what happened at Anthropic.
• 10:47
  SPEAKER_00: Now, on the topic of hiring, who thrives in a place like this?
• 10:51
  SPEAKER_00: Well, in the report, they process interview scorecards for thousands of hires.
• 10:56
  SPEAKER_00: In 2024, hiring managers mentioned a candidate's track record with AI tools in about 5% of technical assessments and 7% of non-technical assessments.
• 11:06
  SPEAKER_00: By 2026, that tripled for technical roles and quadrupled for non-technical roles.
• 11:13
  SPEAKER_00: But the deeper change was that in technical interviews, the biggest jump was comments about autonomy and judgment.
• 11:20
  SPEAKER_00: Domain expertise actually shrank as a share of the conversation.
• 11:24
  SPEAKER_00: One manager commented that they expect models to continue to be jagged.
• 11:28
  SPEAKER_00: So for their team, they want them to be fungible across domains.
• 11:32
  SPEAKER_00: All right, now is the twist.
• 11:34
  SPEAKER_00: Remember, I said there was a twist coming.
• 11:36
  SPEAKER_00: In the past five days, other research came to light elsewhere, and it points to something a little different.
• 11:42
  SPEAKER_00: Kate Niederhofer, Christina Rapuano, and three other co-authors released some research for Harvard Business Review on October 15th.
• 11:51
  SPEAKER_00: And a new study found that when people treat AI like a teammate, trust among the human teammates actually erodes and their ability to coordinate weakens.
• 12:01
  SPEAKER_00: And they gave this a name.
• 12:02
  SPEAKER_00: They called it relation slipping.
• 12:05
  SPEAKER_00: And it happens two ways.
• 12:07
  SPEAKER_00: Sometimes you end up skipping the conversation with your colleague because a quick answer from AI is right there.
• 12:13
  SPEAKER_00: So you stop updating your mental map of who knows what.
• 12:16
  SPEAKER_00: Other times people offload their work to AI and their thinking becomes less visible.
• 12:21
  SPEAKER_00: So the map goes stale.
• 12:23
  SPEAKER_00: Now, the piece doesn't give sample sizes, so treat it as a signal, but their advice is clear.
• 12:28
  SPEAKER_00: They say to use AI to prepare for conversations, but there's a downside if you're using it to replace the conversations altogether.
• 12:36
  SPEAKER_00: And then there's the company Jabra.
• 12:38
  SPEAKER_00: They recently commissioned a survey of more than 6,000 knowledge workers across six countries.
• 12:44
  SPEAKER_00: Among the heaviest AI users, 57% said they were doing more administrative and coordination work.
• 12:50
  SPEAKER_00: Among everyone else, 38% were doing more of these kinds of tasks as well.
• 12:54
  SPEAKER_00: As one write-up put it, AI is making individual tasks easier.
• 12:58
  SPEAKER_00: That doesn't necessarily mean it's making work easier.
• 13:02
  SPEAKER_00: Now, this is self-reported and Jabra cells voice AI, so take it with a grain of salt.
• 13:07
  SPEAKER_00: But even still, it's interesting.
• 13:08
  SPEAKER_00: Now, one other thing from the study stood out to me among heavy AI users, 64% said that they collaborate with colleagues more than they did a year ago.
• 13:18
  SPEAKER_00: And even light AI users saw a 25% increase in that as well.
• 13:21
  SPEAKER_00: So this kind of goes against the relation slipping point that the Harvard Business Review study made.
• 13:26
  SPEAKER_00: So who's right?
• 13:27
  SPEAKER_00: Does AI make teams more connected, less connected?
• 13:30
  SPEAKER_00: My read, maybe both.
• 13:32
  SPEAKER_00: And the difference isn't the model.
• 13:34
  SPEAKER_00: Look at what Anthropic did.
• 13:36
  SPEAKER_00: They made their work visible.
• 13:37
  SPEAKER_00: At Anthropic, agents work for individuals, but people still decide the big stuff together.
• 13:44
  SPEAKER_00: It's a deliberate design that they made.
• 13:46
  SPEAKER_00: Now look at the version that most companies are living today.
• 13:50
  SPEAKER_00: A lot of people are quietly working with their own AI.
• 13:53
  SPEAKER_00: Maybe it's in DMs, it's in private docs.
• 13:56
  SPEAKER_00: Maybe they're skipping the conversation because it's faster.
• 13:59
  SPEAKER_00: That's the relation slipping version.
• 14:02
  SPEAKER_00: So here's what I'd take away from all this.
• 14:04
  SPEAKER_00: How many hours a week does your team spend reading, reviewing, and redirecting what agents produce?
• 14:10
  SPEAKER_00: Anthropic found that it adds up to more than half the work week for non-technical teams.
• 14:15
  SPEAKER_00: Now, if you don't know your number, that is what I would try to find out first.
• 14:19
  SPEAKER_00: And it changes what you build.
• 14:21
  SPEAKER_00: I mean, if you ship anything with an agent in it, the human reading the output is your constraint.
• 14:26
  SPEAKER_00: 80% of an agent session sits idle, waiting on a person.
• 14:30
  SPEAKER_00: That's a design problem and it's yours to solve for.
• 14:33
  SPEAKER_00: Now, summaries that say what's changed and what needs a decision, status views, approval cues ranked by consequence, because the juice box emergency and the bleeding finger cannot look the same on your screen.
• 14:44
  SPEAKER_00: The products that win won't be the ones that produce the most output.
• 14:48
  SPEAKER_00: They'll be the ones that cost the least attention.
• 14:51
  SPEAKER_00: And it changes how you measure too.
• 14:53
  SPEAKER_00: Commits per engineer up five times.
• 14:55
  SPEAKER_00: I mean, does anybody think that your customers got five times the value?
• 14:59
  SPEAKER_00: Anthropic's own report even says more code isn't more impact.
• 15:03
  SPEAKER_00: Now, if your dashboard counts activity per person, AI will conflate it.
• 15:08
  SPEAKER_00: And you'll cheer for a number that maybe means nothing.
• 15:11
  SPEAKER_00: So pick the outcome instead.
• 15:13
  SPEAKER_00: Time to a shipped decision, customer results, whatever your customer would actually notice.
• 15:19
  SPEAKER_00: Then there's your own job.
• 15:22
  SPEAKER_00: 17 engineers for every PM, managers committing code.
• 15:26
  SPEAKER_00: If you're a product manager, the coordination part of your week is the part that's actually shrinking.
• 15:31
  SPEAKER_00: What's left is what hiring scorecards keep saying autonomy, judgment, deciding what's worth building, deciding what done actually means.
• 15:40
  SPEAKER_00: Writing a spec clear enough that an agent can run on for hours, that is the skill.
• 15:46
  SPEAKER_00: And if you're hiring, ask what you should actually be screening for.
• 15:50
  SPEAKER_00: Anthropic scorecards move from domain expertise towards judgment and AI fluency.
• 15:55
  SPEAKER_00: Make sure yours moved on purpose, not by accident.
• 15:58
  SPEAKER_00: And then there's the whole junior employee question.
• 16:00
  SPEAKER_00: I mean, Anthropic says out loud that its protections don't help the people who never get hired.
• 16:05
  SPEAKER_00: Microsoft did some research and they pegged employment for 22 to 25 year olds in highly AI exposed jobs down 16%.
• 16:14
  SPEAKER_00: Now, if your team stopped hiring juniors, somebody has to answer where your next senior comes from.
• 16:20
  SPEAKER_00: Now, I don't have the answer to that, but I will say this.
• 16:23
  SPEAKER_00: Those junior employees that are starting to get hired less, those 22 to 25 year olds, remember, these are probably AI natives.
• 16:31
  SPEAKER_00: These are people that have been growing up with Chat GPT and Claude and Grok.
• 16:36
  SPEAKER_00: To them, AI is just tech that they live with.
• 16:39
  SPEAKER_00: I know a lot of product people that have been at it for 20 plus years who are curious product people, but then there's also some that may not be willing to change the way that they work so fast.
• 16:50
  SPEAKER_00: Those young, hungry, AI native juniors.
• 16:53
  SPEAKER_00: I think if you ignore them, you're doing so at your own peril.
• 16:57
  SPEAKER_00: Now, to close things out here, I do have a task for you, uh kind of based off of the research that we went over today.
• 17:04
  SPEAKER_00: Here's an experiment you could run this week.
• 17:06
  SPEAKER_00: Pick one decision your team makes in a private meeting or a DM and write it down somewhere that everyone can see it.
• 17:13
  SPEAKER_00: Everybody and every agent.
• 17:15
  SPEAKER_00: See what starts to change now that the information is accessible.
• 17:18
  SPEAKER_00: It might not be instant, but if everybody starts doing this, you may find that you and your team, both humans and non-humans, will have a lot more context to work with than the kind that lives in our heads.
• 17:30
  SPEAKER_00: And next time you reach for AI to answer a question, ask yourself if the better move is actually to ask your colleague first.
• 17:37
  SPEAKER_00: Well, that's gonna wrap things up for this week's episode of Now Shipping.
• 17:42
  SPEAKER_00: If you enjoyed this week's episode, please tell a friend, spread the word.
• 17:46
  SPEAKER_00: All of that really, really helps.
• 17:48
  SPEAKER_00: And if you have any feedback or you want to see something new next time, leave a comment below.
• 17:54
  SPEAKER_00: I promise you, I read every comment you leave.
• 17:57
  SPEAKER_00: For the team at Mind the Product, my name is Mike Belcito, and this is now shipping.