Latest / Artificial Developer Intelligence / The Psychology of Software Teams with Dr. Cat Hicks

Transcript

The Psychology of Software Teams with Dr. Cat Hicks · Artificial Developer Intelligence

• Shimin (00:00) Hello and welcome back to Artificial Developer Intelligence, a weekly conversation show where we try to sort out the hype around AI from what it actually delivers. My name is Shimin Zhang, and this week is a little different. No news, no clock, and I'll be on vacation. Instead, Dan and I sat down with Dr. Cat Hicks, psychologist for the humans in tech to talk about the psychological impact of AI on software engineering. You can find links to Cat's book, podcast and newsletter in the show notes. So please enjoy and we'll be back in two weeks. Shimin (00:35) It is my pleasure today to chat with Dr. Cat Hicks, a psychologist for the humans in tech. She is the principal scientist at Catharsis Consulting, where she helps software organizations transform with human-centered evidence strategies. She is also the author of the book, The Psychology of Software Teams. I have a copy of the book here. Is it upside down? Nope. Perfect. I got a copy of the book with my own money. Cat Hicks (01:01) What to Shimin (01:03) earlier this summer. I really enjoyed reading it. So thank you very much for joining us today, Cat. Cat Hicks (01:10) It's my pleasure. Thank you for having me. Shimin (01:14) so as a software dev, when I think of a psychologist, I think child development, birds in boxes, being treated inhumanely, or abnormal psychology. So how did you get into the study of the psychology of software teams? Like should we be considered Cat Hicks (01:31) Yeah. Shimin (01:32) a sub branch of abnormal psychology? Cat Hicks (01:36) So I feel like all of us have spent some time in or around the realm of abnormal psychology. So there's nothing wrong with that. The Shimin (01:44) Ha ha ha. Cat Hicks (01:44) second thing I will say is we hey we have a lot of rules for how we treat those birds, actually. this is really funny. I went to grad school with folks who did bird research and treated their birds incredibly well. and study their bird song and things like that. And we've actually got really, really interesting sort of a AI related insights from, you know, studying the language of other animals. Shimin (02:04) Interesting. Yeah. Cat Hicks (02:04) yeah, so we could spend the whole time on that instead. But you know, for me, I think I'm really, really interested in psychology in the real world and kind of you could be many, many different kinds of scientists, many different kinds of social Shimin (02:17) Mm-hmm. Cat Hicks (02:17) scientists, you know. I kind of consider myself an applied scientist, so someone who is interested in how human beings solve problems in the real world. So, software team, software developers, it's a really interesting and fun example to me of like human cognition in the And so Shimin (02:34) Mm. Cat Hicks (02:34) just like all forms of human cognition in the in the wild, it is sort of specific, like it's shaped by the problems of that world. But we also like use kind of, you know, this the architecture of our minds that we share with every human to do so. So there's both like, yes, you're part of the big world of psychology, but also, yes, I think software developers, sh software development, software engineering, or like the problem solving involved in dealing with technical systems is maybe how I would think of it. It could be its own branch Shimin (03:02) Mm. Mm-hmm. Cat Hicks (03:05) of of psychology. I'm kind of fighting to try to make it more of its own branch of psychology. Yeah. Shimin (03:11) Yeah we Cat Hicks (03:11) But I would I wouldn't classify it as abnormal. Shimin (03:15) So we can f yeah, we can finally get the the recognition that we deserve, right? no, it's not true. We get plenty of recognition. arguably too much recognition. So kinda one Cat Hicks (03:20) Yeah. Maybe not the right kind. Shimin (03:27) Yes, exactly. so I think one image that every reader of the book would definitely take away from is the picture of a brains in jars software team. Cat Hicks (03:40) Mm. Shimin (03:40) I can see it right now. can you tell us a little bit more about like what does it mean to have a brains in jars software team and why it could be harmful? Cat Hicks (03:50) Yeah, that is also an image that I love. I started using it because I was trying to describe to my friends in tech this feeling that I was getting, which was so odd, like that that exactly as you say, like software engineers are very valued, right? They are they are thought of, they're constantly talked about. I mean, honestly, it must be kind of tiring, right? To see your own job like in the headlines all the time. And so it is something we spend a lot of attention on. At the same time, I kept going into these organizations. Or doing these studies and hearing this feeling from developers that they were not being seen as whole people. And so Shimin (04:28) Mm. Cat Hicks (04:28) I started to work on this metaphor of calling it the brains in jars organization. So yeah, we have software developers, yeah, we care about their thinking, but we do it in a very specific way that's kind of bizarre. Like it's kind of stripped of all of its humanity. So I see, you know, let's say I'm a leader and I see my engineering team as just a bunch of fungible parts, like little machine parts or Shimin (04:51) Mm-hmm. Cat Hicks (04:52) little factory parts. And this this brains in jars like metaphor was funny. It was also funny because. Because I am a scientist. So people are always a little bit scared, like, hey, you're a psychologist. Are you gonna treat me like I'm just a brain? You know, Shimin (05:05) Right. Cat Hicks (05:06) and I try to tell people like, listen, I actually really, really care about the environment around you, your larger humanity, all of these pieces. And so it was just a way to throw this out and say, have you ever seen this? Have you felt like this? Have you like been in a meeting where it's been like you've been felt like you just all that matters Shimin (05:23) Right. Cat Hicks (05:23) about you is this sort of reduced abstraction of your brain. And then the interesting kind of science part of this too is like it's not a very accurate model of our cognition actually. It's not very accurate to how innovation happens, what makes us Shimin (05:38) Right. Cat Hicks (05:38) really good thinkers or what makes us kind of leap forward, have like big breaks Shimin (05:43) Mm-hmm. Cat Hicks (05:44) in our solution space. We actually have a ton of social cognition, like ways that we build on our solutions. So it was also a way of to for me to draw attention to like this brains and jars thing is never what produced really good software actually. We just kind of act like it on the business side. But but then you go solve problems and you're in this very different ecosystem with it. Shimin (06:05) Yeah, when I when I think to some of the the most famous teams and companies, I think like Bell Labs, before Cat Hicks (06:10) Mm. Yeah. Shimin (06:13) they they got broken broken up, and I think I see a lot of collaboration, a lot of like flat hierarchy. Cat Hicks (06:19) Mm-hmm. Shimin (06:20) yet, personally at least I've never thought two seconds about this idea of like what actually makes the best kind of software teams, right? And then w if you sp Cat Hicks (06:30) Hmm. Shimin (06:30) if you do spend a couple of minutes thinking about it, Of course this idea of a lone genius in a basement writing just amazing software i it's definitely not the way our best work gets done. But this framing is Exactly. we're all ninjas, Dan (06:43) Like how much harm did the ten X engineer stereotype do to the industry as a whole? Cat Hicks (06:47) Yeah. Yeah. But I Shimin (06:51) yeah. Cat Hicks (06:52) I do think I liked and I tried to make this point in the book. You know, you tell me if it worked, but I I do think we get things out of that, right? And like it wouldn't persist, Shimin (07:00) Mm-hmm. Cat Hicks (07:00) it wouldn't it wouldn't be so sticky if it didn't do something, right? And like I Shimin (07:04) Yep. Yep. Cat Hicks (07:04) mean, there've been times in my own life I've kind of a little bit of a workaholic, you know, and I sort of struggle to find like my own Path and like now I have a family. You know, I can't live that way, right? Life is Shimin (07:15) Mm-hmm. Mm-hmm. Cat Hicks (07:16) more more about more things than that. But there were times when I felt like I was in a really unfair circumstance. Like when I was younger, I had to work really hard and believe that my own intelligence believed that I could kind of do this grind set lone genius thing. So like there's lots and lots of reasons I think that we fall into seeing things that way. But one of the big th points that I try to make is in my work is just that it has a real cost. You know, it has a real cost that's so much bigger than we like to see. And then the good ways of living and working have real up. side actually that they kind of win out evolutionarily speaking across Shimin (07:54) Mm-hmm. Cat Hicks (07:54) groups of people and so if you can have this like faith and this hope, you know, and this courage to kind of not give in to that stereotype, then you could really have an awesome life. So that's what I'm like trying to offer Shimin (08:05) Ha ha ha. Cat Hicks (08:05) to developers, you know. I think I can Shimin (08:08) Right. And and and the first yeah, and the first step is like asking a fish to notice to actually notice the water, right? Before you can determine Cat Hicks (08:15) Mm. Shimin (08:16) what the trade offs are. And and if we're if Cat Hicks (08:18) Yeah. Shimin (08:18) the this framing is so perva pervasive that we don't even notice the waters all around us. Cat Hicks (08:23) Yeah. Shimin (08:24) but Dan as a 10x engineer, how does that how do you feel Dan (08:27) Yeah, right. Shimin (08:28) like it has impacted your career development, this idea of brains in jars? Dan (08:35) my goodness. I mean I definitely appreciate being on cross functional teams more than I do ones that aren't because I feel like when you blur the lines, especially on a small team, that's when you start actually getting work done and really good work usually compared to like if everybody acts in their own little silo, even if they are part of the same team and yeah. So like that's something I've always like really strive to convey to teams that I work on and the way that I do that is maybe a little bit unusual, which is like making a ton of jokes and being super vulnerable about when I'm, you know, upset about something or anything else, because it's like when we go to work and pretend that we're robots, it's not a good way to actually like connect with each other as humans. So and it's unfortunately very common. Cat Hicks (09:22) Mm. I love that. Shimin (09:23) Yeah. Dan (09:24) So shut off all emotions there. There was a cat, have Cat Hicks (09:30) Yeah. Dan (09:30) you ever heard of the core protocol stuff? That was also a a big Cat Hicks (09:34) Tell me tell me what you're thinking of. Yeah. Dan (09:36) influence on me too. So like I had to actually take a team class in college and they did the I think it was McCarthy core protocol stuff. And one of the things that I Cat Hicks (09:46) Mm. Dan (09:46) really took away from that and appreciated that's driven a lot of this for me is that there was the check-in ritual. Which is like, Cat Hicks (09:52) Mm. Dan (09:53) I'm checking into work today and it was optional, which is a little like boof, you know. And then the second thing is you would check in and say if you're mad, sad, glad, or afraid, which is kind Cat Hicks (10:04) Mm. Dan (10:05) of a Shimin (10:04) Mm. Dan (10:05) limiting emotional palette, but I think that's like a good place to start for some people that maybe aren't thinking about that. And you have to pick at least one and you Cat Hicks (10:10) Yeah, no, I love that. Yeah. Yeah. Dan (10:13) know, optionally more, and you don't have to say why unless you want to, which is like kind of a great way to like build safety. Cat Hicks (10:17) Yeah. Yeah. Dan (10:20) I've never seen a team do that in real life, but I I've always thought that was like such a cool ritual to like try to build that type of like I don't know. Team focus. Cat Hicks (10:27) I like that. I think it kinda, you know, regardless of whether it's the the perfect representation of emotions or totally you know what I mean? It's Dan (10:33) Right. Yeah. Cat Hicks (10:35) even just it you don't have to be perfect, right? To actually say this is an acceptable category of thing to bring in. And I think something really cool about what you just said, someone like you doing this, right? Because I'm assuming you have a lot of technical credibility, like where you stand. And so when you're able to joke, show feelings, whatever, you know, you're actually doing something very important in your environment. You're kind of giving this message that's like, none of this stuff, this feelings, you know, these things None of this negates my technical credibility. Like none of it makes me less Shimin (11:09) Mm. Cat Hicks (11:09) of an engineer. And we have a lot of stereotypes, right? That kind of we get formed by. They are in our society that like engineers are cold, you know, engineers don't have feelings. And I I have worked with this group of people for like a long, long time, like many, many developers. And These are not cold people to me. And maybe not everybody likes to go to a party or whatever, neither do I all the time. But we do have all of these kind of really interesting and cool things about the way we process information Shimin (11:41) Mm-hmm. Cat Hicks (11:41) that are counter to these stereotypes. And I think when someone like you, you know, can model that It just it just opens up people's minds because the way our minds work, we kind of have these mental models going all the time, and different aspects of them can get foregrounded or activated. Like nobody really believes software developers are not human at all, you know? But I mean, maybe that's some extreme, there's somebody. Yeah, there's Dan (12:04) That's my wife. I don't know. Let's see what she's Shimin (12:05) Mm-hmm. No. I just play one on Zoom calls, yeah. Cat Hicks (12:10) let's say it's a very tail end of the distribution, but like. But like most of the people around you, when they start treating you like a brain in a jar, it's because like certain stereotypes are getting foregrounded or activated, right? And you kind of can activate different ones in the moment when you do this really cool stuff. Like you bring in humor, you point out like, look, we're all trying to figure this out, or this doesn't feel great today. Those are like actually parts of the problem solving we do, I think. Yeah. Hmm. Dan (12:38) It's always helpful to have a reminder that you exist solely to create shareholder value too, you know. Cat Hicks (12:42) That's right. Shimin (12:45) crunch those tickets. another another thing I really loved from the book is this idea that the kind of the most some of the most meaningful work that we do is based on the idea of effortful learning. and I Cat Hicks (13:00) Mm. Shimin (13:00) I really find that to be true myself that like if I think back on my software development career it's when I really be able to like hanker down and really go deep in a topic and and really spend a lot of time struggling with a problem that I kind of come up with the Cat Hicks (13:12) Mm. Shimin (13:12) the best solutions and also like you know reach my quote unquote full potential or whatnot. So it just also happened to be that you're also the creator of the learning opportunity skill, a Claude and Codex skill for deliberate skill development Cat Hicks (13:27) That's right. Shimin (13:27) using AI assisted coding, that also leans heavily into the idea of using AI for upscaling instead of skill atrophy. Cat Hicks (13:33) Mm-hmm. Shimin (13:34) fun fact, that was one of the first skills that I had anyone at like a IRL AI meetup like pitch the skill to me personally. Like, have you Cat Hicks (13:42) Ha ha Shimin (13:43) heard of the skill called learning opportunities from Dr. Cat Hicks? I was like, Of course I do. of course I do. so Cat Hicks (13:46) amazing. It's a cult actually, yeah. Shimin (13:52) what was the story and history behind behind the learning opportunity skill? Cat Hicks (13:55) Yeah, thank you for asking. Yeah. yeah, it is a cult, so you know, you were recruited, but no, I you know, it's so funny. Shimin (14:02) It worked. Cat Hicks (14:03) I I've worked on learning for a long time. Like before I worked with software teams, I was interested in learning. So I did large scale work with, you know, l like all kinds of learning situations. And I've always been interested in people who are learning despite friction and outside of the norm, like non-traditional students or, you know, online learners trying to change their career paths. Or people trying to break into tech, right? There's a to me like this really beautiful value and culture in software to say you don't always need, you don't need a degree, right? You could teach yourself things. You could if you can engage in this problem solving, like we welcome you. You know, that's a reason Shimin (14:39) Mm-hmm. Cat Hicks (14:40) that I was drawn to working with these folks in the first place, because I just loved that attitude. Along comes AI, and there's so much fear, you know, understandably so. There's so much stress, there's so many. poor decisions being made about certain AI implementations Shimin (14:56) Mm. Cat Hicks (14:56) or adoptions or practices or your teammate is slamming you with a thousand lines of code, you know, or something. You know, there's a lot of churn and turmoil. And Out of that churn and turmoil, I kept hearing what I would describe as this really like deficit mindset about developers. This kind of like, we will just lose all our skills, we will just our brains will melt, you know, we won't be able to learn anything anymore. There's no craft, there's no expertise, right? We're still heavily Shimin (15:23) Right. Cat Hicks (15:23) engaged in that war, by the way. Like these questions are not answered. Shimin (15:26) Ha ha. Cat Hicks (15:27) But I did not think that the answer was going to be. No one can learn anymore, you know, because I've seen how much people are capable of navigating like complex systems and taking like a detective work approach to it. And I was really curious about how you could build a toy version of that into AI. I don't think Shimin (15:46) Mm. Cat Hicks (15:46) the AI tools we've built for developers are necessarily currently built around actually helping developers learning that much. You know, I think they're very like obviously. produ production focused. But I thought it's so interesting to take the affordances that AI gives you, like quickly wayfinding throughout, you know, finding places to start or or lateral pattern matching, like scanning across the code bases you're in. And I knew a lot of like techniques and strategies from learning science that help people Shimin (16:19) Is it? Cat Hicks (16:19) study. And something really useful about this is like, you know, to know is that we're kind of bad at knowing what helps us learn. You know? So Shimin (16:27) Mm. Cat Hicks (16:27) we get like dis when things feel really fluent, we think that means it must mean I'm learning. And so we Shimin (16:33) Right. Cat Hicks (16:33) tend to not do strategies that actually help us a lot. So things like quizzing ourselves, or forcing Shimin (16:39) Mm-hmm. Cat Hicks (16:39) yourself to, you know, if you sketch the architecture of a system that you're diagramming before you actually implement it all. That's a nice thing to do. Yeah. And I know that we have tons of rituals and practices and best practices. You're supposed to do all this stuff. But like it's hard, right, in the actual day to day moment to do it and so Dan (16:58) The I think I'm the only person that I work with that uses like the Zoom markup tools to like the fullest extent humanly possible of Cat Hicks (17:04) Yeah. Yeah. Dan (17:06) them because like if I don't if I'm not drawing on the screen in some way, it's not really a Dan meeting. So that's just kind of validating for me. I'm like awesome. Cat Hicks (17:15) Yeah. Totally. So Dan (17:16) hand drawing architecture and no neat boxes like because then you Cat Hicks (17:19) Yeah. Dan (17:20) the other thing I found with that is then if you talk to it as you're doing it, it's a different experience for everybody because you're hitting them with Cat Hicks (17:26) Yeah. Dan (17:27) two different like learning modalities than Shimin (17:28) Mm. Dan (17:29) looking at Cat Hicks (17:30) Like that. Dan (17:30) a static image and like pointing at pieces of it, you know. Cat Hicks (17:34) Like that. Yeah. And in psychology we might call that unpacking. Like it's really, Shimin (17:39) Mm-hmm. Cat Hicks (17:40) really easy to feel like I mean, our minds are built to process, like, look around this world and be like, Where's the tiger or where's the food? You know, like we are Dan (17:48) Yeah. Cat Hicks (17:49) we are built to move really fast. Like, and that's a cool Shimin (17:49) Mm-hmm. Cat Hicks (17:52) thing about us, but it doesn't make us great at building machines or technology sometimes. Shimin (17:56) Mm. Cat Hicks (17:57) And so we especially struggle sometimes with like mechanistic reasoning. And this is like huge Shimin (18:01) Mm. Cat Hicks (18:02) in engineering, right? You probably both have cultivated So many ways to get yourself to actually think about like cause and effect and mechanism and and this leads to that. And then when you work maybe with a junior, you know, you have to sort of show them how to deepen their understanding and not just stay at the surface level where they think they understand things, but they're actually like skipping glossing over a lot that they are are making an assumption about. So learning ops kind of forces people to do that. It gives you like a 10 or 15 minute exercise, it pulls it from the files you're working with. And I also just wanted to show my friends like look just doing 10 minutes actually does something to your mind. It does something to the way that you think. It doesn't have to be like five hours, you know. So yeah. I've been really pleased at how many people have used it. Yeah. Shimin (18:50) Yeah. again, like like I mentioned earlier, I'm a convert. I've built a lot of personal development skills that have nothing to do with coding. using learning Cat Hicks (18:58) Yeah. Shimin (18:59) opportunities as as kind of a reference, teaching me, you know, anywhere from like topics in AI to communication skills. But speaking of recently we covered on the show a paper on the effects of AI and how it gives everybody the Dunning Kroger effect. Kind of basically tricking Cat Hicks (19:17) Mm. Shimin (19:17) us into thinking that we are more skilled when our actual skill could potentially be atrophying. So the paper called this effect the AI mediated metacognitive decoupling. I'm really happy that we have an actual psychologist on the show, so I can Dan (19:30) Say that four times fast. Cat Hicks (19:29) Mm. Ha ha ha. Shimin (19:34) ask an expert, what exactly is metacognition and how can we develop it? Cat Hicks (19:38) Yeah, yeah. So this is like the skill or really the family of skills to pay attention to right now. Metacognition as is as simple I mean people argue about what it is, but it's like you're thinking about your own thinking. And it is not just it's let me see, how do how do I describe this? It gets really meta, and so I try to break it down in a way that's a little bit more accessible. When we go about thinking in the world, we're using some of our like core cognitive capacities. So things like you're working memory, or you might have a lot of content knowledge, you might be really good at planning, you might be really good at creativity. There's like a lot of different things. It's not one thing that drives our problem solving. And some of us are better at some of them than others. And then there's this whole world, which is basically like how well you use what you have. And that Shimin (20:29) Mm. Cat Hicks (20:29) also means how much are you actually calibrating like your Shimin (20:34) Mm-hmm. Cat Hicks (20:35) idea about what you know. So if you hear people talk about decoupling, they essentially mean you're not getting good feedback about what you actually understand. So it sounds really scary, but I actually think it's not that scary. It's kind of tractable. It means we're starting to produce a lot of stuff and we're feeling like we understand it, but we're actually not Shimin (20:55) Mm. Cat Hicks (20:55) always getting accurate feedback, and you need to be Developing the skills and the strategies that really do give you accurate feedback and help you build the right mental models at the right time. So that's a big challenge. I don't think we Shimin (21:09) Mm-hmm. Cat Hicks (21:09) figured it out 100% for AI, but we see a lot of clues. So, like you see these scary, scary research studies where people say, my gosh, we gave some people math problems and other people math problems and they solved it with AI. And then the people who solved it with AI like couldn't even remember, you know, what they did for the last 15 minutes. Sounds Really, really scary, but also Shimin (21:28) Right. Mm-hmm. Dan (21:29) Yeah. Cat Hicks (21:30) if you look at it, you're kind of like, Well, do you ever remember things you just copy-paste? You know, like not Shimin (21:35) All right. Cat Hicks (21:36) that well. And so if your goal is to really, really understand something, you want to take a few metacognitive strategies to that task. So that doesn't mean I think give up all technology, write everything by hand, but I tell people, you know, I try to write a little bit by hand and then scale it up, Shimin (21:55) Mm-hmm. Cat Hicks (21:55) and then I try to Interrupt my generation, you know, I'll like do agentic coding for 45 minutes and then do 10-15 minutes of learning op where I'm, you know, Shimin (22:04) Mm. Cat Hicks (22:05) producing a mental model, testing it. And there's all kinds of strategies that are really interesting in this bucket of like metacognition. And I'll just leave you with like one more tidbit, which is really cool, I think, is that we can actually see that if you invest in metacognitive strategies, it makes you really much more successful over your life. And it's hard for us to find these like like intervention points that actually make Shimin (22:28) Mm. Cat Hicks (22:28) this big difference for people. It's hard to change people's like working memory. It's hard to change your, you know, like level of creativity. There's all kinds of things that are really hard to change. It's actually not that hard to improve your metacognitive strategies. So it's also exciting just because it is something that we can improve and we see it predict life success for people. Shimin (22:51) is there a follow-up sequel to learning opportunities for metacognitive opportunities coming up? Because I I I will line up for that. Cat Hicks (22:56) Yeah. Yeah. That's probably what I should be doing. Yeah. Well, you know, if somebody wants to fund it, because I there's a couple of things that I'd really like. I think we don't have I'd really like to develop and build myself. I don't think we have a really good ways for developers and and people doing coding or whatever you are, you know, people right working in technical systems, we don't have great ways for you to actually assess your own learning. Like measuring it's hard. You know, people experiment with it in academia. There's some assessment, inventories, things like that for like computer science students, but there's not a lot for like real working professionals. And so if we're kind of like worried about what's happening to us and our skills with AI, I would I used to do a lot of assessment science. I would love to build some assessment tools so that engineers or coders or developers could just have them them themselves, you know, and kinda you could kind of know how you're doing over time and in what with what skills. I think that would take a a little bit of rigorous development work. Yeah. Yeah. Shimin (24:00) Yeah, the maybe s yeah, some fundings to to to Dan (24:00) For sure. And you might solve interviewing with it Shimin (24:03) actually try it out. Dan (24:04) too while you're at it, because I feel like that's it's always been bad, but yeah, particularly the right now. Cat Hicks (24:05) You know, you might interviewing's bad right now, right? Yeah, yeah. Yeah. It's getting worse. Yeah. Shimin (24:06) I was just gonna say, yeah. Cat Hicks (24:15) I do think actually, yeah, the way we Dan (24:16) But I but Cat Hicks (24:17) assess programmers is like bad in many, many places of the lifespan. Yeah. Yeah. Dan (24:21) Yeah, that's that's why that immediately came to mind 'cause it's like we, you know, as an industry don't really know how to judge someone's skills and Cat Hicks (24:29) Yeah. Dan (24:29) there's just been so many different attempts to do that. So it's not surprising to me Cat Hicks (24:32) Yeah. Dan (24:33) that something like you're describing doesn't exist 'cause it's I mean kinda it's a really hard Cat Hicks (24:36) Yeah. Dan (24:37) problem to solve. So Cat Hicks (24:38) It is a really hard problem to solve. You need kind of niche specialist knowledge, like you have to be an assessment scientist to know what works and things like that. But, you know, I think we could do better. I mean, this is a really important workforce doing really important work that like touches all of our lives and it frustrates me. Like I'm curious to ask you two, do you feel like the skills you are actually learning right now are different? Like are you learning new skills? If so, what are they? What should someone like me try Shimin (25:07) Yeah, for me personally, I built a couple of side projects around how do you tell when the AI is being syncophantic. And and I think that is one skill that is gonna Cat Hicks (25:16) Yeah, cool. Yeah. Shimin (25:18) be more valuable, kind of be able to spot the bullshit, so to speak, in in an AI output, and know exactly Cat Hicks (25:24) Yeah. I like that. Shimin (25:25) where it started to go off the rails. I think that's Cat Hicks (25:28) Yeah. Shimin (25:29) gonna be like a common skill going forward. well until they Cat Hicks (25:31) I like that. Shimin (25:32) fix it, right? If that ever Dan (25:34) Yeah. It Shimin (25:35) happens. Dan (25:35) for me the biggest area that I'm like concerned about and trying to skill up and is like I feel like the overall like agentic development stuff came with its own set of like how do you even do this and how do you get good output. So that's like one thing, but I feel like there's, you know, tons and tons and tons and tons of writing about that. So like, you know, half the Cat Hicks (25:53) Yeah. Dan (25:54) internet's filled with it right now. But the thing that I think is less covered and really important is like cognitive debt, right? Around like you Cat Hicks (26:01) Mm, mm. Dan (26:02) okay, so you've, you know, used AI to build this like greenfield project, great. You know, you now Cat Hicks (26:07) Right. Dan (26:07) have two hundred and fifty thousand lines of code. You don't know how any of it works outside of like a mental model that you sort of share with the chat. Cat Hicks (26:15) Mm-hmm. Dan (26:16) and so that's an area where like and I haven't really figured out an answer to that yet, but it's something I care deeply about and it kind of keeps me up at night to Shimin (26:24) Mm-hmm. Dan (26:24) be honest, because it's like Cat Hicks (26:25) I'm I'm right there with you. Yeah. It keeps me up at night too. I mean I have another life where I help people who have health issues use AI. And I started Shimin (26:36) Mm-hmm. Cat Hicks (26:36) doing that because I did a lot of patient advocacy and I I got into that by accident because I had health problems come up in my life that were really scary. And I have like access to science, you know, because I'm I'm married to a neuroscientist and I always go research whatever I'm going through. And not everyone can do that. And so I Shimin (26:53) Mm-hmm. Cat Hicks (26:54) I ended up, you know, kinda like the person who's always trying to fix the TV for their parents or something. I was just always the Dan (27:00) Yeah. Cat Hicks (27:01) person who was like, Well, okay, like I can kind of explain this test and what it means and why your doctor said it, whatever. So I got really involved in these conversations at the same time that AI was like you know, just going like wildfire through these patient communities. And so it was really Shimin (27:16) Mm-hmm. Cat Hicks (27:16) interesting to have software engineers on one hand in my work life and then also very you know, people People who are not very far from tech, you know, trying to use Shimin (27:25) All right. Cat Hicks (27:25) like Chat GPT to ask about their health. But some of the cognitive problems were very similar. Like trying to Shimin (27:32) Hmm. Cat Hicks (27:33) teach trying to teach people to think critically, trying to teach them to get sources, you know, trying to Shimin (27:39) Yeah. Cat Hicks (27:39) teach them like, yeah, like Dan (27:40) like about yeah, what the output is and yeah. Cat Hicks (27:42) what would be like an auditable AI system. Shimin (27:45) Mm. Cat Hicks (27:46) And you know, it was really, really cool, I think, to have those two extremes in my life because, you know, they were both very, like I say, want to say worthy. You know, these were people who have real deep needs, like to to understand my cancer treatment or something like that. And so trying Shimin (28:00) Right, right, right. Cat Hicks (28:02) to what the one of the other Claude skills I built is called informed patient. And it gives people some structure for trying to do a lit review and trying to understand to break this sycophanie, you know, what is the reasoning can I understand it separate from the AI? Like the AI can suggest plausible possible connections between what I've reported and what the literature says, but now I can go check it. And one thing that I found that really helped people was just getting them to generate multiple, like competing possi plausible kind of scenarios. Shimin (28:37) Mm-hmm. Cat Hicks (28:38) And and that sort of thing I see engineering teams move towards as well. Like a little bit of like, can we take advantage of scale? Can we can we take advantage of more adversarial development? You know, more rapid prototyping. Obviously it's still very overwhelming. But you know, if you're gonna like give yourself scale in production, I think you need to give yourself like scale in testing and auditing as well. Like kind of pit those things against each other. Hmm. Dan (29:05) Yeah. Ooh, and code gets cheap too. I feel like that's Cat Hicks (29:08) Yeah. Dan (29:08) really powerful for being able to like ideate and stuff. I don't think we've actually Cat Hicks (29:12) Yeah. Dan (29:12) like even scratched the surface of like what can be done there. So Cat Hicks (29:15) Yeah. You you see that in like computational biology, right? If they're gonna just generate tons of plausible like protein structures and then just Shimin (29:23) Mm-hmm. Cat Hicks (29:24) graduate some of those to the level where a human needs to audit them or something. I mean, I think there's a lot of interesting probabilistic thinking that other fields have had to do that now like software engineering's trying to figure out how to do. Yeah. Dan (29:38) Yeah. Shimin (29:39) Yeah, that's one of my big hypotheses as well is like we actually know how to do probabilistic thinking. We've been doing this in robotics and engineering for like more than a hundred Cat Hicks (29:49) Yeah. Shimin (29:49) years. Like just because it's new to us doesn't mean there aren't tools that can solve these problems. But I think for for our last question, Let's take a step back and think about the second order effect of an entire team using AI natively. Like we already talk on Cat Hicks (30:05) Mm-hmm. Shimin (30:05) the show about we see like devs, sometimes junior, sometimes management, like copying and pasting entire novella worth of AI response to via DM Cat Hicks (30:14) Mm-hmm. Shimin (30:15) or on a code review, right? And then and then everybody's like, I don't have time to read it all. Yeah, the slap Dan (30:17) Slop grenade, my favorite. Shimin (30:20) grenade. So Cat Hicks (30:21) Yeah. Shimin (30:22) on on a team level, Cat how do you see what do you think an healthy AI native team look like when it comes to the teamwork Cat Hicks (30:30) Yeah. Shimin (30:31) aspect? Cat Hicks (30:31) First of all, for healthy, I think everybody's gotta be really concern really dialed in, really concerned, really have as a priority the team mental model. Like we're talking about it, but if everybody has a completely different place where they expect that mental model to be developed, suddenly you feel like you're living inside of hypocrisy. You're living inside of like, you get to produce all this stuff, but then I have to review it. You know, I mean, if there is a conflict that's so unmanageable, right, between our different ways of working, it's going to be experienced Shimin (31:05) Mm-hmm. Cat Hicks (31:05) as like very brutal on both sides, you know, because I mean I've I I see the person receiving this the slot mountain and they feel like my life is now very unjust because I just have to clean up I'm now the rigor police and I'm the cleanup Shimin (31:17) Mm-hmm. Yep. Cat Hicks (31:19) police. But on the other side I've also seen, yeah, yeah. Dan (31:21) Or lonely too, right? If you're the lone person doing that, that can be very Shimin (31:23) No. This is why Dan has been Dan (31:25) kinda Shimin (31:25) so grumpy over the since we started his Dan (31:27) Yeah. Shimin (31:28) pod. Cat Hicks (31:28) I you are not alone, right, in that because the you know, I I'm I'll just say it really bluntly. I think some of these leaders need to grow up, like locking yourself in a room with your three favorite engineers and creating a whole slop mountain, you know, with no plan or strategy, and then just saying, please deal with this to the rest of your team or something. That's what you are communicating there is actually not a technical decision. What you're communicating there is like, you know, that it doesn't matter to you like what these other people's experience is or the quality of their problem solving. Shimin (32:02) Mm-hmm. Cat Hicks (32:03) So we have to care about. the quality of this for everyone. And and software has cared a a little bit too much about like just the feature engineer development and like kind of not solved these difficult problems of maintenance, you know, and and kind of rigor for a while. But I don't Shimin (32:22) Mm-hmm. Cat Hicks (32:23) I'm not pessimistic like some people about AI actually. Like, I really think there are reasons people are very, very excited about this. Shimin (32:30) Yeah. Cat Hicks (32:30) You know, like to your hypothesis, I share it, that there have been other fields that have taken advantage of like models and probability, Shimin (32:37) Mm-hmm. Cat Hicks (32:38) you know, and I think that let me try to make it simpler. God teams need more time to learn right now. Like they need space to really experiment and learn and not make it just like you know, some bizarre guessing game where we're trying to read each other's minds. And when I Shimin (32:54) Mm-hmm. All right. Cat Hicks (32:56) have seen engineering teams lower the overproduction pressure temperature and actually give their teams that space to learn and test and go back and change how they're, you know, working and and make some of it explicit instead of like secret Shimin (33:12) Mm-hmm. Cat Hicks (33:13) or never spoken until it explodes into like a slack argument or something. Actually let's decide beforehand how we're gonna use AI together as a team for the next month. And then let's stop and let's say did that work? You know, let's set up a test for Shimin (33:27) Right. Cat Hicks (33:27) it. That I have seen really, really help some orgs turn around and get united. And they've all been very scared that they were gonna like lose their edge in the AI era if they took the time to do that. And then it turned then they Dan (33:40) Let off the gas bottle Shimin (33:40) Mm-hmm. Dan (33:41) slightly, yeah. Cat Hicks (33:42) Yeah. And then and then actually they were like eighteen months ahead of like everybody else Shimin (33:47) Ha ha. Cat Hicks (33:47) if they did that. So really caught in a lot of false pressures right now. Yeah, does that help? That was like a lot, but that was my slop mountain. Shimin (33:54) No, that makes perfect sense. I think, yeah, in the in the AI age, kind of spending more time to improve your world process probably pays off more. And if you would like to learn how to do that, how to change the culture of your team. this is really sounding like one of those plugs I see on the on other podcasts. I did buy it myself. Cat Hicks (34:13) He did buy it himself. Shimin (34:16) yeah. Read the book, get the book of the psychology of software teams, and it has a lot of great tips on how to talk to your teams and how to fight for a better team culture. so with that, like to thank you again, Cat for joining us today. and also Cat Hicks (34:33) It was my pleasure, yes. Shimin (34:35) you also host the podcast, change technically, about Cat Hicks (34:40) Mm-hmm. Shimin (34:41) cognitive issues. I believe. Cat also does consulting work at Catharsis Consulting and where where's Fight for Humans? 'Cause I think that is a great title. I I I need to pull that up. Yes. Cat Hicks (34:50) yeah, that's my newsletter. Shimin (34:53) There's there's also the newsletter, Fight for Humans, which I'm a subscriber. So listeners if you like it, yeah, Cat Hicks (34:58) Fight for the human, yeah. Shimin (35:00) Fight for the Human. Look it up. yeah, that's I think that's all I got. Dan, do you have any last words? Dan (35:06) no, besides thank you for your time. It's been really cool. I'm also like Cat Hicks (35:08) Thank you both for your yeah. Dan (35:10) super interested in like organizational psychology, so it's interested to see what the the overlap is there and Cat Hicks (35:13) Yeah, yeah. There's so much good stuff. Yeah, yeah. Well, thank you. This was delightful. Thank you for I'm feeling I had c big coffee energy a little bit this morning. So thanks for creating space for that. Yeah. Shimin (35:27) I'll drink my coffee. Of course.