Latest / Code Story | Startup Podcast for CTOs, CEOs and Technical Founders / S2 Favorite: Scaling Fintech & Financial Data Infrastructure, with Ryan Graciano of Credit Karma

Transcript

S2 Favorite: Scaling Fintech & Financial Data Infrastructure, with Ryan Graciano of Credit Karma · Code Story | Startup Podcast for CTOs, CEOs and Technical Founders

• 0:00
  [SPEAKER_00]: Code Story is Mixed in Mastered in One Click with Clip Game.
• 0:03
  [SPEAKER_00]: Make your podcast sound loud, clear, and clean, and only one click with ClipGain.io.
• 0:12
  [SPEAKER_01]: You work at the bugs, you know, the, I think a failure mode is you kind of roll out the MVP to everybody and you expect it to work and then it can give the whole initiative a bad rap, because then you have business leaders and product managers saying like, hey, I just want to make this change and now I'm in a 60 day service cycle and, you know, blew up in my engineers, I'll hate, you know, you don't want to avoid that kind of stuff.
• 0:38
  [SPEAKER_01]: My name's Ryan Grasiano, I'm CTO and co-founder at Cretacarma.
• 0:47
  [SPEAKER_00]: This is Code Story, a podcast bringing you interviews with tech visionaries who share in the critical moments of what it takes to change an industry and build and lead a team that has your back.
• 1:03
  [SPEAKER_00]: I'm your host, Noel Abhart, and today, how Ryan Graciano built the technology behind billions of credit scores delivered to the masses, all this and more, on code story.
• 1:18
  [SPEAKER_00]: Early on in his life, Ryan Graciano aspired to be many things, a lawyer, a writer, and eventually a coder, of course.
• 1:28
  [SPEAKER_00]: Fun fact, he is an accomplished dog trainer, focusing on animal behavior modification, and more recently has gotten into powerlifting.
• 1:37
  [SPEAKER_00]: Despite his love of analog activities, he got started coding right after college.
• 1:43
  [SPEAKER_00]: And tried to avoid joining IBM, yet still did through an acquisition.
• 1:48
  [SPEAKER_00]: After a few years of growth, he met a group of entrepreneurs who had an idea to provide credit scores to millions of users.
• 1:56
  [SPEAKER_00]: for free.
• 1:57
  [SPEAKER_00]: This idea would eventually become credit karma.
• 2:03
  [SPEAKER_01]: We started credit karma in 2007, but the story really begins when I graduated college.
• 2:10
  [SPEAKER_01]: I was looking at all these big companies like IBM and Microsoft and I met this really impassioned technology leader from a small company and he convinced me that joining a 60 person company, it would be bigger impact and we're exciting and better culture.
• 2:28
  [SPEAKER_01]: And so, you know, I totally bought into all that and it was true for about six weeks until IBM bought us.
• 2:35
  [SPEAKER_01]: And then I joined the 300,000 person company anyway.
• 2:39
  [SPEAKER_01]: And it actually was a good experience because I learned a lot of things that I would never have learned otherwise.
• 2:46
  [SPEAKER_01]: I saw what it was like to be on that side of an acquisition.
• 2:49
  [SPEAKER_01]: I saw the difference between
• 2:51
  [SPEAKER_01]: small company in big company culture, you know, I got a lot of process experience I wouldn't have seen, so I turned out to be really great.
• 2:59
  [SPEAKER_01]: You know, after a few years there, I was feeling like I wanted that big impact, small culture job again.
• 3:07
  [SPEAKER_01]: Now, a very close friend of mine, his name's Greg.
• 3:10
  [SPEAKER_01]: He's our Chief Marketing Officer here at Credit Karma now, but back then, he worked with our CEO
• 3:18
  [SPEAKER_01]: And so Craig, my friend, says, hey, you should join our five-person company.
• 3:24
  [SPEAKER_01]: You could lead technology.
• 3:26
  [SPEAKER_01]: And I thought, you know, that's, does sound like, you know, big impact, although I'm not so sure about surgeon and marketing.
• 3:32
  [SPEAKER_01]: And he told me to talk to Ken, his friend, who is the CEO.
• 3:37
  [SPEAKER_01]: And so I talked to Ken and Ken, says, you know, I have a way better idea than this.
• 3:42
  [SPEAKER_01]: And so I'm working on this idea.
• 3:43
  [SPEAKER_01]: I think we're going to call a credit karma.
• 3:47
  [SPEAKER_01]: And he just kind of pitched me on the whole concept right there.
• 3:50
  [SPEAKER_01]: And then that's uh, I thought it was a great idea.
• 3:53
  [SPEAKER_01]: And I was excited about the opportunity.
• 3:54
  [SPEAKER_01]: And so, you know, that's how it started.
• 3:58
  [SPEAKER_00]: For the people that don't know what credit karma is, give me a brief overview about what credit karma does.
• 4:06
  [SPEAKER_01]: Well, we're best known for giving out billions of credit scores to over our 100 million consumers.
• 4:13
  [SPEAKER_01]: We actually do a lot more than that now.
• 4:15
  [SPEAKER_01]: So we do taxes for a much of the country.
• 4:19
  [SPEAKER_01]: We do, we help you buy a car, we help you buy a house, we can help you refinance your debt.
• 4:24
  [SPEAKER_01]: We're kind of everything finance and way.
• 4:27
  [SPEAKER_01]: We've added savings accounts.
• 4:29
  [SPEAKER_01]: And so what we want to do is automate a very complicated financial landscape for the average person.
• 4:37
  [SPEAKER_00]: So you talked about the early days a minute ago.
• 4:41
  [SPEAKER_00]: Tell me about the MVP.
• 4:42
  [SPEAKER_00]: Tell me how you approached building the MVP as the technology leader of the team.
• 4:48
  [SPEAKER_01]: the technology choosing process, so that's startup.
• 4:51
  [SPEAKER_01]: That is, it's religious almost, isn't it?
• 4:54
  [SPEAKER_01]: I, you know, so I had debates and discussions with friends about it.
• 4:59
  [SPEAKER_01]: I had some things going in that I really thought were important.
• 5:04
  [SPEAKER_01]: So one, I was kind of coming out of the server side.com era, and I had a little bit of like pattern PTSD.
• 5:11
  [SPEAKER_01]: You know, these newer, more flexible languages could be exciting.
• 5:16
  [SPEAKER_01]: What I was concerned about with something like Ruby on Rails was the wrap back then was that this thing doesn't scale all that easily.
• 5:26
  [SPEAKER_01]: And so I found one man banned in 2007.
• 5:30
  [SPEAKER_01]: I'm not thinking like, hey, I want to blaze new territory here.
• 5:33
  [SPEAKER_01]: I'm thinking, I want to something try something true.
• 5:36
  [SPEAKER_01]: I want to be able to fall playbook and I want to get this business problem done.
• 5:43
  [SPEAKER_01]: And so that was like number one.
• 5:44
  [SPEAKER_01]: I was like, hey, I want to, I want to path to scaling this thing that I understand easily.
• 5:49
  [SPEAKER_01]: Number two, I was thinking, well, I would really love something to be as stateless as possible, because so many problems in software are caused by mismanagement of state.
• 6:01
  [SPEAKER_01]: And functional programming has been around for years and years and years and years but back then it wasn't so hot as it is today.
• 6:08
  [SPEAKER_01]: I think if it was, I probably would have gone that route, but it wasn't on my radar.
• 6:14
  [SPEAKER_01]: But I was thinking, hey, I want the context to be recreated on every request.
• 6:19
  [SPEAKER_01]: And I want to eliminate a lot of these kind of like concurrency performance issues that I spent a lot of time debugging and Java land back then.
• 6:30
  [SPEAKER_01]: I landed in an interesting place.
• 6:32
  [SPEAKER_01]: The cross section of those things.
• 6:34
  [SPEAKER_01]: I went with lamp, which was like a very common, you know, you choose lamp very commonly back then.
• 6:41
  [SPEAKER_01]: But some of the rationale was very tried and true, like very easy scale.
• 6:46
  [SPEAKER_01]: It pretty obvious playbook.
• 6:49
  [SPEAKER_01]: PHP recreates the context in every request.
• 6:52
  [SPEAKER_01]: Now you have to kind of,
• 6:54
  [SPEAKER_01]: and for some framework level stuff to make sure that people aren't like abusing the global address space and those types of things, but with good hygiene, strong frameworks, you know, you can take off a lot of those sharp edges,
• 7:09
  [SPEAKER_01]: You know, it really worked.
• 7:10
  [SPEAKER_01]: I mean, we had very few scale issues that was never a big deal, not for a long, long time.
• 7:17
  [SPEAKER_01]: And that platform served as well for years.
• 7:21
  [SPEAKER_01]: Of course, now we've, you know, we've changed things, but the first, you know, eight, nine years or so, we're very well served by that stack.
• 7:30
  [SPEAKER_00]: So as you were building that, you picked a tried and true platform or technology you used.
• 7:36
  [SPEAKER_00]: What sort of decisions and trade-offs did you have to make in the early days and how did you cope with those?
• 7:44
  [SPEAKER_01]: Well, I didn't want to take on that as insecurity.
• 7:46
  [SPEAKER_01]: So I spent a ton of time on security, just kneeling down, very opinionated, frameworks around how data is handled and did base management and a lot of aspects of the session management, clean stuff.
• 8:03
  [SPEAKER_01]: just a lot of time, hard even to describe how much of my waking thoughts went into that, but you know, it was a time.
• 8:10
  [SPEAKER_01]: And then there's sort of these, uh, which features are going to survive conversation.
• 8:15
  [SPEAKER_01]: That's not obvious.
• 8:16
  [SPEAKER_01]: You know, back then, 2007, if you wanted to get funded in 2007, you would say that your thing is social.
• 8:23
  [SPEAKER_01]: So basically, use the magic word.
• 8:26
  [SPEAKER_01]: So we had all these social features, right, sharing in comments and bulletin boards and voting.
• 8:32
  [SPEAKER_01]: And we thought that these are going to be that really hot, viral things that we're going to have to scale.
• 8:39
  [SPEAKER_01]: So I put some effort into prematurely scaling those things, which was completely wasted, total waste of time.
• 8:46
  [SPEAKER_01]: Because those aren't the things at all.
• 8:48
  [SPEAKER_01]: It's totally different things that ended up needing to be scaled.
• 8:52
  [SPEAKER_01]: But there was a lot of back and forth on, yeah, which features are even going to live.
• 8:58
  [SPEAKER_01]: So that I tried to be as judicious about that as I could.
• 9:02
  [SPEAKER_01]: But I can't say we made the right calls everywhere.
• 9:04
  [SPEAKER_01]: Some things just had to be written.
• 9:08
  [SPEAKER_00]: This episode is brought to you by Rims, the Risk Management Society.
• 9:11
  [SPEAKER_00]: Rims is the preeminent organization dedicated to the profession of risk management.
• 9:16
  [SPEAKER_00]: The York brings networking, professional development and education opportunities to its members of more than 10,000 people across 60 countries.
• 9:25
  [SPEAKER_00]: Their main goals are to equip risk professionals to succeed in a changing environment, enhance their engagement within the community and expand their influence worldwide.
• 9:35
  [SPEAKER_00]: The RIMS 2020 Annual Conference and Exhibition is your place for the best in education, networking, and solutions to build stronger, more resilient risk programs.
• 9:46
  [SPEAKER_00]: nowhere else in the world will risk professionals have access to three things.
• 9:50
  [SPEAKER_00]: Every major insurer, broker, and the latest solution providers in the space, 180 education sessions that address the biggest risks and challenges facing organizations today.
• 10:01
  [SPEAKER_00]: and risk management peers who are not only succeeding despite market conditions but are ready to share their stories.
• 10:08
  [SPEAKER_00]: For nearly 60 years, rims has delivered the latest in greatest strategies and resources that allow our attendees to grow, innovate and succeed in any business.
• 10:19
  [SPEAKER_00]: Join them May 3rd through 6 in Denver, Colorado, for the ultimate risk management experience.
• 10:27
  [SPEAKER_00]: So you mentioned that the original product lasted for eight or nine years, which is a pretty good amount of time.
• 10:34
  [SPEAKER_00]: Staking with that eight or nine year product, how did you progress and mature credit karma?
• 10:41
  [SPEAKER_01]: The timing of our founding is really important in that discussion.
• 10:47
  [SPEAKER_01]: So we started in 2007, which is like everything in the world is great and everything's going to be up in the right forever, which is how everyone feels at the top, you know, what could go wrong?
• 10:59
  [SPEAKER_01]: And then 2008 came and our business model, right, is we essentially make money when the customer saves money.
• 11:07
  [SPEAKER_01]: So if our members get a better deal on the loan, you know, we get a finder's fee for that loan.
• 11:12
  [SPEAKER_01]: If banks are folding and not lending, that model doesn't work that well.
• 11:18
  [SPEAKER_01]: In 2008, you know, when we're, we just built the product, it's got a lot of traction, product market fit is really strong, people love it, feedback is really good, and then this whole externality happens, and so revenue just is like not possible.
• 11:37
  [SPEAKER_01]: And so we have to kind of rethink how this product is going to develop.
• 11:41
  [SPEAKER_01]: Because our vision is, you know, we want to be able to kind of automate your finances,
• 11:46
  [SPEAKER_01]: and we're gonna need all these banking relationships and bank integrations.
• 11:50
  [SPEAKER_01]: We'll obviously, there's some kind of near term compromise that's gonna have to be made there.
• 11:57
  [SPEAKER_01]: And so what we did was we started, we did some advertising type stuff, we sold cell phones on the site for a while, which sound really disconnected, but actually they check your, when you sign up for a cell phone plan, they're checking your credit to make sure that you can do the, you know, pay the phone.
• 12:18
  [SPEAKER_01]: So we did a lot of that type of stuff and we started actually doing things like white labeling our products.
• 12:24
  [SPEAKER_01]: So I had to develop white label version of our products that we progressed.
• 12:30
  [SPEAKER_01]: And there was a lot of stuff just done to survive.
• 12:33
  [SPEAKER_01]: You know, the live through that 2008 to 2010 timeframe when, you know, nobody's going to give money to a financial
• 12:48
  [SPEAKER_01]: So there was a lot of that and then we were trying to really advance the market here.
• 12:55
  [SPEAKER_01]: So back then it was impossible to even get just a free credit score.
• 12:59
  [SPEAKER_01]: we wanted monitoring, we wanted multiple scores, we wanted reports, we wanted to make this digestible for you, we wanted to actually predict what would happen and tell you how to dodge bullets that hadn't been fired at you yet.
• 13:12
  [SPEAKER_01]: And we wanted you to be able to, hey, not just like view your data about act on it, like correct errors.
• 13:18
  [SPEAKER_01]: And so we were over that period of time working, you know, with our the credit bureaus to try to paint a vision of what this really could be
• 13:28
  [SPEAKER_01]: And how over time it's going to be helpful for the credit bureau to have a better relationship with the, you know, with the American public and for, you know, we could be kind of a facilitator there, we could help and help all sides improve this, you know, this condition and it's a slow process, you know, you have to be very patient, but you know, it worked out.
• 13:50
  [SPEAKER_00]: My next question is around how you built your roadmap.
• 13:53
  [SPEAKER_00]: I can kind of siphon some of that from your response.
• 13:57
  [SPEAKER_00]: Some of it was about the 2008 incident around survival, white labeling, the product.
• 14:03
  [SPEAKER_00]: But was there ever a point, a specific point, where it changed from survival mode to progressing the industry?
• 14:12
  [SPEAKER_01]: Yeah, so I've kind of always thought about this as a generalizable,
• 14:17
  [SPEAKER_01]: problem.
• 14:17
  [SPEAKER_01]: So when I think about building road maps, there's a certain amount of just short game, near-term activities, that I'm doing, that I kind of know we're going to create an impact.
• 14:27
  [SPEAKER_01]: You know, they're going to help me see the next month, the next two months, get me through the quarter, and then I have a set of big bets.
• 14:35
  [SPEAKER_01]: And different teams, depending on where you are in your cycle, we'll say, like, it's 20% big bets, and some will be, like, well, it's 20% short bets, because we're very early in our progression.
• 14:46
  [SPEAKER_01]: I think a lot of the discussion all throughout has been, you know, it's always a mix of the two, but what's the appropriate mix?
• 14:54
  [SPEAKER_01]: And how big are those big bets?
• 14:57
  [SPEAKER_01]: And what is your kind of strategic roadmap look like?
• 15:00
  [SPEAKER_01]: And how do the big bets plan to the strategy?
• 15:02
  [SPEAKER_01]: And so there's, you know, we've always had a way of juggling that and that's evolved, you know, over time, as the business becomes more mature, you have way more capacity for everything.
• 15:15
  [SPEAKER_01]: But here, the short-term bats, you might have more like a 70% short just because, you know, you got to kind of see the company continue to grow in a predictable way.
• 15:28
  [SPEAKER_01]: But then you really want to be,
• 15:30
  [SPEAKER_01]: investing a lot in the future.
• 15:31
  [SPEAKER_01]: You want to make sure that your vision succeeds plays out.
• 15:35
  [SPEAKER_01]: You want more than one big bet.
• 15:37
  [SPEAKER_01]: And so you've got to be thinking about what's the probability of success on those big items and what happens.
• 15:44
  [SPEAKER_01]: If none of them play out, what happens if one plays out and like kind of 2010, 2012 time frame when we really started to see
• 15:53
  [SPEAKER_01]: A ton of success and things really started to pick up, banking started to come back.
• 15:59
  [SPEAKER_01]: We were starting to plan a lot of seeds for the future.
• 16:02
  [SPEAKER_01]: Cause we're like, hey, now that it's kind of raining everywhere, you know, let's plan a bunch of seeds and see what grows.
• 16:10
  [SPEAKER_01]: But that, yeah, that mix is always shifted.
• 16:13
  [SPEAKER_00]: So how did you go about building your team?
• 16:17
  [SPEAKER_00]: Specifically the engineering team.
• 16:19
  [SPEAKER_00]: How did you structure it around product development?
• 16:23
  [SPEAKER_00]: And what did you look for in those people that indicated they were the winning horses?
• 16:29
  [SPEAKER_01]: In the early days, I handpicked, I'm believe it or not.
• 16:32
  [SPEAKER_01]: So I got on Craigslist, actually.
• 16:35
  [SPEAKER_01]: So that's actually what I did.
• 16:37
  [SPEAKER_01]: So that's showing my age, but 2007, people in the Bay Area were on Craigslist.
• 16:44
  [SPEAKER_01]: And engineers would actually post a resume on Craigslist, which is also kind of hard to believe now.
• 16:49
  [SPEAKER_01]: It doesn't seem like it.
• 16:51
  [SPEAKER_01]: But, and so I would kind of go through those, and I would just find stuff that spoke to me, sometimes it would be a side project, sometimes it would be specific experience.
• 17:05
  [SPEAKER_01]: I have like a lot of systems and operating experience in my own background and security experience.
• 17:10
  [SPEAKER_01]: And if I kind of glean that, you know, something very difficult there had been done that would be useful to us, I would reach out to that person about that thing.
• 17:19
  [SPEAKER_01]: And when you have that kind of really targeted contact like that, your response rage is way higher, right?
• 17:25
  [SPEAKER_01]: Because people were like, wow, this person really knows what I do and why my profile is interesting.
• 17:30
  [SPEAKER_01]: And so hand picking while a lot of work, the hit rate was amazing.
• 17:39
  [SPEAKER_01]: Not sustainable forever.
• 17:41
  [SPEAKER_01]: but a very good way to get started.
• 17:43
  [SPEAKER_01]: That's how I built the initial team.
• 17:45
  [SPEAKER_01]: And they stuck around for a long, long time.
• 17:48
  [SPEAKER_01]: I mean, one of my first three or four hires is still here.
• 17:53
  [SPEAKER_01]: He just passed his 10-year mark.
• 17:57
  [SPEAKER_00]: You mentioned earlier a bit about scalability around the 8-9-year mark you switched the platform up a bit.
• 18:04
  [SPEAKER_00]: So I want to touch on that.
• 18:06
  [SPEAKER_00]: How did you make that shift towards a more scalable platform?
• 18:09
  [SPEAKER_00]: You know, everyone has their opinion around monoliths and microservices these days.
• 18:16
  [SPEAKER_00]: But I want to know your opinion.
• 18:17
  [SPEAKER_00]: So how did you approach that problem?
• 18:21
  [SPEAKER_01]: Let's see, the first really major shift we went was towards scaling out the data.
• 18:27
  [SPEAKER_01]: So more sophisticated database sharding, all that kind of stuff.
• 18:31
  [SPEAKER_01]: So we could really just sort of scale infinitely.
• 18:33
  [SPEAKER_01]: And that was pretty early because the scale of our data got pretty big, pretty fast.
• 18:40
  [SPEAKER_01]: Then software scaling, we had a framework that broke things up in a pretty reasonable logical fashion, but it was still one code base, it's still one application.
• 18:51
  [SPEAKER_01]: No matter how good a job you do, modularizing, you're going to get some interconnected code.
• 18:57
  [SPEAKER_01]: You're going to get some dependencies and then over time without enforcing the boundary, you get a lot of problems.
• 19:05
  [SPEAKER_01]: So we move towards a more service-oriented approach,
• 19:11
  [SPEAKER_01]: The devil's in the details, you know, what granularity of service do you go for?
• 19:15
  [SPEAKER_01]: You know, do you do like a you go full on microservice or do you have kind of like these, you know, giant, you know, monolithic services and or everything in between and how opinionated is your framework and what kind of, you know, what's your stack and how, you know, our homogenous or heterogeneous and we wanted to land the principles and it really to well-defined way.
• 19:38
  [SPEAKER_01]: And so the first year of this journey, we were prototyping, you know, our first couple of services that we thought would be good examples for everything else, and we were testing out our hypotheses.
• 19:53
  [SPEAKER_01]: In the first set of technologies we used, we didn't, we actually didn't like, and we just chucked out.
• 19:58
  [SPEAKER_01]: I think back then we were, we had a pretty rest oriented approach that didn't really end up panning out.
• 20:04
  [SPEAKER_01]: We were using Skala, which we did end up likeings, we kept that.
• 20:08
  [SPEAKER_01]: And then some of the guidelines and things, we thought would play out, well, it didn't really play out that well in practice.
• 20:14
  [SPEAKER_01]: And so there was some, there's a learning, kind of like a year of learning.
• 20:18
  [SPEAKER_01]: And at the same time, we were hiring.
• 20:21
  [SPEAKER_01]: So that's the next thing.
• 20:22
  [SPEAKER_01]: You want people who've seen this movie.
• 20:25
  [SPEAKER_01]: You don't want to go through a three-year transformation and be like, oh, no, but it's ever done in this year before.
• 20:30
  [SPEAKER_01]: So we were picking up a lot of talent from companies that had done this.
• 20:35
  [SPEAKER_01]: We in particular picked up a bunch of folks from Twitter, and that was helpful because they had gone through a similar transformation actually not that long ago.
• 20:45
  [SPEAKER_01]: So we got people who'd actually done that and seen some of the same problems.
• 20:49
  [SPEAKER_01]: In the next year, we really kind of defined what we want it.
• 20:53
  [SPEAKER_01]: We had it written down.
• 20:54
  [SPEAKER_01]: So it's like, hey, we're going to be extremely homogeneous.
• 20:57
  [SPEAKER_01]: We're going to be very specific about what technologies you can use at which layers.
• 21:02
  [SPEAKER_01]: And the reason is that, for us,
• 21:04
  [SPEAKER_01]: security is number one and a homogeneous environment is a very good way to include all of the controls and build those in all the way across the stack and you know there is some amount of loss of flexibility there that can be that can be challenging but there's a lot of upside in maintainability, reliability, you're tooling gets so much better because your tooling can just focus on you know this one stack and it just gets better and better deployment gets better
• 21:33
  [SPEAKER_01]: that played out really well for us.
• 21:35
  [SPEAKER_01]: Then we said like, hey, and this kind of year two, we want to go from basically one or two services to like, you know, 20.
• 21:43
  [SPEAKER_01]: So let's get the platform team good at this.
• 21:46
  [SPEAKER_01]: That was the first thought.
• 21:48
  [SPEAKER_01]: And we worked out a lot of kinks, but we're working them out within the platform group.
• 21:53
  [SPEAKER_01]: So the platform group, you know, kind of has like a bat, you know, everyone's got
• 22:01
  [SPEAKER_01]: In the next year, we're like, okay, now it's time for all the application teams to adopt.
• 22:07
  [SPEAKER_01]: And so the goal that year was every app team will build at least one service and move monolithic capability out into that service.
• 22:20
  [SPEAKER_01]: And by the end of the year, at the end of the out year, we had everyone in the company trained and had gone through the process.
• 22:28
  [SPEAKER_01]: And we really knew what the kinks were.
• 22:31
  [SPEAKER_01]: There's a lot of it is technology, but a lot of it is process.
• 22:34
  [SPEAKER_01]: How do you actually get these things out there?
• 22:36
  [SPEAKER_01]: And how do you do security and quality and how do you manage when things go down and who's on call and all that kind of stuff?
• 22:45
  [SPEAKER_01]: And then in the final year, year four, we said, okay, now we're not just gonna be building services and getting good at this, we are gonna get it this.
• 22:54
  [SPEAKER_01]: Now we're gonna kill the monolith, and we're gonna try to move as much out as we possibly can, which is a whole other topic, but that was basically a year long sprint to do as much as we could.
• 23:11
  [SPEAKER_00]: You mentioned something that peaked my interest there.
• 23:14
  [SPEAKER_00]: You started out making sure the team got good at this, meaning good at executing the process, building the services, et cetera.
• 23:25
  [SPEAKER_00]: How did you accomplish that?
• 23:26
  [SPEAKER_00]: How did you make sure they got good at it?
• 23:29
  [SPEAKER_00]: So process, I think, is a lot like building a product.
• 23:34
  [SPEAKER_01]: You come up with an initial version.
• 23:36
  [SPEAKER_01]: You come up with an MVP.
• 23:37
  [SPEAKER_01]: The MVP is not that good, but it may be accomplished as the basic thing you're trying to set out to do.
• 23:43
  [SPEAKER_01]: You need feedback.
• 23:45
  [SPEAKER_01]: You need some kind of feedback loop.
• 23:46
  [SPEAKER_01]: So who's your customer?
• 23:47
  [SPEAKER_01]: We set up the customers to be really internal, like internal to the platform team at first, so the feedback loop would be very tight.
• 23:55
  [SPEAKER_01]: and then you're faireting out the bugs.
• 23:57
  [SPEAKER_01]: And so, you know, kind of like our first service deployment process, you know, and those early days was like, it's like 60 days to get here, your service into production, because so many things are manual, so many people are involved.
• 24:11
  [SPEAKER_01]: But the very first time we measured it, which is like in that first year, we're like, okay, we need to get this down to a week to be viable.
• 24:19
  [SPEAKER_01]: And so,
• 24:20
  [SPEAKER_01]: You know, you set that goal and in a quarter, you know, the whole team is like every time they go through this manual thing they're figuring out how to automate and so then we got good at it and that we just built these processes that way and then we started to go out to everybody else we could say like hey we now have like
• 24:37
  [SPEAKER_01]: a two to three-day kind of deployment timeline for your service and everything's kind of on rails that reviews are easy to set up in the office hours work and the technology we've worked out in the most of the Kingston.
• 24:51
  [SPEAKER_01]: You work at the bugs, I think a failure mode is you kind of roll out the MVP to everybody and you expect it to work.
• 24:59
  [SPEAKER_01]: And then it can give the whole initiative a bad rap as then you have business leaders and product managers saying, like, hey, I just want to make this change.
• 25:07
  [SPEAKER_01]: And I'm like, 60 days service is cycle.
• 25:12
  [SPEAKER_01]: And blue up in my engineers, I'll hate it.
• 25:13
  [SPEAKER_01]: You don't want to avoid that kind of stuff.
• 25:16
  [SPEAKER_00]: So as you step out onto the balcony, and you look across all that you have built at Credit Karma, what do you most proud of?
• 25:26
  [SPEAKER_01]: The thing I'm probably the most proud of is the team.
• 25:31
  [SPEAKER_01]: There's this kind of like emergent behavior thing.
• 25:33
  [SPEAKER_01]: In the early days, you sort of know what everybody's working on.
• 25:37
  [SPEAKER_01]: And in your head, you can kind of picture how stuff's gonna go.
• 25:41
  [SPEAKER_01]: And you have this like just the sense of control.
• 25:44
  [SPEAKER_01]: And as you kind of like progress, you lose that sense of control and you have to learn to trust.
• 25:50
  [SPEAKER_01]: And I think what I'm most proud of now is that just the teams surprise me in such a positive way so often, you know, it just feels good to have created something so much bigger than yourself and to see, you know, how talented the people are and how creative they are.
• 26:08
  [SPEAKER_01]: And that's just really cool, you know, to like look out there and think that, you know, it all started with, you know, us three people.
• 26:18
  [SPEAKER_01]: So let's flip the script a little bit.
• 26:20
  [SPEAKER_00]: Tell me about a mistake you made and how you and your team responded to it.
• 26:27
  [SPEAKER_01]: I'll give you kind of a funny one, I guess, or some people think it's funny.
• 26:30
  [SPEAKER_01]: So if you have, you know, a long run up to being big, you have time to experiment with a bunch of different stuff.
• 26:38
  [SPEAKER_01]: So for a while, you know, I was doing all kinds of stuff.
• 26:42
  [SPEAKER_01]: to think about different ways to manage people and different ways to run the company.
• 26:47
  [SPEAKER_01]: You know, I think you see a lot of little companies do that there's like a blog post about how they're doing this like whatever radical thing that they're doing.
• 26:53
  [SPEAKER_01]: And we were, we had read a bunch of research.
• 26:58
  [SPEAKER_01]: I can't remember what got us excited about it.
• 26:59
  [SPEAKER_01]: We decided, like, hey, what if we actually didn't really have like engineering managers?
• 27:05
  [SPEAKER_01]: per se we had like a system of coaches and the coaches we're taking like all the best elements of management there you know kind of guiding you in your career and encouraging you and pointing out you know where you need to improve and what things are going wrong you know without all like the micro management problems you know telling you what to do right they're not here to tell you what to do they're here to help you and guide you on your path
• 27:35
  [SPEAKER_01]: getting the investment, the coaches were invested in the people, but not that invested in the outcome, right?
• 27:42
  [SPEAKER_01]: Or even that connected to the outcome?
• 27:44
  [SPEAKER_01]: Because if the coaches and feel accountable for what's happening, they don't really pay that much attention to the ultimate inequality of the product that's being developed or the software that we're producing.
• 27:57
  [SPEAKER_01]: And so we just had a lot of issues where coaches are kind of like,
• 28:00
  [SPEAKER_01]: being very encouraging and telling someone something is great and you know we're also seeing that this area is kind of a disaster and nobody is super accountable for it or group whole groups like just don't work that well together and who's really accountable for that you know we had we had a four instance for that stuff we had like actual coaches and whatnot but
• 28:22
  [SPEAKER_01]: just the efficiency of that compared to the more traditional management structure was not great.
• 28:29
  [SPEAKER_01]: But I'm happy we tried it.
• 28:30
  [SPEAKER_01]: I mean, it was interesting to see, you know, some of those things play out.
• 28:34
  [SPEAKER_01]: And everyone was actually, we were always very transparent like, hey, you know, this is something we want to try and
• 28:41
  [SPEAKER_01]: We hope it'll be great, but maybe it won't be, and if it's not, we'll mix things up, and people are actually really good, sports about it all, and people who are, I think I'd fun with it, and everyone gave it their best shot, and when stuff didn't work out, we just said, all right, we're switching it up, you know, as no big deal.
• 29:01
  [SPEAKER_00]: So I'm an active credit karma user, so I'm excited about this next question.
• 29:06
  [SPEAKER_00]: What does the future look like for credit karma the product and the team?
• 29:11
  [SPEAKER_01]: I want to lean a lot more into personalization, so the product that we have is very capable of hyper-personalizing what it shows you, but we just recently put a lot of the technology pieces in place to really leverage that potential, and so now the path where we're headed down is, hey, everything that we should tell you.
• 29:36
  [SPEAKER_01]: you know, some engine should really have thought through what outcomes are in there for you and what is likely to resonate with you kind of at this point in your financial journey and so the going from kind of thinking this kind of like global maxima type approach where like AB tests one experience versus another experience
• 29:57
  [SPEAKER_01]: towards, hey, there's an engine behind all of this that's powering hyper personalized experience, and then people getting kind of out of the mode of I'm creating this feature that's going to reach a hundred million people towards, you know, hey, maybe my future only reaches two million people, but those two million people really need it.
• 30:16
  [SPEAKER_01]: and then having a product that's smart enough to put it in front of the right people at the right time, you know, those two million people really appreciate that.
• 30:23
  [SPEAKER_01]: And if you build 50 of those features, you know, now you have a really great product, but it's still reaching 100 million people, but in a much, a much more interesting way.
• 30:33
  [SPEAKER_01]: And so we've spent a lot of time around here thinking about,
• 30:38
  [SPEAKER_01]: What is that product look like?
• 30:40
  [SPEAKER_01]: How does it work?
• 30:41
  [SPEAKER_01]: There's a lot of internal process stuff.
• 30:44
  [SPEAKER_01]: You know, how do you build those experiences and, you know, how do you make sure that they're doing what they're supposed to be doing and what decides and what does that algorithm even look like to, you know, measure, you know, how effective.
• 30:56
  [SPEAKER_01]: Um, this, you know, this experience is for this person, you know, there's a really hard questions.
• 31:01
  [SPEAKER_01]: Um, we have a lot of smart people here thinking about those things.
• 31:04
  [SPEAKER_01]: A lot of PhDs thinking about objective functions.
• 31:07
  [SPEAKER_01]: Uh, so I'm excited because we're actually, um, you know, really this year.
• 31:10
  [SPEAKER_01]: We're shipping a lot of the fruits of that.
• 31:13
  [SPEAKER_01]: And I'm really interested to see, you know, how people start interacting.
• 31:17
  [SPEAKER_00]: For you, Ryan, who influences the way you work?
• 31:22
  [SPEAKER_00]: Name an architect or CEO, CTO, tech person, non-tech person, it could be anyone.
• 31:29
  [SPEAKER_00]: Who do you look up to and why?
• 31:32
  [SPEAKER_01]: I draw inspiration from a lot of places, I guess.
• 31:34
  [SPEAKER_01]: I have friends in similar but different roles everywhere, cause I think.
• 31:40
  [SPEAKER_01]: you know, the specific like CTO.
• 31:42
  [SPEAKER_01]: It's, it's similar, but it's really different depending on where you are.
• 31:46
  [SPEAKER_01]: So it's just kind of interesting to get other people's take on, you know, what's life like for them, and what's, what's important where they are and, you know, how do they kind of marry, you know, business and culture and technology, and, you know,
• 32:00
  [SPEAKER_01]: So there's always kind of like inspiration to be taken from the experience of others.
• 32:04
  [SPEAKER_01]: As a younger programmer, I was really inspired by like the John Carmax of the world.
• 32:11
  [SPEAKER_01]: You know, these like superstar people that felt like they could invent anything, you know, trailblazing whole new industries.
• 32:18
  [SPEAKER_01]: And what I think is kind of interesting about that, and maybe can be a little unique.
• 32:25
  [SPEAKER_01]: to engineering and sports is that when you have these people who are these like larger than life, incredibly good at this thing, it can almost be demoralizing.
• 32:35
  [SPEAKER_01]: You're like, wow, I can't jump as high as that person.
• 32:38
  [SPEAKER_01]: I'll never code like this guy does.
• 32:43
  [SPEAKER_01]: You can get really focused on that and kind of forget that actually in practice, so much of it is about how do you work together and
• 32:53
  [SPEAKER_01]: You know, how do you marry these ideas with, you know, the business and, you know, there's so much in software beyond leak code.
• 33:03
  [SPEAKER_01]: And so I'm a perfectly fine leak cutter, you know, I'm good at that stuff, but if it's like your only barometer, it's not a good barometer, you know, it's because real the real world is a lot messier than that and we all have something unique to bring to bear.
• 33:18
  [SPEAKER_01]: And so, you know, what I try to get inspiration from people who kind of do stuff well differently, you know, what makes them work.
• 33:27
  [SPEAKER_00]: So, if you could go back to the beginning in the early days of credit karma, what would you do differently, or what would you consider taking a different approach on?
• 33:37
  [SPEAKER_01]: Probably a lot of things in hindsight.
• 33:40
  [SPEAKER_01]: There's definitely a lot of things that we could scale differently, technology choices that we could have made.
• 33:46
  [SPEAKER_01]: differently.
• 33:47
  [SPEAKER_01]: We were, we're at this really awkward spot.
• 33:49
  [SPEAKER_01]: We were like right before cloud, which was so tough.
• 33:52
  [SPEAKER_01]: And sometimes I wonder, like, what if I just kind of just push this in there and try to get the banks and the credit bureaus to bite?
• 34:00
  [SPEAKER_01]: You know, would it?
• 34:01
  [SPEAKER_01]: Would that have been better?
• 34:02
  [SPEAKER_01]: Or would it have been just a disaster?
• 34:03
  [SPEAKER_01]: Because I would have failed every, you know, compliance checkpoint that I ever have for five years until everyone realizes this is a good idea.
• 34:11
  [SPEAKER_01]: I thought about that a lot, the data stuff you always think about, like there's just always in hindsight a better way to have done the data, you know, because data gravity is such a thing.
• 34:22
  [SPEAKER_01]: You know, when you realize later on that you have a different use case or a different way in which you, you know, want to leverage that.
• 34:28
  [SPEAKER_01]: And when you move into service territory,
• 34:31
  [SPEAKER_01]: all that stuff changes.
• 34:32
  [SPEAKER_01]: So once we kind of moved into microservices, we had to kind of rethink data and we're migrating data from kind of one store to the other to work in this new space.
• 34:43
  [SPEAKER_01]: And I think that I really played that out in my head.
• 34:45
  [SPEAKER_01]: I could have tried to do more of that upfront.
• 34:50
  [SPEAKER_01]: I definitely would have disallowed joins.
• 34:52
  [SPEAKER_01]: I would have banned joins.
• 34:54
  [SPEAKER_01]: Yes, uh, you know, just number one.
• 34:56
  [SPEAKER_01]: I'm so sorry, everybody.
• 34:58
  [SPEAKER_01]: I just I liked joining.
• 34:59
  [SPEAKER_01]: I can't I don't know.
• 35:00
  [SPEAKER_01]: I don't have an excuse.
• 35:01
  [SPEAKER_01]: I just it's very expedient.
• 35:04
  [SPEAKER_00]: So you're getting on a plane and you're sitting next to someone who just built the next big thing.
• 35:10
  [SPEAKER_00]: They're a tech founder.
• 35:12
  [SPEAKER_00]: They just shipped their code.
• 35:13
  [SPEAKER_00]: They're excited to show it to you and show it to the world.
• 35:17
  [SPEAKER_00]: What advice do you give them being further down this road?
• 35:21
  [SPEAKER_00]: What do you tell them?
• 35:23
  [SPEAKER_01]: advice I give most people is just stay focused on the member problem or the business problem in this case.
• 35:32
  [SPEAKER_01]: I talk to younger cTOs who are kind of like newer at their startup or whatever all the time and you know happy to give them advice on like technology decisions and you know all that kind of stuff.
• 35:44
  [SPEAKER_01]: Usually, my advice to them is like, what's the best thing for you next year?
• 35:48
  [SPEAKER_01]: If you're not even like, series A yet or if you're just kind of like, in series A, you're really just trying to get to the next letter.
• 35:56
  [SPEAKER_01]: And you gotta keep it going up into the right.
• 35:59
  [SPEAKER_01]: And, you know, if you're talking about, like, repladforming or how, you know, 70% of your team is working on this big technology things to avoid some problem four years in the future.
• 36:12
  [SPEAKER_00]: Usually I'm like, is that really the right thing to do?
• 36:15
  [SPEAKER_01]: Or is it better to just have a problem later and have already succeeded?
• 36:20
  [SPEAKER_01]: And now you have enough people to fix that problem.
• 36:22
  [SPEAKER_01]: I do realize, you know, it's very hard later when you actually have those people.
• 36:26
  [SPEAKER_01]: And you have that problem to have the discipline to go do it.
• 36:29
  [SPEAKER_01]: We would try to foreshadow that stuff for, you know, the executive team and say, like, we're moving to cloud.
• 36:35
  [SPEAKER_01]: It's going to happen.
• 36:36
  [SPEAKER_01]: We're going to, you know, break everything out into services.
• 36:39
  [SPEAKER_01]: It's going to happen.
• 36:39
  [SPEAKER_01]: It's going to be expensive.
• 36:41
  [SPEAKER_01]: And the more you can force you out of that stuff, you can actually win it's time, do it.
• 36:46
  [SPEAKER_01]: But doing it prematurely is usually a huge mistake.
• 36:52
  [SPEAKER_01]: You're much more likely to be killed because your business didn't work than because you had like one too many foul wells.
• 37:00
  [SPEAKER_00]: Well that's great advice.
• 37:01
  [SPEAKER_00]: Well Ryan, thank you for being on code story.
• 37:04
  [SPEAKER_00]: Thank you for your time and telling the creation story of credit karma.
• 37:09
  [SPEAKER_01]: Yeah, it was a pleasure.
• 37:10
  [SPEAKER_01]: Thanks for having me
• 37:12
  [SPEAKER_00]: and this concludes another chapter of Code Story.
• 37:18
  [SPEAKER_00]: Code Story is hosted and produced by Noah Labpart.
• 37:21
  [SPEAKER_00]: Season two episodes are co-produced and edited by Bradley Dinnom.
• 37:25
  [SPEAKER_00]: Be sure to subscribe on Apple Podcasts, Spotify or the podcasting app at your choice.
• 37:30
  [SPEAKER_00]: Support the show on patreon.com slash code story for just five to 10 bucks a month.
• 37:36
  [SPEAKER_00]: And when you get a chance, leave us a review.
• 37:38
  [SPEAKER_00]: Both things help us out tremendously.
• 37:41
  [SPEAKER_00]: and takes again for listening.