Latest / Agentic Conversations (formally mlops.community) / Just when we Started to Solve Software Docs, AI Blew Everything Up // Dave Nunez // #235

Transcript

Just when we Started to Solve Software Docs, AI Blew Everything Up // Dave Nunez // #235 · Agentic Conversations (formally mlops.community)

• 0:00
  I'm Dave Nunez. I'm cofounder of Abstract Group
• 0:04
  and a partner at Stellar Docs. And I love cold brew, making my
• 0:10
  own cold brew. Welcome back to another ML OPS
• 0:15
  community podcast. I'm your host Dimitrios, and
• 0:18
  today I have the pleasure of having a co-host.
• 0:21
  We're bringing it back to the old times.
• 0:23
  I brought on my man Cuba because he is someone that I very much
• 0:28
  respect when it comes to the developer space and I tend to
• 0:33
  ask him a lot of questions around documentation.
• 0:36
  And our guest today, Dave, we geeked out for about an hour
• 0:40
  about documentation and really what makes good documentation,
• 0:44
  why you want to strive for good documentation, and how to
• 0:51
  continuously update that. So you had some fun takeaways,
• 0:55
  Cuba. What were the things that you
• 0:57
  thought Dave said that you particularly enjoyed?
• 1:02
  Yeah, so, so I loved this concept of, of thinking about
• 1:07
  time to phrase value, right, and getting, you know, getting
• 1:10
  having the North Star and then optimising it about, but
• 1:13
  ultimately pushing people to get to that, to that value, get to
• 1:16
  that first dopamine hit. This is a really good metric to
• 1:20
  to optimise and to think about. Yeah, it's funny because I know
• 1:24
  that in the community we had a thread that was happening in
• 1:28
  Slack that said something along the lines of don't conflate
• 1:32
  users and clients. It's not always the same.
• 1:35
  And somebody wrote underneath that clients are the ones who
• 1:40
  pay for the product, users are the ones who suffer from the
• 1:44
  product. And so hopefully because of this
• 1:48
  conversation with Dave that is not going to be commonplace.
• 1:52
  If you listen to this, if you listen to the next hour that we
• 1:56
  talked to him, I mean, he was given value bombs after value
• 2:00
  bombs. In my way of seeing it.
• 2:04
  I really enjoy just what he understands.
• 2:08
  It's almost like a second nature to him.
• 2:11
  And it really feels like he's been steeped in docks for
• 2:14
  whatever, a decade, 2 decades. I don't know exactly how long,
• 2:18
  but it is obvious that he has woken up every day and thought
• 2:23
  about how to make docs better. And so I love getting to speak
• 2:27
  with somebody at that calibre and just learning from him.
• 2:32
  He talked all about these design cues that the docs can have and
• 2:36
  make sure. One thing that I thought was
• 2:38
  particularly cool was this idea of an escape hatch in your
• 2:42
  docks, just to make sure that the person is in the right spot.
• 2:46
  So when you're listening, let your ears perk up during that
• 2:52
  part when he starts to talk about escape hatches.
• 2:55
  And I think that's it. Could the main thing I want to
• 2:58
  say now is if anybody is listening and they have not
• 3:02
  subscribed or liked or given some reviews on Spotify, it
• 3:06
  would mean the world to us if you did that.
• 3:08
  And as always, just tell one friend and we are embedding this
• 3:15
  baby. Let's keep it rocking.
• 3:23
  We gotta start with this, man. Like you spent so much time at
• 3:26
  Stripe. What are good dots?
• 3:30
  What are bad dogs? And how do we intuitively feel
• 3:35
  that? Yeah, Yeah.
• 3:37
  It's good that you said intuitively feel because I spend
• 3:41
  a lot of time talking about their first impression of a doc
• 3:43
  and justice that like subconscious feeling a developer
• 3:47
  gets when you're like, alright, I'm already upset that I got to
• 3:50
  go look at the documentation because I'm blocked on something
• 3:52
  or I have 30 minutes to try and see if I can set up this thing.
• 3:56
  It's my first time using this product.
• 3:58
  I need to go use the docs. And you know that feeling when
• 4:01
  you land on the docks, you don't even know where to go.
• 4:03
  Like there's no, there are no guideposts.
• 4:05
  Like you don't know what to search for because you don't
• 4:07
  know the terminology yet. Or you click on the quick start,
• 4:10
  like get started in 5 minutes and then it's just like 10
• 4:13
  paragraphs of text. That is an awful first
• 4:15
  impression. And it's probably likely the
• 4:18
  developer is going to be like, yeah, let me check my Slack
• 4:21
  messages or I need a cup of coffee.
• 4:23
  And the opposite of that is something that's inviting
• 4:27
  depending on what you're trying to do.
• 4:29
  Like developers also subconsciously knows what a doc
• 4:33
  looks like, what depending on what they're trying to do.
• 4:35
  If they're trying to understand a platform product and the
• 4:39
  architectural pieces, if they land on a nice looking diagram
• 4:44
  where they don't have to read all these things, they can just
• 4:46
  see how the data moves and how the pieces fit together.
• 4:49
  Like you're already inviting them in building that trust.
• 4:53
  If it's a quick start and you see one line of code like, or
• 4:57
  you see like A1 sentence, like, here's what you're going to
• 5:00
  accomplish in this, in this document.
• 5:03
  And then you start seeing some like several lines of code like
• 5:07
  that's pattern matching to them. Simplicity and and like, I want
• 5:11
  to learn more O the documentation shouldn't be about
• 5:16
  like solving all of their problems all at once and being
• 5:19
  perfect across the board. It should be like, how can you
• 5:21
  build that trust and keep getting them to go down that
• 5:24
  path? So ultimately good docs have
• 5:26
  like, if you don't have a good first impression, you're already
• 5:29
  behind with your user. And then once you invite them
• 5:32
  in, the writing needs to be clear.
• 5:34
  It needs to be written for. And it's, it's sad that it's not
• 5:38
  obvious, but like it needs to be written for a human.
• 5:40
  A lot of times companies with a lot of resources, they're mostly
• 5:44
  writing for their translation tools and they're thinking about
• 5:48
  the cost of translation and how can you write for basically a
• 5:51
  robot that can turn this like string of text into a lot of
• 5:55
  different languages instead of like taking on that complexity,
• 5:58
  writing in a more localised language and then dealing with
• 6:01
  like even if it's manual, I'm sure now with LLM tools, it's a
• 6:04
  lot easier to do this well, but that's a user first example.
• 6:09
  And then and then just the, you know, all the little elements
• 6:13
  are there is is the writing good?
• 6:16
  Are there call outs? Are there like the right balance
• 6:19
  of inline links, like putting yourself in the shoes of the
• 6:22
  user and what are their what are they actually trying to do and
• 6:25
  how much time do they have? And then bad docs are the
• 6:28
  opposite. I'm writing for me.
• 6:30
  I think that or here's everything that the tool can do
• 6:34
  and let me just put it onto the page, not worry about
• 6:38
  readability, not worry about how much time the the user has or
• 6:42
  like what are the actual, what's the actual format or
• 6:45
  presentation style like they're going to want in this at this
• 6:48
  point in time. O maybe maybe there's double
• 6:52
  click on it. I'm wondering so two things.
• 6:54
  One is what are those signals, good signals for the dev landing
• 7:00
  on the on the dogs? You mentioned diagrams, you
• 7:02
  mentioned the code. What are some of the other
• 7:04
  things that you that you think are very inviting, you know, to
• 7:11
  to the dev? And then the second question,
• 7:15
  how do you sort of like sanity check what you said that you
• 7:18
  didn't write it for yourself? Yeah, the signals, I think white
• 7:21
  space is a good signal. Like hey, this is an
• 7:23
  approachable piece of documentation.
• 7:25
  No matter what it is, you don't want to see block the text, you
• 7:27
  don't want to see a video in a diagram and a tonne of visuals
• 7:30
  in your face code. Like there should it, it should
• 7:34
  be very clear where you want the user to start and what you want
• 7:37
  them to see. And then after that it should
• 7:40
  also be scannable. So that's where that white space
• 7:42
  is because nobody is like cool, I'm going to crack my knuckles,
• 7:44
  sit down and read this document. It's like, and it's called, you
• 7:47
  know, information foraging. I'm looking for the truffles of
• 7:51
  good information. And then I'll even like
• 7:53
  backtrack. Like there are a lot of eye
• 7:54
  tracking studies that have shown when users are looking at
• 7:57
  documentation, it's an F pattern.
• 7:59
  Like I'm just going boom, boom, down, boom, like a cross.
• 8:02
  I'm not reading it left to right all the way down.
• 8:05
  So, so the white space is really important.
• 8:08
  And then every sentence should be, should mostly stand on its
• 8:13
  own. Like you see, a lot of good
• 8:15
  developers write three line sentences and you have to reread
• 8:19
  it several times in order to understand like what you're what
• 8:22
  they're trying to tell you. But if you're very clear in
• 8:27
  like, here's what this thing is and you're clear about the
• 8:30
  sequencing, like here's how the speeds into the next thing.
• 8:33
  I think those are those are clear signals.
• 8:35
  A lot of it is subconscious too, because again, like developers,
• 8:38
  especially 5-10 years ago, admittedly hated documentation
• 8:42
  and you know, justifiably so, they hated documentation,
• 8:46
  reading documentation, they didn't want to be marketed to,
• 8:48
  But those things are still important.
• 8:50
  So how can you deliver information in a way that
• 8:53
  they're not frustrated? You don't need to make them
• 8:55
  happy. They're not going to be like,
• 8:56
  this is the best documentation I've ever read, Like, let me
• 8:59
  print this out. You just don't want to frustrate
• 9:02
  them. And then the sanity check.
• 9:05
  I mean, the there, there's a lot of qualitative things, like one
• 9:09
  of the best things you can do it's, it's really hard to edit
• 9:12
  yourself. Like when you write a piece of
• 9:14
  documentation and then you go right back to the top and try
• 9:17
  and edit, you can clean up some stuff.
• 9:19
  But to get it really like pared down and and much cleaner, you
• 9:25
  need to like give it a day, like sleep on it, come back to it
• 9:27
  with fresh eyes. You'll forget a lot of what you
• 9:29
  wrote and you'll see, oh, this is bad.
• 9:32
  This doesn't make sense. This can be this can be removed.
• 9:36
  And then another good one too, is like, you give it to a
• 9:40
  whoever the audience type is going to be like a coworker and
• 9:46
  say, like here, read this document.
• 9:48
  Now, what were your takeaways? And like, think in your head,
• 9:52
  OK, I want them to learn this concept or I want them to
• 9:55
  accomplish this task or I want them to know what steps are
• 10:00
  required for this particular user journey.
• 10:04
  And so I asked them like, OK, what are your takeaways?
• 10:07
  And if they're not clear in what, in what those things are,
• 10:11
  what those goals are, then, you know, keep iterating until you
• 10:14
  nail that. So that is better than nothing
• 10:17
  like stripes it. A lot of we did a lot of user
• 10:20
  research where you're asking 510 users, like, what do you think
• 10:24
  about this guide? But you know, like, a lot of
• 10:27
  times just asking the colleague like, Hey, what is what were
• 10:30
  your takeaways from this? And and then just optimising for
• 10:34
  for what you want them to take away.
• 10:36
  So I know Stripe has a very strong writing culture, right?
• 10:40
  And it's almost like part of being indoctrinated into Stripe
• 10:45
  as an employee, you have to write a lot of things down.
• 10:49
  Do you feel like that plays into how Stripe manifested its docs
• 10:56
  and your ability to be able to go to just anyone that you're
• 11:02
  working right next to and say, what do you think of this?
• 11:05
  Where can it be more clear? What could be better on that?
• 11:08
  Yeah, absolutely. The, the writing culture played
• 11:11
  a huge part and I'm trying to think if, if people like, if it
• 11:18
  was self selecting, I think a lot of it was self selecting.
• 11:20
  People knew there was a writing culture.
• 11:22
  You were going to have some form of writing in your during your
• 11:26
  interview. I mean, people are analysing the
• 11:28
  hiring managers are looking at your emails, like when you're
• 11:31
  just corresponding and engaging your kind of, you know,
• 11:34
  intelligence and ability to convey thoughts like in all of
• 11:39
  your written correspondence and your resume.
• 11:43
  And, and so there's some filtering up front and then
• 11:47
  there's some reinforcement on the job.
• 11:49
  The, the career ladders themselves, like documentation
• 11:52
  was included in in the, the engineering job bladder.
• 11:56
  And, and the higher you go up, the more, the, the higher the
• 12:00
  expectation of your, of your writing.
• 12:03
  Like you needed to be able to write a really good design
• 12:06
  proposal, architecture proposal when you're a staff engineer or
• 12:10
  above. And, and so, but also I met a
• 12:14
  lot of people that did not like writing.
• 12:16
  Like they knew it was important and they played along, but that
• 12:19
  wasn't their thing. They were the people that liked
• 12:21
  the multiple choice tests in, in school, like hated the essay
• 12:24
  portion. So it was important to like when
• 12:28
  you're, when you're working with all those people, not assume
• 12:30
  everybody likes to write, but make it.
• 12:33
  So there's a high floor for those people who are like, I
• 12:36
  just want to get the job done. I just want to review or
• 12:39
  contribute my thing and and then go on with my life.
• 12:42
  That is the reality, you know, like the writing culture is not
• 12:46
  very prevalent on the engineering teams all across the
• 12:50
  board. And I'm, I'm wondering like, how
• 12:51
  do you manage those, those staff relationships?
• 12:54
  How do you get more input and collaboration with with
• 13:00
  engineers and yeah, how do you kindle that culture?
• 13:04
  I think it has to be, it has to start with the leadership.
• 13:09
  Because if everybody, if all the developers care about docs, but
• 13:14
  the leaders are writing really terrible emails like that just
• 13:19
  seeps into the culture immediately.
• 13:21
  Or this would happen a lot too. This is a big signal that docs
• 13:24
  don't matter. You have a release and you
• 13:28
  picked an arbitrary date because most, most software deadlines
• 13:31
  are and the docks aren't ready and you ship it anyway.
• 13:34
  But if you say, hey, the docks aren't ready, we're not, we're
• 13:36
  holding back this release until they're ready.
• 13:38
  That's a huge signal that it matters.
• 13:42
  And we, we had a gate or take into account docs for every
• 13:47
  single release at Stripe. There's usually a fire drill at
• 13:50
  the end to get them ready, but the product would like our CTO
• 13:54
  or CEO, our CPO would be reviewing docs like before,
• 13:59
  before launch. So those were, those are great
• 14:02
  signals. So the leaders themselves like
• 14:05
  showing that they're they're they care about their right,
• 14:07
  their own writing and then also making it an equal factor in, in
• 14:12
  your in your launches. I'm wondering sort of what
• 14:17
  changes can you do bottom up when maybe the leadership isn't
• 14:24
  better about writing or but they they are not necessarily great
• 14:27
  or they don't necessarily, you know, care about it as much as
• 14:30
  folks it's tried did. How do you make a change coming
• 14:34
  from from the bottom up, you know, as a technical writer?
• 14:37
  Yeah, with the bottom, with the bottom up, I mean, the company
• 14:40
  should have that component in the hiring process where
• 14:44
  everybody's doing. I mean, I can think of every
• 14:47
  single role in Stripe that I know of that had some type of
• 14:50
  project or take home component to the interview process.
• 14:53
  So you're bringing in people where you're gauging their
• 14:56
  writing. They don't need to be excellent
• 14:57
  writers. They need to be passable and
• 14:59
  they need to know that writing is important.
• 15:01
  And then once they're actually in the door, make sure that come
• 15:06
  performance review time, like that stuff actually matters.
• 15:08
  Like, Hey, you're being praised for the work that you've done
• 15:12
  sharing knowledge or you're being penalised because you have
• 15:15
  not shared any knowledge or you're, you're not taking your
• 15:18
  writing seriously. And then what a technical like
• 15:20
  the professional writer. So the first thing that I did
• 15:22
  when I joined Stripe that I did this late at Uber, but it was
• 15:25
  like seriously, the the first week at at Stripe, I threw up
• 15:30
  the writing bat signal and like figured who all the professional
• 15:34
  writers were across the org and justice.
• 15:36
  We got together and met biweekly to share our resources, our
• 15:41
  relationships, our knowledge, and it's just really helpful to
• 15:45
  kind of create that coop because the marketing writers, the brand
• 15:49
  writers need the same things that you do relationships to the
• 15:54
  to the technical side, the product managers style guides.
• 15:57
  You don't want to have the documentation to have a certain
• 15:59
  style and then the marketing site to have a certain style and
• 16:02
  support site to have a certain style.
• 16:03
  O coming U with those core artefacts and then sharing those
• 16:08
  out with the company. And that makes it easier for the
• 16:10
  engineers or the non writers to know too.
• 16:13
  OK, there's a unified set of guidelines they need to follow
• 16:16
  as I'm writing. So I think that's the base that
• 16:19
  you can do is like get your writers to agree upon like the
• 16:23
  style and the the the writing expectations.
• 16:28
  And then after that, just keep giving as many resources as you
• 16:31
  can. Like we ended up creating like
• 16:33
  an internal, basically like a self-guided book for engineers
• 16:37
  to because we couldn't serve everybody on creating their
• 16:40
  docs. And they were, it was basically
• 16:43
  like how to create docs like a Stripe technical writer.
• 16:46
  We added to it over time, but it was eventually like an end to
• 16:49
  end guide with templates and annotations and, you know,
• 16:52
  checklists. And they could follow that
• 16:55
  process for for their own like self-serve.
• 16:57
  That took years to accumulate. But it's something that if I
• 17:01
  were to join a company, I would that's something I would start
• 17:04
  from the beginning, not like three years in.
• 17:07
  Fascinating. That's super high leverage.
• 17:09
  It's the docs on how to create docs and you don't matter, but
• 17:14
  it is very it. It makes complete sense on why
• 17:17
  that could be so useful and you're empowering.
• 17:20
  You're decentralising the. Ability, So one more thought on
• 17:25
  that too, because this is really useful is everybody says they
• 17:28
  want templates, but templates alone kind of suck when you're
• 17:31
  talking about documentation. It's like, cool, I have like 4
• 17:33
  headers. I'm still, I still don't know
• 17:35
  what to do. So the next iteration of that
• 17:37
  state the templates of OK, in this section, here's generally
• 17:41
  what you want to put. Maybe you link to an example,
• 17:44
  but the best thing that we found is you create an example of
• 17:49
  every content type, let's say like 3 content types, an
• 17:52
  overview guide, a quick start and a how to guide.
• 17:56
  And instead of doing like templates on all these things
• 17:59
  that are fairly blank, you say the, the writing team is going
• 18:04
  to give you an example of a very good overview guide, a very good
• 18:08
  how to guide and a very good quick start.
• 18:11
  And maybe you add some context of like why we made these
• 18:15
  decisions, what goes in here? But that's I think the most
• 18:18
  useful for engineers to pattern match because they can say, all
• 18:21
  right, I'm doing this thing, it's different, but I'm going to
• 18:24
  follow the same structure. I'm going to follow the same
• 18:26
  length and, you know, make sure I'm going into the same level of
• 18:29
  detail when it. Comes to almost like creating a
• 18:34
  movement around docks and having what I mentioned before we hit
• 18:40
  record like a a cult like following with Stripe and the
• 18:44
  docs. I haven't heard people say that
• 18:48
  they like reading docs that much.
• 18:50
  That's not as you mentioned, it's not really a pastime that
• 18:53
  engineers enjoy. However, whenever I ask someone
• 18:57
  show me some examples of good docs, 9 times out of 10, Stripes
• 19:02
  docs get bundled into that. And I like how you were talking
• 19:08
  about like what not to do. It almost reminds me of the
• 19:10
  Charlie Munger thing on like, I don't know what I want, but I
• 19:13
  know what I don't want. So let's start there and then
• 19:15
  let's go. And by knowing what I don't
• 19:18
  want, it'll probably get me closer to what I do want.
• 19:20
  And so there's all kinds of ways that you can make sure you don't
• 19:25
  just do the obvious faux pas and the obvious things that are
• 19:28
  going to mess you up and make people even more painfully or
• 19:35
  more irritated when it comes to using your product.
• 19:37
  But are there things that you feel like you haven't mentioned
• 19:41
  so far that helped the user experience and the developer
• 19:47
  experience at Stripe really thrive?
• 19:51
  Yeah, I think the, I think the mindset because you're saying
• 19:54
  like a cult following this, this started for me back at Uber
• 19:58
  where you could write an RFC and get it approved if you you got a
• 20:06
  couple of yeses from some important people and nobody like
• 20:09
  through under the bus and then you got like unlimited headcount
• 20:13
  to do what you wanted to do. But I had an RFC out for
• 20:18
  creating a unified platform. We were using Confluence at the
• 20:21
  time. Wanted to kill, kill Confluence,
• 20:24
  create our own thing from the ground up.
• 20:25
  It was like a really ambitious, big risky project.
• 20:29
  And I had all of my data, like here's how it would be cheaper,
• 20:33
  here's how it would be a better user experience, here's how
• 20:36
  we've derisked it. And I was still getting some
• 20:38
  pushback and my my mentor, who is an engineering director was
• 20:43
  like, you're not gonna win this argument with logic because it's
• 20:47
  so against what the norm is for everybody.
• 20:50
  And it's against what the norm is for leadership to think like,
• 20:53
  wait, why do we, why do we need an internal content platform for
• 20:57
  engineers? Like, why do we need to build
• 20:58
  that ourselves? And he told me, like, you're
• 21:02
  basically running a campaign. Everybody's wondering when
• 21:05
  you're trying to convince them to do something that they don't
• 21:07
  want to do, like what's in it for me?
• 21:09
  And so he I was asking like, what's the leadership book?
• 21:12
  Like I read all the books on Apple, like, what can I do to be
• 21:15
  more persuasive? And we had Barack Obama's old or
• 21:20
  former campaign manager working for Uber at the at the time
• 21:23
  named David Plouffe. And he's like, you should read
• 21:26
  his book and see like how to win a campaign against all odds.
• 21:29
  And so it was like very grandiose to think of internal
• 21:32
  docs is like, you know, historical president, president
• 21:35
  being elected. But it's saying was really Yeah,
• 21:40
  I think, I think we yeah, it's pretty much the same in history
• 21:44
  in terms of impact. No, if you.
• 21:45
  Ever meet Barack? You're like, I stand in
• 21:47
  solidarity with you. I know what it's like.
• 21:50
  He's probably. Reading my book and telling his
• 21:52
  team like you gotta think about things that he's working on a.
• 21:54
  Digital startup right now for what I know.
• 21:58
  Not crypto, that's good. But yeah, he so that was a huge
• 22:03
  turning point for me. I mean I was still I was pretty
• 22:05
  far into my career already, but thinking about what's in it for
• 22:07
  me and not thinking about it in terms of jamming logic down
• 22:11
  humans throats. And so at strike, a lot of that
• 22:15
  was the same thing. We get mean, mean tweets all the
• 22:19
  time. Like this thing is really
• 22:20
  confusing. And instead of being offended or
• 22:21
  thinking like you're using it wrong, like we would think, OK,
• 22:26
  what is different about this person?
• 22:28
  And so we had a particular user we were struggling with, but it
• 22:31
  was really important, which was like the non-technical
• 22:34
  cofounder. And we, we obsessed over
• 22:37
  startups, we obsessed over technical founders.
• 22:41
  But the non-technical cofounder was also really important
• 22:45
  because when they're installing a billing system, it's beyond
• 22:48
  just like integrating the code. You have to think about what you
• 22:52
  want out of this billing system. What are you, what are the, what
• 22:55
  are the payment options, the subscriptions you're offering
• 22:57
  your users? Who is your backup house finance
• 23:01
  team? And like, what are their needs?
• 23:02
  Like, there's a lot of non-technical stuff that you
• 23:05
  need to, you need to serve in order for them to use the
• 23:09
  product. So putting yourself in the shoes
• 23:11
  of like, what's in it for, like what's in it for me from a
• 23:15
  user's perspective, like you just uncover all these little
• 23:18
  things to add up, add up overtime.
• 23:20
  And there weren't a lot of like Big Bang ships I can think of at
• 23:25
  Stripe with docks of like boom, we unveiled this thing and it
• 23:28
  just changed your life. It was like little things over
• 23:30
  time, like removing, one of the biggest things was removing
• 23:35
  inline links. Like like everybody who cared
• 23:37
  about SEO, we'd stuff every sentence with an inline link.
• 23:41
  And then we did user research and saw that the developer would
• 23:43
  have like 30 tabs open. So we, we minimise that by like
• 23:47
  10X and we immediately saw like a better user experience or we
• 23:52
  redid how, how we, how we did diagrams or we improved search
• 23:57
  or we improved the information architecture.
• 23:59
  So once you start thinking about it from the user's perspective
• 24:03
  in terms of like, all right, what are they, What are the
• 24:06
  small things they're trying to accomplish?
• 24:07
  And what are the little paper cuts and points of friction?
• 24:10
  You can just, you can just smooth over.
• 24:12
  Because I know for me when I'm using a product, if, if somebody
• 24:18
  adds that extra context that I needed to like get over a hump
• 24:22
  super. But it's a great feeling.
• 24:23
  I feel smart. I feel I'm blocked.
• 24:25
  And I think those are the feelings that collectively
• 24:28
  somebody later on is like, ohh, I had a great experience.
• 24:31
  And now I feel much different using this other tool where
• 24:34
  they're not thinking about all of the ergonomics of what I'm
• 24:37
  trying to accomplish. Yes, yeah, exactly.
• 24:40
  I think. I think.
• 24:40
  To that point, to those things you mentioned, funny that you
• 24:43
  didn't mention the one that I, I hear a lot about strike Docks
• 24:47
  actually, you know, speaking to our engineers as well, they said
• 24:50
  that one of the things they loved the most is that, you
• 24:55
  know, you do the smart, you know, inject injection of those
• 24:58
  API tokens somehow or like, you know, you can test it.
• 25:01
  So like you don't have to copy paste it and then everything
• 25:04
  gets messed up or something like that.
• 25:05
  Just like, you know, it's, it's injected just like takes that
• 25:07
  one boring, messy step that is like, yeah, you should do it,
• 25:11
  but but it does it for you. So it you know, you, you
• 25:15
  immediately feel taken care of that.
• 25:18
  One we got so much praise for that was actually done before I
• 25:22
  joined. But like that, I was so happy
• 25:24
  that mindset was there and there was so much feedback from users
• 25:28
  that just reinforced like, well, this small thing that you all
• 25:31
  did were so thrilled about. So how can we recreate that
• 25:35
  experience? And I remember one time at an
• 25:37
  all hands Patrick Carlson was like, I want to talk about this
• 25:41
  really exciting ship in in the industry.
• 25:44
  And he held up his phone and we're like, Oh, the the iPhone
• 25:48
  and he was so excited. Do you remember when ale?
• 25:51
  I don't know if you all iPhones, I'm sure Android does this now
• 25:54
  too. But when you get a two factor
• 25:56
  auth code and you have to go to your text messages, copy and
• 25:59
  paste it, then go back and then sometimes the browser would
• 26:02
  close or whatever. And then it started just showing
• 26:05
  it in the keyboard, like right above your keyboard and you
• 26:07
  could just copy and paste it or it would auto fill it.
• 26:10
  He was so psyched about that. And it's something that
• 26:13
  everybody was just conditioned to deal with and you, everybody
• 26:17
  hated it, but nobody was thinking like, ohh, they should
• 26:19
  think of a feature. But I'm, I'm guessing at some
• 26:22
  point they did use a research and saw like, ohh, I got to do 3
• 26:26
  clicks just to go back and do this thing multiple times a
• 26:29
  week. Like, how can we make that
• 26:31
  better? So that was the mindset we had
• 26:33
  is just like, what are these hidden unsexy things that will
• 26:36
  just make it a a tiny bit easier for for the user speaking?
• 26:41
  About this part of like user research, and you mentioned it
• 26:45
  before, like if you have a chance like speak to your dad's,
• 26:47
  you know, ask them questions or do proper user, which I wonder
• 26:51
  what are your favourite, you know, questions or sort of
• 26:54
  scenarios that you like to run to uncover those things, you
• 26:58
  know, to find those frictions and find those problems with
• 27:01
  your dogs. How do you like to run this kind
• 27:03
  of stuff? Yeah.
• 27:04
  That's a great question. I think the best thing is to get
• 27:08
  on a call with a user, have them share their screen like in the
• 27:11
  environment they're comfortable with.
• 27:13
  And then if you're, so, if we're trying to, for example, was
• 27:17
  helping a client recently with a big information architecture
• 27:20
  overhaul and they had the challenge, which a lot of
• 27:23
  companies do is they had an open source product and a paid
• 27:26
  product, a hosted product. And they wanted to like the
• 27:30
  hosted product was much better obviously.
• 27:31
  And they only made money if people used that one.
• 27:34
  So they wanted to present the information without being,
• 27:39
  without neglecting the open source community.
• 27:42
  It's a, it's a really hard information presentation
• 27:44
  problem. And at the time they were, they
• 27:48
  were going the opposite way, which is like open source 1st
• 27:51
  and then OHH, by the way, we have this paid product and, and
• 27:54
  unsurprisingly, they weren't getting as much traction as they
• 27:56
  wanted to the paid product. It's.
• 27:58
  Totally typical, yeah. It's a hard problem to solve.
• 28:02
  So when when we did user research, we had the user we're
• 28:07
  like fire up a fresh browser and now go to the docs and we told
• 28:12
  them like what docs to look for. But we didn't say like start on
• 28:14
  this page. It was like, we want to see
• 28:16
  everything. Are you starting from, from
• 28:18
  Google search or are you starting from from a different
• 28:22
  browser? Are you, are you going to or
• 28:25
  actually, we'd even say like look for information on this,
• 28:28
  like maybe they're starting with like a pending Reddit to their
• 28:33
  search or going to YouTube. And so once we did eventually
• 28:37
  bring them to the docks, if they didn't start there, it was the
• 28:40
  same thing. We'd be like, think out loud.
• 28:42
  We want you to accomplish this task.
• 28:44
  And just what would you normally do?
• 28:47
  Like don't try and change your behaviour for us.
• 28:51
  We're just going to observe and the thinking, telling them to
• 28:54
  think out loud is is really key. Like you're not going to get all
• 28:59
  the answers. You're going to find really
• 29:01
  obvious stuff after one call, but you're not going to get all
• 29:03
  the answers unless you do a few of those because people have
• 29:06
  different learning styles. Like I never trust search, so
• 29:09
  I'm always browsing. And then I get really mad
• 29:11
  because the browsing experience usually sucks, but a lot of
• 29:14
  people search 1st and so they're not browsing.
• 29:18
  So you get this like experience and this feeling of how your
• 29:21
  users navigate your docs and the different design cues you're
• 29:25
  giving them intentionally or unintentionally.
• 29:27
  And you'll uncover a lot of things there because you may
• 29:30
  have an excellent document, but it's buried or it's, it has a
• 29:34
  poor title. And then like as the, so we
• 29:37
  don't ask a lot of specific questions.
• 29:39
  It's just think out loud. And then you may say, OK, why
• 29:41
  did you do that? Like, tell me more.
• 29:44
  And then a lot of really great themes start to arise after you
• 29:48
  do then just like two or three of those.
• 29:50
  What are? Some design cues.
• 29:53
  Like where you put the text box is, is an important one.
• 29:57
  The like call outs of information are, are
• 30:01
  surprisingly important. Like you, you probably all read
• 30:06
  docs or both red docs where it's like, here's how you do the
• 30:10
  thing. But keep in mind you need this
• 30:11
  installed. And if you're thinking about
• 30:13
  doing this, think about this. And next thing you know you're
• 30:16
  like 3 paragraphs in and you've only read caveats and you can't
• 30:18
  remember what you're trying to do.
• 30:20
  So to break up those queues of like prerequisites, you may
• 30:24
  already have this stuff, but like, here's the checklist of
• 30:26
  things you need before you even start reading because there's no
• 30:29
  worse feeling where you get halfway down the page and you
• 30:31
  didn't realise you needed to like install something that you
• 30:34
  didn't or like here's a tip. And it's a, you know, it's a box
• 30:39
  that's elevated out from the docks.
• 30:41
  It's not buried in a, a paragraph or a warning or
• 30:44
  something like that. And then another thing that we,
• 30:47
  we, we did at Stripe were these things called escape hatches,
• 30:51
  where it's basically a call out, but you're never going to nail
• 30:55
  the audience. 100. Like for example, we had a, we
• 31:01
  had hosted Stripe checkout form where you can just like very
• 31:05
  simple, no code set up a forum, but it's going to be a separate
• 31:09
  window with Stripe in the URL. Or you can create your own, like
• 31:14
  write your own or customise your own.
• 31:16
  And it's, it's code based and it's two very different paths.
• 31:20
  Like a developer who wants very custom white labelled experience
• 31:23
  and somebody who just wants to like up and running in an hour
• 31:25
  or so, but they had the same name.
• 31:27
  They're both Stripe checkout. So we, we were aware of the
• 31:32
  scenario where somebody's Googling and they land on the
• 31:34
  right thing, they land on the wrong place and they don't know
• 31:37
  the other thing even exists. So we made sure that there was
• 31:40
  always an escape hatch somewhere like, oh, you're on, you're
• 31:43
  looking at the checkout, the hosted checkout.
• 31:45
  Do you want to go see Stripe Elements, which is you can
• 31:48
  create your own and pretty much every product there's especially
• 31:53
  when you're, when you're offering choice like there's
• 31:56
  some use case that is you're not serving that particular user.
• 32:01
  So if you give them these escape hatches to feel, not feel like,
• 32:05
  oh, they're not serving me or they don't support what I'm
• 32:08
  looking for, then you have a really good experience to guide
• 32:12
  them down the right the right path it.
• 32:15
  Feels to me like a lot of the magic that people have felt when
• 32:22
  they are using docs that just work is that somebody has had
• 32:28
  that same ANE or gone through that user journey and then
• 32:33
  thought about ways to make that as easy and as useful as
• 32:39
  possible. Right.
• 32:40
  Like you're saying how OK, I want the no code solution.
• 32:44
  I Google it, I see the I click on this it, it says check out,
• 32:51
  but it's actually the code solution.
• 32:53
  I get through a little bit of it and then I go, oh wait, no,
• 32:56
  here's what I really want and making it the best that they
• 33:00
  could and not leaving any stone unturned.
• 33:03
  Yeah, and, and the feeling of, of being OK, being the like
• 33:08
  behind the scenes when you're creating the developer
• 33:10
  experience or content. Like I think the best feeling we
• 33:12
  could give to users is making them feel smart, like they're
• 33:15
  the ones with the accomplishment.
• 33:17
  Because a lot of times that's what it is.
• 33:19
  Like, damn, I crushed that. I just built this thing in 30
• 33:23
  minutes. Like they're not going to go
• 33:24
  say, hey, colleague, the Stripe docs are really great.
• 33:27
  They're like, hey, look, I got this thing up and running in 30
• 33:30
  minutes. Maybe later they'll reflect back
• 33:32
  on like, I want to go back to strike because that was a good
• 33:34
  experience at that moment. If you can make them feel smart
• 33:37
  and make them feel like they're the accomplished one, then you
• 33:40
  earn that trust. It's like probably
• 33:42
  subconsciously to start, but they're going, they're going to
• 33:44
  want to go back versus feeling like that was like pulling
• 33:48
  teeth. I don't want to go back to using
• 33:50
  that platform or those docs and unless I absolutely have to.
• 33:53
  And and one thing I see not still 90% of the time is
• 33:59
  documentation read like read me. They don't describe what The
• 34:02
  thing is. And obviously it's a norm and
• 34:05
  it's fine to name your products or your service something
• 34:08
  memorable, you know, platypus dot JS or something.
• 34:10
  It might actually be a real thing.
• 34:12
  I don't remember mermaid dot JS. Like you can't intuit what these
• 34:15
  things are. And if you go to the README and
• 34:18
  it's just like we had this 3000 microservices at Uber when I
• 34:22
  left and they were all named just random names, which is
• 34:26
  again, totally fine. All right.
• 34:27
  Real quick, some words from our sponsors, Ellis, and then we'll
• 34:31
  be right back into the programme.
• 34:33
  Are you struggling with building your retrieval augmented
• 34:36
  generation AKA RAG applications? You are not alone.
• 34:41
  We all know tracking unstructured data conversion and
• 34:46
  retrieval is no small feat, but. There's a solution.
• 34:49
  Available now Introducing Zelizer Cloud Pipelines from the
• 34:54
  team behind the open source vector database Melvis.
• 34:57
  This robust tool transforms unstructured data into
• 35:01
  searchable vectors in an easy manner.
• 35:04
  It's designed for performance. Zellis Cloud platforms
• 35:09
  pipelines. Zilis Cloud Pipelines also offer
• 35:14
  scalability and reliability. They simplify your workflow,
• 35:17
  eliminating the need for complex customizations or infrastructure
• 35:22
  changes. Elevate your rag with high
• 35:24
  quality vector pipelines. Try Zelis Cloud Pipelines for
• 35:29
  free at soliz.com. You can find the link in the
• 35:32
  description. Let's get back into this
• 35:34
  podcast. So well, on that note, why don't
• 35:38
  we talk a little bit about foundational elements that you
• 35:42
  need in your documentation, especially if you're starting
• 35:45
  out. What are some things that are
• 35:47
  must haves? The onboarding experience needs
• 35:50
  to be really good. The getting started experience,
• 35:53
  like, even if you have nothing else, don't even don't even
• 35:56
  think about end to end tutorials or a lot of reference
• 35:59
  architectures until you've shown like describe like, like I
• 36:05
  mentioned very simply and clearly and succinctly like what
• 36:10
  The thing is, what the product or what your company is and why
• 36:14
  you should care. And then the quick start should
• 36:17
  be a hello world experience where somebody is able to
• 36:19
  accomplish something and you know, a sitting 1520 minutes,
• 36:23
  maybe 30 minutes Max. And if you can't time box it to
• 36:26
  that like figure something out because if developers can help
• 36:31
  it, they're not, they're going to go find something else like
• 36:34
  you can get their attention for 1520 minutes and they can follow
• 36:37
  these steps and accomplish a thing to to now go on to the
• 36:40
  next thing, then that's really key.
• 36:43
  So overview of what your company does in, in, in a short manner
• 36:47
  and then a quick start. And then from there, let's say
• 36:51
  you have a handful of features, maybe two products, you're going
• 36:56
  to have probably 20 to 30 documents.
• 36:59
  The information architecture is really important because the
• 37:05
  it's your chance to create the user journey for the user.
• 37:09
  Like a lot of times it's either all flat, like all the pages in
• 37:12
  whatever order they're created is just listed in the side NAV,
• 37:15
  yes. Or it's like a lot of times
• 37:18
  they'll be generic buckets, which is good.
• 37:20
  Developers are trying to do different content types, but
• 37:23
  it's like, oh, review section, conceptual section, tutorial
• 37:26
  section, troubleshooting section, and I'll like
• 37:29
  developers don't think like that.
• 37:31
  They want to think I'm getting started with your thing, and
• 37:34
  then I'm trying to design the thing, then I'm trying to build
• 37:37
  the thing, then I'm trying to deploy the thing, then I'm
• 37:40
  trying to maintain the thing. So if you can get that
• 37:42
  sequential ordering like to mimic the user journey that you
• 37:46
  want your users to follow in your information architecture in
• 37:49
  your side NAV, you're going to create a pretty good experience
• 37:55
  for them. Wow.
• 37:58
  So does it get started? Design the thing?
• 38:01
  What was the? Because I want to write that
• 38:03
  down, Yeah. Design, Build, Deploy, Manage
• 38:08
  are usually like the steps, but figure out what those phases are
• 38:13
  for your product and architect your documentation around that
• 38:18
  to mimic that user journey. You're just removing cognitive
• 38:22
  load. Yeah, just.
• 38:23
  Taking maybe a step back, you know, because you mentioned that
• 38:27
  sort of like the, the minimal infrastructure of information in
• 38:31
  the README. And to me, I don't know how you
• 38:33
  look at it, but to me, read me sort of like this.
• 38:35
  I guess that like the, the first documentation, you know, the
• 38:38
  first doc you write, it's probably to read me, right?
• 38:40
  And then, umm, you know, step from read me to those five steps
• 38:45
  for potentially multiple products.
• 38:46
  That's that's a big step, step obviously, right.
• 38:49
  So I'm wondering how do you think about transition?
• 38:52
  What does she do even transition from that, you know, read me to
• 38:57
  an actual, you know, doc, whether MK dogs or some platform
• 39:00
  that you use or whatever it is. And then, you know, build those
• 39:03
  sections in there, right? Because I think it's like easy
• 39:07
  to, you know, yeah, please, when you maybe don't need to.
• 39:10
  Yeah, that's a good question. I think you can probably tell
• 39:13
  when you're spending a lot of time manually helping your
• 39:15
  users, especially early on, it's pretty common.
• 39:18
  You don't want, you don't want to set up your docs platform
• 39:20
  before you even have traction or know what your users need.
• 39:23
  But let's say at first the README is users are like, cool,
• 39:27
  I got up and running. You see your usage spike and now
• 39:31
  you're getting a lot of inbound questions.
• 39:32
  OK, but what about this use case or what about this feature?
• 39:35
  And you realise, oh, we support that feature, but it's not
• 39:38
  documented. So I think the read me
• 39:41
  accomplishes the like front page of the of the repo, like what
• 39:45
  The thing is and how to install it.
• 39:47
  But you need to graduate once the users are now beyond that,
• 39:51
  like you have some use cases you want them to try out.
• 39:54
  There's some maintenance, there's some upgrading, like
• 39:56
  whatever those things are that should not be in the in the
• 40:00
  README. The README should be contained
• 40:03
  to like a short overview, quick installation steps and maybe
• 40:07
  some troubleshooting. But like once you graduate
• 40:09
  beyond that and you're that and you're manually.
• 40:11
  Shorting users to get those things done.
• 40:15
  Put it in. Put it into Documentation 1.
• 40:17
  Thing Cooper and I talk at length about is how you actually
• 40:23
  measure with Okr's documentation.
• 40:27
  How can you say, alright, you're someone whose full time job is
• 40:31
  to work on the docks, How do you know what you're doing is being
• 40:36
  effective? Yeah, I would say that's the
• 40:39
  easiest way to like, if you want to end a conversation with a
• 40:42
  technical writer, just ask them that question and then they'll
• 40:44
  leave you alone because that's a really hard question to answer.
• 40:47
  So I've done it. I've tried a lot of things over
• 40:51
  the years because I think there's two ways to measure.
• 40:55
  One is like the efficacy of your docs, like how how can we
• 40:58
  quantitatively measure the usefulness for our users?
• 41:01
  And then also internally, like what's the ROI of putting
• 41:04
  resources on this? Like are we actually, are we
• 41:08
  actually justifying the cost of putting engineers on the docks
• 41:11
  or putting full time technical writers on the docks?
• 41:14
  So most of my career was, you know, measuring page views,
• 41:19
  bounce rate, search queries, like driving clicks to the
• 41:23
  signups, like those are all important, but it's not it's not
• 41:26
  showing you the whole picture and then the support ticket
• 41:31
  deflection to justify our costs internally, like how many
• 41:35
  tickets are being booked after they read the docs, like, or how
• 41:39
  many took, how many tickets in general are being booked?
• 41:41
  And how can we we, we block tickets from being created or
• 41:46
  make them faster to be resolved through better documentation.
• 41:49
  But I've, I've also changed the conversation or like the mindset
• 41:54
  a bit, because if you just focus on those metrics, you're going
• 41:57
  to get really terrible developer docs like you're optimising for
• 42:00
  the wrong things. You're making it really
• 42:02
  transactional. O I've thought about it and
• 42:05
  frame it more like design. Like it's really hard to come up
• 42:09
  with hard metrics for design, like, yes, driving traffic and
• 42:13
  driving sign ups, but you know, good design when you see it.
• 42:17
  And a lot of that is, is, is qualitative.
• 42:19
  So you have to have some trust there.
• 42:22
  And, but then some other things that are a mix of quantitative
• 42:26
  and qualitative driving like customer or a customer
• 42:31
  satisfaction is good. Like pretty much every dock set
• 42:34
  has page ratings on the bottom. And those are good to, to make
• 42:37
  sure. I really just think there's a
• 42:38
  threshold you should care about like 60% or higher means it's,
• 42:42
  it's a solid dock. You're, I've never seen anything
• 42:45
  be 100% unless it was like how to reset my password.
• 42:48
  And then the doc, you know, the doc actually worked.
• 42:50
  But 60 to 80%, you're probably in really good shape.
• 42:53
  Nothing should be below 60% or it's probably terrible.
• 42:56
  And then, but most importantly, especially when you're at
• 43:00
  Vaughn, you're getting like decent volume.
• 43:02
  You can see how long does it take for a user to accomplish a
• 43:05
  particular task. That's really important.
• 43:08
  And it could be like mimicking whatever's in your quick start.
• 43:11
  Is it just installing the your thing or is it creating a test
• 43:17
  charge with, with Stripe? And we were lucky enough to have
• 43:22
  data scientists that could like tell us exactly how long it took
• 43:26
  and, you know, have different cohorts based off of their,
• 43:29
  their user demographic. But that was the thing we
• 43:32
  obsessed the most about was that that time to 1st charge, that
• 43:36
  time to live. And then we mimic that for every
• 43:39
  product, you know, like time to 1st creating a subscription,
• 43:43
  time to 1st creating an invoice in a, in a dashboard.
• 43:47
  So I would say those things that are not hard Google Analytics
• 43:53
  type metrics, but things that your docs obviously heavily
• 43:56
  influence but. So, so on that front, as I think
• 43:59
  that's a really good one, is the one you mentioned is basically
• 44:04
  the sort of like a north star, right?
• 44:06
  Like you're thinking that dog should support is if if if that
• 44:09
  page is basically, you know, quick answer to an FAQ.
• 44:12
  You want a high bounce, like you want people to go there, solve,
• 44:14
  go, right. But, umm, I'm sort of like
• 44:17
  thinking what? I'll do those, you know, let's
• 44:22
  see some mentioned Google Analytics metrics.
• 44:24
  Let's say you actually found valuable for improving pages,
• 44:28
  right? For improving those local things
• 44:30
  while keeping the North Star in mind.
• 44:33
  Yeah, I think 100% what you said is the IT there are signals,
• 44:38
  they're not like they're signals towards your North Star and you
• 44:41
  should treat them as such because you're totally right.
• 44:43
  I use that example a lot as well as like the bounce rate on its
• 44:46
  own doesn't tell you anything. Like are you trying to have them
• 44:49
  reset their password and go away?
• 44:51
  Or are they like you don't want, you don't want to brag that you
• 44:53
  know, time spent on pages 10 minutes for resetting your
• 44:57
  password. So but those are signals once
• 44:59
  you know your business. And I think CSAT is a great
• 45:02
  signal as well. Like it's just an example of
• 45:06
  like it's shining a light on the things you should care about.
• 45:08
  It's not giving you all the answers, but OK, I have 1000
• 45:11
  pages. These are the top ten worst
• 45:15
  rated pages. I should probably take a look on
• 45:17
  if I need to improve these. Another thing too, that's, I
• 45:21
  guess measuring, but it's super crucial and it gives you a lot
• 45:27
  of good product insight. What are people searching for in
• 45:29
  your site? Like what are the top 10 search
• 45:32
  terms? That is so illuminating.
• 45:34
  It's never what you think it's going to be like.
• 45:37
  You think that they're looking for information on all your new
• 45:39
  whizbang features? Like probably not like we, we
• 45:43
  didn't have any, we weren't using any metrics when I, when I
• 45:47
  joined stripe like on for the docs.
• 45:49
  And so that was one of the first things I did was set U all that
• 45:52
  instrumentation and we looked at the top ten search terms and
• 45:55
  we're thinking, Ohh, we just launched this.
• 45:57
  We just launched or not launched, but we're investing in
• 46:00
  connect, our marketplace platform and it's really complex
• 46:02
  and we just need to provide more guides and better docs.
• 46:05
  And then we go look at the search terms and it's like
• 46:07
  connects not in the top ten. Paypal's number one, like, OK,
• 46:10
  well we're not supporting PayPal.
• 46:12
  But then interestingly enough, number two, I think was like net
• 46:17
  examples, like people were looking, we didn't have, we
• 46:19
  didn't support net. And that was feedback we can, we
• 46:22
  were able to give to our product team and we're like, oh, we
• 46:25
  would have done this, we just didn't know.
• 46:27
  And then we updated all the docs, you know, we had the
• 46:29
  tabbed examples with the different languages and we're
• 46:31
  obviously showing the ones or whatever the hottest new
• 46:33
  languages are. But really our users were they
• 46:37
  really wanted net. So those search terms are super
• 46:41
  crucial. And then also you're probably
• 46:44
  not using the terminology that your users are using.
• 46:49
  Like you might have fancy feature names for your stuff,
• 46:52
  which is totally fine, but you need to know how your users
• 46:56
  think about your product and what they're searching for.
• 46:59
  And make sure that you're, if not writing in that way in your
• 47:04
  docs, like on the back end, you know, setting up your search
• 47:07
  weights and your synonyms. So they're, they're routed to
• 47:10
  the right place because again, they're going to be like, OK, I
• 47:13
  want point of sale and then I go to strike point of sale and it's
• 47:16
  like terminal. I don't want terminal.
• 47:18
  That sounds like a developer product, like see you later.
• 47:21
  But Stripe terminal was the name of our point of sale product.
• 47:23
  So there there are a lot of examples like you can just find
• 47:27
  gold in the in the search terms in the top search queries.
• 47:32
  Yeah, I think on the search wrong, like the like Santos
• 47:37
  interesting to me was that I assumed that people searching
• 47:41
  the dogs, but but you know, most of the folks, they actually go
• 47:43
  to Google and do branded searches, you know, so like, you
• 47:46
  know, something's tried Docs, right and and that was super
• 47:51
  interesting to me, you know, like that.
• 47:53
  And then actually you can do the same thing, right?
• 47:54
  Like you, you just look what do people do when it comes to
• 47:58
  branded searches, right? Or especially branded with the
• 48:01
  dogs in there, right? It's the same, sorry, but I
• 48:03
  think that that is interesting, right?
• 48:05
  Because you're landing on some page through Google and that's
• 48:08
  where people start in the docs who don't start on docs home,
• 48:11
  right? So you gotta assume every page
• 48:13
  is page one, like speaking in that information foraging
• 48:17
  language that you mentioned at the beginning I.
• 48:21
  Rarely used our stripe doc search.
• 48:22
  I would. I just had the muscle memory of
• 48:24
  Google search. Like I would start 95% of my
• 48:27
  search queries with Google for striped docs like I didn't.
• 48:31
  I don't know, it's just embedded.
• 48:32
  I didn't trust the search. One, one question I had on the
• 48:36
  basically time to 1st action and how were you measuring that?
• 48:42
  Because if somebody's on the docks, are you using tools to
• 48:48
  know who they are or is it just that you're seeing them spin it
• 48:53
  U like how can I, as someone who is less sophisticated,
• 48:58
  understand that? And I guess another piece to it
• 49:01
  is I have assumptions about what's important or what the
• 49:06
  first action that I want somebody to take, but then maybe
• 49:10
  there's ways that I can see what better first actions are.
• 49:14
  And so I can ideally, with help from the docs, encourage people
• 49:18
  to take that first action. Yeah.
• 49:20
  One of the best things that Strike had that looking back was
• 49:24
  huge for us, is I don't remember seeing the login experience for
• 49:28
  documentation before Stripe, OK, having a logged in experience,
• 49:33
  you got some tremendous data from your users going back for
• 49:37
  back and forth between the dashboard and the docs.
• 49:39
  So that's one where you can see exactly who the users are and
• 49:43
  their entire user journey. And then we just had world class
• 49:45
  data scientists too, where we would get good anonymous data.
• 49:48
  Like we've seen their where their IP address was from and,
• 49:53
  and, and the like their user session, like all that stuff
• 49:57
  where you could create like a shadow profile of like this
• 50:00
  anonymous user session. And then in aggregate, you can
• 50:05
  see, you can get information that way.
• 50:08
  But creating a login experience with the docs, which I see is a
• 50:11
  lot more common now, you get really, really good stuff there.
• 50:14
  And then, yeah, it's deciding what to choose.
• 50:16
  I like to simplify it of what is your company's strategy?
• 50:20
  What do you want to accomplish? What do you want your users to
• 50:22
  see? And then your user, what is your
• 50:25
  user research telling you? Like, are they, it's great if
• 50:29
  that allies, but what are they actually using your product for?
• 50:32
  And you should figure out like between those two inputs like
• 50:35
  what are the signals you wanna focus on and measure.
• 50:39
  Sure. And you know.
• 50:40
  As a technical founder, you know, what was the first option
• 50:43
  that really matters? Obviously later things get, you
• 50:45
  know, more complicated into your point.
• 50:46
  You know, you need research, you need, you know, you need to look
• 50:49
  into those things. But I guess at the very
• 50:51
  beginning, it's sort of, I mean, there are not that many
• 50:54
  candidates right at the beginning, right?
• 50:56
  Like, it's sort of sort of clear, you know?
• 51:00
  Yeah, but developers have so much choice now.
• 51:04
  And a lot of times when they're playing around with something,
• 51:06
  it's because they saw it on Hacker News or somebody tweeted
• 51:09
  about it or they're Googling. Like, I am not an expert in this
• 51:12
  thing, but I need to use a tool for this and let me just Google
• 51:16
  or let me search on Reddit and let me get, I have 5 minutes to
• 51:19
  give it a try. Because rarely are they like
• 51:24
  today, are they forced to sit down and use this thing.
• 51:26
  Like, OK, my CTO purchased this product, now I got to use it.
• 51:31
  A lot of times it's bottoms up experimentation and there's so
• 51:35
  much choice. So how can you like create that
• 51:37
  really quick dopamine hit to make them come back or not say
• 51:42
  negative things about you, maybe even say positive things.
• 51:44
  And that's really how this flywheel starts for building a
• 51:46
  great dev brand where you hear 22 developers talk positively
• 51:51
  about a thing like that's very rare.
• 51:53
  You're going to be interested to go try it out yourself and it's
• 51:57
  already going to have this where you're going.
• 51:58
  You're going to give it more trust because it's been promoted
• 52:02
  by people you trust. The dopamine hit idea is exactly
• 52:07
  what I was thinking is can you get someone that dopamine hit as
• 52:10
  quickly as possible and then boom you don't.
• 52:13
  You don't necessarily got them, but you're on your way.
• 52:18
  Yeah, like the the Oracle or the the Java docs, like learning,
• 52:23
  it's just like like the equivalent of like brutalist
• 52:25
  architecture. It's just like, what are you
• 52:27
  going to do about it? Read these, you have to.
• 52:29
  And they're just awful. They make you miserable.
• 52:33
  The way they look, the way the font is, the way the kerning is,
• 52:37
  the way like everything is just miserable.
• 52:41
  But so how can you try and achieve the opposite of that?
• 52:44
  Exactly. Going back to what not to do
• 52:47
  that is. That is very helpful.
• 52:49
  So do the opposite of those Java docs.
• 52:51
  Well dude, Dave, I appreciate this so much.
• 52:55
  I want to know what you've been U to now like what are what are
• 52:58
  yours your day in day out look like these days and what are you
• 53:01
  excited about? Yeah, I when I left Stripe, it
• 53:05
  was July 2022 and a lot of like, I felt that I was getting
• 53:12
  repetitive a little bit. Like we didn't talk.
• 53:14
  We could do an into another episode on just internal docs,
• 53:17
  which I may even be more excited about than external docs because
• 53:20
  it's a it's it's a new frontier, but but a lot of it felt like,
• 53:26
  all right, there's just basically maintenance like that.
• 53:30
  I don't know what I want to do next.
• 53:31
  I just did this twice in a row. Do I want to start over again?
• 53:34
  And then three months later, chat show BT dropped and like, I
• 53:38
  was like, oh, I'm going to be out of a job soon.
• 53:39
  Like that was, I remember my father lost setting an article.
• 53:43
  He's like, look, what's number one on jobs to be replaced by AI
• 53:45
  1st. And he thought it was hilarious
• 53:47
  that it was technical writer. And so we all had this,
• 53:50
  everybody in my field had that thought like, what does this
• 53:53
  mean for us? But I was, so I was doing Angel
• 53:56
  investing and advising, investing that whole, you know,
• 54:00
  pseudo sabbatical. And I started getting a tonne of
• 54:04
  inbound from AI focused companies that were like, we
• 54:07
  have some traction. We have a product.
• 54:09
  We have no idea how to educate our users.
• 54:12
  So going on a year and a half of that, like I have actually two
• 54:16
  consultancies, one with the first technical writer at
• 54:21
  Stripe, Guy named Larry Ullman was there nine years.
• 54:24
  Like we do, we take warm referrals for companies that
• 54:30
  want to do like improve their developer experience and their
• 54:33
  documentation, obviously. And that's just been so exciting
• 54:37
  to like learn and try and create the new way to present
• 54:41
  information because OM focused products, it's it's gotta be
• 54:44
  different. Like you can't deal with a
• 54:46
  really real time multimodal interface.
• 54:51
  And now I'm going to go look at static docs like that just feels
• 54:54
  so wrong. So we're spending a lot of time
• 54:57
  rethinking how to present information for, for LM AI
• 55:01
  focused products. And then also doing consulting
• 55:06
  for, I founded a group called Abstract Group.
• 55:10
  And we're we're also doing just go to market overall.
• 55:14
  Like, hey, you're a researcher, you raised a lot of money.
• 55:17
  You've never like worked at a real company before.
• 55:19
  Like how do you actually get your product to market and how
• 55:23
  do you get traction? So I've just been all in like
• 55:26
  very well, maybe if I didn't just have a kid, I would have
• 55:29
  done this with the crypto phase. Like, all right, I'm all in on
• 55:32
  NFT. That's so exciting to be part of
• 55:33
  a technology wave, but I'm I'm, I'm just, yeah, unashamedly so
• 55:39
  excited about AI and trying to figure out how to help startups.
• 55:43
  Like with all those messy problems that I think a lot of
• 55:46
  us kind of felt like, alright, we mostly have the answers for
• 55:48
  SAS. Like we may not do everything
• 55:51
  how we're supposed to, but we know what like we have good
• 55:54
  examples, we know how to get there and that just does not
• 55:56
  exist for AI companies. And I think that's really
• 55:59
  exciting for me, The. Idea of internal docs out of
• 56:04
  everything you just said is what got me so excited.
• 56:08
  I love the LM stuff, but that honestly is what we like play in
• 56:12
  every day. I've been reading two books that
• 56:16
  I can't get enough of and it's by the same author.
• 56:19
  One is writing to learn and the other is like on writing well
• 56:25
  and it just talks about the power, basically the superpower
• 56:30
  of writing. And that was one of the reasons
• 56:32
  that I was so excited to talk to you too, because you can
• 56:37
  accomplish so much in the written word.
• 56:41
  And it's funny that you're mentioning how now, all right,
• 56:44
  you can't just have only the written word.
• 56:46
  You do need to accompany that with other ways to keep people
• 56:51
  engaged and keep things more interesting.
• 56:54
  But when you have very complex ideas that are put simply on a
• 57:01
  page, it is so incredible. And internal documentation is
• 57:10
  like the last thing to get that. And especially with the remote
• 57:14
  culture, I am so on board with that idea.
• 57:16
  So yeah, I would love to get you on here again and talk 100%
• 57:21
  about that. It's.
• 57:22
  Great because you and and hopefully Stripe open sources
• 57:27
  what we built. But we we took Mark doc, which
• 57:30
  we created for external docs and we took the learning and the
• 57:33
  philosophy and applied it to internal docs like nothing
• 57:36
  crazy. The hard part was convincing
• 57:38
  people internally that this approach would work and luckily
• 57:41
  it did. But there are also less rules.
• 57:44
  Like there are so many things you need to worry about
• 57:48
  justifiably so with user information.
• 57:51
  Like you can, you have all the access to your users internally
• 57:54
  because they're coworkers and you don't have to have this
• 57:57
  level of Polish that you need for external.
• 57:59
  You have to worry about mean tweets and you work side by side
• 58:02
  with your users. So you can say like, hey,
• 58:05
  engineer, this is for you. Like you need to help me or this
• 58:07
  team, I'm helping you out so you can do some phenomenal things.
• 58:11
  And like there, I, I hope they, they do a blog post or something
• 58:15
  on how amazing the internal docs platform is.
• 58:18
  I think it's rivalling the external docs.
• 58:20
  Ur not strip. Excellent dude.
• 58:23
  The the other thing too, that I guess it's a little self
• 58:27
  promotion, but I am excited about it because I wanted to
• 58:29
  exist. I'm working on a book about.
• 58:32
  I mean, it's not a it's not about strike product philosophy.
• 58:36
  A lot of people were talking to are Stripe are from Stripe, but
• 58:39
  it's more about how you build great products with like a
• 58:43
  particular mindset, not a framework, not a not a process.
• 58:47
  It's more where the war stories from accomplished startup people
• 58:51
  on building things that developers and users love.
• 58:56
  So that should be coming out next year.
• 58:59
  And I'm working on it with Cider Orlando, who was the editor of
• 59:02
  Strike, editor in chief of Strike Press for a while.
• 59:04
  Phenomenal writer. And we've been interviewing a
• 59:07
  bunch of people from across the industry.
• 59:09
  And because that's how I've learned the best is like reading
• 59:11
  these books from even companies that like failed, you know, or,
• 59:16
  or projects that failed, like Xerox Park or or Silicon
• 59:19
  Graphics, like what did they do? And just tell me the stories.
• 59:22
  Don't try and distil it into a philosophy or a framework.
• 59:25
  Tell me the stories and I'll derive my own findings.
• 59:27
  So I'm really, really excited about that because I don't think
• 59:30
  and going back to the beginning of the conversation, Cuba, when
• 59:34
  we were talking about like, it's all the all the functions, it's
• 59:37
  marketing, it's it's sales, it's it's developer experience, it's
• 59:41
  engineering, it's technical writing.
• 59:42
  And that's the core thesis of the book is like, we're going to
• 59:46
  try and tell the stories from all these perspectives because
• 59:49
  you need all of these functions in order to build great
• 59:52
  developer products. Awesome dude, I really
• 59:55
  appreciate you coming on here and chatting with us about this.
• 59:58
  It was very self-serving from my part because I got both of you
• 1:00:02
  who I very much respect on the dark side of things to get in a
• 1:00:07
  virtual room and chat all about it and really geek out.
• 1:00:11
  So I learned a tonne. I mean, I have notes and notes,
• 1:00:15
  so just going from that idea of how.
• 1:00:19
  You structure your dogs to not be just like throw it U there
• 1:00:23
  take takes a little bit of ride and thoughtfulness in what every
• 1:00:30
  bit and piece of the DOT looks like.
• 1:00:33
  So going from what you were saying how it's getting started,
• 1:00:36
  design, build, deploy, manage that I had never heard before.
• 1:00:41
  I thought that was really cool. The dopamine hit just try and
• 1:00:44
  get that quick dopamine hit as fast as possible and make sure
• 1:00:48
  that you can get that and then what not to do like don't write
• 1:00:52
  complex sentences. Don't put a bunch of don't make
• 1:00:55
  it like Web 1.0 where you're getting banner ads all over the
• 1:00:59
  place and everything is trying to vie for your attention.
• 1:01:02
  Make sure you are thoughtful in how you are doing white space
• 1:01:05
  because people are doing that F shape scanning pattern.
• 1:01:10
  Which I fully agree. Whenever I will read stuff I'm
• 1:01:14
  not. I'm mainly looking at the
• 1:01:15
  headers to see if it's relevant and then I'll dive in.
• 1:01:18
  So big same. Yeah, both of you.
• 1:01:20
  All. Thank you.
• 1:01:21
  Yeah. Good materials.
• 1:01:23
  Thank you for having me. Thank you SO.
• 1:01:25
  Much.