Latest / Agentic Conversations (formally mlops.community) / The Impact of UX Research in the AI Space // Lauren Kaplan // #272

Transcript

The Impact of UX Research in the AI Space // Lauren Kaplan // #272 · Agentic Conversations (formally mlops.community)

0:00
My name is Lauren Kaplan. I'm a mixed methods UX
0:03
researcher focused on AII. Take my coffee with some cream
0:10
and that's about it. Welcome back to the ML OPS
0:17
Community Podcast. I am your host Demetrios, and
0:20
today I'm happy as a lark because we confirmed Thomas
0:26
Wolfe, the Co founder of Hugging Face, will be in person with me
0:30
in Amsterdam for the AI Agents in production virtual
0:34
conference. Don't let that in person part
0:36
that I just said confuse you. We're going to be streaming live
0:40
from a really cool museum in Amsterdam.
0:43
If you want to come, hit me up. But you don't have to be in
0:47
Amsterdam to enjoy the fun. That's right, it's all going to
0:51
be in the Internet. On the Internet everywhere that
0:57
you can find a web page, we will be there.
1:00
And the specific web page to register for the AI Agents in
1:04
Production conference on November 13th will be in the
1:08
description below. Highly encourage it.
1:11
We've got Thomas Wolfe, Co founder of Hugging Face, as one
1:15
of our keynotes and he is just one of 30 incredible speakers.
1:21
I don't want to waste your time naming them all off, but they're
1:25
good. And I'm trying really hard to
1:28
just talk to you about the conversation I just had with
1:32
Lauren. This conversation was all about
1:36
UX research, and she has a bit of experience with this because
1:43
she worked on Pytorch as a UX researcher to know how to just
1:50
make that framework and that open source project Excel.
1:56
And I think they did a pretty good job.
1:59
So we go through what it means to be a UX researcher, why it's
2:03
important, why we should even be thinking about this, and how if
2:09
you have a UX researcher at your disposal, should you interface
2:14
with them? How can you be a good
2:16
stakeholder in the research? Let's jump into it with Lauren.
2:20
And as always, if you enjoy this podcast, just share it with one
2:26
friend. That's all I ask today for the
2:29
musical recommendation. We've got something very
2:31
special. Let's listen to my man MKG.
2:34
He's been blowing up lately and this song is just a beauty.
4:05
We have to set the scene. I want to know what user
4:09
experience research is and the TLDR of of the space and what
4:15
you've been up to. All right.
4:17
So I think starting with the first piece is generally the
4:20
process of getting insights into how people are interacting with
4:26
an experience which that can be a product we can just like start
4:30
there and really interesting in their perspective in terms of
4:33
like their needs, their pain points, their motivations with
4:36
that. Secondly, it was like what am I?
4:41
What have you been up to? Well, yeah, I mean, I've, I
4:45
guess there's some background into like what I've worked on,
4:48
right? Like in the past, worked at
4:51
Facebook AI, which turned into Meta, working on research for
4:55
Pie, Torch and also on privacy preserving machine learning.
4:59
And then most recently, I've been at AI chip startup and
5:03
right now sort of exploring other opportunities.
5:08
Nice. So there is something that I
5:11
think about when it comes to like user experience research,
5:18
and that is the typical way that I feel like it gets done, which
5:24
is kind of, as you mentioned, folks sitting together trying to
5:29
explore where there's pain points, where there's friction.
5:33
Does that just mean that like, the job is you watching people
5:38
do their jobs? There's different methods,
5:41
right? So I think like what you just
5:44
mentioned is sort of like thinking of like what you call
5:47
it kind of qualitative, right? Like you can do things called
5:50
like a contextual inquiry where you're going, where people are
5:53
actually using a product and sitting down with them and
5:55
observing them. But you can also do things like
5:59
more quantitatively, right? Like you can collect data even
6:03
like analysing it with some like machine learning tools, not
6:07
right? Like doing sentiment analysis or
6:09
analysing like behavioural data and looking that up with surveys
6:13
and that sort of a thing. So there's like a big range of
6:16
approaches you can use to get insight into what people need
6:22
and what they're doing. Yeah, I can see that.
6:26
So you have a lot of tools in your tool belt to try to
6:31
ultimately make the experience as painless as possible.
6:36
Do you feel like there are certain things that have worked
6:42
better for you in your your past that have given you or like
6:48
helped you gain those insights? I think it's very like context
6:52
specific, right? Because you have to start on
6:54
like, and that's what like, I think it was like a maybe at the
6:58
workflow for a researcher, right?
7:00
It's like really defining, like, what is the problem that you're
7:03
tackling? Like identifying all of your
7:06
stakeholders And like, that's all of the context that goes
7:09
into it. Like who are you actually
7:10
including and the research and why?
7:13
And like, what would people even feel comfortable with?
7:15
Like in the past, some of the topics I focused on people were
7:20
pretty private about, right? So some methods like, hey, let
7:22
me sit and follow you at Google while you're working on some
7:27
confidential projects. Like no one's going to do that,
7:30
right? But will they maybe do?
7:32
Not saying anyone was necessarily from Google in my
7:34
research, but maybe they'd feel more comfortable doing an
7:38
interview right where they are clear on like, hey, like these
7:43
are your rights as a researcher. You don't have to talk about
7:46
this or that. Like this is going to be
7:48
anonymized. There's a lot of like privacy
7:51
concerns for people sometimes or generally.
7:54
And also like best practises as a researcher that you have to
7:57
follow, that will also kind of shape what you can do.
8:03
The other thing that as you were talking about, hey, I would love
8:07
to have someone get to the end of an interview and say, wow,
8:12
that was amazing. I want to do more with you.
8:15
What are you thinking about as you're trying to prep questions
8:18
or you're trying to like go deeper?
8:20
Is it just trying to ask someone to tell you about their pains
8:26
when using a product or when like where does the product not
8:31
hold up? Is it feature request?
8:35
So I think we're getting kind of like into like what the goals of
8:40
like what you're the insights you're trying to get right.
8:42
So before you even get to the point where you're doing, let's
8:46
say I'm just going to anchor in an interview, you've already
8:49
ideally like met with all your stakeholders internally.
8:53
You're very clear on like what the impacts, like everything
8:56
should be very solution oriented.
8:58
What is the impact of doing this research?
9:00
Why are we doing it? Who do we want to talk to and
9:03
why these are this is what we're going to focus on on this work.
9:07
And then when we sit down, you start to create what's called
9:09
like an interview guide, right? And ideally and like a
9:13
successful collaboration, right, with, you know, you'll work with
9:17
engineers, designers, product, depending on the team, the
9:20
company you're with. And ideally, like a really good
9:23
collaboration between an engineer and a researcher would
9:27
be like they're giving you feedback in that talk, right?
9:31
They're being very engaged, like, hey, we want to know about
9:34
this or you draught out of questions like, yeah, we want to
9:36
get like drilled down into, I don't know, like what percents,
9:41
you know, what's what does it mean to have like a good
9:44
experience with like their inferencing or like they just
9:46
give like very specific feedback?
9:48
Because if you don't give specific feedback and you don't
9:51
engage with a researcher, you're going to get like really high
9:53
level results and you're not going to be happy able.
9:56
Why do we do all this work? It's just all high level.
9:59
And so in order to have like really targeted insights, you
10:03
need a lot of engagement and collaboration.
10:06
And I think that's what I saw at Meta that was very positive was
10:13
the engineers were just like super engaged.
10:15
Like if you left to dock with them, you'd see like hundreds of
10:19
people like can go like commenting.
10:22
So that's a really positive sign, like and positive way to
10:25
work when you're a researcher. If you don't have that, it's
10:28
going to be tough. And do you think at Meta, you
10:32
saw that because folks could see the outcomes of the research,
10:40
They would see a product or a experience that they were having
10:46
get better? And so it's almost like, you
10:48
know, it's easier for me to pay taxes in Germany because the
10:51
roads are really smooth. And so I see my taxes at work in
10:55
a way. I think so.
10:57
And I think also the engineering teams like had experience
11:01
working with researchers. So even if they hadn't worked
11:04
with you, they might have worked with someone else.
11:06
I also think the culture was just really strong.
11:11
It was very open, like no stupid questions, right?
11:15
Because as like from my background being sociologist and
11:19
then Amber, like one of my first projects, they're like really
11:22
going deep into AI and I was a little intimidated, right?
11:28
But I like, I had to like, dive in and like, learn really more
11:31
deeply about the topic. And something that was really
11:34
positive was like, I remember, like, drafting out an analysis
11:37
and like sitting down with the products and engineering lead.
11:40
And like, I'd have like, a couple questions.
11:42
And they'd be like, yeah, that's a total reasonable question
11:45
because that's an internal tool you would have never even known
11:47
about, right? Like, hey, you're asking good
11:50
questions. And I think having that culture
11:51
of like encouraging people to learn is really important as
11:56
well because especially now that AI has grown so much, I feel
12:02
like more and more researchers are going to have to on board,
12:06
on set AI. And so not everyone like has
12:09
that experience yet. So that's of a ramble, but.
12:14
And one thing that you told me was that UX or user experience
12:19
can be a much broader thing than just like product.
12:26
And what exactly does that mean? Yeah.
12:29
So the reason I started like this, like working definition
12:33
with product is because I feel like it's the more traditional
12:36
and more concrete thing that people think of.
12:40
But in reality, I think you're working much more broad than
12:44
that. Like, for example, you might be
12:46
as a researcher working on understanding people's
12:50
experiences, like of like, for example, with AI, like how do we
12:54
even communicate about AI to consumers?
12:57
Or how do we meet the needs of like what's the developer
13:01
community and what do they need? Or like, what are the ways that
13:05
like an AI framework can be competitive with other AI
13:08
frameworks? And how can we, what are people
13:10
really working on when it comes to different model architectures
13:14
or what language support do we need?
13:16
Like it becomes like very broad in that sense.
13:20
And then I also feel like researchers will cover not only
13:23
product research, but also things like market research.
13:26
So like you might not necessarily have someone who's
13:28
just like a dedicated market researcher because maybe you're,
13:32
you're a marketing team is like, you know, tied up with other
13:35
things. So like in my work I've had,
13:37
I've done work that it's impacting on both the
13:40
engineering side, but also for marketing as well, right?
13:43
So there's like a pretty broad set of stakeholders that you
13:47
need to make happy. I think that there might be, you
13:51
know, some folks that are more focused on like the design
13:54
component, but my work has been a bit more broad than that.
14:01
And when you're doing this market research, are you pulling
14:04
from sources that you yourself did not create?
14:10
Like are you just trying to get information from anywhere?
14:13
So going back to the interview or surveys, that is what I, you
14:19
probably have a technical term for it, but I, I would consider
14:21
that's like something that you've created data that you
14:24
created while doing the research.
14:25
But then you can go out and you can look at into communities
14:30
like the ML OPS community and kind of see what's the sentiment
14:32
around this. And so that's not necessarily
14:35
data that you've created, but you're still researching it and
14:37
you're almost like letting your tentacles go out into the world
14:41
and then bring back things that are of relevance to this
14:48
research. Yeah, it's like primary, I think
14:51
the right term will be like primary and secondary research.
14:54
So primaries like when you're like collecting your own data
14:57
and then secondary you're kind of leveraging existing data sets
15:00
in a way. So I guess you could say the
15:01
discourse on like the ML OPS community, like channels could
15:05
be an existing data source, which I guarantee you many
15:08
researchers are super interested in your community, right?
15:12
I think that either way, you need to be systematic.
15:17
And this is The thing is that you have to always anchor again
15:20
and like what is the impact? Why are we doing this?
15:24
You have a plan like your workflow would be, you have your
15:27
like research plan. You're working through that with
15:30
your stakeholders. And if there's going to be a
15:32
marketing component that your marketing stakeholders are there
15:35
and they give their voice and their opinion of what they want.
15:38
And then also the engineering team is going to be there giving
15:40
their voice and opinion of what they need.
15:42
And then you'll have to come to an alignment from the beginning
15:45
before you even start because that's going to shape the type
15:49
of data you're trying to get and deeper.
15:52
And so also how you're going to design things, if that makes
15:56
sense. Yeah.
15:58
And in that plan are you mapping out different hypothesis?
16:02
You can do that or you can like generally like every plan should
16:06
have like their your questions or research questions, right?
16:10
And so there's a difference between your research questions,
16:12
which are like the broader goals of what you want to learn.
16:16
And then like the questions you actually, let's say, ask people
16:19
in a survey or an interview, that's like separate and notice
16:21
people kind of mix those together.
16:24
Yeah. And so this is free going out
16:29
there and, and starting the research.
16:31
Then afterwards once you've gathered all this data and you
16:35
are working through it. One thing that I found,
16:38
especially when we did just at the surveys within the ML OPS
16:42
community, which I would say is very, very adolescent compared
16:48
to the stuff that you've done, I had the hardest time
16:53
synthesising the data and going through and drawing conclusions.
16:59
And, and so I, I don't know if you have advice on how to go
17:05
about, OK, I've gone through and I've done this, I have this plan
17:10
and here's all the different stakeholders and what they're
17:12
looking for and their needs and I've gathered all this data and
17:16
now how do I connect all those dots?
17:20
That's a really important skill for a researcher to have.
17:25
I think that's something that helps is you could have
17:28
something like an analysis plan from so that like when you're
17:33
actually going through the data, you at the outset sort of know
17:38
like how you're going to be using it.
17:40
So I'm trying to think of how to describe it.
17:42
Like, for example, let's say you're like don't have a lot of
17:45
budget and you're like using a, even a Google form, right, to
17:49
like basic questions, regardless of what you're using.
17:52
Let's say using that and you're, you can only ask a few
17:56
questions, right? Because you want to have people
17:57
sending 20 minutes answering questions.
17:59
Every single thing you ask needs to be strategic and maed to what
18:04
you're looking for so that when you have your data collected,
18:08
you have a plan. Like let's say you're looking at
18:10
ML OPS community, like, hey, I want to know like people's
18:13
backgrounds in terms of like what roles they are like are
18:15
they looking for work? What's their biggest challenges
18:18
in finding that? Like how do they feel about the
18:20
content of my podcast? Like you would plan all of that
18:23
in advance so that once you get the data, you already have a
18:28
plan of like how you're going to analyse that data and the like
18:31
patterns that you're generally looking for.
18:34
And sometimes like the data might surprise you and you'll,
18:37
you'll get something you weren't anticipating, but at least
18:40
you're going in a systematic way.
18:42
And I think also the types of questions you're going to, the
18:45
way you frame them, right? You can have like structures,
18:48
right? Let's say you're looking for
18:49
like, Hey, I want to get a sense of like the overall, like, I
18:52
don't know, satisfaction with my community, right?
18:55
You can ask a question that you're like, you know, calculate
18:58
a mean or something like that. But then you can also design a
19:01
question that's more open-ended. And you could either like
19:04
manually like open code that to see what's going on.
19:07
But you could also like couple it with like a pre trained
19:10
bottle to analyse it and see how it how it works.
19:14
I've done that as well. I'm always paranoid that the
19:20
people that fill out these surveys, they're the ones, if
19:25
they're taking the time to fill it out, they're going to give me
19:27
more or less good reviews on it of how they feel about the
19:32
community because they, they spend the time to do it.
19:34
And so it's like, how can I get those folks that aren't taking
19:39
the time and the diverse opinions, right?
19:42
Like how do you cover the robustness of the data that
19:45
you're collecting? Yeah, like that's definitely a
19:48
good point because you're kind of getting into like bias,
19:50
right? There's like a social
19:51
desirability bias there where people are going to be more
19:54
engaged saying more positive things.
19:57
I mean, some ways you can do that is try to incentivize
20:00
people by like giving them, you know, gift cards or something
20:04
that maybe someone who maybe even they're not as engaged,
20:07
like, hey, cool, I can get a $25.00 gift card.
20:10
I guess I'll do it. I, you know, could help kind of
20:13
get more people to respond, but it can be challenging.
20:16
And I like even now I, there was a study that came out, it's like
20:20
it's not peer reviewed yet, I don't think, but they actually
20:23
like analysed. One of these commercial
20:25
platforms where people are doing research and like 35% of the
20:29
responses were they found that people were actually using AI to
20:33
help them just fill out the surveys to get better.
20:37
Like my so like that. But that's like a whole another
20:41
thing because that's going to affect methods because like, I
20:44
think something that researchers have in common or like
20:48
researchers have in common with like AI researchers and also
20:53
like that kind of community is like we have to think about
20:55
things like missing data, like is there a bias to that?
20:58
And like accounting for all these technical components?
21:01
And so if we're having people use AI to write their questions,
21:06
use AI to respond to the questions, like how do you
21:08
handle that data? And I don't think they're an
21:12
answer right now. And they're kind of needs to be
21:14
an answer. We need to know how to like, you
21:17
know? So let's go over some scenarios
21:23
and and like why, why we should care about this, right?
21:26
And maybe you have a few examples of things that we could
21:30
look at from the ML and AI lens. OK.
21:37
I think like one example would be if you are like one obvious
21:41
one for like an ML engineer is like if you are deploying models
21:47
and you have like a user researcher who is like focused
21:50
on understanding, like developer tools, like that person is going
21:54
to be able to like bring a voice to the pain points you've been
21:57
having. And you can point to that and
21:59
say, yes, see if we do need support for this to make this
22:03
easier for people. So like I think that's one very
22:06
clear place where it can be a strong partnership and benefit
22:11
to people. And that's always fascinating
22:16
because the what I'm hearing you say, especially in that free
22:23
data gathering phase when you have to have your plan in place
22:27
is you really want to know what the impact is going to be.
22:31
Have you been able to measure impact in like monetary ways?
22:39
Is that the most impactful way of putting it for someone to
22:43
say, look, if we do XYZ, it's going to save us or make us
22:48
millions and millions of dollars or billions, right?
22:51
And or it's going to, are you doing it more in ways that are a
22:56
few steps down from money that you can kind of extrapolate out
23:02
like, hey, if we can save time on the deployment process or
23:07
have our ML engineers dealing with deployment processes less,
23:13
that is going to save money in the form of their salary and
23:16
also in the iteration speed that we can get models out etcetera?
23:23
Yeah. I mean, I think that
23:25
organisations are generally driven by efficiency.
23:32
So I do think that the efficiency argument definitely
23:37
is useful for organisations, of course.
23:40
Yeah, like the monetary component can be there and is
23:45
definitely convincing. If you have like a billion
23:47
dollar problem that you can show like that you've made like a 4%
23:51
difference on this. Pretty big for companies as
23:54
well. Yeah, I mean, I think a lot of
23:58
like the work that I have done is like really focusing on like
24:02
actually understanding like the ML engineering workflow.
24:04
So. And I think a big part of the
24:07
reason of that is I'm trying to be mindful of like what I can
24:10
say, but it's like it's really, really important.
24:14
I hope, like, like community members, like, no, like, if
24:17
you're not having like, a good experience, people aren't going
24:19
to use, like, your products and they're going to switch over to
24:21
something else. Yeah.
24:24
And so the you have to find as a researcher the synergy between
24:28
what's good for a company and what's good for the people who
24:30
are using their products. And you have to advocate for
24:34
that and find that balance. OK, So you're, you're gathering
24:39
this information, ideally you're telling it to a monetary number,
24:44
but efficiency is also a great number that is going to get you
24:48
far and workplaces tend to understand that gain and that
24:54
trade off. Are there other, I'm not, I'm
24:59
not going to say buzzwords, but just like words or things that
25:03
you could anchor on that is going to help folks understand
25:08
the value of if we do this research and we are able to
25:14
create an impact in X way, whatever it is efficiency gain
25:20
or monetary gain or Blake, then it's worth worth it for us to do
25:26
that research. Yeah, I mean, I think that, I
25:30
mean, it's funny because this is actually like a big topic among
25:33
researchers is like how do you measure impacts?
25:36
How do you track it? Active research.
25:40
Yeah, I think that like another component could be like more
25:45
anchoring and just, I don't know if I'm answering your question,
25:50
but things like that show that you have like a very intuitive
25:55
product that people are enjoying using and understanding.
25:59
Like what are the strengths of what you're offering and where
26:03
are the gaps so that you can maintain like being competitive.
26:08
That's very valuable. And I think in AI, people know
26:13
like if you were in a very AI driven organisation, how
26:17
valuable that is. I don't know if that's getting
26:20
into tracking the impact, but I think that might just be more
26:23
into the value of understanding people's needs.
26:28
I think also just like companies really struggle, I think to
26:31
understand who their audience even is.
26:34
So I think if you're able to get insight into people that have
26:38
really like a deeper level, it's super valuable for teams, right,
26:45
to know like who is, who are, who are users.
26:48
Like a lot of companies actually don't know that or for our
26:52
customers, they're trying to struggle and figure that out as
26:56
AI is changing, which it changes very quickly.
26:58
And so you have to cover like if you're focused on like a
27:02
products, like for example, Pythorch, you have to be
27:04
focusing like the entire for me, I felt like like all, like all
27:09
domains. Are you doing video?
27:10
Are you doing speech? What languages are you using?
27:13
Like it's super broad. And so you have to be able to
27:16
like really prioritise and like go both broad and deep to, I
27:21
think really give a signal that's like accurate and useful
27:25
for people. That's going to be good for like
27:28
potentially even like a couple of years.
27:31
Like you'll be good. You'll like know who your
27:33
audience is for a while. But now things are moving even
27:35
faster. So you might need to do it more
27:37
often, but. How did you prioritise?
27:42
I think by having really strong partners, it's super useful to
27:50
help me to prioritise, right? Like what is important to them
27:54
and why, and then taking that as a whole and putting that
27:58
together. It's almost like you're doing
28:00
internal synthesis or analysis of your stakeholders in a way to
28:05
be able to figure out what you really need.
28:08
And it can be really hard actually.
28:10
You ask good questions because, well, of course you ask good
28:13
question, you're good, have a good podcast.
28:15
But I think that that's something that people really
28:19
struggle with too, is like, especially when you're a
28:22
researcher that comes, I think from like a back, like my
28:25
background, like you're having a PhD, like you're used to doing
28:28
these like really broad, deep, comprehensive studies.
28:32
But in industry, you have to move a lot faster.
28:34
You're going to have to make trade-offs.
28:36
You're going to have to streamline it so you can't cover
28:38
as much as you want. And you might walk away feeling,
28:41
even if everyone thinks you did an awesome job feeling like, and
28:43
I guess it was good enough. So it is a struggle that
28:46
researchers have. Like sometimes you just have to
28:48
like cut certain things out and like, like don't like to say
28:52
like ruthlessly prioritised, but like sometimes you have to do
28:56
that, right? In some ways you do that.
28:58
It's like maybe you pilot what you're doing and you realise,
29:00
like, you know what, like this just doesn't.
29:03
Or you learn. Sometimes you don't learn until
29:05
you're done and you share insights and you're like, yeah,
29:07
I thought that people would really care about X topic, but
29:11
actually, yeah, no, that was a waste of time.
29:14
Next time, lesson learned. It feels like potentially some
29:20
of that could be combated by having almost like a three phase
29:24
rollout or hey, we're going to go out there and phase one is
29:28
we're going to try and validate this assumption.
29:33
And then when we have a little bit of data, we're going to go
29:37
into phase two, which is playing or building off of that
29:41
assumption. Do you tend to set up projects
29:45
in that way at all or not? Yes, actually a lot of my work
29:50
has been like multi phased work, right.
29:52
So you say, hey, you know, first phase we're going to do like a
29:55
pilot based on like whatever your situation, let's say you
29:57
start off with like a survey, right?
29:59
OK. And then after we get that,
30:01
we're going to go and dive deeper into, I don't know, let's
30:04
say like ML, like model monitoring, let's say.
30:07
And we'll do a deep dive with that.
30:09
But first we want to get a sense of like what models people are
30:11
using or like getting more context into who they are.
30:14
And so you sort of plan things out that way and that helps you
30:17
to like cover more grounds as well.
30:21
And it can also help for you to see like what's working versus
30:23
not like, hey, yeah, we asked some of these things in the
30:26
first phase, but what actually become refine those questions
30:30
even more. So you're actually spot on with
30:34
that. Yeah, because you don't want to
30:38
spend 6 months only to realise, oh, we're not asking the right
30:43
questions and now we recognise that, which probably still
30:47
happens even with these phases, that you gain this insight.
30:51
I really like how you say that is that you gain an insight
30:55
after everything's all done, but that insight is probably so
31:00
valuable even if it is the insight that you were completely
31:05
wrong. Yeah.
31:07
And I think that you don't really even have that amount of
31:10
time realistically. Like I think in the past, you
31:14
might be doing like more foundational research, which is
31:17
work that you do when there's like not really any research in
31:21
your problem space and there's just like a really broad scope.
31:25
So with like those type of projects in the past, I feel
31:28
like researchers were given resources to do those and they'd
31:31
maybe have 1/4 like 3 months to get them done and that would be
31:35
considered a long time frame. Now I feel like researchers are
31:40
doing things. So maybe now I'm getting to a
31:43
different topic. But then there's also like more
31:45
iterative research where you have like some understanding,
31:47
but you're iterating on like a certain experience, right?
31:50
So you might be doing like have a couple weeks and then I see it
31:54
going down to like, I've had things where it's like, can I
31:57
have it in a couple days? And so I actually think that
32:00
kind of going on a tangent, but like, I think AI moves really
32:03
quickly. I think everyone's workflows
32:06
have been like organisations are just really making things very
32:10
efficient. And so I think that the
32:11
efficiency pressures that engineering teams have been
32:14
facing have also like trickled into other people's workflows
32:18
where we're all just working super, super quickly and have
32:22
less time to do our work. And so it's going to have to be
32:25
able to learn and iterate very, very quickly in your work.
32:30
And that was always important, but I think it's like even more
32:33
important now. And that's why I think the
32:35
collaboration piece between different people and different
32:39
functions is so important, because without that it can take
32:42
longer to even figure out you're not and going in the right
32:45
direction. God, but the value prop there is
32:49
so nice when you talk about, hey, can we have some kind of a
32:55
question and get some early signals in a few days?
32:59
That is really powerful to be able to go back and say there's
33:03
worth US spending time on this problem or no, we weren't able
33:09
to do it. So we can double down and spend
33:12
more time like spend another week on it trying to see if
33:15
there are signals or we can just leave it and have it be
33:18
something that we tried and then we'll table it for now, there is
33:23
something else that I think is worth talking about.
33:26
And that's like you've been working at gigantic companies
33:32
and then startups. And the thing that I have been
33:36
wondering as we're having this conversation that's been going
33:39
on in my head is I see the value of having a researcher.
33:45
I would love to have a researcher that I could have on
33:47
the team because I have so many questions.
33:50
And it would be great to be able to say like, can we go and find
33:53
signal? Is there anything there?
33:54
Or am I just hallucinating this? Right.
33:59
But it feels like the user experience researchers that I
34:04
know are all that big companies. Have you seen that trend where
34:08
it is more of a big company type of role?
34:14
It's hard because actually like Speaking of data, like I just
34:18
feel like I see like this like anecdotal data of like what's
34:21
going on in terms of the user experience, like research, like
34:24
what's the market and what's going on 'cause people like
34:28
struggling to really land rolls right now and like they want to
34:32
kind of know like what's going on.
34:35
So it's kind of hard to like, say, concrete things.
34:42
Do you feel like there is a when it comes to this user?
34:49
Research it. It is really interesting to look
34:54
at this idea of you're researching a product versus you
35:00
are researching a framework per SE.
35:04
When we're in the developer sphere, right, there's like the
35:09
end product that a user is using that uses AI maybe or there is
35:16
some developer tool and be that a framework that a developer is
35:21
using. And do those two worlds differ
35:25
that much or where do they have their overlap?
35:28
Like what does that Venn diagram look like?
35:31
I mean, I could start with like the framework piece.
35:33
I think I kind of touched on it before where it's like you have
35:36
to cover a lot of ground in terms of what's going on in AI
35:40
and like understanding people's workflows and really deeply
35:43
understanding what they need. I think like the more like B to
35:48
C, like let's like build something on top of ChatGPT and
35:53
then we'll have, I don't know, like a recruiting tool, a tool
35:56
for recruiters to automate their like thing like that might be a
36:02
bit different. Where like you might be doing
36:04
like shorter, like design sprints and have some iteration
36:08
of the product. And maybe your scope might be a
36:12
little bit more narrow and like focused in on like a very
36:15
specific niche, niche area, which can be useful in a way
36:20
because it could help you to prioritise.
36:22
But then there can also be a lot of challenges in terms of like
36:24
alignment. Like I was talking to a company,
36:27
an enterprise company, and they were working to like integrate
36:31
AI right into their products. And it's like, it was like
36:36
really surprised to hear. It's like kind of sad to hear.
36:39
It's like, yeah, you know, we have sometimes we do research
36:42
here and like the products already launched and nothing
36:45
gets integrated. And actually we're using this AI
36:47
model and when people ask the question to this model, it just
36:51
says, I don't know, it's like, OK.
36:55
And like, you know, so I I'm kind of going in circles in a
36:59
way I feel. But I feel like the challenge,
37:00
what I was trying to get to is like this alignment because I
37:03
was like, oh, yeah. Well, you know, are you talking
37:05
to engineering? They're like, no, we don't talk
37:07
to engineering. Like, OK, so you're creating a
37:09
product that's built with AI, but no one's talking to
37:13
engineering. It sounds like an issue, right?
37:17
Big issue. So I feel like I don't know.
37:21
It also will be very organisational dependent and
37:24
like how advanced like people are in terms of like both
37:29
research and AI and just like their processes at the company.
37:33
And like, I think a lot of companies are actually really
37:35
struggling unless they're in the lead of like, how do we
37:38
integrate AI into our products? Because they feel like they need
37:42
to do that to stay competitive, but they don't actually know
37:47
enough about. You don't even have to finish
37:49
that sentence. I know exactly what you mean,
37:51
that is. And that's how I've been
37:54
feeling. And funny enough, I wrote this
37:57
whole talk because I was going to a conference last week and I
38:00
thought I had a talk. Turns out I didn't have a talk.
38:02
I had a fireside chat. So I didn't need to spend the
38:06
six hours on the plane to the conference frantically trying to
38:11
write this talk and be like, is this good enough?
38:13
I don't know. There needs to be more substance
38:16
here. And the talk was all about how
38:20
in my eyes, the best strategy right now for companies that are
38:27
not on the full cutting edge, like not your Google Meta type
38:31
companies, but companies that you were just mentioning that
38:35
feel pressure to implement AI in any way, shape or form.
38:39
But they don't have the understanding or maybe they
38:42
don't have the culture and they think they can just skip over
38:46
all of the data foundations. In my eyes, the best strategy
38:50
for them is to play the patients game because in one or two years
38:57
it's going to be a lot easier and it's going to be a lot more
39:00
clear for what use cases are really effective and what ways
39:07
you can leverage AI. So that's like from a high
39:11
level, but then from a, from the engineering level, all of these
39:17
frameworks, we'll see, we'll kind of shake out and you'll
39:20
have one or two or three that are very used and there's a lot
39:27
of support for the community has kind of rallied around.
39:31
So that's going to be easier for the engineers because right now
39:34
you have this sprawl that's happening.
39:36
And so maybe you choose this one and then in a year or two, it
39:41
turns out that company got bought by another company and
39:45
they got discontinued. And so you have to do a whole
39:47
lift and shift or whatever. And those were a few of my my
39:52
arguments. We can say as to why it might be
39:55
better just to, I don't want to use the word do a fast follow,
40:00
but really be patient and almost like a cat that's waiting for
40:06
that mouse. Like pounce when the time is
40:09
right, but don't just pounce because you can.
40:15
Yeah. I mean, I feel like they're
40:16
doing like the bladder and like tying it back, I think to
40:20
research. Like that's why the research is
40:21
valuable, right? Like companies like like tech
40:25
leading companies have already like done research to know the
40:28
direction that these things are going to go to a certain extent.
40:33
And I can't like say too much about that.
40:35
But I mean, if you take the time to invest and to understand, OK,
40:41
we want to implement again, our AI for recruiting or our AI for
40:46
HR, I've seen a lot of company, like few companies doing that
40:49
with enterprise, right? Like how do we talent pipeline
40:53
and people who are working at the company, it's like you have
40:56
to really sit down and understand like, what are the
40:59
use cases at this specific company?
41:01
I might be using use cases in a different way than you right
41:04
now, but you know, like, what are the like top like 4 use
41:07
cases We want to prioritise, for example, like what support are
41:11
we going to need to actually implement it?
41:13
Like on the engineering side and all that is stuff that like you
41:16
can do with research, right? Like, oh, do we want to
41:18
prioritise support for Pytorch? I feel like this answer is going
41:23
to be yes for that generally, but there's other like pieces
41:28
too, like what cloud service providers are people using?
41:31
And like, I feel a lot of empathy for engineering
41:35
community because it's like, I, I get targeted with these things
41:38
all the time. It's like be a cloud engineer
41:40
because then I OK, you're like syllabus, like what are you
41:43
offering to teach me? It's like, well, you're going to
41:46
learn, you know, Google cloud and it's like, well, OK, well,
41:48
what if you want to use different cloud?
41:50
It's like, oh, never mind, you have to learn this.
41:53
Like what are the commonalities like across those experiences
41:56
that would like make it easier for people to like switch
41:58
between these different tools, right?
42:01
It's like going on topic, but I think it's really like a lot of
42:05
companies don't have an understanding of like AI
42:07
infrastructure and how to build that and what's going to be in
42:12
house versus not. And so they just, I don't know
42:16
what's going on. Yeah, yeah.
42:19
And that's where you get my favourite job title ever, which
42:23
is like centre of AI excellence. And I feel like that's their job
42:28
is to go and figure out all right, how can we implement AI?
42:31
What are the top use cases here? And so despite having the funny
42:35
name it, there is some value in that.
42:38
But the other piece that I was wondering about when you were
42:42
talking about how there's these research projects that you have
42:47
and sometimes they're going to be very broad and then other
42:51
times it's going to be like super pointed and very short
42:55
lived. How, how would you kind of
43:00
juxtapose the two? How much weight is it going to
43:03
take us to actually put this research project out there or
43:10
like gather all the data and then go through all the motions
43:13
etcetera, etcetera? Yeah, I feel like you're asking
43:15
questions about like prioritising and like balancing
43:18
research. And it's funny because that's
43:19
often like a interview question. It's like, tell me how.
43:23
Oh yeah, tell me how you like prioritise, you know, these
43:27
different things. I mean, I think ultimately it,
43:32
it also has like it's hard to answer it without context, but
43:35
like, let's say you're in a company where it's, they haven't
43:38
done research before. I feel like you'll find yourself
43:42
maybe saying yes more often to some of these, like shorter,
43:46
like iterative, narrow focused things because you're trying to
43:50
build trust with folks and should do things and that you
43:54
can that you're not going to be a bird and you're not going to
43:57
be moving too slow. But at the same time, you have
43:59
to also like be meeting with leadership and across the
44:03
company to understand like, yes, we're doing these like iterative
44:06
smaller things, but bigger picture, what we really need to
44:08
know is this. We don't know, let's say, who
44:11
our audience is or like what our market actually is or like we
44:14
don't know what tools people are actually using.
44:17
We don't know like how like how can we meet?
44:20
I don't know, with model survey, it's a big pain point.
44:22
How can we make that easier for folks Like you have to be able
44:26
to execute and advocate at the same time as a researcher.
44:33
And I think that's pretty challenging.
44:34
And then also layered on top of that, you have what we mentioned
44:38
before, which is like tracking the impact.
44:41
So you have sort of like executing then landing your
44:45
research, which is like really like sharing it out to different
44:48
stakeholders, like, hey, these are the insights we found,
44:50
here's the supporting data, these are the recommendations,
44:53
these are the solutions. And then you have to after
44:56
you're landing it. It's almost like monitoring,
44:58
like what's happened since then, while like keeping the next
45:01
things moving. And it sounds like it's a bit of
45:07
education that you have to do like from your side to
45:13
leadership to help them understand what the impact is.
45:19
Have you seen that? Or is it a two way street where
45:23
sometimes it's like leadership is telling you what the impact
45:26
is? And so it's very clear.
45:30
I think it's like a bit of a mix, but I feel like it's almost
45:34
more like you actually as a researcher sometimes have to
45:36
really explain what you even do in the 1st place.
45:40
Like I feel like. Like this call?
45:43
Just send them this podcast that's.
45:45
I don't know, I mean, it depends how this I want to see it after
45:47
I wonder how it's coming out. But I mean part of the because
45:52
like when you invited me, it's like, hey, like, is this going
45:54
to be valuable to people? Like what do we want?
45:56
Like as outcomes as I could be helpful to have some clarity of
45:59
to like what do we actually do? How can we now there for like
46:03
the ultimate goal of like improving like our work for one
46:07
another in a context where everyone struggling and like
46:10
taking on sometimes like the job of three different people across
46:14
different functions? Yeah, yeah.
46:20
Speaking of which, you mentioned how engineering, especially when
46:25
you're at Meta, they were great partners.
46:28
Are there certain things that you look for or that that are
46:32
like green flags with a partner when you're going and you're
46:36
trying to get this research done?
46:38
What are things besides than being very enthusiastic on a
46:42
dock? Well, I think when I, when I see
46:46
things like in terms of like, are they active and like getting
46:50
feedback? Are they like, are they engaged
46:55
like with external communities, right?
46:58
Like are they like parts of like are they contribute to open
47:01
source communities? Like that's also like a signal
47:03
that they really want to stay close and connected right with
47:08
users. So that's definitely a like a
47:12
very important piece. And I feel like engineers are
47:15
actually like very can be a very good at that, right?
47:18
Like when you're, when you're doing well with a, you know,
47:23
with a team and an engineer goes out and gets feedback, what
47:28
you're going to hear is going to be really close to what you're
47:31
finding with your research because there's just such a
47:33
strong connection between the engineering team and like the
47:38
external audience. And so like, that's powerful to
47:41
have because then they, the engineers also will see the
47:45
value of like, yeah, like, of course we need to stay close to
47:49
what people are doing and we need partners and like,
47:52
resources to do that. So like, they see you as a
47:55
resource and they have that hunger and appetite to like
47:59
mutually succeed together, right?
48:02
And you create the tangible assets from these research
48:09
projects, but then how do you make sure that something is
48:15
actioned on them or does, is it kind of out of your hands once
48:18
once you've created that, it's like, yeah, I've done my part.
48:21
Because I, I know you mentioned before where there was that
48:25
disconnect, there was the user research team, but then nobody
48:28
was doing anything with any of the stuff that they were
48:31
creating. Yeah.
48:33
I mean, I didn't work there, so I don't know what's going on
48:35
exactly at that. Sounds dysfunctional, yeah.
48:39
Yeah. Well, something, you know,
48:41
they're just facing challenges and there were some really great
48:43
people that I talked to. So, you know, I don't want to
48:46
like say anything negative, but I think a lot of it's just
48:50
organisational. And like, that's why I like, I
48:52
think part of what we had planned to talk about too was
48:55
like, when does research just seed succeed versus not?
48:58
And it's like organisational. And then there's also like
49:00
components of like as a researcher, like are you going
49:04
into a situation where there's like what we would say if
49:06
there's like a high maturity and awareness of what you actually
49:09
do, Are you going into a situation where there's a very
49:12
low awareness and understanding of what you do and having to
49:15
like build all that up? Because if you're going into a
49:18
situation where the research maturity is high, I feel like
49:23
you have a lot more potential to have impacts because you've
49:27
already established this function, right?
49:31
Among the other like organisational factors that have
49:33
to have alignment, right? And also it depends too, because
49:36
it can be a mix of like top down thinking where it's like, you
49:39
know, executives, there's only so much you're going to be able
49:41
to change or you're just kind of going with what they need versus
49:46
like bottom up. And so there's I think also
49:48
dynamics with that within organisations to navigate as
49:52
well. When you've seen low maturity in
49:56
this field, are there things that you feel like can help
50:03
bring the maturity up? Is it just a matter of time and
50:09
more experiences with the user experience researchers or are
50:14
there other things that we can do?
50:16
Yeah. I mean, I think one piece is
50:18
like as a researcher, if you're in that context, you almost have
50:21
like a second job of like doing a lot of extra work to really
50:26
make sure that you're like meeting with people throughout
50:30
the company, like aligning on like what they need and what's
50:34
important to them. Explaining what you do at a
50:37
level that you might never have had to do before.
50:41
You might even like be building up the operations for the
50:44
company as well and trying to convince your stakeholders of
50:47
like the value of investing and having like platforms to even be
50:51
able to do your work. And you're going to have to be
50:53
able to really explain things in ways that you, like repeating
50:58
myself, you haven't had to do before.
51:00
And so I think when it comes down to it, it's also the same
51:03
thing with products and with AI, like people don't engage unless
51:07
there's trust. And so I think exceed you need
51:10
to have be and the right fit for a company and they have to be
51:15
the right fit for you ultimately.
51:17
Like you can do all of that work, but if people don't have
51:21
that trust through collaborating with you over time, then it's
51:26
like a good signal to like move on.
51:29
But there's definitely, you need to be very patient, self
51:32
reflective and, and empathetic of like, Hey, not everyone's
51:36
going to know what I think is this very simple thing and I
51:39
need to explain it and why it's important and just be very
51:44
patient. I think is advice that I would
51:46
give to a researcher. But I realise like this audience
51:49
is not necessarily researchers, but I think the collaboration
51:55
piece and working closely is important.
51:57
And I think it's been really challenging because like some
51:59
organisations just had a little infrastructure like with like,
52:02
you know, the pandemic when they went remote, that they just had
52:05
like strong culture and collaboration as other companies
52:08
didn't. So like any work is going to
52:11
suffer if you don't have the culture, right?
52:14
And you don't have that opportunities for people to work
52:18
together, whether it's remote or in person.
52:20
And some cultures are more in person and you don't really know
52:23
that until you join, right? So.
52:28
Yeah, it's, it's almost like you have to evangelise for yourself
52:33
as you are explaining the role in in a way and help see the
52:41
other person's. I imagine with time you will get
52:44
to see what each stakeholder is looking for and how they are
52:48
working or what, what's really impactful for them.
52:51
But at the beginning, if it's all on you and you're the one
52:54
who has to do this across the organisation, that can be a bit
52:58
daunting and overwhelming. And it it will take time for you
53:01
to understand each stakeholders needs and and wants.
53:08
Now let's talk about the career path into being a user
53:12
experience researcher. I think there are some
53:15
fascinating for me, one thing that stands out or that feels
53:20
like there's like this bi directional, probably road or
53:27
superhighway is someone in the product position that would go
53:32
to a UUX researcher position and then vice versa.
53:39
Have you seen that and what are some other ways that you've seen
53:44
people come into this position? Yeah, I mean product management
53:48
and user experience researchers are very similar and that
53:51
they're very collaborative and need to stay really close to the
53:57
users of a product and like drive the direction in terms of
54:01
like the vision and strategy for a product.
54:04
But I think there is so some like, you know, rough pass I
54:08
would say is like I could start with mine, right.
54:10
It's like, it's like, let's say you're a PhD and either like a
54:13
social scientist or you could be like a neuroscientist, like, or
54:17
you could be an anthropologist, you know, from a range of
54:20
different backgrounds, right? And you've just been, maybe
54:23
you've been working on like Policy Research that has impact
54:26
that's choose something, right? Like so I've got to work on
54:28
homelessness and got to do work that impacted on that.
54:30
And so that sort of helped me to transition into like industry,
54:35
right? And then in terms of like
54:38
another path I would see is like people for a very focus on like
54:41
human computer interaction. So like very like human factors
54:45
focused, like makes me think of like kind of like a Stanford
54:48
person. And then a third path I would
54:50
say is like you could even be a designer.
54:52
So because designers do like a lot of work, especially UX
54:56
design of like understanding like user flows and like, you
55:00
know, the product. And so they're like, you know,
55:03
you see a lot of roles, like design researcher roles.
55:06
And I've also noticed a lot of roles want you to be both a
55:08
designer and a researcher. So that's interesting.
55:13
And then, yeah, you could be in marketing as well because again,
55:16
that gets into like using some of the methods to understand
55:19
audience. And I think maybe for this, you
55:22
know, for engineers, I've definitely met people who used
55:25
to be engineers that like switched into UX.
55:30
So there's definitely like a range of different paths.
55:36
I could see the value in a marketer wanting to do this or
55:43
coming into this because if you are able in and it's almost like
55:49
the way that I look at it is, you know how there's sometimes
55:55
politicians that will go and work for the big companies that
55:59
are paying them and when they're a politician and then all of a
56:04
sudden they retire from being a politician and then they go and
56:06
sit on the board of one of these big companies.
56:09
I see that with like the value of being a marketer and then
56:13
going and being in being a UX researcher and then going back
56:17
to being a marketer type thing would be huge because you go and
56:21
you're trying to speak to the audience, you're trying to
56:24
understand what hits. And if you ideally are learning
56:28
different marketing chops and the nuts and bolts of just doing
56:33
marketing, that's great. But it doesn't matter if you
56:38
don't truly understand the customer.
56:41
And so that's why I could see the value in being a user
56:46
experience researcher, understanding the customer and
56:50
then going back and like levelling up if you ever wanted
56:53
to go back into marketing, assuming you're in the same
56:57
field or the same space for as you go through these different
57:01
positions. Yeah, and something with the
57:03
engineers too, like I noticed there's a few that I've met that
57:07
are like having their own consultancies, like with AI,
57:11
like helping companies to figure out how they're going to
57:13
implement AI. And I've noticed that some of
57:16
them are like, oh wow, I'm realising that like this
57:18
projects actually like could really use someone on the UX
57:22
side as I'm figuring out like an implementing like what this
57:26
client wants. And so that could also be like a
57:29
potential point of like collaboration.
57:31
Like if you have your own company as a consultant and
57:34
you're like, you might notice that you like people are asking
57:37
you questions are very like UX like.
57:40
There is a little bit tangentially related to that.
57:46
There's so much more than the tech when it comes to the user
57:52
experience, and we kind of touched on that before, but the
57:57
user experience, maybe you can break down all the different
58:00
areas that aren't tech that still are part of the user
58:06
experience. So like not the intro but like
58:10
what would make a good experience?
58:13
Yeah, exactly. Not, not the exact like, Oh
58:17
yeah, you can deploy this and then it creates low latency.
58:20
But how does the how does that actually look when you're I?
58:27
I'm just thinking about for example, in this consultancy
58:31
issue where it would be nice to have somebody that is, is a user
58:38
experience part. And there's a whole lot that
58:41
goes into. I get, I guess it could be
58:46
solved with tech, but I'm trying not to be like techno centric on
58:50
it. But there's a whole lot of just
58:52
like the human aspect of, hey, we are keeping the client
58:58
updated on how we're moving and where we're going and what we're
59:02
doing. And we're also making sure that
59:07
these things are checked off like the visibility and the
59:11
communication. And those are those are some
59:14
things that come to mind, the transparency of of the project.
59:19
Yeah, I mean getting into like staying in communication with
59:23
folks and the reporting and then like having making it reminds me
59:26
like having like dash words and like showing things like
59:29
visually the folks of like what's going on.
59:32
It's like definitely something you could collaborate on.
59:35
I think in terms of like getting you back to the question of like
59:37
what's a good experience? I think like some like measures
59:41
that people use are things like getting back to again, like it's
59:45
pretty like kind of like core things are like satisfaction,
59:49
ease of use. What are the pain points I feel
59:51
like and how can we improve? I think if you have those like 4
59:55
pieces, that's like a good place to start if you haven't done it
1:00:00
before. And I think we've said also is
1:00:02
like. Making things people centric,
1:00:05
like in the case of being like a consultant in AI, yes, you have
1:00:09
to be keep both your client as a person human centric, but also
1:00:15
what they're actually making themselves.
1:00:17
And you have to have a knowledge of like, all right, my client
1:00:20
wants to implement this AI for again, I'll just go back to the
1:00:24
HR thing, but they don't really like have a like clear
1:00:28
understanding of like their use cases or people's needs.
1:00:31
And that's like exactly what UX research does is like our role
1:00:35
is to advocate for people so that things are built around
1:00:39
them and that AI is people centric.
1:00:41
Like that's sort of the core of it, right?
1:00:44
Because without that ultimately, like long term things will fail,
1:00:48
right? You'll implement, you'll think
1:00:49
of short term things are working, but then you might have
1:00:52
backlash, right? And a lot of companies and you
1:00:55
know, that are more well known know like how the damage to
1:00:59
their brand that that can happen, right?
1:01:01
Like if you roll something out and people have a bad experience
1:01:05
with like let's say Gemini or something, it's going to be all
1:01:08
over the news, right? So being able to like have the
1:01:13
foresight and like have a long term view is also super
1:01:17
important there. The thing that you said, four
1:01:24
kind of pillars there that I wanted to write down, but I
1:01:27
didn't catch them all. It was satisfaction, ease of
1:01:29
use. What are the pain points and
1:01:32
like what are the areas to improve?
1:01:35
Have you seen folks that are not user experience researchers
1:01:42
still go out there and do this type of research?
1:01:48
Because it's almost like they're, it's not necessarily
1:01:51
that their position is calling upon them to do it, but they see
1:01:58
the need. And before they go and they
1:02:00
invest or they make a decision to invest all this time and
1:02:05
energy into building out an ML platform or building out or
1:02:11
hiring teams and then choosing a centralised way of using a
1:02:20
platform, etcetera, etcetera. I'm just trying to like bring it
1:02:24
back to what engineers probably will do the IT feels like this
1:02:31
can be done by engineering teams and this is almost like a new
1:02:39
tool in their tool belt as they're trying to think through
1:02:43
some of these hard decisions and recognising, Oh yeah, like you
1:02:48
said, it's not the most comfortable, but we could get
1:02:51
some signal in a week and maybe we can really dive into knowing
1:03:00
that we make the right decision here.
1:03:02
Yeah, I mean, engineers definitely can and and do do
1:03:05
that. And also, I think like when you
1:03:07
were talking, founders came to mind, right?
1:03:10
Because if you're starting your own company, right, you're going
1:03:13
to be really, I mean, I might be generalising, but what I've seen
1:03:16
is a human being really, really focused on like having the best
1:03:19
possible products. And a big part of that is
1:03:22
talking to people. So I've had people who are, you
1:03:25
know, founded a company and they want like feedback, right?
1:03:30
And they're reaching out and they want to talk.
1:03:31
And so you see a lot of that as well because they know they're
1:03:36
their companies aren't going to be successful if they haven't
1:03:39
like gotten a sense of like their market and how they could
1:03:42
be positioning themselves. Yes, awesome.
1:03:48
Well, I'm so glad that we did this.
1:03:51
This, the whole reason that I, you know, like I find this
1:03:56
fascinating is for that very fact is that I've heard too many
1:04:01
people come on here and say, we built a great product internally
1:04:08
and we launched it to crickets. And really, it feels like some
1:04:13
of these tips and tricks on becoming a better researcher and
1:04:18
understanding the pain points and the ease of use can be ways
1:04:25
that you will avoid these problems.
1:04:28
Because if people are not using your tool, there's a few reasons
1:04:34
that that could be for right? Like it could be that you just
1:04:39
didn't do a good enough job letting them know that the tool
1:04:42
is there so you have an awareness problem.
1:04:45
Or it could be because the tool sucks and so nobody wants to use
1:04:47
it because when they use it, it's like, this is a lot harder
1:04:52
than when I do it on my own. And so for that problem, the
1:04:57
tool sucks. This is a great way to
1:05:01
counterbalance. That, yeah, I agree.
1:05:07
Is there anything else like I know that we had thought about
1:05:11
talking about developer experience and how we apply
1:05:17
everything to developer experience.
1:05:19
Is that like, do you feel like there's things that you want to
1:05:24
touch on there? I think when I had plans to talk
1:05:28
about was really like, you're just taking all of what we
1:05:31
talked about and applying that specifically to developer
1:05:34
focused tools and also like their experiences, for example,
1:05:38
like both and just their careers or at the organisation they're
1:05:42
working at. And I'm really understanding
1:05:44
people's needs. And I think that Google, Google
1:05:49
does a pretty good job of that with Dora.
1:05:51
They have a team that's like they publish their annual report
1:05:56
and they're really focused on that developer experience.
1:05:58
And I think it's a good resource.
1:06:00
I linked it into the doc I shared with you because that's
1:06:06
another area too of like, if let's say you're, I don't know,
1:06:10
you're engineering managing, you're trying to think of ways
1:06:12
of like mitigating pain points for your team.
1:06:16
Like you could do like an internal, like you can actually
1:06:19
do that internally, right? And do like a developer
1:06:22
experience research and do like a survey throughout your company
1:06:25
and understand like, hey, OK, my team, most of my team uses this.
1:06:30
So a gap here is that we need to have more support for learning
1:06:34
like this other tool if we want to switch or like, you know,
1:06:38
people aren't feeling that great about this thing.
1:06:40
This is a solution that I can do to support my team.
1:06:42
So I think that's another important piece that could
1:06:46
support developer teams as well. God, as you say that it's so
1:06:52
true that just the Hopefully every engineering manager is
1:06:58
doing this, but taking the time to survey, especially if you
1:07:02
have big teams and or you have a, a big engineering
1:07:06
organisation to recognise where pain points are for the teams.
1:07:13
And that's where you can spend your energy and focus.
1:07:16
It's if there's signals coming out of a survey saying, Oh yeah,
1:07:21
we're having, we're spending a lot of time because we're always
1:07:24
having to go back and fix the data quality.
1:07:28
It's like maybe we should start figuring out how we can get
1:07:34
better data governance, right? Or if we are spending a lot of
1:07:38
time or there's, there's a lot of pain because the open source
1:07:42
tools that we're using on our ML platform are not reliable or
1:07:47
doing something. Oh, should we maybe look at how
1:07:51
to fix that? And so yeah, doing doing this
1:07:55
research from a managerial perspective or a leadership
1:07:58
perspective, it just feels like something that should and
1:08:02
hopefully is constantly happening.
1:08:06
Exactly.