Latest / The DEX Show: A Show for IT Change Makers / Reality Bytes #30: Synthetic Data and the Digital Workplace

Transcript

Reality Bytes #30: Synthetic Data and the Digital Workplace · The DEX Show: A Show for IT Change Makers

• 0:01
  Welcome it, change makers to the Deck Show with Tim Flower and Tom McGraw.
• 0:07
  Let's get into it. Hello, everybody, welcome back. So the
• 0:11
  Deck Show show within the show, it is, of course reality bites.
• 0:14
  We've got a great topic of discussion for today's episode, and I'm joined for
• 0:19
  it by Tim Flower. Hy Tim, how's it going? Good time?
• 0:23
  Welcome back from vacation. I hope you're rested and relaxed and ready to go
• 0:27
  and shop. As attacked him, I promise in actual fact, it was.
• 0:32
  Actually it's been hotter in London while I've been away, and it was
• 0:35
  in Lanza Rotti, Spain. If you can, if you can believe that,
• 0:39
  it's the way of the way of the world, isn't it. And
• 0:41
  we've got Megan break with us. Hey Megan, how are you doing?
• 0:44
  Hey Tom, I'm doing good. And we have got Dina. I'll show
• 0:48
  off how you doing, Dina. I'm all good. Tom, welcome back,
• 0:52
  Thank you so much, Thank you so much. And everybody's got to
• 0:55
  bear with me. I'm a little little blunt and and and and foggy headed
• 0:58
  today, but we have got a very sharp and absorbing topic to discuss.
• 1:03
  It is all about synthetic data and its implications on deck. So synthetic data
• 1:11
  is data that's artificially generated rather than produced by real world events, and obviously
• 1:18
  that's going to become more and more prevalent in all kinds of areas. You
• 1:21
  could say we've reached a bit of a crossroads even between real and synthetic data
• 1:26
  and data modeling. Of course, that's going to be precipitated in large part
• 1:30
  by the growth of AI, and Gartner in fact predicts that by twenty twenty
• 1:34
  four more than sixty percent of AI model training data is going to be synthetic.
• 1:42
  But what we have to ask, because I say, is, of
• 1:44
  course, is what are the implications going to be for digital employee experience.
• 1:49
  It's an area, after all, not only defined by data collection and analysis,
• 1:53
  but by definitively organic data. It's real world user based endpoint data.
• 2:00
  It's never been based on anything other than that. That's been its definition.
• 2:06
  So we're going to get into those implications. But but first of all,
• 2:08
  Tim, I wonder if you could help us set the scene a bit with
• 2:12
  some with some positives. First of all, because obviously this is very very
• 2:15
  powerful, pretty new technology and very presumably some sectors and some areas of it
• 2:22
  that are going to benefit first of all. Yeah, so my experience my
• 2:28
  previous employers at the Hartford for twenty years and we had a lot of internally
• 2:31
  developed applications. Companies want to differentiate themselves. They don't want to buy software
• 2:37
  off the shelf, and many times software off the shelf doesn't do what they
• 2:39
  needed to do, so they develop their own applications that are specific to not
• 2:45
  only to their company, but right down to very specific job functions. So
• 2:49
  in those scenarios where you're creating your own applications, you need some way on
• 2:55
  a net new application to test it, and synthetic transactions do a couple of
• 3:02
  things. They validate that each function within the app actually functions. You've got
• 3:09
  to be able to exercise the whole application and make sure that all of your
• 3:14
  coding is actually working and that the application is performing as it's intended to,
• 3:21
  and not just for one employee doing one set of tests, but underload.
• 3:27
  So synthetic transactions can not only give you employee less testing, but it can
• 3:35
  place a significant load on the back end to also simulate lots of employees doing
• 3:40
  the same thing at the same time. And you just can't do that in
• 3:46
  a human way. You can't have ten thousand people all sitting in a keyboard,
• 3:51
  all pressing the same key at the same time. So the synthetic transactions
• 3:53
  and synthetic testing form a very unique and very valuable function in application development and
• 4:00
  rollout. I'm curious to see how the synthetic transactions and synthetic data now start
• 4:09
  to inform not only app developers and where they need to take their applications,
• 4:13
  but the but the AI models now the AI data that we rely on when
• 4:18
  we ask chat GPT a question, how much of that data is real data
• 4:24
  versus something that was synthetically created. I think it's an interesting I think it's
• 4:29
  interesting question to ask, super interesting. And actually, when we were in
• 4:33
  the chat earlier sort of discussing this before the show, Megan, you brought
• 4:39
  up the point that you potentially find a repetition or a reiteration of the famous
• 4:46
  watermelon effect. I wonder if you maybe maybe kind of explore that a little
• 4:49
  bit, first of all, and some of the pitfalls that could come from
• 4:54
  a sudden insertion of so much synthetic data into this space. Yeah, So,
• 5:00
  as I was reading about this, synthetic data just more generally after the
• 5:04
  topic I brought up. You know, most of the examples that are provided
• 5:08
  are about testing for autonomous vehicles and testing for healthcare, not so much in
• 5:15
  it. So I was kind of trying to think of more along what Tim
• 5:17
  was just saying of what would it actually how would it be applied, how
• 5:20
  would it be used in it? And something that came up from me was
• 5:25
  this idea of like, everything that Tim just described is an obvious benefit,
• 5:30
  but when we extend it beyond the testing process and we take those conclusions and
• 5:35
  we pushed them into a real environment, I think the thing to keep in
• 5:39
  mind is this idea of bias, right, and so bias coming into data
• 5:44
  sets has been a point of discussion almost as long as from my point of
• 5:48
  view, this kind of synthetic data to AI has been in point of discussion,
• 5:53
  right, Like, and machine learning trained on white mail faces is going
• 5:57
  to be more likely to select white mail face is a recognize white male faces
• 6:00
  than it is to recognize female faces or faces from other races. Same thing
• 6:04
  when we look at this in the digital workplace. So let's say you know
• 6:09
  you're doing you're running one of these tests. Tim was talking about testing to
• 6:12
  see what's going to happen before we deploy this new software or new huge transformation
• 6:17
  project. The tests, I'll come back and say, yeah, it's all
• 6:20
  going to be good. Here's the kinks. Okay, you iron out the
• 6:23
  Kink's great, It's all going to be good. You roll it out and
• 6:26
  it comes back and it looks like it's all really great. Well, what
• 6:29
  about the real experience of employees? I feel like that's the thing that's getting
• 6:32
  left out again here, And that's the whole reason that this deck's approach that
• 6:36
  we're always talking about what got put in place in the first place, right
• 6:42
  is a system can tell you that everything is fine, but an employee might
• 6:46
  be telling a different story and their experience isn't reflected in the data. So
• 6:51
  as extremely valid and exciting as this kind of application of synthetic data, maybe
• 6:58
  I think we have to be careful or like be aware of are we just
• 7:01
  going back to a world where it's that watermelon effect. On the outside,
• 7:05
  everything looks green and wonderful, but inside it's all read So are we really
• 7:10
  are we still keeping in mind the real lived experience of employees? I think
• 7:15
  synthetic data always needs to be balanced with real data if it's going to be
• 7:18
  used in the digital workplace. And I'm used to ask the question, how
• 7:21
  do you know right you've rolled out, you've tested this application, you've used
• 7:26
  synthetic transactions, now you roll it out, And the question to the opt
• 7:29
  developers is how do you know that the employees are actually functioning the way that
• 7:32
  you demonstrated they would in test? And the answer typically was, well,
• 7:38
  we've deployed robots, we've deployed monitors, we've got synthetic transactions out in production,
• 7:42
  but it's still not the employee transactions. It's the synthetic transaction. So
• 7:46
  at certain point you've got to marry the two and make sure that reality matches
• 7:50
  what you did in a synthesized way before reality by its him before I want
• 7:57
  to get sort of in a way, we've got kind of infusions of two
• 8:01
  relatively recent, completely different kinds of data, and I think, you know,
• 8:05
  one hand, synthetic on the other hand, sort of organic sentiment data,
• 8:09
  and I want to I want to ask Dina about that, that kind
• 8:13
  of how they can potentially complement one another going forward. But first of all,
• 8:18
  Tim I want this point of analogy about the kind of origins of next
• 8:22
  Think itself, where you know, very kind of abstract data, very distant
• 8:26
  data, was suddenly supplanted and kind of kind of undermined, if you will,
• 8:31
  by by much more meaningful endpoint data. And I was interested in your
• 8:37
  view of the relevancy of that analogy for this kind of present moment in technological
• 8:43
  development. Do you see a lot of parallels there? Yeah, So I
• 8:48
  go back to my as a as a pre customer, as a prospect looking
• 8:52
  at next Think, and I go back to my Holy Cow. This changes
• 8:56
  everything kind of a moment when I first saw it, and I think there's
• 9:01
  a similar from an APPTEV standpoint, there's a similar reaction when they say,
• 9:07
  oh, holy cow, look what I can synthesize. Look what I can
• 9:11
  create without bothering the employees. Now I can actually do this all on my
• 9:15
  own. And I think there's a little bit of there's a little bit of
• 9:20
  a danger there in looking at that synthetic datas as real It is technically it
• 9:26
  is real data. It is exercising the application. I look at the holy
• 9:31
  Cow moment that I haven't and honestly, almost every customer that I've talked to
• 9:37
  also has in a little different way because that holy Cow moment is based on
• 9:43
  to go back to the R word, reality, it's based on what's actually
• 9:46
  happening. And I think the holy Cow moment for app developers if they take
• 9:52
  it too far a lot like AI the holy Cow moment with AI. If
• 9:54
  you take it too far or trust in it too much, it becomes a
• 10:01
  false security blank at a false sense of security. So the analogy I think
• 10:05
  plays in that there is a holy Cow moment, this changes everything, but
• 10:09
  it's got to be approached with some level of skepticism and caution. Right,
• 10:13
  let's get into a little deep event, Dina. In your view, how
• 10:18
  predictable are employee computing habits at large? Any view? So, if we're
• 10:24
  going to look at experience, the whole word itself describes itself. We can
• 10:28
  look at the hard data, we can look at this synthetic data that we
• 10:33
  keep on collecting. However, if we are not considering the sentiment aspect,
• 10:39
  how is it really put into place? And then we understand the predictability of
• 10:43
  the user behavior. Marry them both together, put a one big picture where
• 10:48
  you can see both of them Apple to Apple, You're going to get the
• 10:50
  real picture. This is the real picture. Otherwise, like Tim and Megan
• 10:54
  mentioned earlier, we're going to go back to the watermelon effect. We're going
• 10:58
  to go back to the fact that we have data that is way too much
• 11:03
  artificialized that you wouldn't know where is the user in it? And I think
• 11:07
  we're When we're talking about experience, one of the first points that we can
• 11:13
  start with is the user. That's a user journey. How can you tell
• 11:16
  a user journey without the user interaction? Yes, the purpose is to minimize
• 11:22
  the disruptions, but increase the communication, increase the connection. And there is
• 11:28
  a lot that would come into that end user experience that you're trying to measure,
• 11:35
  because you're not trying to measure the automated experience, or how good are
• 11:39
  you at collecting the data without the user or ignoring them altogether from the whole
• 11:46
  equation. So the accuracy of how the habits are being predictable, that's a
• 11:52
  very big topic that we need to talk about because it's very hard. Even
• 11:56
  if you give people scripts to type these in or put in those steps or
• 12:01
  processes, everybody would add one little bit of flavor even if they just mistype
• 12:07
  something. So that kind of user input you still need to measure. You
• 12:11
  still need to keep including into measuring experience. Otherwise we're only enhancing the productivity
• 12:18
  of the machine and we're not looking at increasing the user productivity as we want
• 12:22
  to and enhancing their performance. So it's the man versus machine kind of thing.
• 12:28
  But for you to measure experience, I think we still need to maintain
• 12:31
  the user input and to the equation. There's other intangibles there too, the
• 12:37
  variability of an employee running and using an application versus synthetic transactor. Synthetics synthetic
• 12:43
  will execute based on a plan and a schedule and a certain step set of
• 12:48
  steps, whereas an employee will do things in a similar way but in an
• 12:52
  unstructured in unstructured timing. Oh and by the way, I also had Chrome
• 12:58
  running in the background, and oh, by the way, I was also
• 13:01
  streaming the football game, and oh, by the way, I also had
• 13:05
  you know, my personal application up at the time, which caused the conflict.
• 13:07
  So there's so many things that can vary in a user experience and an
• 13:11
  user execution of an app that's synthetic testing and synthetic transactions just can't anticipate.
• 13:18
  I agree, it's the full story behind the whole scene that would give you
• 13:22
  the insight on how you want to measure that, or even the experienced numbers
• 13:26
  that we see or the data that's being truthfully collected from the endpoint, etc.
• 13:31
  But not knowing the full story of the background of what the user is
• 13:35
  actually trying to achieve or do, or have running in at the same time
• 13:41
  consecutively or parallel, it would be the missing part or the missing piece of
• 13:46
  you having the colors of the picture. Do you feel, Dean of there's
• 13:52
  a potential dynamic where the two types of data increasingly complement one another and inform
• 13:58
  one another one hundred percent. If you have the potential increase the amount of
• 14:05
  data being collected synthetically or dynamically or organically being generated from the endpoint directly streamlined
• 14:13
  into a gigantic data lake, and you keep on having the inputs of the
• 14:16
  users, you will have better measurements for predictability of how they could be happening
• 14:22
  in the future, so you can have a bit of a preemptive analysis in
• 14:26
  the background, but still you would need the user input. Without the user
• 14:31
  input. After a while, it will just become a matrix of data and
• 14:33
  data and synthetic data without really having the capability to go and measure experience or
• 14:39
  taking full actions that would correlate with the real story in the user's eyes.
• 14:46
  Megan, if this is obviously an epiphenomenon, if you will, of a
• 14:52
  larger sort of societal and technological shift to what extent based upon what you've seen
• 14:58
  so far in what is looking like the era of AI's general explosion, how
• 15:05
  many how much grounds for sort of optimism are there in your view in terms
• 15:09
  of this potential for a a kind of complimentarity between man and machine, between
• 15:16
  the organic and the synthetic. Or do you see just sort of the form
• 15:20
  of pulling off in it in a sort of aggressive direction, or do you
• 15:24
  see indications of there being some kind of conversation between what we can now do
• 15:28
  and what we want to see out of this technology? M asking me the
• 15:33
  really big question stuff reserve for you, Man, I think that's a really
• 15:41
  I think that's an interesting question, right as much as I think that really
• 15:46
  analytical minded of us want to believe that everything can be programmed into data sets
• 15:50
  and everything can be imagined with AI. I don't know. I feel like
• 15:54
  you can't ever underestimate the truly randomness of humans and the ability for groups of
• 16:00
  humans to act in incredibly unpredictable ways. So, you know, I think
• 16:07
  that this explosion of AI, if harnessed in a thoughtful way, has of
• 16:15
  course the ability to revolutionize. I mean it's like another revolution, right,
• 16:19
  another industrial revolution. I mean to say, yeah, has the ability to
• 16:26
  make our lives enhance them in so many, many, many many different ways.
• 16:30
  I mean, from making it easier to predict how your digital workplace is
• 16:33
  going to behave to the sort of the first use of this, right,
• 16:37
  which is autonomous vehicles, getting a car that can actually read the road and
• 16:41
  anticipate what's going to happen, so that, you know, maybe we do
• 16:45
  away with traffic forever and the roads are always running smoothly because nobody's driving a
• 16:49
  car anymore. Obviously, that's like an idealistic and beautiful future. I think,
• 16:55
  though, that this conversation that we've had here today, you know,
• 16:57
  hopefully the scientists who are working with this are much smarter than I am,
• 17:02
  and I've already thought of this, But you always need to be larnering in
• 17:04
  that real experiential data. How are human beings really acting in this perfect setup
• 17:10
  that you've made with your synthetic data. How much would you trust your tesla,
• 17:15
  for example, if all of its anticipations in its autopilot we're based on
• 17:19
  synthetic transactions of highway exits and speed limits and traffic. I wouldn't get in
• 17:25
  that car. Yeah, nor would I be a pedestrian in that world.
• 17:30
  What do you think, Dana, how taken abroad of you? I know
• 17:33
  I said I reserve for big questions only for men. I don't really everyone
• 17:37
  gets. What do you think, taking a broad of view, if it's
• 17:41
  kind of dynamic between as you put it, man and machine, how do
• 17:44
  you think things are panning out so far? And where do you see them
• 17:48
  go? I think we can use it nicely and wisely, but excluding the
• 17:52
  human interaction or the human opinion, then we're just going to be giving in.
• 17:59
  I am all pro finding trends, executing computations, finding all of the
• 18:06
  best and greatest stuff that we can into, the data analysis and the data
• 18:10
  science that we can through GPTs and through eyes and all sorts of new caled
• 18:17
  things. But we still need to have the human power as the supreme being.
• 18:22
  Right, So we're still here. So as long as we're here,
• 18:23
  we need to be included in the equation. If we keep onjust being into
• 18:29
  the synthetic data, then we're going to be part of a big simulation that
• 18:33
  we are not controlling. Yeah, well said, well said. And just
• 18:37
  bringing it back to my new Shire the issue a little bit more, I
• 18:42
  wonder if you could anyone can explain to me sort of some of the things
• 18:47
  that become more possible, more viable for IT teams not already covered with the
• 18:52
  advent of synthetic datas of synthetic data specific areas, you can see it really
• 18:57
  starting to make a new difference at what I hear from customers when they're talking
• 19:03
  about like our product. One of the new elements I think in infinity right
• 19:07
  is that it's based off of our data pool, which comes from anonymous data
• 19:11
  from our customers using our product. But it offers like AI assistant suggestions if
• 19:17
  here's what to do next, here's what to do next, here's a common
• 19:19
  fix for this issue that you're singing. So maybe a use of synthetic data
• 19:23
  in the digital workplace at least is aggregating. Okay, here's all these things
• 19:27
  that could happen. Here are the top three fixes you could try to address
• 19:33
  this issue that's been plaguing you for however long. Agree, and you would
• 19:37
  still need a human to be validating those suggestions at the end of the day,
• 19:41
  to say is it worth it to chase, is it wrong? Is
• 19:45
  it right? And what would be the best consulting or recommendation that you can
• 19:49
  have So you can have the AIS in the background to do the cross check
• 19:52
  for you, find the trends, quickly, gather the anomalies in the customers
• 19:59
  environments, but you still need the human aspect to validate if this is valid
• 20:03
  to be done or it's just going to be a waste of time. I
• 20:08
  think it's an interesting question, Tom. I think the value of synthetic data,
• 20:15
  and I mentioned it at the outset, is just the sheer volume of
• 20:19
  data that you can get without depending on employees to do the work. In
• 20:26
  our space and in our product, especially with anonymized data over fifteen million end
• 20:33
  points, you can now leverage that same benefit of a huge volume of data
• 20:41
  without relying on employees to do extra work. But it's based on real data.
• 20:45
  It's based on real employee interactions and not synthesize synthetic transactions, so you
• 20:49
  get the same benefit of huge volumes of data, very reliable data, no
• 20:56
  extra burden on employees, and now reliability to IT teams to take action on
• 21:00
  that huge volume of data. Brilliant and look, thanks for your insights everyone,
• 21:07
  and thanks for bearing with me. I should have added at the top
• 21:08
  of the show. Actually I actually got in with a family from the airport
• 21:12
  from to the house around midnight last night and then got their first day of
• 21:17
  school at about six fifteen six thirty. So thanks for your patience with my
• 21:22
  very organic personal data set today and really some really good insights there. And
• 21:29
  thanks to Sean for suggesting this topic for this show. And we will have
• 21:33
  some links in the show notes to Infinity, which I think was very properly
• 21:38
  plugged by the various panelists today because I think it's super relevant. And look,
• 21:44
  thanks for joining us and we will see you at the next Reality bites
• 21:48
  by. To make sure that you never miss an episode, subscribe to the
• 21:52
  show in Apple podcast Spotify, your favorite podcast player, and if you're listening
• 21:57
  on Apple podcasts, make sure to leave a rating at the show just have
• 22:02
  the number of stars you think the podcast deserves. If you'd like to learn
• 22:04
  more about how next Think can help you improve your digital employee experience, head
• 22:10
  over to nextthink dot com. Thank you so much for listening. Until next time,