Latest / The In-Between Tech and Trust Podcast / Trust as an Operating Metric in AI Companions (EP 15)

Transcript

Trust as an Operating Metric in AI Companions (EP 15) · The In-Between Tech and Trust Podcast

0:14
Hello, this is Eva and you're listening to the In Between Tech
0:18
and Trust podcast. It's the first.
0:20
Episode of 2026 and it's going to be a quite interesting one
0:24
because we're diving into relationships and the kind of
0:28
relationships that we can form with AI companions in
0:31
particular. And I had an expert on that
0:34
regard with me in conversation and that's Leo Oren.
0:37
And Leo is the Chief Technology Officer at Replica.
0:41
And some of you might have heard of Replica before because it's
0:44
one of the largest AI companion companies with a community of
0:49
over 35 million people that interact with their platform.
0:54
And what made our conversation? So special is that we.
0:58
Had an intersection and look into the way of how we form
1:02
trust in conversations from a human and then again from an AI
1:08
perspective, but also from a tech perspective in general,
1:11
which means that we talked a lot about also so the way that
1:15
infrastructure needs to be formed and built.
1:18
So that it also is. Flexible enough with user
1:21
requirements and guiding along the experience that we have and
1:25
also the design principles that one must follow when you really
1:30
want to put together a platform that intersects with all three
1:33
of them. So the conversation really
1:35
covered many. Aspects of.
1:37
The intersection of tech and trust and also made us reflect
1:42
of what actually is valuable and having human relationships
1:47
overall. Hello Lior.
1:51
And welcome to the in. Between Tech and Trust Podcast.
1:54
Hi Eva, thank you for having me and special thanks to Grisha for
1:58
Miro for introducing us. Shout out to you Grisha.
2:01
It's so great for the connection and I'm happy to now talking to
2:05
you Lior I I do want to start with you personally also with
2:11
your background and like engineering and technology part
2:13
and dive right into the topic of trust.
2:16
What does it mean for you personally?
2:18
Also when technology becomes a companion rather than a tool?
2:22
So trust is, is is. Everything we have in
2:24
relationships, right? I I have a trust in my
2:27
relationship with humans. I have trust in my relationship
2:30
with myself. I trust myself.
2:32
I trust with. The police officer around the
2:35
corner. And and with everything then
2:37
comes kind our our tools. I have a trust in my hammer.
2:41
To do its. Job and I have trust in in
2:43
technology now. Sometimes with technology I can
2:47
have more. Trust because I have much.
2:48
More control than humans even than myself and and when we come
2:52
to to companion to AI companion. It it blurs the borders a bit,
2:57
but there's still. Very strong ability to enforce.
3:00
The trust I have. I can have much more
3:03
transparency, I can have much more controls, and AI Companion
3:07
is actually designed to. Serve our well.
3:09
Being right, it's a tool that is designed to serve us and to make
3:14
us feel better. So in in this term, I think that
3:16
while AI companion is kind of blurring the the borders between
3:19
humans and and and machines, it's actually a more trustworthy
3:24
machine than what we used to. And if we now talk about
3:27
blurried lines and that replica particularly oversee like the
3:31
systems that interact with people's emotions and like their
3:34
identity and also the private thoughts, how do you see the
3:38
boundaries between human and humans, but then again, also
3:42
compared to human and machines? And where does trust lived in?
3:46
Like the in between of it all? It's again.
3:49
Blurry as as as a. Person who grew into the social
3:52
media revolution and also worked for a while at at my time in the
3:56
integrity teams. That's supposed to protect.
3:59
The community the line is already.
4:01
As a blurred so I have a. Relationship with the machine,
4:05
with the platform, but my relationship is actually with
4:07
humans on the other side of the platform.
4:10
So I have expectations. And this leads a bit to a
4:12
discussion about regulations and, and, and our whole
4:15
discussion about what we expect from the platforms.
4:18
We expect certain level of trust and actionables, but eventually
4:23
there is a human from the other side.
4:25
So I'm, I'm trusting a machine which is easier to trust
4:28
sometimes, which is more capable because it's, it's, it's
4:32
involved heuristics. But eventually there are humans
4:34
on the other side. So this is already very blurry.
4:38
Border now sometimes when it comes to to an AI, to a
4:41
companion, I can have better relationship with with my
4:45
companion than the human on the other side.
4:48
You know this bully on the other side?
4:50
Of of Twitter so these borders. Are moving and I'm not.
4:55
Sure that there is a. Concrete border, it's really
4:57
different who's on the other side?
4:58
Which service on the other side? And and of course me and how I
5:02
understand the technology, my borders, how to set borders in
5:06
front of other people or other machines and so.
5:09
Yeah, these borders are very. Vague to my opinion.
5:13
And when it now comes into the design of them and also
5:17
particularly with your background and like building up
5:20
large scale conversational AI platforms and intriguingly also
5:25
emphasizing the reliability, scalability, also the measurable
5:29
performance of it. If you now look into the tech
5:31
architecture and like from a point of view of that, what are
5:35
key technical design decisions that underpin trustworthy or
5:41
reliable conversational AI experiences?
5:44
So the there are few aspects for it, but don't forget that in the
5:50
in the in the place of of companion AI, one of the main
5:56
KPIs or metrics. Is user, is, is, is trust
6:00
because your. Dissatisfaction is how user
6:02
feels comfortable with the conversation because just won't
6:05
come back if they won't feel comfortable with the
6:07
conversation. When you have AI bolts or
6:11
assistants, how they? Call they just help you buy a
6:13
car or help you. Take a service ticket.
6:17
This is very clear success. But in our case success is a
6:21
well done conversation is is a conversation I want to continue.
6:25
It is sickness. I will come back and that by the
6:28
way, very hard to measure in a companion wall.
6:30
It's very hard to measure what is a good conversation, but.
6:33
Trust. Is, is one of the things that
6:35
keeps user in and coming back. So it's built in a KPI, built in
6:40
metric in what we're doing daily.
6:42
And I think that like any other software, and I've been for a
6:45
while in the industry, you have things you need always to have
6:48
on the road map like security and Dev X and reliability and,
6:52
and cost efficiency and with the AI with, with conversational AI.
6:56
Trustworthy guardrails. How they're called today to put.
7:00
Things so that the the conversation won't go off or you
7:03
can detect when it's going off is just another new thing we
7:08
need to to clear our road map for and to put inside exactly
7:11
like security and and and data protection so.
7:15
This is a it's it's, it's a part of our daily work and it's a
7:19
part. Of the road map and I think
7:20
every team, every engineer, every engineering manager, every
7:24
product manager need to know what are the tools in the
7:27
market. What the big?
7:28
Players are doing to protect, to make the conversation more
7:30
trustworthy. It's a very young place, the
7:33
whole AI. So.
7:35
We are a bit more reactive than proactive tools are there.
7:38
They're not yet perfect, but you need to always try the newest
7:42
tools and to know what what are the standards on top of that.
7:45
I think that's me. It's the last thing I'm going to
7:47
say about it, but that's maybe the most important thing is.
7:49
Observability if observability used to.
7:52
Be something help our trust in the machine help us as engineers
7:56
to trust our systems here observability in this unknown
8:00
world of of AI that things are still new in this statistical
8:04
machines are are still a black box and unknown.
8:08
The ability to know that things went off and to immediately.
8:12
Be able to see what? Is going on and most of the
8:15
times why and then to take action is super important.
8:19
So I wouldn't build any platform without full observability from
8:23
day one. Just because these these are.
8:25
Not heuristics you you cannot predict, but things it will
8:29
happen and you need to. Have the ability to.
8:31
See what happened and to try to. Figure out why.
8:33
And could you maybe make it a bit tangible in a way that you
8:38
shared one to two examples of how exactly a design principle
8:43
could look like for like a platform or a companion to
8:48
provide a good conversation? Well.
8:51
There are many, many, many aspects.
8:53
Right, because good. Conversation means it should be
8:56
fast. It should be intuitive.
8:58
So the whole. Normal system and UX decisions.
9:02
There is a. Lot.
9:03
A lot. That is being invested, of
9:05
course, in the prompt, right? You you you prompt the normal
9:09
assistant. To to just.
9:11
Solve the user's problems and you and a companion needs to
9:15
more take care of the user, be more empathetic as the user
9:17
about them. So it's it's.
9:20
The different approach on the prompt, but again the ability
9:24
to. Observe to see what is.
9:26
Going on to see when things went off, the ability to measure our
9:30
core principles. For.
9:32
A conversation of the, I think and that's my approach in last
9:35
three platforms that I've been involved in.
9:39
Because this is. Evolving all the time and keep
9:43
changing. And you know, I had talking cold
9:46
talks few weeks ago in Humboldt and I and I was talking about
9:50
how when I submitted. The talk that was 6.
9:52
Months ago, until when the conference started everything
9:56
changing our world so whatever I submitted was completely
9:58
irrelevant in six months. So everything we build now needs
10:02
to be. Very.
10:03
Flexible for changes and that's also very, very important.
10:06
So we cannot put guardrails and how good conversation looks like
10:10
now when tomorrow there will be another model.
10:13
Out there which will be. Better Now if I will add to
10:16
that, think about in conversation, in companion.
10:20
Tone of voice is something that one model has.
10:23
You cannot upgrade the model or replace the model.
10:26
The user will get a different tone of voice, a different
10:29
companion. This is something you wouldn't
10:30
expect from your friends or your relatives.
10:32
You cannot expect it from your companion, so the system must be
10:37
so flexible to changes that you need to figure out on the.
10:41
Way what you'll do? OK, what do I do with all these
10:43
users we used? To talk with this model, I now
10:46
found there is a new model, cheaper, better, more
10:49
conversational, but I cannot replace it for them because they
10:52
will get a new companion. What do I do?
10:54
So you need to build a platform that will be very flexible for
10:57
changes because they keep happening and very, very fast.
11:00
And. The fastness of it all and let's
11:03
call it velocity of also the innovation part is a particular
11:07
aspect of like your role at Replica as well.
11:10
And the company's story also shows both like the potential
11:14
and like the real ethical regulatory challenges that you
11:17
might have when interfering or like looking into emotional
11:21
attachment style with data privacy aspects or also
11:25
particularly in Europe, the regulatory aspects of it.
11:30
Can you share some examples or insights of how you and your
11:34
team balance innovation speed with like the Safeguard rails
11:39
also when designing for user emotional engagement with AI?
11:44
Yes. So as said earlier in our.
11:47
Case we're a bit lucky. Because safety and all the.
11:51
Stuff are part of our. KPI, our product won't be
11:55
successful if those things won't be embedded if, if and one of
12:00
the things that helps a lot, our engineers are engaged very much
12:03
with what's going on with the users.
12:05
I think I I've been advocating for years that engineering
12:08
needs. To see the closest to.
12:10
Customer support. Because usually they are so.
12:12
Far away from the users, they don't hear the stories, they
12:14
don't know what's going on. And in our case, our engineers
12:17
are so on Slack, they are pasting stories from Reddit.
12:20
We have Reddit community very, very active.
12:22
They're pasting story from Reddit.
12:24
The good. The bad ones, they're very much
12:26
connected to what's going on with the users and how they're
12:30
using the product. So that will be the.
12:32
First thing, it is embedded in our culture, the good and the
12:35
bad, how emotionally people attached to their product.
12:39
And when it's embedded in the culture, it's embedded in the
12:41
code eventually and it will be embedded in the road map and in
12:44
the demand. So that's first thing it really
12:46
needs to be a part of the culture.
12:47
And you know, when people are emotionally connected to a
12:51
platform, crazy things are happening.
12:53
For example, when we have a downtime and we had because.
12:57
It's software, people took it really bad.
12:59
And it's on one hand it's scary, on the other hand, it's, it's
13:03
amazing to see how the system influencing.
13:06
People how this product is. Really, really helping people
13:08
and how connected. They are to it.
13:10
So it's a part of the culture, it's a part of the.
13:12
KPI and we, we keep, we are listening.
13:14
Of course, I said earlier. You need to put place on your
13:18
Rd. not especially at scale. Maybe when you're startup you
13:21
doing MVP POC it's you don't need to care about this thing so
13:24
much your. User is only the VC.
13:26
That needs. To to to support.
13:29
But at scale, you must put a place on the road map.
13:32
For security for all these. Things and and on top
13:36
regulations I remember how old the.
13:38
The the years of the. Cookie banner and the GDPR
13:41
companies needed to invest so much engineering firepower and
13:44
road map. Space just for this cookie.
13:46
Banner and and maintaining it, and sometimes it looked
13:49
ridiculous and sometimes. It feels like regulators.
13:52
Don't really understand how engineering works when they
13:54
implement these things, but the intentions are good and
13:57
eventually we want to protect the users and we want a better
13:59
Internet. So you must clear and invest in
14:02
that. I'll give again an example from
14:05
Meta. Actually that Meta has huge
14:07
integrity teams and. It just to protect users.
14:10
Right. And and part of it is because
14:12
regulators and also the media are kind of demanding that, but
14:16
eventually a company is investing a lot in that.
14:19
Now this is not necessarily killing innovation.
14:23
It's just an extra investment that could have been invested in
14:26
innovation, you could say, but it's it's not when it's embedded
14:29
in the culture, when a company decide to invest, it's not
14:33
necessarily kill innovation. And by the way, I've seen
14:35
amazing innovation in this area. Exactly.
14:38
And for us investing in innovation, in safety, again, it
14:42
pays because eventually a good conversation, safe conversation
14:46
is something that will make user come back to us.
14:49
Earlier you just said one sentence that I loved and I want
14:53
to pick up on that. You said if it's embedded in the
14:57
culture, it ideally and eventually is embedded in the
15:00
code. Is that so and why?
15:04
On purpose, go to another example which is easier.
15:07
For example, when you have engineers who care so much about
15:11
security, when you have engineers who care so much about
15:14
testing, you know some companies.
15:15
Say test coverage. Must must be 80% and stuff like
15:19
that. It's a.
15:20
Part of the culture. Right, we are doing only test
15:22
driven development part of our culture.
15:24
It is eventually embedded in the code it is built in.
15:27
You can say oh develop this feature and.
15:29
After that, write the tests. Or you can have just a.
15:32
Culture that. Writing a test is a part of
15:34
implementing the feature and it's not even negotiable.
15:37
It's it's not a separate numbers on the ticket on JIRA.
15:40
And that's exactly it. It engineers while they're
15:43
building their thinking about it, while we are writing the
15:46
prompt, we are thinking about it.
15:48
It's not a separate boring task when it's a part of the culture,
15:52
when people care, when people see what they wrote make people
15:55
having very negative emotional effect when they know it, when
15:58
they see it, they will think about it next time.
16:00
So. It's it's really a part of the
16:02
culture and a. Part of the engineer's way of
16:04
thinking. And if we look into the way that
16:07
AI companions are engineered and knowing that they are or
16:12
eventually become part of like the everyday life that we also
16:16
explore. And we enter new spaces where
16:18
like the culture or the ownership of how the code shapes
16:23
the companionship of AI within our lives is like required to
16:28
have a trustworthy also systems and setup.
16:32
And but in our own agency, it's also hard to adapt it in the way
16:37
that we interact with like exactly those companions.
16:41
How do you see user agency in the context of Replica and also
16:46
at the intersection of influential AI responsibility
16:51
and the adoption of it all? So we if we'll start with a.
16:55
Simple. Software.
16:56
We have ways to. Control.
17:00
Most of the time the service provider will need to build.
17:02
This with space. For it.
17:03
But if I'm talking just about engineering, we can control
17:06
everything and that's the beauty of it, we can decide.
17:09
What to do with the? Data if to store.
17:12
It what to delete? Now if you implement this.
17:14
Feature for the user. Allowing the user to delete.
17:16
Their their data to delete. Their name, the right to be
17:19
forgotten and everything regulators are really trying to
17:22
to implement and. You give the users this.
17:24
Control. Now there are more features into
17:27
that. For example, we allowed to we
17:29
can show you what we remember about your relationship.
17:32
With with the replica. Or or in other platforms that I
17:35
build the same. We show you what we.
17:37
We remember about you and we allow you not only to delete it,
17:40
but sometimes to. Edit it which is.
17:42
Feeling much more control. So you have first transparency,
17:46
then you have control. And these are the engineering
17:49
parts of it. But if you think about it,
17:51
what's so special about conversational AI?
17:55
You're in control where the conversation will go.
17:57
Okay, you can tell the AI, hey, I don't like when you mention my
18:03
ex-girlfriend or I don't like when you speak with this slang
18:08
word. So we had the discussion at the
18:10
beginning. So you need adaptable
18:12
concessional AI. So if a business person use use
18:15
the companion, we cannot use some words like lol and stuff
18:18
like that. Just an example.
18:20
You're in control where this conversation will go, and the
18:22
challenge generally today in the AI world is the memory.
18:26
If we do good job. With remembering it always,
18:29
you're in control from now on that I will never mention your
18:32
ex-girlfriend. And that's amazing.
18:34
That's another mean of control that you have that we never had
18:37
before. And you don't even notice it.
18:39
It's not something you click and say, delete my data.
18:41
You're just saying it. And the conversation shifted.
18:44
And I find this quite amazing and from the engineering
18:47
perspective, very challenging to remember that you asked me these
18:51
things. And there is another layer of
18:53
trying under the hood to adapt to what you want.
18:56
And this is kind of you're in control, but you don't even know
18:58
that you're in control. This is even more complicated,
19:01
but we're also trying to implement that.
19:03
If I may challenge that on the other side, what you describe,
19:07
you could also say that you're playing with reality.
19:11
So you could say me, let's say asking the AI to forget about my
19:16
ex-boyfriend or to adapt the style that like things that I've
19:21
previously written shouldn't be remembered even though it was
19:24
part of like the earlier life that I lived through.
19:26
Isn't that also influencing the way that I perceive or the way
19:31
that I have experienced interactions beforehand?
19:36
And wouldn't that make it artificial?
19:38
And so let's compare it to you talking to a friend.
19:42
You would expect that once you asked your best friend to stop
19:46
talking about a certain topic. That that.
19:49
Person will consider your request and and be empathetic to
19:52
to your needs and to stop talking about it.
19:55
Would that be artificial? You will say no now if you're in
19:58
a crisis, and that's the time to remind you this, although you've
20:02
asked sometimes something we do as humans or you need to
20:05
escalate it and let's say some. Things which are very severe.
20:08
Someone went through a very severe event and that you need
20:13
to escalate even. This is a very good.
20:16
Question. Maybe that's a completely new
20:18
topic that we can discuss. We are not there yet, and again,
20:23
it's matter. That they have.
20:24
Escalations and they need to escalate to law enforcement when
20:27
when stuff are going off. But in this case it's a very
20:30
extreme case. But generally it's not
20:32
artificial. It's what I'm expecting, a
20:34
person or a companion, human or machine that care about me or
20:38
about what I. If we now take that on a like
20:41
even larger level and look into the regional dependencies that
20:46
we have with technology in itself and not only on a
20:49
personal level, when we as a society and trust technology
20:53
with personal and emotional domains and also the
20:56
interactions that we have with it.
20:57
What do you say? And from your experience that
21:00
you've had, what role does digital sovereignty play?
21:03
And are users losing control of potentially also gaining a new
21:07
form of agency through it? So first of all, we are.
21:10
Always in controlling what we share.
21:12
This is true to our. Friends, this is true to any,
21:15
any system. This is true to social media and
21:18
this is also. True to to social.
21:21
Too sorry to conversational AI and we can share emotional
21:24
private stuff we can be influenced by.
21:27
The feedback that we get. From the other.
21:28
Side about what we shared. And this is always true now.
21:32
First of all, we are. In control and it's very
21:35
important that. People, of course, will first.
21:37
Know that. It starts with you.
21:39
Now, on top of that, AI, you can always stop things.
21:43
I can close the conversation, I can shut down the computer, I
21:47
can ask for help, OK, I can change direction like I
21:52
mentioned earlier, Hey, stop talking about that something
21:55
that is not always possible with human to human.
21:57
So I actually gain more control to an extent.
22:01
I know that sometimes things can be.
22:03
Harder to ask to stop. And I know that not.
22:05
All people are always. On top of things and understand
22:08
that. Hey, I have this ability.
22:10
Do I know that when I have a conversation with another human,
22:13
it could be the same difficulty, right?
22:16
So I think we're gaining more control and we decide what the
22:20
AI know or the system knows or forgets and what are we can
22:24
define the boundaries now the. Beauty of it is is we have
22:28
control as a user but we also have control.
22:30
As an employee of the company that provide a service, and
22:33
that's very important to know. There is a human in the loop,
22:35
There are several humans in the loop here that can stop
22:38
something, that can give control because we can get alerts at
22:42
certain points or user can alert us as something is going off and
22:45
we can also stop things and in really bad case even escalate
22:49
them. So I think there is much more
22:51
control, but the awareness is maybe much more limited.
22:56
And when we. And look ahead.
22:59
On the way that conversations are shaped and also the way that
23:02
AI is scaled rapidly, what do you believe are the three non
23:07
negotiable capabilities that you yourself as ACTO must bring to
23:11
today to build up like new systems, but also to ensure the
23:16
trusted users have in future interactions with companions
23:20
tools or the systems that you build up?
23:23
Reflecting upon all that you've just said and also shared, when
23:28
it comes to the sovereignty of that potentially users want to
23:31
keep as well. It's user trusting this the
23:34
product. It's also when engineers or the
23:37
company employees are trusting the system, so I believe I
23:42
mentioned that. Before that, the first.
23:44
Thing, especially in this unknown field and again, it's
23:48
developing so fast and I'm lucky enough to.
23:51
To to already build several. Of these platforms but but
23:54
always helped and what was so important when things are
23:57
developing so fast. Is to have observability, so.
24:00
I want to know that when something has happened, I can
24:04
immediately see or even get alerts immediately see what
24:07
happened and be able to to to understand and to fix it if
24:10
needed. I don't think you can in this.
24:14
Field where there are black. Boxes and unknowns and new
24:18
models all the time, and new systems without observability.
24:21
You're just blind. And so that's the first thing.
24:23
Always needs to be in the system from day one.
24:26
The whole topic. Of human in the loop is
24:29
important we touch it in the last question so.
24:32
The user is always able. To control the conversation.
24:36
To stop the conversation to. Ask for help.
24:40
And then the other human, the employees, we can do something
24:44
about it and it's certain cases as certain products also law
24:48
enforcement and or teachers, parents, etcetera, etcetera.
24:52
So this human in the loop, there are several humans that could be
24:55
in the loop and that's very important to allow that and not
24:58
just that, you know, autopilot and, and hope that it will work
25:02
and everything will be OK. And third thing, and I mentioned
25:05
that before also and I tend to build platforms in this case.
25:10
Which are very very flexible. Which means I don't know what
25:13
will happen tomorrow, I don't know which regulation policy
25:16
will come tomorrow, and regulators don't always
25:19
understand the engineering as mentioned so.
25:21
So I need to. Be able to be.
25:22
Flexible, adjustable. So every new model, every new
25:27
system, every new memory. I will be.
25:30
Able to to experiment with and implement if needed or remove
25:33
when I say that something is not working.
25:35
So we touched observability, human in the loop and
25:38
flexibility and. May I just briefly look with you
25:41
into the flexibility part a bit further because from my
25:45
experience, that is the hardest part for organizations because
25:49
like in the past time we've set teams, we've set structures,
25:54
we've set control systems and so on.
25:57
And now if you say that we need to be flexible and adapt with
26:01
the evolving technology or also the user requirements that are
26:04
set, what does that actually imply for teams and particularly
26:09
in the technology area of how they will collaborate and of
26:14
also how they will be set up from a skill set perspective?
26:18
That's. Super complex.
26:20
So first of all, I think that any organization today, even big
26:24
ones that is not an adoptable organization, that doesn't.
26:27
Work like a start. Up in a.
26:28
Way will die. It's, it's really, really it's,
26:31
it's a very fast. Growing or fast changing
26:34
environment and and organizations cannot allow
26:38
themselves to to be slow and not adoptable.
26:40
And it's it's it's just impossible.
26:42
It doesn't mean that you need to adopt every new fancy thing and
26:45
the market, but this? Is where you put.
26:47
Professionals who knows what should be experimented with and
26:51
and what doesn't fit and and and is not needed.
26:54
So flexible. Flexibility.
26:56
An organization. It's a mindset.
26:58
It's a culture that always. That that is needed having roles
27:02
in engineering teams. Which are very very.
27:05
Complex. Today, what is the type?
27:07
Of engineer that I need in an AI team to it's very complex
27:11
context engineering and you know the traditional Python.
27:15
People. Coming from data engineering
27:18
teams, we need software engineers that did client
27:22
facing, API client facing products for many years with
27:25
Python. They're really rare to find and
27:27
and then not mention all the new vibing tools that are coming in
27:30
agents. So people who enjoy playing with
27:32
all these configurations and all these new tools that there is no
27:35
name for it yet. It's very complex to build a
27:39
team with not real defined roles.
27:42
So you need to find talents. You need to find talents which
27:46
are. Flexible that likes to explore.
27:48
Things and experiment with things and are not afraid from
27:51
changes. From that.
27:52
Of course you get the software that eventually is very flexible
27:55
and software which is very thing and and and you can throw parts
27:59
and implement new parts. Daily.
28:01
And if you now look. Ahead and I know in AI and in
28:04
technology, it's super hard to look ahead and like let's say at
28:07
least two years from now. But if we do, and if you look
28:11
into the field of AI companions and their evolution and how they
28:15
integrate deeper into our lives, what would you like to see
28:19
changed or also regarded when we talk about trust and when we
28:24
talk about AI? And what would you you want this
28:28
shift or change to feel like? I have.
28:31
Opinions from my my my learnings and experience and perspective.
28:35
I'm not sure that they are right.
28:37
It's a very complex topic I think.
28:39
I think the discussion about AI. Should should go back.
28:45
A bit to then to to what are the engineering?
28:48
Capabilities of these? Things It's still a statistical
28:51
machine. We sometimes expected to have
28:53
100% success in being trustworthy or not to do.
28:58
Stuff we don't expect it to do for.
29:00
Example if we want to prevent all the false positive so we
29:04
want some some AI system not to mention certain things or not
29:08
to. Suggest people to do.
29:11
Bad things, we need to eliminate all the false positives or it's
29:15
also when when we want to catch fraud or catch crime over social
29:18
media, we want to eliminate. All the false positive.
29:21
But then we will also hurt innocent people and block them
29:24
or even send the police to their door.
29:26
So it's a game. With these classifiers that
29:29
that. That you need to play with and
29:30
it can never be 100%. And sometimes people have
29:33
expectations of a 100% and they forget that these things are
29:37
statistics. So this discussion is A and and
29:40
B. We will have much.
29:41
Healthier products also if. Journalists, regulators will
29:45
will understand how these things are working and BI think we
29:49
should have a discussion. I don't really have a full
29:52
opinion about it. I had a very good discussion
29:55
about. It with with with a.
29:57
Friend the other day, he's a journalist and he said, I think.
30:02
We should just make laws that should.
30:06
Force all these companies. To have age.
30:09
Protection and that AI will never talk about sexual things
30:12
etcetera, etcetera and always like.
30:15
The law will. Prevent it and we had I think 10
30:18
minutes discussion. I tried to explain to him, but.
30:21
And that's a very good friend. Yeah.
30:22
So we have this opening discussion, but you don't
30:24
understand it's. Impossible.
30:26
This is not how. Engineering works, no, I expect
30:28
my the the law to to protect me and my child and this is quite
30:33
problematic thing to expect and I think we should have also a
30:37
discussion around. What we as society expect.
30:42
AI can can can do things that we want users not to do with
30:46
humans. Okay, for example, sex talks.
30:49
I will be. Happier if people.
30:50
Will go and talk about it with AI then people who are getting
30:53
exploited in only fence or things like that.
30:56
But do we want it? Do we want to encourage that?
30:58
Now we kind of open this thing and said okay.
31:01
That's okay because it's with AI.
31:02
Where is the border? If they print.
31:05
Pictures fake. Pictures not of real people.
31:08
Do we want to allow that or we don't want to allow the time?
31:11
I think we need to have this discussion, but I don't think it
31:14
is discussion that has a real solution, but it's a very
31:17
important one. That we as a society.
31:19
Should have shouldn't end immediately.
31:21
With with the law regulators. It's so interesting because we
31:24
look into, if we look into those discussions.
31:28
For me, it always comes down to questions that are as old as
31:33
humanity is like, what is, you know, right, what is wrong?
31:37
Where do you start? Where do you also frame it?
31:40
And what is real or what is also like empathetic towards each
31:45
other. And I'm just wondering how you
31:48
would like how you would open up the space to have the discussion
31:53
on a way that there can be a result.
31:55
Do you have an opinion on that? I think.
31:57
We are having this discussion as I said.
31:59
For a long time, but for me kind of from from what I've been
32:04
dealing with for for many years, again, social media is a great,
32:08
great example. We had these discussions all the
32:11
time, what to allow on social media, who is responsible to
32:15
what happens on social media there and.
32:17
There's a very big. Difference between how the
32:20
European Union see it and how the the the US regulator seeing
32:23
it, the platform is responsible or the humans at the edge.
32:29
It's it's it's a very problematic.
32:31
I I don't think there is an end solution.
32:33
I think you can say that the market.
32:35
Will will solve it. So good products that will
32:37
protect the users will will get. Gain the trust and they will win
32:40
the market. It's not always true.
32:42
You also need a good balance of regulations and I think what I
32:47
find the best is the discussions on, on, you know, the blog post
32:50
and newspapers and journalism and.
32:52
Podcasts, of course, think this is where it should happen.
32:54
Because in this place you have people from all sides.
32:57
People could listen, make up their opinion, learn.
33:00
And that's the most important thing.
33:01
We actually didn't talk enough. About it, but educate people.
33:05
Learn that I have control. Learn that.
33:08
This is right. This is wrong.
33:09
But my kids can still do those things and I need to to to watch
33:13
for it and. Decide what what is good?
33:15
For my kid, you know, it's like the decision kids at certain age
33:18
getting an iPhone or not. It's my decision as a parent
33:21
many times as much as other things won't influence it.
33:24
But but I I can still be. I can be.
33:29
Educated that this. IPhone has some protection
33:32
measures. For kids.
33:33
So I can use it. If we will have these
33:35
discussions openly. On the television, with
33:38
podcasts. Blog posts everywhere.
33:40
People will, will, will. Get educated will know what they
33:43
can, what they cannot, and. That's hopefully will push.
33:47
Also the industry and the. Products to a good.
33:50
Place you know we will. Always have products that will
33:53
do. Less good things and people.
33:55
Will use it. I don't see a way to avoid it.
33:57
You can have some regulations in the EU and then another country.
34:01
Will will serve. From from their.
34:03
Server as a product. That is doing something against
34:06
it and you cannot really prevent it.
34:07
It's the Internet. Yeah.
34:09
I was just in my head, I was running down the rabbit hole of
34:12
like global decision and regulations versus local ones.
34:16
But that will be a whole different conversation and we're
34:21
already. Yeah, let this topic I always.
34:24
Have in my head the example of the.
34:26
So I I always have this example in my head and and in the past
34:30
there was. This whole topic of Dolly the
34:32
Ship. That it was not allowed to to
34:35
clone animals in the US, in Europe, in the whole world and
34:39
then somewhere in Asia. I don't remember in which
34:40
country it happened, so I don't see a.
34:43
World where we. Can enforce regulation globally.
34:46
It's just impossible. I I just don't see this
34:48
happening. We need to be smarter a bit.
34:50
Yeah, I would. I'm super cute.
34:52
Curious if that's gonna turn out true.
34:54
Because also like on a European level, there is like a whole
34:57
conversation about how to regulate on European level when
35:00
there is like national interest and then globally as well.
35:03
So I think it's gonna be super interesting the next coming
35:07
years to define and also a trade on the way forward.
35:12
What happened now is the age gate in Australia that it was
35:15
implemented, but OK, You're not allowed to log in to register as
35:18
a team, but you can still see the feed when you're not logged
35:21
in. So what?
35:21
Did we do there but. So then I'm if I have a website
35:25
that sits in Europe, I'm not. Allowed to serve it to.
35:28
Australia or I need to serve it differently to Australia or it's
35:31
only Australian website? It's so complex on the Internet.
35:34
Who is not allowed the? User or the website or?
35:37
How do you? Enforce it Really.
35:38
Globally Super. Complex.
35:40
Absolutely. And I'm coming to the end of our
35:43
podcast episode. I always have those so-called in
35:46
between moments with my guests, which I love, which are also
35:49
sometimes a bit tricky because I ask you in particular to answer
35:54
like a prompt that I give quickly, intuitively and just
35:58
with a reflection of what comes to your mind instantly.
36:02
And I want to start with what trust feels like to you.
36:06
Perfect relationship. And what should an AI companion
36:10
not do? I think it shouldn't be what I
36:13
don't want it to be. I think it it should be what I
36:17
want myself to be, the best thing of me and plus.
36:20
The the superpowers of of. Machines like searching stuff
36:24
really fast and answering me. What is agency for you?
36:27
Having control over my boundaries.
36:30
What is the biggest risk in AI companionship?
36:33
Unrealistic expectations from all sides.
36:36
And when we look into 20-30 and that's five year heads now, what
36:41
will users still need? First, humans around them.
36:44
This will never change. We won't become so lonely.
36:47
We'll need technology to help. US.
36:50
Thank you for a perfect ending. I really appreciate the
36:52
conversation and the time that you took and all the insights
36:55
that you share was great having you Lear.
36:58
Thank you very much, Eva. Thank you.