Latest / Show Me The Evidence / The end of "see one, do one, teach one" — Prof Stefano Puliatti | Show Me The Evidence Episode 9

Transcript

The end of "see one, do one, teach one" — Prof Stefano Puliatti | Show Me The Evidence Episode 9 · Show Me The Evidence

0:07
My guest today is Stefano Puliatti.
0:09
Stefano is a professor of urology at the University of Modena in Reggio Emilia.
0:15
He's a robotic surgeon, he's a faculty member of SurgeQuest with my colleague Ruben De Groote.
0:24
And he's former medical director at Orsi Academy in Belgium where I met him in 2019.
0:30
Very welcome, Stefano.
0:32
Thank you, Prof. It's a pleasure for me.
0:35
You were a resident in Modena during a time when robotic surgery was expanding rapidly.
0:41
Did you and the surgical leadership in Modena realise how pivotal
0:45
our robotics would be for the future of surgical practice?
0:49
Yes. We had the fortune to start robotics in 2007 in Modena.
0:58
It was one of the first robotic platforms in Italy
1:03
And the robotic leadership at
1:05
the time was my mentor, Professor Gianpaolo Bianchi,
1:10
full professor in urology and the former director of the urological clinic here in Modena.
1:17
And he saw very soon the potential of this new technology applied to the urological field.
1:27
Because the urological field is a very, very
1:30
specific field of surgery, we have the pubic bone
1:34
that obscures the anatomical view. And the robot is really able to
1:43
avoid conflicts to improve the visibility and to improve the technical possibility to do,
1:51
for example, an anastomosis. So really from the very first moment, he saw the potential
1:59
and we followed up with the implementation of the robotic surgery in our city and in urology.
2:09
So robotics is quite different, but I mean, something that you started out
2:14
your surgical experience learning laparoscopic and open procedures. So how did you learn
2:22
a laparoscopic and open procedure? Traditional Halstedian?
2:26
Yes. It was a kind of a Halstedian model. It was typically still the standard way of training
2:37
a lot of centres in Europe. So I had the fortune
2:41
to have a great surgeon teaching to me. So I was
2:48
able to learn watching and then doing when I was allowed to do directly on the patient,
2:57
directly in the OR. And this was the setting.
3:03
So we saw the procedure done by a very, very good
3:07
surgeon and we were allowed step by step to do some part of the procedure done under the
3:15
supervision of the trainer directly on the patient directly in the OR.
3:22
And did you attend subsidiary courses in laparoscopic or were they ran in the hospital or
3:30
they were apprenticeship type training as well?
3:33
Yeah, there were some courses run in Europe or
3:39
in Italy and I attended some courses in my first and second year of the residency school in
3:46
laparoscopic and endo-urology around Europe and around Italy, mainly based on theoretical
3:58
lessons and some life surgeries that we were allowed to follow and some surgical skills
4:10
but not with a methodology, a theoretical methodology on the base of these courses.
4:19
So during your time as medical director at ORSI,
4:22
looking back on it now, what are the key lessons
4:25
that you learned about surgical training and learning skills? Yeah, it was really
4:34
mind-changing because it was something really different what we did in ORSI Academy because
4:45
there were some very important key points, I
4:50
think. First of all, there was ORSI Academy. ORSI Academy
4:54
is a fantastic place in which we have a lot of simulation operative rooms, we have a lot of
5:05
robotic platforms, we have everything to simulate
5:09
a real operatory room without the risk of operating
5:14
directly on a patient. This is a kind of
5:20
wonderful place in which you have all the video games,
5:26
all the possibilities that you wanted to learn
5:30
but I think this is not enough. The other key point
5:35
that I found was a methodology. That was, I think,
5:39
the thing that is missing in Europe, also nowadays,
5:45
in the training setting. So we learn watching
5:49
other surgeons and doing steps of the procedures
5:54
directly on the patient but without a
5:58
methodology that is on the base. It means that if you have
6:03
a very very good surgeon that trains you, you can
6:06
learn really well how to do surgery. But if you have
6:10
a quality that is not enough in your trainer ship probably you will be not a very very good
6:21
surgeon because it is difficult to implement
6:25
your skills if you don't have very very very good surgeons
6:29
teaching to you. What I found in ORSI Academy
6:33
was a methodology. So the methodology is a kind of
6:37
way that standardised the way we do training and
6:42
we have methods, so we have rules to respect and
6:45
are the same for all the trainees. And this means
6:49
that it is possible to reach a very very high level
6:53
of skills at the end of the training. So it was
6:57
not only about the training setting and in that
7:01
case was ORSI Academy that is a very very high level training setting but was also about the
7:07
methodology applied to the training that you do.
7:12
And this methodology, those metrics, those rules
7:15
that you have to respect, those arrows that you have to avoid are really the key points that
7:21
are probably in my mind able to democratise and
7:25
standardise the level of the training and make
7:30
the possibility for everybody to reach a certain
7:33
level of performance at the end of the training.
7:38
So at the start of your time in ORSI if you can think back to 2019,
7:43
what did you make of proficiency based progression training when you first encountered it?
7:50
At the beginning it was very difficult because you were the leader and I was a little bit
7:58
difficult to understand your accent but this
8:03
was probably the first big problem but after three
8:08
months I had a feeling with your accent, I had a feeling with you and I had a feeling with
8:19
the methodology and this was for me really
8:22
something crazy, really something very good because I saw
8:31
a methodology that in the beginning seemed not so simple but when you enter into it and you
8:41
understand the methodology in depth it becomes
8:46
really simple and really straightforward and it
8:52
was really simple for me to understand how to apply the methodology in the different training
9:00
models and to really understand quickly that if
9:06
you are able to focus the errors and the critical
9:11
errors and to avoid the errors and the critical
9:15
errors then your performance becomes straightforward,
9:20
becomes faster, becomes more, I don't know if it
9:26
is possible to say like this but it is good looking.
9:28
So you can see that there is a flow that is more fast, more beauty to see and at the end
9:40
the results represents this beauty and you have
9:45
an improvement in the results and at the end it
9:48
is really fast to reach a proficiency level so a
9:51
very very high level of performance. So to make
9:55
a long story short at the beginning seems complicated the methodology but it is not,
10:01
it is simple, it is easy to apply on different tasks and the results are really fast
10:09
to be recognised at the end of the training performance. So five to six years later, okay
10:20
you were working with robotics in ORSI academy, so
10:24
five to six years later what do you make a PBP now?
10:29
Now it is a guidance and it is a must to have. I have the fortune that all the robotic groups
10:41
here in Modena was allowed to have a one-year fellowship in Orsi academy and they were all
10:51
trained accordingly to the PBP methodology so
10:55
also when we start here in the OR to do part of the
10:59
procedures when they come back from the
11:01
fellowship and they start to do part of the procedure it is
11:04
accordingly to the PBP methodology and only who
11:08
has a benchmark proficiency already demonstrated
11:12
in the anastomosis task and in the other task of the procedure we know that the robot assist
11:21
the radical prostatectomy is probably one of the
11:23
first robotic procedures that was metricised with
11:26
the PBP methodology and so we have all the stats
11:29
with the metrics and we follow this methodology
11:34
and those rules in the OR. This makes the things
11:38
really easier for the trainer and for the trainees
11:44
because it is more smooth the learning curve on
11:50
the patient and without or with really less risks
11:56
so it is a win-to-win for the surgeons for the school and also for the patients.
12:02
I think Modena have had a fabulous engagement
12:09
with the proficiency-based progression methodology
12:11
and I think it bodes very well for the future in terms of the development of the department
12:16
and for the new professor when they come in. So what for you are the essential features of
12:24
the proficiency-based progression methodology that
12:27
make it effective? I think there are three points.
12:31
That makes PBP methodology a winning methodology.
12:37
The first point are the metrics. The metrics are
12:42
the rules, are the receipt that you have to
12:46
follow to have good results at the end of the training.
12:51
More than that are errors and critical errors that you have to avoid. So when you focus the
12:58
critical errors that you have to avoid it is really more simple to do a good procedure.
13:04
The second point is the deliberate practice because the deliberate practice for me is key.
13:13
We did a study when we were in ORSI also with Luca Sarkey and Pietro Piazza and other
13:21
Rui Farina other colleagues in which we did a tele-mentoring.
13:27
The tele-mentoring with PBP, PBP-based was really
13:30
successful. Why was that? Because we were able to speak a really direct language based on the
13:39
metrics and this is the deliberate practice. So
13:42
you give feedbacks that are based on the metrics
13:47
with a clear language that is already written on the metrics. This makes the feedback
13:56
direct, really understandable for the trainee
13:59
and deliverable also in a tele-mentoring setting.
14:09
This is something different in comparison to
14:12
the Halstedian model because in the Halstedian model
14:15
you use a language that is your own language. So
14:18
it is different if I do the training, if you do the
14:21
training or if another person does the training.
14:24
With the metrics you have the same language for
14:27
everybody and you can give really a direct
14:30
feedback to the trainee that is able really to understand
14:34
in a very very short period of time what is your message and how to fix the problem.
14:44
The third point that I think is key in the PBP methodology is the proficiency benchmark
14:51
because we have a really high level that the trainee has to reach because it is the mean
14:59
level of experienced surgeons, very very good surgeons that did the same task. So at the
15:08
beginning when you see it you say okay but it is
15:10
impossible that they reach the same level of this
15:13
fantastic surgeons doing that task and it is
15:19
impossible that they reach that level in one day,
15:22
nine hours or one day and half. But with the metrics it is possible to reach this proficiency
15:30
benchmark really fast and at the end you reach this kind of technical skills in the lab,
15:38
not directly on the patient and this is
15:42
something that allows you then when you go on the patient
15:46
to have the confidence and the safety to
15:50
operate directly on the patient with a very very high
15:53
level of technical skills and this makes the trainee more confident but also the trainer
16:01
is responsible of the surgical, more confident that we will not have damages on the patient.
16:10
Yeah I think one of the lessons that we learned from your studies and from Ruben's studies
16:17
is that you also have to train the trainers how to
16:21
deliver the delivered practice because I remember
16:24
walking around the skills lab and watching the
16:27
different individuals deliver delivered practice
16:30
and some of them are really good at it and some of them are really shy and some of them
16:35
they didn't understand what proximate feedback
16:37
was. So it was I think it was a huge, your time
16:41
in ORSI think it was a huge developmental phase
16:45
for proficiency based progression where we learned
16:48
a lot. One of the things you talked about you know
16:52
practicing in the skills lab, I mean do you think
16:55
simulation models for training skills or robotic
17:00
surgical skills, do you think they by default have
17:03
to be expensive? We tried to find something
17:09
that was less expensive because there is a kind of
17:19
market also around the training models. There are some really expensive virtual reality
17:29
simulators, there are plastic models that are
17:33
really expensive. What we tried to do was to find
17:38
a model that is available worldwide, it is not
17:43
expensive and we found it using a chicken model
17:49
and the chicken model in my opinion that was already
17:54
acquired and studied as a training model but without a training methodology.
18:05
It was not PBP based before. To apply the PBP methodology to acquire the technical skills
18:14
using an inexpensive model like the chicken
18:16
model I think was a very very good idea because the
18:20
chicken model is available worldwide. We use
18:23
the chicken that you can find in the supermarket,
18:27
it is the same chicken that you buy to eat at lunch or at dinner. So instead of using it for
18:37
feeding we use it for training that I think is a
18:42
very very important thing to do and the chicken
18:48
model has some advantages. It is a biological
18:52
model so has some elasticities, some characteristics
18:59
of the tissue that really are similar to the human tissue and it is more or less inexpensive.
19:09
The most difficult part in my opinion was to find using a single model the different tasks
19:18
that were able to reproduce the basic skills that you have to acquire to do then a very very
19:27
important surgery like radical prostatectomy or
19:30
some genealogical or general surgical procedures.
19:34
We found three main basic skills suturing and knotting, dissection and coagulation tasks
19:48
that are really important to acquire and using the chicken model we were able to find
19:55
different exercises that were able to train these three skills that are the
20:04
elementary skills that you have to acquire before going into the OR So I think it is
20:13
inexpensive, available worldwide, really similar
20:16
to the human tissue training model. It is really
20:20
important to have it and to use it. Obviously the biological tissue and the biological animal
20:29
probably in the future will be not allowed. Now
20:33
we have really technological advanced 3D printers.
20:40
Probably this will be the way in the future to
20:42
try to really find not biological tissues that are
20:48
able to reproduce the bio-mechanical information and the bio-mechanical characteristics and
20:55
features of the human tissue. Yeah I agree with you. I mean a lot of the models that I see
21:03
are very expensive and I'm not too sure they're
21:07
very functional particularly for the suturing and
21:09
knotting. I've yet to see, I am a huge fan of physics based virtual reality simulation
21:15
but I have yet to see a simulator that
21:17
accurately or realistically simulates needle thread tissue
21:22
dynamics, particularly one tie-in and knot.
21:25
Dwight Maglin and I were talking some time ago and he
21:29
agrees that we don't seem to be at that point yet
21:32
but he thinks that AI may help in the development
21:35
of that but that could be years off. So was the
21:39
development and validation of the vesicoeurithral
21:42
anastomosis chicken model in Orsi? Was it an important milestone for you? You've combined,
21:48
you've got this check, you've got this inexpensive
21:50
chicken model for the vesicoeurithral urethral anastomosis,
21:54
you've got a training methodology and you've combined these so was it an important milestone
22:00
for you? Yeah it was super important. It was for
22:04
me really a very important school because with this
22:09
training
22:12
exercise. With this training exercise we were
22:15
able to apply the methodology on the basic skill
22:18
setting in roboticsurgery. This was not done before we did it. So it was a first time
22:28
and it was a first of all personally a very very nice moment because we cooperated with your
22:35
guidance with other colleagues working on the metrics, working together thinking about the
22:43
errors, the mistakes, the critical errors that
22:45
you can commit when you do this kind of training
22:49
exercise and to apply the methodology on a
22:53
model. And this model is very nice because you have a
22:56
stomach and the cloaca of a chicken that really simulates the vesicle and the bladder and the
23:05
so the vesicle retral anastomosis it is really similar to the stomach cloaca anastomosis that
23:12
you do in the chicken. So it was a combination of
23:18
things and I think about that as one of the most
23:23
important research I have done so far and one of
23:26
the most important experiences that I have done.
23:30
It taught me a lot about the application of a methodology, about scientific research,
23:37
but also it is something that I still using day by day in terms of really deliberate practice
23:46
and information that as a trainer now I give to
23:49
the trainees and to the fellows and it was really
23:53
useful for me. I remember the type, I thought it was a very powerful vehicle for teaching
24:03
you guys about the methodology but also how the
24:06
methodology applies to real surgical procedures
24:09
and it was probably one of the most productive times for me and probably for ORSI in terms of
24:17
the training models that we developed, the publications that we had and so on and it
24:22
was fun doing it. So I mean in relation to that and the papers you were the first
24:30
or the joint first author and the first systematic review and meta-analysis
24:35
of all prospective randomised and blinded studies that had used a proficiency based
24:40
progression methodology to train one arm and
24:43
compared it to the gold standard, the conventional
24:47
Halsteadian approach to training robotic surgical skills or skills in general,
24:53
systematic review covered all of them. So what were the main findings of that study?
24:58
Yeah, I remember it as another fantastic period, it was COVID period and we had,
25:06
it was really how to learn how to do science. Again under your guidance we were I think in
25:15
Zoom calls every night thinking together and one idea came because yeah we were applied at
25:25
the moment the PBP methodology in robotic surgery
25:28
and the robotic surgery was more or less the first
25:31
time that the methodology was applied but the
25:33
methodology was around from 20 years at the time
25:38
if I'm not mistaken and it was already used in the medical field. So there were a lot of
25:47
studies in which PBP methodology was applied in training for example in anesthesia,
25:53
in orthopedics, in a lot of other specialties. So what we thought was to analyze the results
26:03
that PBP already demonstrated in other medical fields in comparison to the standard way of
26:10
training because there were a lot of data already
26:13
on the table and at the time nobody analysed all
26:19
those data together. So the idea to do a meta-analysis and a systematic review on those data
26:28
was I think a very very good idea and the
26:31
results were really something crazy because we expected
26:39
something good in the comparison between the PBP methodology and the standard training
26:46
but not with this magnitude of difference and we found that with the PBP methodology applied
26:55
in the medical field the trainees who completed
26:59
the training, the PBP trainees in comparison to
27:04
the traditional trainees that were following the traditional way of training were faster.
27:13
So there was a 50% difference in terms of velocity in doing the exercise but we know that
27:24
to be fast is not the same to be high quality surgeons but it was something. So the trainees
27:34
were faster in completing the task, completed more procedural steps, 47% of difference and
27:45
more than that and it was really the thing that was for us the most important. They
27:53
committed the PBP trainees, committed the 60% less operative errors. It means that at the end
28:01
PBP allows the trainees following this methodology
28:05
to be faster, to complete more steps and to do less
28:09
errors. So we had already at the time before starting really to apply it in robotic surgery
28:19
the demonstration that the PBP methodology was
28:22
working really well in the medical field and was
28:25
really improving the results and the data were all data coming for prospective randomised
28:34
trials. So it was not, it was probably all papers were level 1A of evidence. So
28:47
we're really robust data that we cannot ignore. Yeah and I mean what I was impressed was okay
28:54
it was a simple idea but it was a fabulous idea
28:58
and I remember how well we actually worked and how
29:01
strictly we adhered to the science including the evaluation of the quality of the papers
29:08
and it forced me to rethink about and you the steps of the procedure, the time it takes to do
29:15
a procedure, those are measures of process and
29:17
we know the process on its own will not get you
29:20
quality and what that study forced me to
29:23
realize was that the errors were the indicator of the
29:29
quality of that process and so that's how they
29:32
actually fit together. So I mean it was fabulous
29:35
and I remember how well we worked together at
29:37
the time. I spent five years in the US working in
29:42
surgery and cardiology and Frank Lewis was the chief executive or executive director of the
29:51
American Board of Surgery. The American Board of
29:53
Surgery they run American surgical credentialing and so on
29:58
Frank was an old-fashioned trauma surgeon, he was very smart, I liked him a lot,
30:02
we would get on really well. He liked the proficiency based progression approach, he says
30:07
Tony I like that but number one you're never going to get people, you talked about it earlier
30:14
setting the benchmark on the main performance
30:16
of good surgeons, he says you're not going to get
30:18
everybody all of the trainees to the proficiency benchmark. He said that it's really that the
30:24
benchmark is too high, so one benchmark is too high and two that we were never going to get
30:31
everybody into proficiency. Now we're 20 years, we're more than 20 years on, was he right?
30:38
I think that the PBP methodology can be applied in all the fields and I think that it is a
30:51
methodology that really as I said is able to
30:54
democratise the quality of the performance at the
31:00
end. If you think about the proficiency benchmark
31:04
and you look at it before you start training you
31:09
say it is impossible, so or almost impossible
31:12
because you think that the benchmark is too high,
31:17
the benchmark is the same quality level of a
31:21
surgeon that has done thousands of that kind of proceed.
31:28
But then you realise that if you understand the
31:32
rules that are on the base of the surgery, on the
31:35
base of the acquisition of the technical skills, then it is possible to focus the errors, the
31:40
critical errors and then it is possible to avoid
31:43
that. Obviously you have to train, obviously you
31:46
have to fatigue, you have to put effort in the training, it is effort for the trainees, it is
31:54
effort for the trainer, but it is a process
31:58
that makes the trainees really in an easy way up to
32:06
a proficiency level. And so the benchmark in my
32:10
opinion, the criticism that were moved at the time
32:17
are understandable, are logics probably, but if you try to do that accordingly to the PBP you
32:29
understand really fast that are not true
32:34
because it is possible to reach that benchmark, it is
32:37
possible to reach it quickly and probably
32:42
faster than with a traditional training methodology.
32:50
Probably or definitely. I think that that are
32:53
there, so you know that my English is an Italian
32:57
so you have to take it a little bit with, you have to understand me, but the data are there.
33:04
So the probability, probably the era of the
33:08
probability is finished and now we have data that
33:11
cannot be ignored. Again our all data coming from
33:16
a prospective randomised trials so it is level 1A
33:21
of evidence and if we don't trust level 1A of evidence probably we have to trash more than
33:27
half of the medical literature. Yeah, I think your papers are some of the most important
33:33
in the proficiency based progression
33:35
methodology history because I mean the question that Frank
33:39
asked me, I mean I like Frank, I still have
33:41
enormous respect for him and he asked good questions
33:45
and you unambiguously answered the question, okay
33:49
can we train everybody to proficiency? Yes we can
33:52
and it or in comparison to the conventional training it takes about one third the time
33:57
to get everybody to proficiency and only 58% over three days got to proficiency with the
34:05
conventional approach to training. I would say that's definitely and unambiguously answered
34:10
those two questions. Yes, the other thing, so go ahead. No, when we started this trial in
34:20
ORSI Academy and we had as trainees naive medical students in the trial that I
34:30
leaded the idea was that was really difficult to
34:34
have everybody reaching the proficiency benchmark
34:38
and with the PBP model so we were able to do
34:41
that. So probably if you are able to do that with
34:46
medical students it will be really more easy if you have more prepared people.
34:55
Yeah, the other issue that surgical leaders and
35:00
industry leaders who actively fund a lot of the
35:03
courses what they say to me is totally yes we like the idea of proficiency based progression
35:08
but it's too expensive. It's more expensive than
35:13
the conventional approach to training the way that
35:15
you do. Is that correct? This was the other very important point and really important
35:24
criticism that we tried to answer together in my
35:28
period in ORSI was also another very, very nice
35:32
study and it taught me another approach because other than the medical and surgical
35:42
field that daily I have to inquire there was the
35:47
economical field applied to training that is not
35:51
less important than the technical field because you can have the best technical methodology,
35:58
the best model but if it is too expensive it is not applicable in the daily life.
36:05
And what we found again was a kind of surprise because yes if you look at that
36:14
before starting to analyse it in depth it seems
36:18
that the PBP is more expensive. Why? Because you
36:22
have the cost of the metrics. Obviously you
36:25
have to develop the metrics and you need a team of
36:29
experienced surgeons, you need a behavioural scientist, someone that really knows how to do
36:37
science or how to do statistics and you have to
36:41
put this team together. It should be a team that
36:45
cooperates really well because otherwise you can
36:48
have issues in the developing of the metrics and
36:52
the application of the methodology on a training task and it is effort and it is economically
37:01
costly at the beginning but then what we showed is that using the PBP methodology you have
37:09
everybody reaching the benchmark so you have a
37:11
100% of success in having people reaching a certain
37:17
level, a very very high level of performance and you are faster. So the same task to reach
37:25
the proficiency benchmark in the same task with
37:27
the traditional model, in the suturing and knocking
37:31
task using the chicken model it was three and a half days to have the 58% of the participants
37:39
in the traditional model reaching the proficiency benchmark and using a full PBP methodology
37:45
we were able to have the same results for the PBP
37:49
trainees in nine hours. It means that you have two
37:55
and half days of difference in terms of time
38:00
spent in the training lab and the training lab is
38:04
really costly. This was the magnitude of difference in terms of time and as in life
38:13
time means money so if you are able to spare
38:18
time in the lab you spare money. So the money that
38:24
you invest to produce the metrics if you have a large scale of people that will be trained
38:31
using that methodology at the end becomes saving of money and what we found is that the point
38:41
of equivalence is 25 trainees. So if you train less than 25 trainees we used as a model the
38:53
suturing and knocking chicken model. So if you think about that model so to learn how to do
38:59
suturing and knocking using the PPP methodology
39:02
it is more costly if you train less than 25 people
39:08
in comparison to the traditional model but when
39:10
you arrive at 25 people you start to spare money
39:17
and you think you have to think that a training center as Orsi Academy or other important
39:24
training centers train normally train thousands
39:29
of trainees per year. So at the end you have a
39:34
you are able to save billions and we found in
39:40
our study that it was not statistically significant
39:46
but if you think about a company a big company that delivers training at the end of the days
39:53
to spare billions per year in deliver the
39:58
training for their devices or for the surgical skills
40:04
acquisition yes it is not probably statistically
40:07
significant in the study that we are on but it is
40:11
economically significant for companies and if we think about the training that can be used
40:17
in the European community it will be really prepared better surgeons with less economically
40:27
expensive. Yeah I was to be honest I was surprised at the cost equivalence point
40:34
because I knew that question needed to be quantitatively answered and I just assumed it
40:38
would be 100 or 200 or something but the 25 for the cost equivalence was a surprise to me
40:46
and I remember we modeled using your results we
40:50
modeled okay what's the cost equivalence point
40:52
for the development of the metrics and training
40:56
using PBP for prostatectomy and the metrics for
41:00
that procedure were much more expensive they
41:02
ran into the hundreds of thousands and I think the
41:06
cost of equivalence point was about 45 so I mean
41:09
I think the data that that you published were very
41:13
powerful and the BJUI published it as an open article so I think they must have thought it was
41:18
important too. So does the dilution of the PBP
41:24
methodology does an impact on training outcomes
41:29
mean the PBP methodology is fairly strict in what it prescribes you're supposed to reach the
41:34
proficiency benchmark well you're supposed to do
41:36
the online learning you're supposed to demonstrate
41:38
the proficiency benchmark and the online learning
41:41
you're supposed to pass the benchmark before you
41:44
get to the skills lab and so on what happens if you dilute the methodology even just a little
41:52
yeah if you dilute the methodology then the methodology becomes less powerful and less
42:01
efficient so there isn't honestly no reason to dilute it but it was a study that we did
42:11
to understand if we needed all the parts that you cited already of the PBP methodology or
42:21
there were some parts that were not useful or not
42:28
important the answer was no we have really to use
42:32
the PBP methodology the full PBP methodology if
42:35
we dilute it for example instead of delivering
42:41
the learning theoretical preparation with a benchmark at the end if we take out the benchmark
42:50
we already lose efficiency in the methodology
42:54
what it means it means that then we need more time
43:00
to have people reaching the proficiency benchmark
43:05
at the end of the training and it means that it is
43:08
less efficient and more expensive imagine to deliver the theoretical information about the
43:20
methodology the PVP methodology applied to that
43:23
technical training skill in the lab in and not
43:29
online it is an inefficient way to do that because
43:32
you use time in the lab so you have to rent the lab
43:36
you are probably less efficient in the
43:40
communication of the information and also you don't know really
43:47
if people are focused on the theoretical information that you are giving to them
43:54
they cannot review it online you can as a trainee you can review what you have already
44:03
watched you have a benchmark that you have to
44:06
pass at the end you don't need a training facility
44:12
to do that so it is really inexpensive it is efficient and there is no reasons to not use
44:20
this setting so what we found it is that the methodology should be used in the more
44:31
completed form as possible if you dilute if you remove some part of the methodology then it
44:37
becomes less efficient and in the background we
44:42
had also comparison with the traditional way of
44:45
training and we saw that it was by far the less efficient way to deliver the training so yes
44:57
the PVP methodology also if diluted is better than the traditional training but if we use the
45:03
PVP methodology why not use it in the complete
45:06
form yeah I remember discussing the results with
45:11
Yurio Masuroni who was working in cardiology and I was telling him I've shown him the data
45:18
and it took 14% longer to get to proficiency if you if he just removed the benchmark from the
45:25
online didactics and he said well it wasn't statistically significant he said yeah it might
45:30
have been significant for you he said but if you're a company like Medtronic and you know 14%
45:36
of five to seven hundred and fifty million a year which is what they spend on education and
45:43
training it doesn't have to be statistically
45:46
significant for you that's a huge amount of money
45:48
and that was an important point for me so proficiency based progression methodology is
45:57
in your experience a very powerful methodology for training robotic surgical skills
46:03
and probably surgical skills does PBP have a role to play outside robotic skills and surgical
46:11
training do other disciplines do they will they
46:14
accrue a benefit from using it yes I think it is
46:20
a methodology that can be applied in all the different surgical medical and also
46:28
non-medical disciplines there are already the
46:31
data demonstrating the capability of this methodology
46:36
of implement the way to do to do training to be honest I was really happy that we developed
46:46
together the basic surgical skills because that
46:50
that basic surgical skills using an inexpensive
46:53
model as the chicken model are valid and are important not only for urology are really
47:01
important for are for for all the robotic surgical
47:08
disciplines because also if you are a gynaecologist
47:11
also if you are a general surgeon also if you
47:14
are a cardiac surgeon using robotics it is really
47:19
required to acquire the suturing and knotting the coagulation the dissection so are really
47:27
training exercises that are valid for all the surgical disciplines and this is in my opinion
47:35
is a very very nice thing that we developed and
47:38
we did but other than the technical skills then
47:43
you have really to metricise that all the single
47:48
procedures this is obviously an effort that the
47:52
different surgical disciplines have to do they
47:56
have to find that surgeons that are really able to
48:03
create and to apply the methodology on the different surgical procedures but for sure
48:11
methodology can be used for all the surgical disciplines this will be a little bit a long
48:20
pathway because obviously the procedures that we
48:22
have to metricise that are a lot but in my opinion
48:27
when we will have the metrics for the most important the most common procedures at that
48:34
moment we will have already a decreasing in the
48:39
errors that can be done and in the learning curve
48:43
that is for for the most common procedures and it will be an implementation of the results
48:53
both for surgeons and for patients in the
48:56
different surgical disciplines yeah yeah i mean a lot of
49:00
clinicians think that we have to develop the metrics for every procedure not initially what
49:05
we need is index procedures you know that that
49:08
teach the important skills that are benchmarked
49:11
and so on and i think but i think that's going to take quite some effort what I've
49:17
what I've said before what probably is required
49:20
with the level of evidence of PBP and how effective
49:23
it is what's probably required is an effort something similar to the genome where we
49:29
metricise the index procedures for the different surgical disciplines for the different
49:34
procedure based disciplines and so on somebody,
49:38
I gave a keynote in Melbourne a few years ago and
49:43
somebody said to me Tony, if you've got all this evidence it's how come PBP is not wider
49:50
practiced and I said to them, I think that's
49:53
because of a failure of leadership but genuinely thats my view
49:57
and but I think that you're one of the
49:59
younger new generations that will come along that have
50:02
they're here that come along who will lead
50:06
the application and implementation of a proficiency
50:09
based progression approach to training Stefano as
50:13
always it's been a pleasure to talk to you thank you
50:17
Thank you Prof, for your guidance in all those years thank you for your support thank you
50:23
for this interview and i hope to see you soon in person