Latest / Lab to Lives / Towards a more Agile Vendor Qualification: A Discussion with Dele Babalola and Rani Naik

Transcript

Towards a more Agile Vendor Qualification: A Discussion with Dele Babalola and Rani Naik · Lab to Lives

• 0:21
  Speaker 1: Welcome to Transformation in Trials.
• 0:23
  Speaker 1: This is a podcast exploring all things transformational in clinical trials.
• 0:27
  Speaker 1: Everything is off limits on the show and we will have guests from the whole spectrum of the clinical trials community, and we're your hosts, Ivana and Sam.
• 0:35
  Speaker 1: Welcome to another episode of Transformation in Trials.
• 0:40
  Speaker 1: In this episode, we turn everything a little bit on its head.
• 0:43
  Speaker 1: I have two guests, Deli and Rani, and they'll introduce themselves.
• 0:47
  Speaker 1: One of them is Rani Naik, and she has a very unique vision for what a quality organization is supposed to be.
• 0:54
  Speaker 1: Can you tell us more about that, Rani?
• 0:57
  Speaker 2: Sure.
• 0:57
  Speaker 2: Thank you, ivana.
• 0:58
  Speaker 2: The quality organization that I aim to build is more of a GXP focused quality and it's got to be agile and phase appropriate to support the cross-functional teams in potential risk that you might see throughout clinical development and address them in a timely fashion.
• 1:22
  Speaker 2: Rather than building out a quality infrastructure that's too rigid or focused only on GMP or GCP In most organizations, you need a quality system that can span across the breadth and talk to each other so that any risks that are potential risks that are identified can be acted on by the cross-functional teams to mitigate or eliminate.
• 1:53
  Speaker 1: That is a very interesting vision for quality that differs from at least some of the quality organizations that I have witnessed in companies.
• 2:00
  Speaker 1: I really like that you take upon yourself the actual role of ensuring quality and not just pointing out the potential risks or problems that there may be to the quality.
• 2:13
  Speaker 1: The second guest on the show is Deli Deli.
• 2:18
  Speaker 1: Can you tell us more about yourself?
• 2:20
  Speaker 3: Hi Deli Babalala.
• 2:21
  Speaker 3: I have worked in the industry now for about 15 years and all of that for 15 years has been in data management, kind of stumbled into the industry, as seems to be the case for most people.
• 2:32
  Speaker 3: I've stayed since.
• 2:34
  Speaker 3: I have been fortunate to work in organizations that have pretty good data management setups.
• 2:42
  Speaker 3: I've seen a lot of the right way to approach things, but it's also interesting.
• 2:49
  Speaker 3: I've gone from that to the start of space and, more recently, been really surprised at how behind other organizations tend to be.
• 3:01
  Speaker 3: I remember coming to Boston in 2018 and finding out some people still collect their essays on paper.
• 3:06
  Speaker 3: I was just mind-blowing because I was like I haven't done this in 10 years.
• 3:09
  Speaker 3: Why are we doing this?
• 3:11
  Speaker 3: I think for me, just in general, it's seeing how we can move from being a reactive industry to a more proactive one the DM space.
• 3:25
  Speaker 3: One thing that's really huge a lot, which I think almost everybody does now is to have databases live before the first patient in.
• 3:33
  Speaker 3: For me, I think where we need to be is not just to have the EDC stood up, but all the systems, all the backend systems, should be stood up before the first patient comes in.
• 3:44
  Speaker 3: What tends to happen is, yes, you have database live before the first patient in, but then you don't have your SaaS listings or whatever you're using to check the data.
• 3:56
  Speaker 3: You don't have those running until months afterwards.
• 4:00
  Speaker 3: Then it's like now you're finding issues and then you're finding trends.
• 4:04
  Speaker 3: It's like trends you probably could have fixed early on, but it's like, okay, this is one year later and we found all these things that we really could have at least mitigated months ago, but all that data has gone to waste.
• 4:18
  Speaker 3: Now it's trying to figure out ways to be able to have all those things up and running before the first patient is screened.
• 4:28
  Speaker 3: It's getting harder and harder to do as studies get more and more complicated, because now there's a lot more data coming in through studies.
• 4:35
  Speaker 3: It's not impossible, but it definitely feels like that's not the priority in industry, because the focus is always on oh, just get these studies live.
• 4:44
  Speaker 3: Then the next thing is just get these studies locked.
• 4:47
  Speaker 3: It definitely feels like we're not really investing in getting right.
• 4:52
  Speaker 3: Coming back to quality is I still think in general we're still trying to inspect quality into our studies where we really should be pushing for that quality by design.
• 5:06
  Speaker 3: I think doing that would help that goal Absolutely.
• 5:14
  Speaker 1: Today we are carrying of marrying data management and quality, bringing you together to talk about a topic that emerged during the pre-call that you and I had, where we're just kicking the can around and talked about inefficiencies in the life sciences industry, specifically in the GXP space what could we do differently?
• 5:36
  Speaker 1: Where we both bonded about this idea that why is it that each individual from a studio company qualifies all their vendors?
• 5:43
  Speaker 1: That's a lot of redundant work.
• 5:46
  Speaker 1: Could it be done differently?
• 5:49
  Speaker 1: Why hasn't it?
• 5:50
  Speaker 1: That's the topic that we're going to be basing this conversation about, but daily maybe, in your own words, why do you think there's a potential for improvement there?
• 6:03
  Speaker 3: I think one of the definitions for quality and quality gets defined in a lot of different ways is the ability to meet requirements, something that's quality if you're able to meet requirements.
• 6:14
  Speaker 3: In a sense, if somebody asks you for three things, it's quality if you meet all three things Technically.
• 6:24
  Speaker 3: If you meet five, the additional two is a waste Technically.
• 6:30
  Speaker 3: They didn't ask you for that.
• 6:30
  Speaker 3: In that sense, I find if the goal is to have requirements match, then we really should ask ourselves are the things we're doing helping to meet those requirements, or they or they not?
• 6:47
  Speaker 3: Because if they're not, then it's just a waste.
• 6:49
  Speaker 3: And I find, like you know what's the value in different companies going to one company and asking them the same questions over and over again, like there can't be any value in it.
• 7:02
  Speaker 3: And for me that's what I find with you know audits right is even the quality folks on the vendor side it's like they're just jumping from one meeting to another meeting answering the same questions, right, and for me it just feels like we're just wasting our time.
• 7:23
  Speaker 1: And, Ronnie, for you.
• 7:23
  Speaker 1: Is there good reason for why all pharmaceutical companies ask vendors the same questions to get them qualified?
• 7:31
  Speaker 2: I think it.
• 7:31
  Speaker 2: Sometimes it becomes a box checking exercise, right?
• 7:34
  Speaker 2: So vendor management and we live in an environment where virtually everything is outsourced.
• 7:43
  Speaker 2: Most companies have an outsourced model, whether it's a smaller or a larger organization.
• 7:49
  Speaker 2: There's components that are outsourced, but we're all using that vendor for the same capabilities and services.
• 7:55
  Speaker 2: So there's no, there's very little value added in going and exploring a vendor to see if they have an infrastructure the QMS infrastructure that you want to use for the services that you're going to delegate to them.
• 8:08
  Speaker 2: It won't tell you anything that a questionnaire wouldn't do, or remote, even an abbreviated remote audit if you really wanted to make sure that it's an actual entity that's functioning.
• 8:21
  Speaker 2: I think there's more value added in making sure that they're they have the business continuity, they're they have the infrastructure in place.
• 8:30
  Speaker 2: How it's going to perform for your program, that's yet to be seen at a qualification.
• 8:36
  Speaker 2: So there's more value added evaluating them once you've started engaging with them and generating data, versus going in and checking a box.
• 8:44
  Speaker 2: So this is where I think you can be innovative in how you design your vendor management program.
• 8:52
  Speaker 2: So typically, a qualification might be done using a questionnaire or just a evaluation remotely, and then, once you actually engage with them.
• 9:06
  Speaker 2: You can do an onsite where you're evaluating your data against the QMS that should have performed for your program Right.
• 9:15
  Speaker 2: So there are different ways.
• 9:17
  Speaker 2: I think it can be approached, but right now I'm not sure of the percentage.
• 9:22
  Speaker 2: But majority of the industry tends to do a qualification where you're all going in asking the same questions to Deli's point and evaluating the same capabilities.
• 9:36
  Speaker 3: Right, and it's kind of like I think you're going to hit us on something I was thinking about which was you kind of it's like when you interview people like I forget where they were talking about if you had a chance to interview a person or other people that interact with that person, who would you rather speak to?
• 9:55
  Speaker 3: Right, because if you asked me about me, I would tell you I was the best data manager in the world.
• 10:01
  Speaker 3: Right, my upstains might feel different, my statisticians might feel different, and I think in the end you know, because I tell people this all the time when they say you know, have you worked with this CRO before you work with that CRO?
• 10:11
  Speaker 3: And I think, look, everybody's SOPs are pretty much saying the same thing, like we're all trying to do the same thing.
• 10:16
  Speaker 3: The real, you know, the taste of the pudding is in eating.
• 10:20
  Speaker 3: It's when we kind of get in this relationship.
• 10:23
  Speaker 3: That's when I find out are those just words on paper in your SOPs or you know, how do you guys actually do execute?
• 10:31
  Speaker 3: And what it got me thinking was we have qualification on it and then we have a re-qualification on it.
• 10:37
  Speaker 3: You know where we're kind of ordering them again and I wonder, like.
• 10:40
  Speaker 3: Doesn't that then prove that we don't have the right metrics in place to be kind of continuously monitoring our vendors so that, like I always say like for me when I have meetings with my team, I don't want I shouldn't be asking how we're doing.
• 10:57
  Speaker 3: I should know how we're doing.
• 10:59
  Speaker 3: The conversations should be more around why are we where we are?
• 11:04
  Speaker 3: Because I know where we are and I think it's funny that re-qualification audits tend to be where we find out issues, which for me, is a red flag, because I think you know, like two years in a relationship, we should already know where we are and you know.
• 11:20
  Speaker 3: Those conversations should be around.
• 11:22
  Speaker 3: You know how we fix this or how do we improve on this?
• 11:26
  Speaker 3: Do you get what I mean?
• 11:27
  Speaker 3: So I really think we're kind of putting all our energy in the qualification process and not enough of it in the how do we actually track what we're doing in real time?
• 11:40
  Speaker 3: Yeah.
• 11:42
  Speaker 3: And again, being reactive and not, you know like not being proactive with these engagements.
• 11:50
  Speaker 2: Absolutely.
• 11:50
  Speaker 2: And I think this goes back to what we were saying Vendor qualification can't be confused with vendor oversight.
• 11:57
  Speaker 2: There are different activities.
• 12:00
  Speaker 2: Qualification, again, sometimes can be just that checking the box.
• 12:04
  Speaker 2: The oversight is where you should be defining your metrics for how you're going to actually monitor the quality of what you're getting Right.
• 12:13
  Speaker 2: So I think sometimes to deli's point.
• 12:15
  Speaker 2: If we can refocus that energy that goes into a qualification into defining some of those metrics, again being more proactive, then you're hopefully aligned between the sponsor and the vendor On the expectations of what's going to be so, what's going to be delivered, how it's going to be delivered, the quality of what's going to be what it's going to be delivered.
• 12:41
  Speaker 2: And again, qualification, why you're all looking at the same thing, the other piece that it picks up, it should pick up on.
• 12:48
  Speaker 2: But again, even the qualification audit is now source activity, so it's usually a contractor, so they're not going to know the details of how your infrastructure as a sponsor is set up, right.
• 13:01
  Speaker 2: But I, in an ideal world, a qualification audit could potentially detect where those gaps are between your infrastructure as a sponsor and the vendors infrastructure.
• 13:14
  Speaker 2: Because if they're not seeing, I die.
• 13:16
  Speaker 2: You're going to have those, those gaps.
• 13:19
  Speaker 2: You're going to see those deficiencies start showing up as you execute with activities with them.
• 13:26
  Speaker 2: In order to mitigate that, one way is you could define those metrics, whatever they are going to be.
• 13:35
  Speaker 2: So that's one way to mitigate that potential gap.
• 13:39
  Speaker 2: But that's again, it's a proactive way of thinking, versus waiting for the data to roll out and then start seeing a trend and saying, hey, there's a potential trend here that we need to investigate.
• 13:53
  Speaker 1: But could there also be a difference between vendor qualification, where the usual is the quality function that performs this and performs this well or has an outsourced model for performing it, and vendor oversight, which I assume would fall to the department that's actually responsible for the vendor and may be less experienced in vendor management than this audit machine that we may have built in quality?
• 14:20
  Speaker 2: I think an ideal qualification should include both the quality as well as the functional subject matter expert, because that would be then your quality individuals evaluating the QMS infrastructure.
• 14:34
  Speaker 2: The functional subject matter expert would be looking at the technical details.
• 14:40
  Speaker 2: So in an ideal world that's how it would work.
• 14:43
  Speaker 2: But then again it's always a timeline driven industry so you don't always have that luxury because the functional group is involved in either getting the systems up and ready or the project up and ready, so it's usually left to the quality and vendor management or procurement whatever the group is titled in any organization to perform that audit as a qualification.
• 15:10
  Speaker 2: But the most successful audits I've seen are when it's a collaboration between the quality and the functional group to evaluate, with either a qualification or a review of the data, because you're both looking at it from different perspectives and you'll have a, I think, more valuable outcome when it's the two functions.
• 15:35
  Speaker 1: Dilly, when have you seen the strongest outcome of a vendor onboarding situation?
• 15:42
  Speaker 3: For me.
• 15:42
  Speaker 3: Most times I find the qualification piece like so, owners, and again it's a resource thing, right.
• 15:50
  Speaker 3: It's like there's only so much time.
• 15:52
  Speaker 3: You have to do a million things all at once, right.
• 15:55
  Speaker 3: And I think that's the whole point of if we have only this much time, how do we most efficiently use that time to do all the things that need to get done?
• 16:06
  Speaker 3: So, like Ronnie was saying, typically when we do the audits we'll sit with the quality folks.
• 16:13
  Speaker 3: We'll kind of tell them pay attention to this, pay attention to that.
• 16:18
  Speaker 3: But one thing I try to do, even through the audits, is I try to get familiar with a vendor's process Again, to kind of find those gaps where it's like, okay, this is what they do.
• 16:28
  Speaker 3: There seems to be something they don't do that I think that they should be doing.
• 16:32
  Speaker 3: So maybe that's something either we get them to fix in their process or that's something that we'll need to make sure that we're doing an R end so that we're kind of covered.
• 16:46
  Speaker 3: And it's interesting because I think I'm thinking about it now.
• 16:51
  Speaker 3: It's like the qualification.
• 16:52
  Speaker 3: We can rethink how we do that, maybe to try to reduce the burden from it.
• 16:58
  Speaker 3: But I almost feel like the oversight piece is probably the most critical part, and not even so much the qualification, because in the end, I think what we tend to use the qualification for is like those are the gotcha moments where it's like we want to go see if they're really doing what they should be doing, which is like that's what we should be doing every day, right, not like every two years trying to see if they kind of like drop the ball in there.
• 17:23
  Speaker 3: And I think, especially like if we're in that outsource model, we need to figure out how to make the oversight process kind of almost automatic, where it's kind of like baked into what we're doing.
• 17:37
  Speaker 3: However, you're able to do that, that you're able to be on top of what your vendors are supposed to be doing, so that I almost feel like if you're doing great oversight, you probably should never have to re-qualify anyone.
• 17:53
  Speaker 3: Like if you really think about it, like what's a re-qualification?
• 17:56
  Speaker 3: Like really, for me it suggests that you're not really you know, like you're not doing oversight.
• 18:04
  Speaker 3: That's why you need to re-qualify them, right, you gotta make, because it's one thing to qualify someone because this is the first time you want to kind of get involved with them.
• 18:12
  Speaker 3: But if you're working with them, why are you re-qualifying them, like really you know?
• 18:18
  Speaker 3: And then if you're re-qualifying them, why are you finding out things you haven't noted the whole time you're working with them?
• 18:24
  Speaker 3: You're not doing something that's going wrong somewhere, right?
• 18:28
  Speaker 3: So I really think it's about figuring out what kind of processes we need to have in place as the sponsor that actually add value, that help us achieve what we want to achieve, which is we want to be able to be certain that our vendors are doing you know, like, what they should be doing.
• 18:47
  Speaker 3: And I was, while you were kind of saying we usually sometimes even outsource the auditing, I kind of was thinking of a biggie rhyme and I'm gonna.
• 18:55
  Speaker 3: He goes.
• 18:57
  Speaker 4: I've got lawyers watching lawyers but that's what we're doing, right, we've got vendors watching vendors.
• 19:04
  Speaker 3: You know, and it's really interesting, you know, come to think of it which I mean there is some safety there.
• 19:08
  Speaker 3: You know, you kind of go to one person to, but it doesn't absorb us of our responsibility At the end of the day, where the ones, the regulators, are going to come to if something gets missed.
• 19:18
  Speaker 3: So I think we need to figure out how our processes, either, you know, address gaps in our vendors processes or you know we figure out processes that help us, you know, like find a really efficient way to oversee what our vendors are doing.
• 19:37
  Speaker 1: And what if we had a pool of vendors that were?
• 19:40
  Speaker 1: This is funny.
• 19:42
  Speaker 1: I see this every time I go to a conference.
• 19:44
  Speaker 1: Some vendor puts on a poster that says we are FDA approved.
• 19:47
  Speaker 1: Like well, there's no such thing, but okay.
• 19:51
  Speaker 1: But what if there was?
• 19:53
  Speaker 1: What if there was a pool of vendors that were maybe not FDA approved but somehow a life science industry approved?
• 20:01
  Speaker 1: Would that help in establishing the initial collaboration?
• 20:06
  Speaker 3: Right, I mean for me, like I see that I think maybe that would help us get rid of a lot of luck, right, we?
• 20:14
  Speaker 3: You know we may still have questions that maybe have not been answered, but at least it would.
• 20:19
  Speaker 3: I'm trying to think of it like.
• 20:21
  Speaker 3: A good analogy here is like if I go and I buy a car, typically a car will kind of cover you know, I don't will cover certain things.
• 20:30
  Speaker 3: If I need something additional, then I can ask for, like an upgrade and say, you know, I want it to also do this, and I think that's what that would allow, like if something was like ISO certified, you kind of know what you're getting from it to a certain extent, and if you need something more, then you could do that.
• 20:45
  Speaker 3: So I definitely think that might be a way to kind of reduce the burden of qualifications.
• 20:51
  Speaker 3: And you know, because the way I'm looking at it is, if a vendor only needs to go through this really thorough qualification from this body that certifies them, then they can, you know, put all that effort into that one qualification exercise.
• 21:05
  Speaker 3: And if me as a sponsor wants to use that vendor and they say, hey, you know, like we were audited by this body and you know, like these are all the vendor responses.
• 21:15
  Speaker 3: Then I could just kind of flip through that and say, okay, they've answered all the questions I have.
• 21:20
  Speaker 3: Or maybe I have an additional question that's not on kind of like the menu, and then they can just speak to that.
• 21:27
  Speaker 3: You know, like specifically right, I think you kind of save a lot of people, a lot of time, and then maybe quality groups then can focus more on putting together effective vendor oversight plans instead of just kind of managing audits all the time.
• 21:47
  Speaker 3: Right, and for me, like the whole targeted Sdb thing, it's a great analogy, right, because Ronnie and I went, went, but with the same company we had a conversation about TSDB and I said, look, every time I've seen TSDB implemented, they say, oh, we're only going to Sdb, the primary endpoints, the secondary endpoints and the safety endpoints.
• 22:04
  Speaker 3: By the time you write all that step down, you're already at 95%.
• 22:07
  Speaker 3: So it's like, what target are we talking about?
• 22:10
  Speaker 3: Like, we're already, it's almost 100%.
• 22:12
  Speaker 3: So like, this is obviously not working.
• 22:14
  Speaker 3: So you have to figure out a way to really scale it back and say we're not going to look at vitals.
• 22:19
  Speaker 3: And the reason we're not going to look at vitals is because we have a gazillion edit checks that look at that data so that it's highly unlikely that we're going to have discrepant data.
• 22:32
  Speaker 3: That's that's critical.
• 22:35
  Speaker 3: That's a huge deal.
• 22:36
  Speaker 3: You try to then find ways to free up a CRA so that when they go to the site maybe now they're doing 30% SdV and then the additional time they have they can then spend that time doing stuff.
• 22:50
  Speaker 3: That's arguably more value add.
• 22:53
  Speaker 3: I think it's the same idea I think I have with qualifications.
• 22:56
  Speaker 3: It's like let's try to reduce the burden on vendor qualifications so that we can actually focus on what I think is more important, which is the vendor oversight piece.
• 23:08
  Speaker 3: I think that's where we're really going to get back for a buck, not like this 21 questions before I get in a relationship with you, which for me it's kind of what we're doing with the qualification audits.
• 23:19
  Speaker 1: But Rani, why doesn't this exist already?
• 23:21
  Speaker 1: I think it exists in.
• 23:23
  Speaker 2: There's different forms that do have audits of vendors out there that you can purchase, but I think there's a reluctance from the industry from a sponsor oversight we like going back to my earlier comment we equate sponsor oversight with vendor qualification.
• 23:44
  Speaker 2: That is not the only piece.
• 23:48
  Speaker 2: I think the more value added it's the entire vendor management oversight.
• 23:53
  Speaker 2: Qualification is such a small piece.
• 23:56
  Speaker 2: Even the due diligence going through an RFP for a vendor, evaluating whether they have the capabilities, what the costing structure might be, that whole process.
• 24:07
  Speaker 2: Vendor qualification is just a single dot on that line but we pay a lot of emphasis to the qualification, whereas all of the other pieces if you've put in enough effort and put in the right evaluation pieces, components should work.
• 24:31
  Speaker 2: The qualification tells you again that they have a QMS, but you already know that because you've shortlisted the vendors and either it's through prior experience, either as a team or through the individuals that have worked with them, you know that they're a viable entity.
• 24:53
  Speaker 2: They're not just a website that's not going to perform.
• 24:58
  Speaker 2: You know that they've been in the business because they're credible.
• 25:04
  Speaker 2: So you're not assessing any of those elements, I think where even the due diligence conversations, and that should give you enough oversight into what their SOP infrastructure looks like Not the details, but you might you'll know that they have a framework in place.
• 25:23
  Speaker 2: So qualification at that point is essentially done as you have gone through the RFP process, where there's probably more value added is now agreeing on, and you put in additional plans or agreements in place, like quality agreements with some of the critical vendors, where you're saying, okay, this is how we expect to engage, this is escalation.
• 25:48
  Speaker 2: But we never define metrics.
• 25:50
  Speaker 2: The metrics are never defined in a quality agreement.
• 25:55
  Speaker 2: The metrics are never defined in study plans Because we're hesitant to start putting those into establishing thresholds or even just the framework for metrics.
• 26:10
  Speaker 2: So I think there's more value added in saying, okay, this is how we're going to engage, these are the things that we're going to start monitoring.
• 26:16
  Speaker 2: So then there's alignment from both parties to say, okay, this is how we're going to have to perform, this is our threshold and those are better quality defining procedures, I guess, versus the qualification audit.
• 26:36
  Speaker 2: So there are reports out there.
• 26:37
  Speaker 2: We'll go back to your question.
• 26:38
  Speaker 2: There are reports out there you can purchase for different vendors, both in the GCP and GMP space, and again, if they've been audited by a regulatory agency, they usually have that as a statement that they've been inspected.
• 26:57
  Speaker 2: So if they have a prior audit, regulatory inspection history, is there value added in you going in and then the sponsor qualifying them.
• 27:10
  Speaker 2: Those are the things to take into consideration when you risk rank them as well, to say okay, if they're a viable, credible organization, they've had inspection history.
• 27:22
  Speaker 2: If they've been in business for a while, there's different criteria you can use to say okay, while they're a very key or a critical vendor for us, their risk rank is still low because of all of these factors that you know are there already.
• 27:46
  Speaker 2: So you can design your vendor program how you need it to be, where your focus is going to be on the right activities rather than just check boxing, checking a box or having saying okay, this is now done and we can consider them qualified.
• 28:11
  Speaker 1: It also comes down to the collaboration with the vendor, like you were saying before how we shouldn't find things that we didn't already know.
• 28:19
  Speaker 1: When we re-qualify a vendor, then something's gone wrong In your experience.
• 28:24
  Speaker 1: How can you build that sort of collaboration where you actually work together and understand well where are the gaps and how can we work around those?
• 28:36
  Speaker 3: And I think for me it's about relationships, it's about data, because if everybody knows it's a running sport, everybody knows what metrics we're measuring, then when we come to these conversations, then we already know are we doing good or are we not doing good?
• 28:55
  Speaker 3: Right, because we set the expectations, like, we want queries answered within five days, we want data entered within five days, and if we're not tracking, then the issue is what the issue is.
• 29:08
  Speaker 3: So it's not me surprising my vendor asking for something that wasn't a clearly defined requirement.
• 29:19
  Speaker 3: So I think you define what the expectations are and then you work with your teams to address whatever issues that come up as studies unfold.
• 29:31
  Speaker 3: And I think for me, what's most important to me with vendors is just how collaborative they are.
• 29:38
  Speaker 3: Right, we'll always have issues.
• 29:41
  Speaker 3: So, looking for a vendor that everything's just going to go smooth, that never happens.
• 29:46
  Speaker 3: So it's like, yeah, we're going to run into issues, but I want transparency.
• 29:51
  Speaker 3: I want you guys to let me know as soon as you find something.
• 29:54
  Speaker 3: I don't want to be you on finding it when you really should have caught it.
• 29:59
  Speaker 3: You get what I mean.
• 29:59
  Speaker 3: Or maybe I'm catching something that we didn't tell you to be looking for, then that's on me Now, you know, to look for it.
• 30:05
  Speaker 3: Then you can kind of I wouldn't have to bring it up the next time.
• 30:09
  Speaker 3: And I think that that's just something to you kind of have to build which of vendors is set the expectations and kind of like work through that relationship.
• 30:17
  Speaker 3: Because a lot of times I've been in companies where it's like, oh, it's not working, we want to go with somebody else, yeah, right.
• 30:23
  Speaker 3: And it's like, yeah, like there's no, you can't be a Goldilocks with the three there.
• 30:28
  Speaker 3: It's like there's no perfect vendor out there, right, you know you're going to have to work through some issues and hopefully come out, come up better for it.
• 30:38
  Speaker 2: And long that.
• 30:39
  Speaker 2: I think one of the key pieces that we expect the qualification audit to do is to evaluate the infrastructure and know what SOPs are there.
• 30:49
  Speaker 2: But going back to my earlier point, this is not somebody that's part of the team.
• 30:54
  Speaker 2: This is a contract auditor that's evaluating the SOPs.
• 30:57
  Speaker 2: So I think where there's more value added is the subject matter as a subject matter expert, or the functional team, to know the SOPs on how you're going to engage with the vendor, right?
• 31:09
  Speaker 2: So if you're looking at a process from end to end, as a sponsor, you have oversight of that entire process.
• 31:16
  Speaker 2: But there might be key pieces where you're going to be also a reviewer or signatory or saying, okay, this milestone now has been met.
• 31:25
  Speaker 2: You're not going to know that unless you know the SOPs of the vendor that you're overseeing.
• 31:31
  Speaker 2: So the qualification is an independent assessment, but it's also removed from the functional area.
• 31:41
  Speaker 2: So while it's an independent assessment, there's no connection there, there's a gap there.
• 31:50
  Speaker 2: So I think it's more value added to Deli's point when you're looking at it as a collaboration or strategic partnership, to know the SOPs, to know the process, how it's going to work, from the even the sponsor vendor interactions.
• 32:07
  Speaker 2: So defining metrics, aligning on expectations that all has to happen from an oversight perspective rather than a qualification.
• 32:20
  Speaker 1: Well, I would like to ask both of you the same question.
• 32:23
  Speaker 1: That is well.
• 32:24
  Speaker 1: If it is pretty clear that a lot of the qualifications that we do are more or less a qualification theater, what is the key barrier for removing that?
• 32:37
  Speaker 1: What thing could we stop doing?
• 32:39
  Speaker 1: Whose minds do we need to change?
• 32:41
  Speaker 1: What is keeping us from arriving at a better way of operating?
• 32:46
  Speaker 3: Honestly, I would say time.
• 32:48
  Speaker 3: There's just never enough time to do all the things that we really need to do.
• 32:56
  Speaker 3: And I think the point of running just made, where, for you to really work well with a vendor, you really have to know their process Right.
• 33:07
  Speaker 3: So it's not just good enough for us to write R and SOPs and no R and SOPs.
• 33:11
  Speaker 3: You really have to know their SOPs Right.
• 33:14
  Speaker 3: And that also then lends to how many SOPs do you want to read, right?
• 33:22
  Speaker 3: So the idea that you're going to keep switching vendors already is telling me you're setting yourself up for failure, right.
• 33:32
  Speaker 3: So, again, we're always managing risk, right.
• 33:35
  Speaker 3: You never eliminate risk, because if you stick with one vendor, then you go with your eggs and one basket.
• 33:39
  Speaker 3: But there's a pretty thick line between one vendor and having a vendor for every single project that you run.
• 33:47
  Speaker 3: So I definitely think it's really kind of sit down and really thinking of this vendor management process.
• 33:54
  Speaker 3: Qualification is part of it, but then the oversight I think it's the biggest part of it.
• 33:59
  Speaker 3: So it's trying to figure out how do you put processes in place that we can actually follow right, and taking a risk approach to it, because we can't do everything we would like to do, but then we kind of really need to think about what are the things that we should absolutely do and I do still come back to time, though and what we choose to prioritize right, because a lot of times it's go go, go, go, get the next project alive, get this thing done and because of that, we're not taking the time to think through these processes, think through how can we work better, and for me, I think that's the biggest constraint is there's just never enough time to put everything that's needed in place for things to work as they should.
• 34:51
  Speaker 3: At least from a startup company space, that's usually the challenge, where there just isn't enough time, because you're juggling so many different things.
• 35:02
  Speaker 2: I think from that's an interesting perspective.
• 35:05
  Speaker 2: I hadn't thought of it actually from that angle, but I do believe time is one of the key factors, but I think it's also as an industry.
• 35:15
  Speaker 2: We just need to change slowly, evolve.
• 35:18
  Speaker 2: That takes time and it also takes momentum, but I do see it, especially with the risk-based approach to qualifying vendors.
• 35:28
  Speaker 2: I think more, more organizations are willing to, you know, embrace questionnaires as an audit medium, With COVID obviously being, I think, a necessary disruptor for a lot of ways of how we work.
• 35:49
  Speaker 2: Remote audits now are a viable option where they weren't before, With the exception of labs or manufacturing facilities where you want boots on the ground.
• 35:58
  Speaker 2: But again, you can also argue if they've already been qualified by a regulatory agency, is there any value added?
• 36:08
  Speaker 2: You know majority of the labs are part of the BiMO program where they're evaluated preclinical labs, bioanalytical labs.
• 36:16
  Speaker 2: Majority of the CMOs have gone through a PAI inspection.
• 36:21
  Speaker 2: So is there really value added?
• 36:24
  Speaker 2: So I think those are the questions you need to ask yourself as an organization and then evaluate is it more valuable to do a qualification or do an audit of an organization six months into a project where you now have some viable data to evaluate?
• 36:42
  Speaker 2: So I think, as an industry, we need to ask those questions of ourselves.
• 36:48
  Speaker 2: And going back to Delia's point on time building in those timeframes where, rather than allotting the timeframe for a qualification, we repurpose that now to say, cross-functionally.
• 37:02
  Speaker 2: These are the metrics we need to agree on for this vendor that are going to be crucial for how we're going to start gauging this is vendor performing.
• 37:12
  Speaker 2: So those are different ways of, I think, looking at vendor management oversight that we should be evaluating.
• 37:25
  Speaker 3: And I kind of want you to speak out of this as well is are the regulators doing enough?
• 37:29
  Speaker 3: Right, because at the end of the day, it's their requirements we try to meet.
• 37:35
  Speaker 3: Do you know what I mean?
• 37:37
  Speaker 3: And a lot of times you kind of like FDA guidance and it's like, okay, tell us what we should do.
• 37:41
  Speaker 3: And it's like, oh, we don't want to tell you what you should do.
• 37:43
  Speaker 3: And it's like do you know what I mean?
• 37:45
  Speaker 3: And for me, I think it's a bit I don't want to use the word cowardly, but everyone's trying to kind of not be the one who takes the blame if something doesn't work out.
• 37:57
  Speaker 3: So then you know, like, if they're saying, oh, we want you guys to take a risk-based approach, maybe they need to provide the framework and not be too timid with saying this is what you should do, like they really should be able to give some sort of you know like a framework and say you know what, you absolutely need to do this.
• 38:17
  Speaker 3: Like you know they've said, hey, you absolutely need to have PIs sign up in casebooks, which I think is total nonsense.
• 38:22
  Speaker 3: But you know, like, think about guidance that you can provide these companies with so they feel more comfortable taking some of these risks that you're saying it's okay for them to take the risk.
• 38:34
  Speaker 3: I mean, I don't know what guidance you provide, but I think a lot of times you know it's their guidance that we want to meet, so that whether all this fluff we're doing, it's not just because we like doing fluff.
• 38:47
  Speaker 3: We're doing the fluff because that's what the regulators seem to want, right, and so maybe there's something that's lost in translation there, but we wouldn't be doing fluff if they didn't seem to like fluff, right.
• 39:01
  Speaker 3: That's the reality there.
• 39:02
  Speaker 2: So I think one of the things that, with the regulations and the guidelines that are provided, they're intentionally pretty vague or give you the boundaries within which you need to operate, right, when you talk about from a risk-based risk tolerance as individual, it's based on the organization and it's there's a business component to it, there's the program and then there's the data and the safety pieces, the elements, the compliance aspects that we should be considering.
• 39:37
  Speaker 2: So when you, when you talk about risk-based approach, I think it's evaluating your internal program and how can you defend it from from end to end, right, I think that's that's where, when you say risk-based, it's defining your risk tolerance, knowing your risk registry and journaling, knowing your potential risks at a program level, business level, study level, data level, and then tying them all together and then Creating your own program.
• 40:12
  Speaker 2: Is how I look at it.
• 40:14
  Speaker 2: While we may want the regulators to be more prescriptive, then it's going to work for some, not all.
• 40:22
  Speaker 2: Some of the guidelines are vague because it's intended to work for the entire industry as a whole.
• 40:30
  Speaker 2: If you're a virtual company versus a large pharma, you want to have the flexibility to say, ok, this is the program that I want to defend.
• 40:40
  Speaker 2: So I think that's where I would say, looking for the regulators, rather than looking to the regulators to say this is how we want you to do it.
• 40:50
  Speaker 2: I think we as an industry, we can always say, ok, this is the industry.
• 40:55
  Speaker 2: There's also the industry standard you should be adhering to.
• 40:57
  Speaker 2: So I think we have that.
• 41:00
  Speaker 2: We can define that, what that looks like.
• 41:05
  Speaker 2: And then the regulators obviously, that gives them confidence that we as an industry have given this some enough thought to say this is the process that we want to follow.
• 41:17
  Speaker 2: So that's how I look at it from a quality perspective.
• 41:21
  Speaker 2: Rather than having it prescribed down to me, I want the flexibility to say, ok, this is the framework and this is the process, this is the overall infrastructure, and then defend it.
• 41:32
  Speaker 2: Up to the regulators to say does this now meet your expectations?
• 41:37
  Speaker 2: And in majority of inspections that have either participated in or led teams on, that is what has successfully worked, versus saying, ok, this is your regulations.
• 41:51
  Speaker 2: But the regulations are always at the forefront, you are always aligning to them.
• 41:59
  Speaker 2: But I think it's better to be able to define what that looks like rather than having it superscripted.
• 42:08
  Speaker 3: Yeah, I'm not saying to like, if you look at that RBQM model, how long have we been doing this RBQM piece and everybody's still kind of figuring it out.
• 42:18
  Speaker 3: If it's true.
• 42:21
  Speaker 3: So you may know it's kind of like the FDA guidance on UC trials where it's a recommendation you don't have to do it.
• 42:31
  Speaker 3: But we kind of know If we don't do it then we need to come up with a really strong rationale.
• 42:36
  Speaker 3: So maybe it would be useful to have use cases and say, hey, here's an example of how you could run it and you translate may have certain solutions for certain things.
• 42:49
  Speaker 3: Avoka might hurt certain things.
• 42:51
  Speaker 3: So you can kind of go through Avoka and kind of pull up a bunch of like KRIs and stuff.
• 42:57
  Speaker 3: So I'm not saying go to the extreme where it's top down and they say this is exactly what you want to do.
• 43:03
  Speaker 3: But I think the other extreme of it, where it's like, oh, we're just going to be loosey-goosey with the whole thing and kind of just tell you liquid, that's wet and that's the only guidance you give, there's a lot of room in between those two approaches.
• 43:19
  Speaker 3: So it's just like if we are serious about because I just felt like, if we're all kind of just like working in the dark, like we're wasting a ton of time, we're wasting a ton of money, drug prices are not going to come down if everyone keeps making the same mistakes over and over and over again.
• 43:39
  Speaker 3: So something needs to change somewhere.
• 43:43
  Speaker 1: It sounds like running your approach depends on having that internal person who can take a stand and say well, this is how we're going to do it here, this is what we believe is sufficient or enough, or going to get us the quality you need.
• 43:56
  Speaker 2: Yeah, and I think this again goes back to the time, because how many times do we do an internal process flow?
• 44:04
  Speaker 2: Right, but unless there's an issue that comes up, that's when the investigations happen and we actually like backtrack to say, ok, what is our process, how did we arrive at this?
• 44:16
  Speaker 2: That's when we identify the potential gaps.
• 44:18
  Speaker 2: If we all spent energy in those kind of activities to say internally, this is our process, that, back to your point, will also help us design an overall program, even the risk base.
• 44:33
  Speaker 2: That's if you do an entire process flow.
• 44:39
  Speaker 2: That's when you could say, hey, there's redundancy here, can we eliminate this?
• 44:42
  Speaker 2: There's a gap here.
• 44:44
  Speaker 2: So we wait for the event or the issue to happen before we actually go back and evaluate.
• 44:51
  Speaker 2: So getting into, I think, as an industry, if we get into more of a proactive industry where we're actually giving enough time and energy to the activities, that will add value versus time-based, there might be some value in that.
• 45:16
  Speaker 1: Well, as we start rounding off this episode, we always ask our guests the same question in the end, and that is and I'm going to ask both of you this question If we gave you the special transformation in trials magic wand that has the ability to change one thing in the life sciences industry, what would you wish for?
• 45:37
  Speaker 2: I think the one thing I might ask for is more transparency and sharing of challenges and issues, because we, as an industry, we all face the same challenges.
• 45:49
  Speaker 2: Quality events by design are happening because we're all making the same mistakes.
• 45:56
  Speaker 2: So maybe not so much on the data side the data sharing happens because there's a scientific aspect to it but the challenges we face or the issues that we might encounter from a process perspective more transparency so we can all learn from each other.
• 46:20
  Speaker 3: No, we don't even need a magic wand for that, but things being what they are looks like we do.
• 46:28
  Speaker 3: I would honestly wish I would try to get people to be a bit more open-minded, because sometimes when I like conflict because for me if you agree with me, then it's boring I want to hear something else.
• 46:46
  Speaker 3: If I say A, I want you to say B and I want you to tell me why you think it's B.
• 46:50
  Speaker 3: And a lot of times I've been in positions where people say we're not going to do that and I'm like why?
• 46:56
  Speaker 3: And they can't tell me oh, this is why we think that's a bad idea and a lot of ways it's a lot of oh, we've always done it like this or that's just totally different.
• 47:09
  Speaker 3: But I really wish, as an industry, we did more than talk about innovation and we actually tried things I mean, obviously within the bounds of not too crazy, but that people really put their money with it.
• 47:33
  Speaker 3: Mouths are in terms of trying to really push the envelope, to speed things up.
• 47:38
  Speaker 3: I think that I definitely would like to see a lot more of and there are people doing that, but I would say probably not enough.
• 47:47
  Speaker 1: Well, those are both very good wishes and transformation trials.
• 47:53
  Speaker 1: Grant your wishes, at least to our ability.
• 47:56
  Speaker 1: If our listeners have follow-up questions or want to reach out to both of you, where can they find you?
• 48:05
  Speaker 2: Mon LinkedIn.
• 48:05
  Speaker 2: You can find me and ask any questions through messages.
• 48:10
  Speaker 3: Yeah, castify me on LinkedIn if you're looking for clinical research stuff.
• 48:14
  Speaker 3: I don't talk, walk or any other platform stuff.
• 48:20
  Speaker 1: OK, well, thank you so much both for joining this episode.
• 48:23
  Speaker 1: This was an awesome conversation.
• 48:25
  Speaker 3: Yeah, thanks, ivana, and welcome to the US.
• 48:30
  Speaker 1: Thank you.
• 48:31
  Speaker 3: Hope you enjoy your stay.
• 48:32
  Speaker 3: Is this your first time Staying?
• 48:36
  Speaker 1: Staying Actually, no, I used to.
• 48:37
  Speaker 1: I lived in Texas as a child.
• 48:39
  Speaker 1: Oh wow, Completely different experience.
• 48:44
  Speaker 3: Very different.
• 48:45
  Speaker 3: Definitely not San Francisco.
• 48:49
  Speaker 1: Could be different countries.
• 48:52
  Speaker 2: Oh, thank you very much.
• 48:53
  Speaker 2: This was a great conversation.
• 48:56
  Speaker 3: Yeah, this was fun.
• 49:02
  Speaker 1: You're listening to transformation in trials.
• 49:04
  Speaker 1: If you have a suggestion for a guest for our show, reach out to Sam Parnell or Ivana Rosendale on LinkedIn.
• 49:12
  Speaker 1: You can find more episodes on Apple Podcasts, spotify, google Podcasts or in any other player.
• 49:18
  Speaker 1: Remember to subscribe and get the episodes hot off the editor.