Latest / find your cause / The Missing Piece in Most Nonprofits With Tania Rempert

Transcript

The Missing Piece in Most Nonprofits With Tania Rempert · find your cause

• Joseph Smalzer: Everybody was just running on their own hamster wheels or everybody was just working in their own silos and working very hard and doing their work every day, but not in a cohesive, impactful way where they're able to improve their impact and prove their... So great. So thanks everyone for tuning in to this episode. I had a really awesome opportunity here to interview somebody that's in the neighborhood that I live in, which is amazing. So Tanya Rumpert is the founder and board president of Planning Implementation and Evaluation org or PI. And Tanya and I are to have a great conversation. Tanya, thanks so much for your time today. Thank you for having me. Yes. So per usual to start our interviews, we like asking about the mission of the organizations that we're interviewing. So what is the mission of PI? Yeah. So Planning Implementation and Evaluation org builds the capacity of mission-driven organizations to evaluate impact, promote learning, facilitate equitable change, all of that to mean we help other nonprofit organizations identify, define, measure, report, and improve their outcomes. Awesome. And as we were talking about before I hit record, we've interviewed a lot of nonprofits and I feel like this is one of the most common reoccurring challenges that a lot of these organizations face. So I'm excited to dive into it. But before we do, why you? Why is Pi one of the organizations, the organization that you're spending so much of your time and dedicating towards? Oh, that's so funny. So I grew up in a family of professional volunteers, and my mom and dad volunteered for all these organizations. My dad was a disabled American veteran. My mom volunteered a lot with the church. then for a lot of organizations in the neighborhoods where we grew up on the Southwest side of the city. And every weekend, we were either canning on the street corner for some organization, or we were doing a fashion show as a fundraiser for something, or we were at some kind of county fair at some table advertising, some nonprofit. And so I became really aware of how to do service and different organizations that made an impact. And there were some organizations that even as like a young preteen and teenager, I could tell that there were some organizations that just gave away t-shirts and cookies, right? But there was some organizations that really made a difference. And you could tell by the community that it created and by the level of investment in human capital, right? I'm not talking about big dollars. I'm talking about, you know, like small. organizations in neighborhoods, right? But people were really invested with their time, you or their talent, even though they didn't have a lot of money to give. I felt that there was a difference, right? So anyways, I go off to college and I think I'm gonna, you know, do something pre-med bio. But then I decided that I didn't care if a cell died and I ended up going to public health. And then I found out that, you know, with an undergrad degree in public health, I couldn't do much. So then I decided that I wanted to go and get a master's degree, but I had a really strong science and statistics background. So I wanted to get a master's degree in something hardcore that would really help let me make a difference. And so I ended up at the University of Chicago School of Social Service Administration. And I tested out of the statistics classes for social workers. And so I ended up doing measurement across the midway in the education department where I was trained to create standardized tests like the SAT and the ACT. And then I had a really strong background in social service administration from my coursework there. And so I had this unique skill set by my second year of my master's degree where I was a social worker who also happened to be good at statistics. And so my second year internship was at Youth Guidance in their research and evaluation department. My job was organizing all of their surveys into SPSS and figuring out how to put, you know, reports together. And my first week, I discovered that there were so many different surveys and, you know, 40 different surveys, you know, and this was just at one time point, you know, they had all these different programs all over the city in, you know, grades, kindergarten through 12th grade and all of these various questions. So it was not much for me to summarize because there, you know, there, there wasn't a consistent thread that you could weave through all the various surveys or do about time because in different years it would be different supervisors or different administrators or different program planners or different ground level people. doing the work and they would like put together their own survey. So I started working with the director of the research and evaluation department to create logic models for all of the various 40 programs that they had at the time in all of their departments, in all of the schools across the city of Chicago. And so what a logic model is, some people know, some people think like it's antiquated, but I think that there is no better way to figure out how do we identify what changes we want to make, right? Changes in attitudes, knowledge, behaviors, and status, and what research-based activities do we have at our disposal in our talent pool, in our repertoire to achieve those outcomes that we are hoping to create in a neighborhood, in a school, in a classroom or in the world. And so I created, I met with all of these workers from all across the city over the course of several months and created all of these logic models. And lo and behold, because they were And what is the logic model? Sorry, Dana, just for people that aren't familiar. Yeah. Thank you. Yeah. It's the logic of how the program works from beginning. Like, what are your inputs? What are your activities? What- what outputs, so what things can you measure? Like you can measure the number of t-shirts, you can measure the number of cookies you hand out, you can measure the number of people that show up, you can measure the number of people who finish, you could measure those types of tangible things, and then how do those things you can measure yield outcomes that are real changes in how people act? or think or behave or how they just are in this world, like a change of status. And so it's the logic of how all of those components are integrated into actual practice in the field. And it's the process of putting together a logic model is even more valuable than the end product of having it on paper so that you could share it with outsiders. But the process of making sure that all of the program planners and administrators and stakeholders are on the same page is really invaluable because oftentimes at the beginning of the process, people assume that we're all on the same page. We all think that this organization is important. We're all working towards the same thing, but sometimes they use different language to get to the same place. Sometimes they have different ideas of how to get to that place. Sometimes they think that everything is in order, but they find out that they don't have the resources to do all the things that they think they're actually doing. so all of this is good information to working towards the changes that you want to see occur. Yeah. And and I've in my time the past five plus years interviewing a lot of these nonprofit executive directors and leaders, I feel like this is one of the most common challenges that I see them facing is working smart in addition to working hard. I think everyone in this space are crazy hard workers. They're fueled by passion that's very different from the for-profit space. But a lot of times, they don't know where to go of how to do those initial logic models or they might not have the time or the experience, understand the value or haven't really connected the dots of doing these activities to maybe getting more fundraising or grants or telling a better story that connects and resonates with their audience or the community better. How have you seen that kind of evolve for Pi and the organizations you've worked with? Is that also a challenge you're seeing? ⁓ absolutely. So people usually come to us for one thing that they're being told that they need. in order to get funding, right? some funder might say, well, you need to measure A, B or C in order to get this funding. And then they'll come to us and they'll search us up on Google or something, nonprofit measurement, they'll look up evaluation in Chicago. And they'll find us and they'll say, how do we measure this thing so that we could report it to foundation Z? will start talking to them about it. And then we end up doing like a very thorough process that really helps them with their strategic planning. It really helps them do better training so that they could implement research-based strategies that maybe they've all maybe they're just doing what they've always done and it might be good, but it might not be with the specific intent of achieving the outcomes that they've stated. And so all of a sudden they're so much more effective. And everything seems to flow so much better because all of a sudden everybody's on the same page when they didn't even realize they weren't on the same page. Everybody was just running on their own hamster wheels or everybody just working in their own silos and working very hard and doing their work every day, but not in a cohesive, impactful way where they're able to improve their impact and prove their impact. Yeah. I would love to, from the early stages of Pi, so like when you guys, when you first founded the organization, I would love to hear an example of an organization you worked with in the early days and kind of their process and their experience with you and maybe what are some of the targeted outcomes or focuses they had and what was kind of some of the transformation that they experienced. Yeah, so one of the examples is, you youth guidance. It's this one that was really transformational for me because I got to see when I did the logic models for all these 40 programs within one organization, there was this through line of all their outcomes. It all fell under this idea of resiliency and they were all practicing the same kind of research based strategies, but they didn't have like a word or a theory that they were all connected with with the s one person thought they we dance. The other person th doing, you know, scho know, before school breac you know, group councelin was this through line of th theory of change and the same outcomes, but they weren't all specifically speaking the same language. Got it. Got it. Soon as they started speaking the same language and acknowledged that, yes, we're all working for these groups of outcomes, whether the kids are in kindergarten or whether they're in 12th grade, we might be measuring different outputs, but we're working towards the same outcomes, whether it's working with teachers to better support the students or whether we're working with the parents so that they experience more stability so that the students can achieve these outcomes, right? So all of all of these various activities that seemed like this crazy web really started to fall into buckets that could be measured, defined, talked about. and reported and then improved because all of a sudden we could do staff training that was relevant to all these different staff members who thought they were working on their own program and their own project in their own school with their own students. But all of a sudden we see how they all are intertwined really doing the same good work. And now we could use measurement tools that were similar and consistent year after year, grade after grade. So we were not only able to follow one program over time, but we could follow one student, whether they went from program to program or grade to grade over time to see how individual students were improving, as well as how the program was improving its services over time. I think that just describing that one process is illustrative of many of our clients where he was hired to put all the surveys into, know, SPSS and create reports for all the surveys. But then we went through this process and out comes this way of evaluative thinking where we share the same language and we're thinking about how can we improve. And how could we provide better services? How can we do our mission better? And then really being planful, thoughtful, and efficient about it, really. Yeah. And if I were to try to summarize back to you at the aha moment that you helped that organization experiences develop that single common thread of resilience as the connective tissue across all those siloed activities across the ages of these kids. And then once you have that resilience as kind of that connective tissue, being able to benchmark a resilience metric in each of those separate components and then track over time of a kid's experience in that, like how has their resilience changed, improved, declined throughout these different components? And now let's use this resilience as that story, that, I'd be surprised if they didn't even change their mission or update their mission statement after like having that resilience component. And then that. connected the entire story of the organization and the activities. Is that a good summary? Did I miss something? No, it's excellent. And also the training aspect of training all their staff. You know, we really then had this language of strengths based resiliency, professional development. Every single training was through that lens. So then you didn't have all this disparate, you know, a Christmas tree programming. You really had a workbook that we could then put together, you know, for new workers. It changed how they did their onboarding. It even changed how they did their reviews. It improved how they were able to provide supervision. And so, yeah, it was really life changing for the organization. Yeah. Yeah. Wow. That makes total sense. And it's probably hard because Like all these NGOs and nonprofit organizations are very different in what they provide sometimes. There are like categories of nonprofits I've seen, but every human is different. So every community will be different. And so their needs will be different. Have you seen any, like if you were to be speaking in front of a hundred nonprofit leaders, trying to have, or have an impact on all of them or explain something in a way that would resonate across all of them. Have you seen any common threads across the nonprofit sector of challenges that people are facing in simplifying their way of measuring the impact that they're having? maybe help us demystify maybe what people think about measuring the impact that you've seen that people maybe in their heads make too complex. Well, think that people think that research is really something that universities do, right? But what we do is we bring the PhD to these nonprofits, right? It's kind of like rent a PhD. It's a fractional researcher, you know, that is applied statistics, applied research. And like in the one example I gave of youth guidance, was resiliency, strengths-based, but in every organization, that thread will be different. And so we're able to go in and meet people where they're at and help them identify what a behaviors and uh status th and come to consensus on to to consensus on th how to measure it. And the coaching process that rea we work our way out of a time, you know, we've gone planning process, we've i We've gone through and measured it. And then by the third year, we've put together this reporting strategy where internally people are able, organizations are able to report on their own. And maybe they just need us for like the real messy data or some like hardcore statistics analysis to put into their annual report, but then they're able to own it, you know? And so I think that sometimes people think that doing research or evaluation is like a one-time thing that only is done at a university with professors and somebody working on their dissertation. And then it goes into a three-ring binder and sits on a shelf. But the way that we work with people is collaborative and shoulder-to-shoulder where we coach them and build their own internal capacity to evaluate their own programs and have an evaluative way of thinking. So that way they're able to, you know, determine and discern what strategies really fit and help them meet their outcomes in a way that maybe before they were just looking to say yes to the money and the dollar signs and do whatever the foundation asked them to do. ⁓ gives them the power and the knowledge and the documentation and the background and the research to go back to a foundation that asks for something very specific and say, hey, listen, we actually measure this in this way and this is why we do it. And do you think that this does meet this criteria for you? And in that way, they don't have to know, respond to every Christmas tree type ornament that somebody might be offering them for money, right? But really they're able to say, well, we know this works. are you interested in willing and funding us? And this is how we measure it. And this is the information we can give back to you at the foundation to show your foundation board what impact your money is making in our community, right? A lot of times we work with foundations who hire us to work with their whole docket of green fleas. Because in that way, not only are we able to build the capacity of all of their grantee organizations, but also then the foundation gets a more cohesive, organized, description of the impact of how their dollars are making an impact in a variety of fields, right? Because one foundation might have a mission somewhere along the lines of providing healthcare for underserved youth, right? Period. And so they might provide money and support. lots of different types of organizations because that is purposefully broad so that organizations can provide the services that they think are necessary, right? There might be one organization they support that provides dentistry work. There might be another organization they provide that does mental health services. They might do another one that does, you know, healthcare in a high school setting as a, like an urgent care. And they serve all these. various programs that all fit within their mission. However, they don't have a way to cohesively report the impact, right? Then their impact seems like it's all over when really we could help those organizations define and report their outcomes in a way that is useful to those organizations getting additional funding from other foundations. as well as help the original foundation, you know, report to their own funders what kind of impact they're having and tell that story because now they have a through line also that is more thick and is more obvious than healthcare for underserved youth, right? Right. No, that makes total sense. How have you seen the requirement? like thinking of a small mom and pop shop, maybe one or two ⁓ person nonprofit and much larger organizations. Where have you seen this data, this reporting needed more so? I would assume large organizations obviously, but where does it stop? Do you see this being a value of activity for those very, very small nonprofits as well? And what could they do themselves? At what point should they start working with an organization like Pi? Yeah, ⁓ I definitely think that working with an outside organization is helpful for holding up a mirror and helping people get to the next step, right? Like if they knew what the next step was, they would have already done it. So I definitely feel like any nonprofit who doesn't know what the next step is should go ahead and. try to identify a partner who will help them get over whatever that hump is. Right? So if, if the hump is we have surveys and they're all in boxes under my desk, we don't know what to do with them. If that's your home, then you call us. Right? But if you're maybe some other organization is like, we have these surveys and they're all in Excel databases. and they're all in own files and we can show you any survey from any year. We could pull them back for 30 years, but we don't know how to put them together in a longitudinal story. Well, then you could call us and statistically we could figure out how to tell that story with the data that you have. Right. Or sometimes it's we have our mission, but we measure too many things. We don't know how to put all those measures into a story, or we don't consistently measure the same thing because every year our funders are asking for different things. So we just have year after year after year of measuring different things. Well, then we can help do two things. We can not only help take the annual data and tell a story with that by putting things into categories and doing statistics with numbers to show some cool graphs and things like that. But also we can help with the strategic planning process and even bring those funders into the fold. largest and organizations, largest funders want you to succeed. So they will often pay for like a special grant just to do strategic planning or just new evaluation planning or just to create a database that will forever hold your outcomes data going forward. So that way you could better own your own data and be able to tell your own story and use it for whatever your needs are. We try to use a lot of free programs like we'll use macros in Excel. So that way an organization doesn't have to buy some kind of expensive statistical program, you know, or there's some Google. products that are free for use and they are really powerful and they're like databases that you could purchase but they're more user friendly and the nonprofit can own it themselves without having like an annual payment to make, know, like a ⁓ subscription or something like that. We try to meet organizations where they're at. Some organizations are very sophisticated and they can pay for a very fancy thing. And you know, like an Excel database that has some macros in it to automatically create a graph that has been determined to be useful in alignment with their mission and their outcomes. And all they have to do is, you know, pick the group, pick the year. and then their graph changes for that filter, then they could use it and own it themselves for whatever grant proposal they're applying for for the next three years. Yeah. Wow. What would you say for a nonprofit that founded by a single mom who maybe lost one of her sons to gun violence and she wanted to create an afterschool program just to have a space for kids to have to go to and she's been doing it for a decade and it's just set up a nonprofit and that's about it. And then, you know, she provides food, she provides a place, maybe some programming, but has not invested any time for any reason. Well, not for any because she doesn't have time to do it or doesn't have the capacity to measure the impact that has had over the past 10 years. Like, where would you say somebody like that, like is a starting point? Because that'd be overwhelming. Like, how the heck do you measure the impact of that when You know, you have maybe names of people that have gone through your program. You haven't really tracked them maybe afterwards, but you want to have some sort of story to tell of the impact that has had. Well, we would definitely start with the logic. That's just how we roll. And we'd start with what are the outcomes you're hoping to achieve? What type of knowledge are you hoping to impart on these kids while they are spending time with you? either on a daily basis or over the course of three years that they come to visit your after school program. What do you want them to know differently then than they do now? And then we move on to attitudes. What attitudes do you want to change? What attitudes do you see them coming in with that you're like, I want them to think about this differently, you know? And then we get to that and we really refine it. And what behaviors? How do you want graduates of your program to behave differently than they would have had they never known you? What changes in behaviors are you hoping to achieve? Right? And then we do a really good job of refining that. Then we say, okay, for each of these, for the knowledge, attitudes, and behaviors that you want to change, and maybe it's status too. Maybe she's working in an area where she really wants to improve the graduation rate. So maybe she wants to change their status from a community of people who get their GED to a community of people who graduate in four years, right? So maybe she wants to change that as a status and really refine what that means. And then we start from the end and then we say, okay, what activities are you doing? I know you're doing all of them. You're doing all the great things. What are they? Tell me, tell me what activities, what words are you saying? What things are you doing? What programs are you running? What are you doing to change each of these knowledge, attitudes, behaviors, and status, right? She probably has some tutors coming in. She probably has, you know, she probably shows, you know, store movies. They probably have movie night, right? They probably do a great job at developing relationships, right? And how do you, what are the activities you do that are building those relationships? Who are you bringing in for those afterschool kids to develop relationships with, right? What is the, what is the, the, professional development that you have your staff and volunteers going through to change these knowledge, attitudes, behaviors, and status, right? And we get all of those things on paper and we identify how often are they happening? How many kids are they happening for? How long does each kid receive this type of afterschool program, right? And then... eventually we'll get to place of saying, okay, now what kind of data do we have? And then we can say, ⁓ we have these names of kids who have been in our program for 15 years. You said, I don't know how many years you said, but 15 years, right? We have names of kids. So we can do something retrospective. We can meet with those kids who were there at the beginning and say, what impact did this afterschool program have on your life? How are you different now than you would have been? Right? So that's like one. Yeah. Another idea is actually measuring kids who went through her after school program and what were their graduation rates and their their college outcomes or their career outcomes or their recidivism outcomes. than the other kids in the neighborhood and do a comparison of her students over time versus the annual ⁓ rates of the rest of the community who didn't have the benefit of her afterschool program, right? So that's another way to do it retrospectively. And then going forward, on a regular basis, you k some surveys that kids can on phones, you know, on a weekly basis or on an annual basis, spec in the outcomes that we i with the end in mind in those will give her so muc to show all of that information to future funders, right? So that she's able to perhaps grow the program or even pass on the program to someone else who can now take the reins. But they will now have this knowledge bank of how did this program run? Why did it run this way? And what was the impact that this program had? And how could we improve this program going forward, right? So it gives the next person who's going to be taking over her role so much more information so that they can continue the good work, right? Yeah. Wow. That's amazing. I followed that completely and that makes total sense. Do you also advise on or make recommendations on what areas are better or worse to measure? Like you said, you can measure everything, but what are the needle moving things to measure? And what's usually your decision logic for what typically should definitely be measured and what other areas might not make as much sense? I don't know if those could be categorized and it's probably, again, organization specific. Of course it's organization specific, but oftentimes it's, what's easiest to measure is sometimes the answer. But sometimes the answer is, what's important to measure. And we will then put effort towards measuring things that are important, even if they're not easy, right? And sometimes you determine that by identifying what the outcomes are that you want to measure. And sometimes it won't be possible for a small organization to measure that specific outcome because it's difficult to measure. However, we can go to the research and we can say there have been double blind peer reviewed journal articles that say these outcomes are created by doing these processes. Then we can go ahead and measure these processes, make sure we're doing them with fidelity and say, we know based on this research and peer reviewed journal. It's like finding a transit transitive property outcome that you could connect things to. Yeah. Okay. So sometimes an organization does not have the resources to measure the outcomes they are hoping to achieve, but we can measure the fidelity to the program to make sure we are doing what other people have already identified works. And then in that way, we become a sure thing for a foundation who wants to make a good impact or a particular impact. They want to... They an organization that is doing good work. They want to fund an organization that's doing research-based strategies. And then this small organization that doesn't have the resources to measure those long-term outcomes, but they do have the resources to demonstrate that they do particular things with great fidelity. Wow. I just looked at the time and we are cruising. But the one question I want to ask before we move on to the tail end of this is how is your team and your organization at Pi using all this boom in AI and AI tools and chat bots to just level up what you guys are delivering? So... Because I feel like, and I ask that question because I could put myself in the shoes of organizational leaders that say, I'm just going to like... upload all the documents I have to chat to you and say, like, give me a story that I could help sell my organization to grant XYZ. Here's the grant. Like, help me respond to it. I do think that a lot of fidelity is a question. Yeah. Yeah. So I do think a lot of people are using, you know, AI to write grants these days. But I think the people who are reading the grants still want them to be specific, specific, specific and responsive, responsive, responsive to exactly what they're asking for. Right. So using a generic AI isn't that useful for that. And also, you know, of course AI can make things up that aren't actually in the data that you're putting in there just because the AI wants to tell a good story too. So, you know, you definitely want to make sure before you send something out there that what the AI said you did, you actually did. That's my main suggestion. But we are... We don't use AI for analysis. We have been able to use AI for like summarizing, you know, perhaps interviews and focus groups, right? And in that way we could do it, you know, maybe quicker or differently, right? Because oftentimes we'll do both and we will, we can do our normal human analysis of what we would identify and then like supplement it. You know, with the AI right now at this point, I don't think that it is reliable enough to do our analysis. can't, you know, you can't put, you know, infer you can't put financial information into AI and expect it to spit out truthful, you know, insights into what you should do next. And I don't think that we're there yet for actual, you know, statistics on what kind of programming people should do next. So I think that we and others are currently using AI in a way that is helpful in particular components at a time, certainly not for the whole process, because the whole process is very person related and it has to make sense to the people who are still implementing it in the field. Right. so, you know, we are not getting worked out of a job yet. guess. Yes, no, totally, totally. No, that's amazing. Thank you so much. People that are listening and we have I mentioned before we started rolling that we have a large audience that are nonprofit leaders, people that are listening and want to get engaged with you and learn more at the very least. What's their process? What should they do? Yeah. So you should go to our website. It's P-I-E-O-R-G dot O-R-G. So pieorg dot org. And you can contact us there and you can learn all about us. We are located in Chicago, but we've always done projects all over the country. You know, we've done projects in Alaska and California and New York, all over. We work with nationwide organizations like the American Library Association. We also work with very small organizations. And we also work with foundations that need ⁓ some help working with other foundations when they are working together to make a large impact in collaborating and identifying what those threads are and how do we make a collective impact. So we work with organizations at every stage in their development. We are even helping some organizations sundown now and helping their mission live on. So. Whether you're just starting or you're wrapping up or you're somewhere in between, you know, we help people with all stages of the planning, the implementation and the evaluation. Well, Tanya, thank you so much. Is there anything that we missed or you want to leave us with before we wrap up here? No, but you know, my personal mission is helping other people do more good work. So I look forward to meeting people and doing more good work together. So thank you for the opportunity. Yes, thank you as well. And thanks for going over. We went a lot longer than we anticipated, but great conversation. And I'm excited. I'm so excited to share the work that you're doing. So people that are listening and want to get engaged, pyorg.org. And yeah, thanks again, Tanya. Thank you. Bye. Thanks.