Latest / Consumerpedia / 81 - Algorithms Secretly Control Your Financial Fate
Transcript
- 0:02Whether you know it or not, whether you like it or not, an
- 0:05increasing number of important decisions about your life for
- 0:08now being made by computer algorithms and artificial
- 0:11intelligence. Algorithm suggests music or
- 0:14movies you might like. Algorithms customize your social
- 0:17media feeds and help power safety features in your car.
- 0:20Algorithms and AI help landlords screen potential tenants, help
- 0:24banks make lending decisions and help employers screen job
- 0:28applications. Consumer advocates acknowledge
- 0:30the benefits of AI, but they also see the potential harm.
- 0:34They want lawmakers and regulators to put some
- 0:37guardrails in place to protect you.
- 0:39I'm Herb Weisbaum, the consumer Man, a contributing editor at
- 0:43checkbook.org. Welcome to
- 0:45consumerpedia@checkbook.org. We're the nonprofit that helps
- 0:49consumers select services, avoid trouble, and save money.
- 0:53Because we don't accept any advertising or take money from
- 0:55any business we recommend, you can rely on checkbook.org to be
- 0:59completely independent and objective.
- 1:02Now here's the host of Consumer Pedia America's consumer expert,
- 1:06the Consumer man Herb Y Spom. In this episode, the many ways
- 1:11companies are now using algorithms in AI, how these
- 1:14powerful computer programs make the marketplace less transparent
- 1:18and in some cases, less fair. And how the massive amount of
- 1:21data being collected and used to make decisions about lending,
- 1:25employment, housing, and insurance coverage can infringe
- 1:28on your privacy rights. Here to talk about all that, the
- 1:31CEO of the Consumer Federation of America, Susan Weinstock.
- 1:34Susan's an old friend who's been fighting for consumer rights for
- 1:37generations. So nice to have you with us.
- 1:39Thank you, Herb. It's always great to talk to
- 1:41you. And Jennifer Chen, a senior
- 1:43policy counsel in the Financial fairness team at Consumer
- 1:46Reports. Hi, Herb, Great to be here.
- 1:48For listeners from here on out, I'm just going to say AI because
- 1:51algorithms and AI are becoming so intertwined.
- 1:54Susan, I'm going to start with you.
- 1:56To be clear, consumer groups are not opposed to businesses using
- 2:00AI technology. You just want to make sure it
- 2:02doesn't discriminate or violate consumer rights.
- 2:04That's absolutely right. We actually see that there could
- 2:07be a lot of benefits to the use of AI.
- 2:09It could help with wealth building opportunities, help for
- 2:13people to age in place by having, you know, tools and you
- 2:17can see if someone falls and make sure that they're OK.
- 2:20It could prevent some fraud. It could expand access to
- 2:23credit. Depending on how it's done.
- 2:25It might reduce human bias or reduce closing costs on a house
- 2:29because things will become more automated.
- 2:31At the same time, though, there's lots to be concerned
- 2:34about. There's discriminatory
- 2:36practices, fraud, privacy breaches.
- 2:39There's a black box problem because we don't know exactly
- 2:42how AI decision making is happening.
- 2:45And so that poses a fundamental problem in actually even
- 2:48overseeing it. Companies could use it to be
- 2:51more aggressive in outreach at lower costs.
- 2:54Think about robocalls and things like that could get even worse
- 2:58than they are now. And during this episode, we're
- 3:00going to talk about a number of those concerns should point out
- 3:02that without AI, companies could make instant decisions about
- 3:06approving credit card applications at checkout or
- 3:08providing price quotes for insurance policies, which are
- 3:11things that people really like. So there is some benefit to
- 3:13these algorithms in AI. Yes, that's absolutely right.
- 3:16There is no question about it. It could remove human bias from
- 3:20important decision making. If you train the models on which
- 3:23the AI is being made to make sure that it's both fair and
- 3:27accurate, and those are key parts of this, it actually could
- 3:31reduce discrimination. It could scale monitoring and
- 3:34compliance on fair housing issues, help identify and
- 3:38prosecute fair housing violators more quickly, greater automation
- 3:43to the paperwork. Heavy housing finance industry
- 3:45put tools in the hands of consumers could they could
- 3:48actually make better decisions, comparison shop, understand the
- 3:51benefits and advantages of particularly products and
- 3:53services and make the decision that's right for them, which in
- 3:57some parts of the economy is very hard to do.
- 4:00So yes, there is no question about it that there are those
- 4:03potential benefits. Consumer Reports conducted a
- 4:05survey to find out how average Americans feel about AI.
- 4:09This is the nationally representative sample of more
- 4:11than 2000 adults and it found that the majority of those
- 4:14surveyed are not comfortable with the use of AI and high
- 4:17stakes decisions about their lives, such as landlords using
- 4:21AI to screen potential tenants, employers using AI to analyze
- 4:25video job interviews, and banks using AI to make lending
- 4:28decisions. Jennifer, based on CRS survey,
- 4:31talk a little bit more about what's making people so
- 4:34uncomfortable. Well, as you noticed, Herb,
- 4:36it's, it's not surprising. People are pretty uncomfortable
- 4:39about the thought of these AI systems that they don't
- 4:43understand making important decisions about their lives.
- 4:46And this is really not surprising because it's not
- 4:49clear how these models are being used.
- 4:52Consumers don't know if their decisions being made that are
- 4:55impacted by AI. They don't know what data is
- 4:58going into this information and they feel powerless to do
- 5:01anything about these decisions. CR survey also found that many
- 5:05people are concerned that the computer programs making these
- 5:08important decisions could be using erroneous information and
- 5:11as you pointed out, they wouldn't know it and therefore
- 5:14would have no way to challenge it.
- 5:15That's. Right.
- 5:16So right now there are not clear safeguards in place to really
- 5:19ensure that consumers are informed when AI is being used
- 5:23to make these consequential decisions.
- 5:25But more importantly, the bigger issue is that these models, they
- 5:29have their pros and cons. They're very complex.
- 5:32They're often called black box models, and they can do a lot in
- 5:36terms of analyzing huge amounts of data and providing more
- 5:41ability to predict and make decisions and increase
- 5:45efficiency for companies than ever before.
- 5:47And those do have benefits for consumers.
- 5:49But one of the big issues is that even companies who use
- 5:52these models, they don't necessarily know how they work
- 5:55and they don't know how they reach their decisions.
- 5:58So it's difficult for consumers to be able to really understand
- 6:02what's going on and to know what data was used for a decision,
- 6:06but how it was used and how they might be able to address a
- 6:09decision that they don't agree with.
- 6:11Time out. So you're saying that companies
- 6:13may be using AI and algorithms and they don't even know what's
- 6:16going on to make that decision? That's going to be a business
- 6:19decision for them. That's right.
- 6:21And that is probably one of the core issues with these models.
- 6:24It's very much a double edged sword.
- 6:27And we're talking specifically in the financial sector about
- 6:30machine learning models, which is a more sophisticated advanced
- 6:34use of AI and algorithms of the past.
- 6:38And the machine learning models, they essentially they can learn
- 6:41on their own. And again, they analyze huge
- 6:44amounts of data and they can learn and improve over time and
- 6:49develop their own ability to find connections, make
- 6:53predictions. As I said, one classic example
- 6:57is the use of machine learning for credit underwriting.
- 7:00And it's a double edged sword in all cases.
- 7:03If it's used well, you can expand access to finance for
- 7:06consumers who don't have a very well established credit history,
- 7:10for example, because you can analyze different types of data
- 7:13in a much more efficient manner to provide loans to more people.
- 7:17But if it's used badly, it can result in bias and exclude
- 7:22consumers from credit that they deserve.
- 7:25And you won't be able to pinpoint why because you can't
- 7:27see into the model itself. I find that incredibly scary.
- 7:31I assume you do too, yes? Anyone who's used one of these
- 7:34chat bots now available to play around with to write things or
- 7:38make drawings or whatever, realize that artificial
- 7:41intelligence can do some amazing things, but these programs don't
- 7:44always get it right. Sometimes they make up stuff
- 7:46what's known as hallucinations. If you want some examples, just
- 7:49head to tech.com and check out the story AI Gone Wrong and
- 7:53updated list of AI errors, mistakes, and failures.
- 7:56And having read some of these horror stories, Susan, it makes
- 7:59me nervous about trusting machines to make decisions
- 8:02without human oversight. Really important decisions.
- 8:04Not just what movie to watch, but whether I get approved for a
- 8:07loan, or how much I pay on an insurance premium, or whether
- 8:10the landlord agrees to let me rent the apartment.
- 8:12That's absolutely right. Think about somebody who's wants
- 8:15to be an investor. It could induce an investor to
- 8:18behave in certain ways, manipulate them into making
- 8:22decisions that actually yield higher profits for the firms or
- 8:25as a or a scam as opposed to being in the best interest of
- 8:28consumers. So there's all kinds of things
- 8:31out there that could make the decision making process for a
- 8:34human much more difficult and steer folks in ways that are
- 8:39really detrimental to their best interest.
- 8:41But I'm concerned about the decision being made by the AI
- 8:44that's using erroneous information to make a decision
- 8:46about me. Like I didn't have a default on
- 8:49my mortgage or whatever and I have no idea the AI is using
- 8:52that. That's right, that's definitely
- 8:54a problem. It could make it really hard as
- 8:56well and just make it really difficult for consumers to get
- 8:59their due and to make sure that they're getting accurate and
- 9:02complete information. If a loan is turned down for
- 9:05some reason, can they get an explanation for that?
- 9:08What if the data going into that decision was faulty in the 1st
- 9:12place? Is there a way for a consumer to
- 9:14find out that is what happened and to check the data going in?
- 9:18Right now, no. Mistakes are one thing,
- 9:20discrimination is something quite different.
- 9:22Can AI discriminate, and if so, can it be hidden from view?
- 9:26We're going to talk about that next.
- 9:27I'm Herb Weisbaum, the consumer man, and you're listening to
- 9:30Consumerpedia, powered by checkbook.org.
- 9:35From tech gadgets to financial tips, travel hacks to health
- 9:38trends, Consumerpedia provides expert insights and practical
- 9:42advice to help you save time and money, avoid the scams, and stay
- 9:46healthy. With dozens of episodes to
- 9:48choose from, Consumerpedia is your guide to making informed
- 9:51decisions. Just go to consumerpedia.org and
- 9:55search by episode, title or topic such as home, travel, auto
- 9:59finance, consumer protection or technology checkbooks.
- 10:02Consumer Pedia Podcast Consumer pedia.org.
- 10:06Listen and learn and share with your friends.
- 10:11So what about the argument? The decisions based on
- 10:13algorithms are better, fairer, because it eliminates the
- 10:16potential biases humans have. After all, these are decisions
- 10:19made by machines and based on mathematical formulas.
- 10:22Susan Can algorithms be Biased? Well, in theory, they actually
- 10:27could be helpful so long as the data that goes into the decision
- 10:31in the algorithm is unbiased to begin with.
- 10:33But the problem is we don't know that.
- 10:35So if there's bad data going in, if it's garbage data going in,
- 10:39you're going to get garbage data coming out and and the consumer
- 10:42is completely at the mercy of that bad data.
- 10:45And the consumer doesn't know if the algorithm is biased, like
- 10:48somebody who might make a racial slur if you're applying for a
- 10:50home loan in person or something like that.
- 10:52Herb, the regulators may not even know that the algorithm is
- 10:56biased. This is the whole black box
- 10:58problem is what data is going in to making this decision and can
- 11:03that be put out there so that somebody can actually know the
- 11:06answer to that very important question?
- 11:09Jennifer is AI able to suck up data from the Internet that no
- 11:12company would be allowed to ask for or ask about when someone
- 11:16fills out an application for credit or employment or rental
- 11:19housing. In some ways, yes.
- 11:20What AI and machine learning models in particular allow
- 11:24companies to do is to use unstructured data, data that is
- 11:28not typically used for credit underwriting and not typically
- 11:32requested. So it could be things like your
- 11:36social media data or your mobile phone transactions.
- 11:40It's not typical that's used. And again, it can have benefits
- 11:44in terms of expanding access to finance, but it can also have
- 11:48drawbacks if there is discrimination in these models
- 11:51and it's hard to pinpoint where the causes.
- 11:53So explain to us how algorithms can make discriminatory
- 11:56decisions. What's going on here?
- 11:58Algorithms. They seem on the surface to be
- 12:00more objective and impartial, and in some ways there is
- 12:04potential for algorithms and AI machine learning models to
- 12:07improve on past models that were discriminatory.
- 12:11But in in reality, there are many ways that algorithms and
- 12:14machine learning models can end up discriminating the data
- 12:17that's used to train these models.
- 12:19It can be unrepresentative for certain segments of the
- 12:22population, for example, which means that the results of that
- 12:25model would be less accurate. There is historical bias in data
- 12:29that's fed into these models, and as many folks say, the bad
- 12:33data in means bad results out, so that could result in
- 12:37decisions that are systematically worse for certain
- 12:40segments of the population. Can data be used that would not
- 12:43normally be allowed by federal regulators because it's a
- 12:45machine sucking it up? So the issue with machine
- 12:49learning, one of the challenges is that especially with credit,
- 12:52you're not allowed to use protected characteristics such
- 12:56as race or gender or religion. But these models are so complex,
- 13:01they pull in and make their decisions based on so many
- 13:05different variables. And some of these variables end
- 13:08up being proxies for these protected characteristics.
- 13:11So for example, you might have a variable like zip codes or some
- 13:16other variable that on its face doesn't seem like it's a
- 13:19protected characteristic, protected by anti discrimination
- 13:22laws, but it ends up being a very close proxy for that
- 13:25characteristic. And if that is a key factor in
- 13:29the model and how it makes a decision, then it could have
- 13:32negative impacts on certain population.
- 13:35So the rules that were put in place when we filled out forms
- 13:37on paper and decisions were made by humans.
- 13:39The rules about what information could be collected and how it's
- 13:42used, Those rules were created to prevent discrimination based
- 13:45on sex or race or ethnic background.
- 13:48How can these rules be enforced when the regulators and class
- 13:51action lawyers don't have a clue as to what went into making that
- 13:55black box decision and probably if they even got the code,
- 13:57couldn't even figure out what was going on?
- 13:59It's a huge problem. And so we're going to have to
- 14:01figure out a way to find out what is the data that goes into
- 14:05this algorithm, what's going into the decisions that have
- 14:08been made. We haven't passed any laws or
- 14:10really any regulations that looking at AI and how it's
- 14:14making decisions and how that could impact people and to make
- 14:18sure that is not discriminating against people, you know, in
- 14:21fields where they aren't supposed to be protected.
- 14:23And it's not even, as I said before, it's not even just a
- 14:27challenge to regulators. It's a challenge that companies
- 14:29themselves, and frankly, I think many companies are well aware of
- 14:32this. And some institutions are being
- 14:34cautious about how they employ these models and others are
- 14:37being, I think, a little too aggressive about it.
- 14:40There are some safeguards and some recommended emerging good
- 14:45practices that both companies and regulators can use to try to
- 14:49ensure that when they use these machine learning models, they
- 14:53don't result in discrimination. So, for example, you can take
- 14:57active steps to ensure that the data that you feed into the
- 15:00model is as complete and representative as possible for
- 15:05all different demographics. You can actively test your model
- 15:09to ensure that it's not using proxies for protected
- 15:13characteristics like race or religion or gender.
- 15:16And you can very actively build your model to not only optimize,
- 15:22for example, the performance, so the accuracy you want in terms
- 15:26of predicting default, for example, you can also optimize
- 15:30it to try to minimize the amount of disparate impact that it
- 15:34causes. So we at Consumer Reports, we
- 15:36strongly feel that these measures, these safeguards
- 15:39should be required of all institutions that are using
- 15:41machine learning. And as Jennifer pointed out,
- 15:44because artificial intelligence can suck everything up from the
- 15:47Internet, the decisions could be made on information or factors
- 15:50that clearly aren't allowed by regulations, but you'd never
- 15:53know that the machine was using that information because you
- 15:56have no idea what's going on in this black box.
- 15:58That's exactly right. I mean, think about all of the
- 16:00junk that's on the Internet. That is useless information that
- 16:03steers people in the wrong direction, that gives bad
- 16:05information. Now throw that into an algorithm
- 16:08and think about how that compounds the problem even more
- 16:11for a consumer. So what can be done to protect
- 16:13us from AI run amok? We're going to talk about that
- 16:16next. I'm Herb Weisbaum, the consumer
- 16:18man, and this is Consumerpedia, powered by checkbook.org.
- 16:23Consumerpedia Fast Facts An algorithm is a set of
- 16:27instructions that must be followed in the correct order to
- 16:30complete a task or solve a problem.
- 16:32The first algorithms were created more than 4000 years ago
- 16:35by the Sumerians, Babylonians, and Assyrians who lived in
- 16:39Mesopotamia. What is Iraq today?
- 16:41Algorithms don't need to be complex mathematical formulas,
- 16:45according to the website learning.com.
- 16:47Any step by step process that is completed the same way every
- 16:51time is an algorithm. For example, tying your shoes.
- 16:55There are a limited number of steps that need to be done in a
- 16:57specific order to get the task done.
- 17:00Or using a recipe when you cook. That recipe is an algorithm
- 17:04designed to make it possible for anyone to create specific food
- 17:07by following a set of detailed instructions.
- 17:12Clearly, individuals don't have any real way to fight back
- 17:15against a world run by algorithms and AI.
- 17:18That's why it's up to lawmakers and regulators to create rules
- 17:22that protect us. Susan, what would the Consumer
- 17:24Federation of America like to see done?
- 17:26Well, there's got to be at the right balance between innovation
- 17:29and responsible deployment of AI.
- 17:32So there needs to be a robust regulatory framework that covers
- 17:35this that makes the two work together in the best interest of
- 17:39consumers. So there's a few things that we
- 17:41need. We need to disputability,
- 17:43limiting the ability of the AI to make a final decision needs
- 17:47to be reviewed by regulators or experts to make sure that the
- 17:50decision that comes out of there is correct.
- 17:52There has to be fairness that they're trained on the data is
- 17:56good data and it's the data is done in a way that consumers can
- 18:00figure out what's going on. The companies that are using.
- 18:03I need to be able to explain and validate the decisions that are
- 18:06being made so that if a consumer has a question, there's a way to
- 18:10answer that question. So would you test drive?
- 18:13Would regulators test drive algorithms or set up a process
- 18:16to make sure that they're not doing discriminatory things?
- 18:19Or how in reality would this work?
- 18:21They've got to hold the companies that are using the AI
- 18:24responsible and liable when scams or frauds or harm are
- 18:29perpetrated on consumers. So that's, you know, that's a
- 18:32fundamental piece to this because that's going to get them
- 18:35to make sure that their algorithms that they're using
- 18:38and the AI that they're using is doing it in a responsible
- 18:41manner. And how would they go about
- 18:43finding out that the company is using information in lieu of
- 18:47things like race and sex or whatever?
- 18:49If the AI is sucking up zip codes and that zip code happens
- 18:52to be a community where it tends to be African American people,
- 18:55you don't have to ask, are you black?
- 18:56You found out by the zip code. How are we going to deal with
- 18:59that kind of thing? Right.
- 19:00Well, that's why regulars have to provide guidance and rules
- 19:04and advisory opinions to make sure that companies understand
- 19:10that this is how they need to do their work.
- 19:12So that regulators can go back in and make sure that it's being
- 19:15done fairly and without discrimination.
- 19:17If a company is using information other than what the
- 19:19consumer is providing, you know, we fill out an application form
- 19:22or whatever. If it's the artificial
- 19:24intelligence, it's just grabbing stuff, scraping the Internet.
- 19:26This stuff could be wrong. Do you think the company should
- 19:29have to provide? Here's what we base this
- 19:31decision on, above and beyond what you, the consumer, provided
- 19:34to us. Absolutely.
- 19:35That's a no brainer because otherwise how will we ever know
- 19:38what information they're using? How will a consumer be able to
- 19:41dispute the decision and that's been made?
- 19:43So, Jennifer, Consumer Reports recently published
- 19:45recommendations for new rules and new laws that you believe
- 19:48are needed to protect consumers. The two big keys were
- 19:51transparency and increased privacy.
- 19:53Let's start with transparency. You want companies to make it
- 19:55clear to consumers that an algorithmic tool or AI is being
- 19:59used to make decisions about them.
- 20:00Yes, there's multiple steps and safeguards that need to be put
- 20:04in place at the same time to safeguard consumers 1.
- 20:07Is transparency to consumers and that involves a number of
- 20:11components. 1 And is that a consumer?
- 20:13They should be made aware when they are subject to a decision
- 20:17that has been driven by AI or machine learning models.
- 20:20And not only that, but they should be told the specific
- 20:24factors that led to a decision that may have been called
- 20:28adverse actions. So what if they were rejected
- 20:31for a loan, for example, or they received a loan at a unfavorable
- 20:35price? A company should be able to tell
- 20:37them what are the main factors that led to that decision, so
- 20:41the consumer is empowered to do something about that and try to
- 20:45get a better decision in the future.
- 20:47We advocate for consumers to have the right to appeal to a
- 20:52human, a live person at that company.
- 20:55If they're subject to automated decision by a machine learning
- 20:59model, they should have the right to appeal for human
- 21:01review. So how can a company tell you
- 21:03the reason you were denied or didn't get the best rate or
- 21:06whatever if they don't even know what's going into making the
- 21:08decision? That is a great question.
- 21:10So what I was just describing, we sometimes call that external
- 21:14transparency, so transparency to consumers.
- 21:17There's another set of safeguards that need to be put
- 21:20in place around internal transparency, which means a
- 21:23level of transparency into the model itself.
- 21:26None of these safeguards are well established currently, but
- 21:30there's a clear need to ensure that financial institutions and
- 21:33companies overall they have a sufficient level of transparency
- 21:37into the models they're using so they can pinpoint what these key
- 21:41factors are that led to a decision and and more broadly,
- 21:44so they know how their models are working.
- 21:46There are increasingly techniques to try to achieve
- 21:50some level of transparency into these black box models.
- 21:54There's not yet really clear guidance on what is the
- 21:57sufficient level of transparency that you need to really achieve
- 22:01these policy objectives and to protect consumers, nor is there
- 22:04any clear guidance on which of the techniques are sufficient,
- 22:08and we think more guidance from regulators would be helpful in
- 22:11this area. And what about the issue of
- 22:13protecting people's privacy? Privacy is one of the underlying
- 22:17factors behind so many of these models.
- 22:19All the machine learning models are driven by huge amount of
- 22:23data that exists in the digital marketplace about all of us,
- 22:26about which we have little control over.
- 22:29What we really advocate for is strong data privacy rules that
- 22:33are put in place. And in particular, the best
- 22:36practice across the board would be to have rules on data
- 22:40minimization. That means any company, when
- 22:43you're interacting with that company, they should follow a
- 22:46principle of only collecting and retaining and using from you
- 22:52data that is necessary for the service that you are signing up
- 22:55for, which you would think is a very obvious and common sense
- 22:58approach, but that's not the standard right now.
- 23:01Right now, many times when you're signing up for a product,
- 23:03by default a company may be collecting, as you were saying,
- 23:07all this data about you that is really beyond what is needed for
- 23:11that particular product. You're purchasing, so my fear is
- 23:13the genie's already out of the bottle and it's going to be
- 23:16really hard to put guard rules in place because this thing is
- 23:18already happening. Are you concerned that while the
- 23:21consumer world wants things done, it's important, but it's
- 23:24probably not going to go anywhere?
- 23:25Well, it's time. I mean, the European Union has
- 23:28looked at it. The European Union has passed a
- 23:30set of rules and they are implementing those rules.
- 23:32There is no reason why the US can't do the same and shouldn't
- 23:35do the same. And why are we behind and
- 23:37Europeans, we should be equally looking at what is in the best
- 23:41interests of consumers and what is going to be the best interest
- 23:44for all of us and for the, you know, for the economy.
- 23:46I mean, it's such a broad issue and could be so widely deployed
- 23:50throughout the economy. It's really incredibly important
- 23:54that these products are used responsibly and with the
- 23:57consumer and use in mind. So we always try to end these
- 24:00podcasts with practical information, what the listener
- 24:03can do to protect themselves. But with AI and algorithms,
- 24:06there really is a whole lot you can do.
- 24:08We have to rely on regulators to protect us.
- 24:10I think that's right. Maybe the thing to do is write
- 24:12your congressman and your senators and tell them they need
- 24:15to do something about passing laws and rules and regulations
- 24:18about AI. Well, we just scratched the
- 24:21surface on this issue, one that's just starting to get the
- 24:23attention it deserves, and we're going to keep following it for
- 24:26you as we move forward. I want to thank our guests,
- 24:28Jennifer Chen as Senior Policy Counsel and the Financial
- 24:31Fairness team at Consumer Reports.
- 24:33My pleasure. And Susan Weinstock, CEO of The
- 24:36Consumer Federation of America. Thank you so much.
- 24:39Always a pleasure to talk to you, Herb.
- 24:40And with that, we come to the end of this episode of
- 24:42Consumerpedia. Remember, we release new
- 24:45episodes every other Thursday. If you like what you heard, we
- 24:48hope you'll rate this episode and follow us on Apple Podcast,
- 24:51Spotify or wherever you listen to your podcasts.
- 24:55Another way you can support this show is to follow us on
- 24:57Consumerpedia on Facebook and Instagram and at my
- 25:00Consumerpedia on X, formerly Twitter.
- 25:03I'm Herb Weisbaum. Thanks for listening.
- 25:06Consumer Pedia is a public service of checkbook.org.
- 25:09We're a unique nonprofit that helps you save money and make
- 25:12smarter choices. You can count on Checkbook to
- 25:15help you find the best services and avoid the worst with local
- 25:18ratings that are accurate and unbiased.
- 25:21If you live in or around these seven cities and haven't joined
- 25:24Checkbook yet, check us out Boston, Chicago, Philadelphia,
- 25:29Seattle, San Francisco, Minneapolis, Saint Paul, and
- 25:32Washington DC. To get your free 30 day
- 25:35subscription, go to checkbook.org/consumer Pedia
- 25:38Consumer Pedia. Empowering consumers to save
- 25:41money and make smarter choices.