Debt Collectors Intend To Exploit People For Money By Using AI Chatbots

An industry that has a history of harassing and frightening people who are struggling financially is switching to AI-powered debt collectors, a move that one company claims would “change debt collections forever.”

According to Skit.ai, a company with offices in New York and Bangalore, AI tools will usher in “a new era of debt collection” for the sector. The company claimed that a digital voice agent, the new iteration of a robocall, could make millions of outbound calls in just a few days, contacting and requesting payment from a collection agency’s entire portfolio of debtors at a much lower cost than human staff. It did this by using AI chatbots and text-to-speech capabilities for dynamic, responsive conversations.

According to the company’s blog, human agents might apply AI at every stage of the collection process, enabling “instant scalability” through “end-to-end automation,” which would undoubtedly increase productivity and reduce expenses. On the other side, the possibility of speaking with a real person during the procedure grows increasingly remote for the person on the other end of the call.

Motherboard asked Skit.ai about utilising AI for debt collection, but they didn’t react. But as machine learning and even generative AI become more prevalent, software applications targeted at debt collectors promise to maximise the recovery of money from debtors. These uses are only expected to increase, especially at a time when debts are at an all-time high and AI hype is flourishing.

An industry that has historically exploited the poor and marginalised adds another nightmarish element with the possibility of automated AI systems calling people in distress. Black communities are much more likely than white ones to experience debt collection and enforcement, and studies have shown that predatory debt and interest rates make poverty worse by keeping individuals caught in a never-ending cycle.

Borrowers in the US have been taking on more debt recently. According to the New York Federal Reserve, household debt reached a new high of $16.9 trillion in the fourth quarter of 2022. This increase was accompanied by a spike in the delinquency rates for major debt obligations, including mortgages and auto loans. The number of unpaid credit card bills is also at an all-time high. The epidemic caused a significant increase in online spending, and younger consumers were drawn in by fintech firms pushing new financial products like the wildly popular “buy now, pay later” business models like Klarna, Sezzle, Quadpay, and others. This was in addition to using regular credit cards.

As a result of rising interest rates and accumulating debt, more and more people are skipping payments. This means that more unpaid debts will be turned over for collection, offering the sector the potential to add some AI to the time-honoured practise of nudging, nagging, and forcing debtors to pay up.

We need look no further than the sales copy of businesses that produce debt collection software for an understanding of how this operates. The following product descriptions mix typical corporate jargon with nightmarish portent: Similar to Skit, SmartAction is a conversational AI product that offers debt collection services. Its goal is to “alleviate the negative feelings customers might experience with a human during an uncomfortable process” because customers will undoubtedly feel more at ease negotiating with a robot than a human.

While Katabat offers “full omni-channel orchestration, true machine learning,” and a “powerful collection strategy engine,” Latitude “resolves gaps in functionality while reducing the pressure on your agents and increasing recovery rates,” TrueAccord operates an “industry-leading recovery and collections platform powered by machine learning and a consumer-friendly digital experience,” and Latitude “resolves gaps in functionality while increasing recovery rates.” Additionally, TrueAccord brags that it provides “experimentation in A/B testing, consumer research, and machine learning” to deliver more compassionate debt collection experiences.

The underlying promise is the same as that of many AI-powered products: work more quickly and with fewer humans present; feed the data you get back into the system; adjust, improve, and repeat as necessary.

Timnit Gebru, the founder of the Distributed AI Research Institute (DAIR), said in an email to Motherboard that using AI for debt collection is “punishing those who are already struggling.”

“Are we really attempting to design instruments to put even more pressure on individuals who are struggling at a time when income disparity is off the charts, when we should be cutting things like student debt? This holds true even if the software functions as intended, according to Gebru.

Gebru continued, “In addition to this, we know that there are so many biases that these LLM-based systems have, encoding hegemonic and stereotypical views,” pointing to the findings of the work on massive AI models that she co-authored with a number of other researchers. “It is also deeply troubling that we have no idea what they are doing and that they are not required to tell us.”

Some businesses that just exist to collect debt are among those that stand to gain the most from AI integration. These businesses, referred to as debt purchasers, buy “distressed” debt from other creditors at deep discounts—typically pennies on the dollar—and then make every effort to encourage debtors to settle their debt in full. They don’t offer loans or perform any other services that customers would owe them money for; instead, their business strategy is to capitalise on consumers who have fallen behind on payments to others. They also rely largely on the civil court system, which some experts predict may eventually be overrun by cases brought about by AI for unpaid debt.

To these third-party purchasers, many providers of debt collection software market their goods, but Arrears.com is one of the few that also specifically emphasises huge language models. A recent blog post gushed over the exciting world of GPT-4 debt collection, which will allegedly be more personalised, effective, and emotionally intelligent when asking people to pay their bills. Although the digital collections platform currently claims to integrate GPT-3, it clearly has its sights set on the newer update.

According to the company’s blog post, GPT-4 has the capacity to be “firm and compassionate” with clients. “Striking the right balance between assertiveness and empathy is a significant challenge in debt collection,” the blog post states.

There is a significant danger that bias is being unintentionally created when algorithmic, dynamically optimised systems are used in sensitive domains like credit and finance. According to a McKinsey paper on digital collection strategies, AI may be used to identify and categorise consumers according to their risk profiles—which include their credit score and any other data the lender can take into account—and to adjust their contact methods accordingly.

According to Odette Williamson, a senior attorney at the National Consumer Law Centre, training data reflecting the lengthy history of lending discrimination against low-income groups and communities of colour can help AI models detect systematic bias.

“Is the [training] data complete, or is it inaccurate and misleading?” stated Williamson. “Given our country’s history of racism… Are there any discriminatory trends in this data, and if so, how may it affect future judgements a system might make on which individuals to target and how aggressively to pursue debt collection?

In 2016, Australia provided a case study of the negative effects of automated debt collection when the government overpaid for welfare benefits and then attempted to recover the money using an automated system, with disastrous outcomes.

Careful auditing is required because of the knowledge of biassed outcomes, which have been documented in areas including prison sentences, school dropout prediction, and welfare fraud monitoring.

“At the end of the day, we need to ensure that these models are statistically sound, that they are evaluated during the creation process, and that they are also tested when they are deployed… And if you can’t alter models that are producing discriminating or biassed results, you shouldn’t employ them, she added.

Regulatory bodies are well aware of the numerous potential threats AI poses to consumer finance. Late in April, just a few weeks ago, the Consumer Financial Protection Bureau (CFPB) announced its intention to target discriminatory practises that arise from the use of automated systems in a joint statement with the Department of Justice, Federal Trade Commission, and Equal Employment Opportunity Commission.

The Consumer Financial Protection Bureau (CFPB) will expect debt collectors to abide by all Fair Debt Collection Practises Act requirements and the Consumer Financial Protection Act’s prohibitions against unfair, deceptive, and abusive practises, regardless of the type of tools used, a CFPB spokesperson told Motherboard when contacted by email.