ChatGPT Scams Are Infiltrating the App Store and Google Play

Any major trend or world event, from the coronavirus pandemic to the cryptocurrency frenzy, will quickly be used as fodder in digital phishing attacks and other online scams. In recent months, it has become clear that the same would happen for large language models and generative AI. Today, researchers from the security firm Sophos are warning that the latest incarnation of this is showing up in Google Play and Apple’s App Store, where scammy apps are pretending to offer access to OpenAI’s chatbot service ChatGPT through free trials that eventually start charging subscription fees.

There are paid versions of OpenAI’s GPT and ChatGPT for regular users and developers, but anyone can try the AI chatbot for free on the company’s website. The scam apps take advantage of people who have heard about this new technology—and perhaps the frenzy of people clamoring to use it—but don’t have much additional context for how to try it themselves. The researchers first learned about the scam apps after seeing ads for them in news apps and on social networks, but users may also encounter them by searching in Google Play and the App Store.

“I saw multiple ads for these types of apps on social media platforms where it’s cheap to advertise, and sometimes they use tactics like typos in the name—calling the app ‘Chat GBT’ or others—to screen out people who might be a bit more savvy,” says Sean Gallagher, a senior threat researcher at Sophos. “They’re trying to screen out people who would do the free trial and then cancel it because it’s crap. They want the people who are not focused enough to know how to unsubscribe.”

Such scams are known as fleeceware. And these apps, which hook victims into paying a regular weekly or monthly fee, are difficult to stamp out, because they typically don't exhibit the technically invasive and malicious behavior that would get more explicit malware booted. When scammers submit their apps to Apple and Google for review, the researchers note, they may not include all of the details on the subscription pricing and when users will have to pay to continue receiving functionality. Later, they can revise their demands without changing anything about how the app is engineered.

Google and Apple provide mechanisms for developers to offer in-app purchases, both one-time fees and recurring charges. And these companies get a cut every time apps in their app stores collect payments from users.

In the case of the Android app Open Chat GBT, users could download the app for free but were quickly confronted with huge quantities of ads and could try the chatbot only three times before losing access to its functionality and receiving a prompt to subscribe. By default, users could sign up for a three-day free trial to continue using the app, which would then become a monthly $10 subscription. Open Chat GBT also offered a $30 annual subscription. The researchers found a very similar app with a different name by the same developer for iOS in the App Store.

The Sophos researchers note that Apple and Google took down some of the fake AI chatbot apps they were looking at before disclosure. There were others, though, that remained available after the researchers flagged them to Google and Apple. Both companies acknowledged receipt of the submissions, and Google took down one more app. Google and Apple did not immediately respond to requests for comment about the findings.

The researchers say they suspect that some of the apps use OpenAI’s ChatGPT 3 application programming interface to generate content for users while others use lower-quality chatbot functionalities. And instead of limiting the user to a small number of queries, some of the apps would truncate responses and give users only a snippet until they started a subscription.

Gallagher says that one of the biggest problems with fleeceware is that users don't always know how to manage their subscriptions and don't realize that even when they delete an app, their recurring payments will continue to be active with the service.

“We define fleeceware as something that charges an extraordinary amount of money for a feature that is available freely or at very low cost elsewhere,” he says. “And it’s effective, because even I sometimes wonder, why am I getting charged this much by Apple every month? And it's like, OK, there’s the shared family storage, there's AppleCare for my phone, there’s Duolingo. You have to be very careful—you have to actively manage subscriptions to apps.”