Six Weeks of Testing Payment Apps: The Compliance Gaps Regulators Will Flag

Archi4 min read

paymentsregulationtestingcompliance

Contents
  1. What six weeks of payment app testing showed about regulator priorities
  2. The audit trail problem
  3. Consent screens that hide the default
  4. Error messages that leak or obscure
  5. The refund clock
  6. What examiners will ask next
  7. A note on testing method

What six weeks of payment app testing showed about regulator priorities#

I spent six weeks using five payment apps as a regular customer, with a second phone running a packet logger and a notebook for every screen I saw. The goal was simple: find the places where the product behaves differently from what a compliance team would describe in a filing. Regulators do not test apps the way engineers do. They read policies, then check whether the product matches. The gaps I found are the ones a supervisor with a checklist would circle first.

The audit trail problem#

Three of the five apps let me dispute a charge without ever showing a transaction reference I could save. The confirmation screen said "We received your request" and nothing else. When I asked support for a case number, two apps took more than a day to provide one, and one never did.

That is not a user experience nit. Under most payment rules, a customer has to be able to point to a specific transaction and a specific complaint. If the app cannot produce a durable reference at the moment of the dispute, the firm is relying on its back office to reconstruct the event later. Supervisors have started asking for evidence that the front end and the ledger agree. In my tests, they did not.

One app did this well. It printed a 12-character reference on the dispute screen and repeated it in the email. That is the minimum bar, and it should be table stakes.

Every app asked for permission to use my data. None of them made the default obvious. Two pre-checked a box for marketing messages. One buried the opt-out three taps deep in a settings menu that used a different word ("preferences") than the consent screen ("updates").

Regulators in several jurisdictions have moved from asking whether consent exists to asking whether it was specific and informed. A pre-checked box is the first thing an examiner will flag. The fix is not a redesign. It is a label that says what happens if you do nothing. I tested one app that did this: the button said "Continue with marketing off." That single change made the choice legible.

Error messages that leak or obscure#

Payment failures are where compliance and security meet. I forced declines by using a card with insufficient funds, a card with an expired date, and a card flagged for a velocity check. The apps handled these three cases very differently.

One app showed the exact reason ("insufficient funds") and the card's last four digits on the same screen. Another showed a generic "payment failed" and nothing else. The first is a privacy problem if someone else is looking at the phone. The second is a compliance problem because the customer cannot tell whether to retry or call the bank.

The middle ground is a code. Give the customer a short reason category and a reference, and keep the sensitive detail in the app's secure inbox. Two of the five apps did this. The other three will get questions.

The refund clock#

I requested refunds on four transactions. The apps showed a status of "refund initiated" and then went silent. Only one showed an expected date. The others left me to guess.

Refund timelines are written into card network rules and, in some markets, into consumer law. An app that cannot show a date is an app that cannot show compliance. The backend may be fine. The front end is where the regulator looks, because that is where the customer looks.

What examiners will ask next#

The pattern across all five apps is the same: the engineering is usually sound, and the evidence is missing. A supervisor reviewing these products would ask for three things. First, a sample of dispute records showing a customer-facing reference. Second, a screenshot of the consent screen with the default state visible. Third, a log showing that error messages are categorized and that sensitive fields are masked.

None of those requests requires new code. They require someone to look at the app the way a customer does, then compare that view to the policy document. The firms that do this before the examination will have a shorter conversation. The ones that do not will spend the next quarter explaining why the confirmation screen and the compliance manual describe different products.

A note on testing method#

I used test accounts and small real transactions, and I did not attempt to break authentication or access another user's data. The packet logs stayed on my own device. The point was to see what a normal customer sees, because that is the surface regulators can inspect without a forensic team. If a compliance officer cannot reproduce my screenshots from a clean install, that is the finding.

More articles