Security Notes: The Use And Abuse of AI

Some years ago, I hired a young chap who right away lifted a heavy administrative burden off my shoulders. In a short while, he also came up with some good design ideas, and within a year he was correcting my coding. With a heavy heart, I realized I had unwittingly hired my own replacement. This is the case with artificial intelligence (AI) today.

Much as I should have hired my brilliant assistant even if I suspected he would threaten my cushy place in the company, so we ought to negotiate our future with AI present everywhere, risky as it may be.

The bane of small payments is that when you dispute a $6.75 charge on your account, it is cheaper for the bank to remove the charge than to place the item in the dispute-resolution protocol, especially to the extent that live people are involved (even if they are working from a low-wage country). But if this problem can be negotiated with a serene, human-like voice conducting a human-like dialogue, with beyond-human patience—and without the cost of salary and benefits—then by all means!

Alas, big-data technology and neural networks go farther. Artificial-intelligence sales­persons can outsmart a used-car salesman, and outright artificial-intelligence fraud isn’t far behind. Innocent victims have been known to carry on extensive email correspondence with no human on the other side, only a malicious AI program.

Some malicious AI packages come as orphans. When they are caught, whom do you prosecute?

For years, we have had cause to regret the lack of foresight that led us to design the Internet with security as an afterthought, and now here we go again. At every industry conference, speakers salivate about the rosy future of AI payments, with no mention of security as a top priority. Scenarios involving cost savings, speed, and convenience are charted with great enthusiasm, but scenarios involving potential abuse go unnoticed—except by the schemers.

The advantages of AI payments, including their contribution to reduced friction, are dramatic. But you can expect headline-grabbing AI fraud tales in the near future.

The best scenario for AI involves handling routine payments. But we should promote this application by equipping the public with “AI shadows” of themselves, also known as avatars.

Let’s say you meet your banker for lunch. The two of you iterate your respective positions and concerns, and then send your AI and the bank’s AI to negotiate the pesky details of the transactions. You will task your avatar to purchase the TV screen you desire, letting the software check the various stores, join any shoppers’ groups, and otherwise optimize the deal on your behalf. We already see investment AI, for example, and Internet of Things AI, as well.

By the way, whenever you give payment authority to a non-human entity, you’d better entrust it with a limited amount of cash rather than with access to an account.

Here I will add a subtle point that is quite difficult to explain: AI creatures may be much smarter than humans as measured by intelligence tests, but AI does not have the quintessential human attribute of common sense or reasonability.

Yes, advanced AI digs into raw data, discerns patterns, and forms its own “rules.” It deals with much more data than any human being can ever hope to handle, and its conclusions are so often much better than those of a human. But to the extent that a situation is unprecedented, AI acts randomly, which means stupidly, at least for half the cases.

With payments AI, proceed with caution. Have plans to disengage. And never, ever design a payments system too complicated for humans to take over and manage without reliance on AI.

—By Gideon Samid gideon@bitmint.com