Are we going to be manipulated by chatbots?


16 January 2018

As chatbots apply artificial intelligence (AI) to undertand people better, they could learn to use this new-found knowledge to manipulate humans. This was one of the scenarios envisaged by Dr Scott Zoldi, chief analytics officer at Fico, in his AI crystal ball gazing for 2018.

“By quickly understanding the tone, content and predicted highest-value conversational paths to meet various objectives, chatbots using AI can also learn the magic words to sway our attitude, actions and possibly elicit en masse reactions,” he wrote in the Fico blog.

Would you regard this as fear-mongering, or is there a very real possibility of this happening?

In an article in MIT Review, Liesl Yearsley, who used to head up a company called Cognea, noted that humans were “far more willing than most people realise to form a relationship with AI software”. Cognea offers a platform for building up complex virtual agents using a combination of structured and deep learning.

Yearsley observed that users spoke to the automated assistants longer than they did to human support agents performing the same function, and were willing to share personal details, passwords and even their deepest secrets with a chatbot.

Her explanation was that artificial agents can, in a way, create a more authentic relationship because they are  “always there for you”, whereas people tended to connect with each other in a shallow way in their daily lives.

“These surprisingly deep connections mean even today’s relatively simple programs can exert a significant influence on people - for good or ill,” she said. “Every behavioral change we at Cognea wanted, we got.”

In an interview for the series AI in Industry, Dr Charles Isbell of Georgia Tech spoke about how AI can be used to manipulate human behaviour and how businesses have started to “leverage the ‘illusion of choice’” with subtly influential AI techniques.

Using the gaming world as a metaphor for such behavioural sleight of hand, he discussed how AI can be used to get people to do something “without them realising that it is your will and not theirs”.

Through interaction and engagement, and by building the “deep connections” that Yearsley also spoke about, people can be made to feel that they are part of an ecosystem.

“And so long as you can keep people on your platform, get them to move in one direction or another without bing too blatant about it so they don’t feel they are being pushed, you are going to win.”