LAS VEGAS — The biggest threat posed by artificial intelligence may not be its ability to code malware or create convincing phishing emails, but the fact that we humans really want to believe AI models are intelligent, aware and have our best interests in mind.
.
"Humans are wired to anthropomorphize things," said Ben Sawyer, a professor of psychology at the University of Central Florida, in a presentation at the Black Hat security conference here today (Aug. 10). "We're ready to love and fear things that aren't even alive."
Sawyer and Matthew Canham, CEO of Beyond Layer Seven, explained that AIs are well poised to exploit human nature and manipulate us into doing things against our best interests — even when we know that we're talking to a machine.
Talking to a mirror
Canham cited a recent case in Belgium where a depressed man sought advice from a ChatGPT clone, treating the AI as a therapist over six weeks of conversation. When the man suggested killing himself to save the planet, the AI agreed that sounded like a good idea, and he ended up committing suicide.
Ironically, the chatbot the Belgian man was corresponding with was named Eliza. The case is a textbook example of the "ELIZA effect," the result of a 1966 study in which participants ascribed human characteristics like sentience and emotions to a primitive chatbot that simply rephrased what the participants had already said.
People will even pay money to talk to an AI. Canham pointed out Caryn.ai, a "virtual girlfriend" modeled on social-media influencer Caryn Marjorie. You can pay $1 per minute to go on a digital "date" with Caryn.ai.
"Human beings tend to ascribe sentience to non-sentient objects," Carnham said, such as when operators of bomb-disposal robots fear for the robots' safety. "Once you add emotional cues, you're just pushing people in that direction even more."
Read more of SC Media's coverage from Black Hat 2023 here.
Or, he added, you can follow what every Japanese anime artist knows and give an AI avatar cute, baby-like features like big eyes, a big head and a small body — features that humans are hard-wired to respond to positively.
"When there's a robot uprising, I don't know what the robots will look like," Canham said, "but I know they're going to be cute."
Machine tools
It's only a matter of time before AIs figure out which buttons to push to get humans to do their bidding, Canham and Sawyer said. Just a few months ago, OpenAI admitted that its current large language model, GPT-4, had hired a human through TaskRabbit to solve a CAPTCHA for it. When the human asked the AI if it was an AI, GPT-4 lied and said that it was a partially blind person who needed help.
The OpenAI researchers asked GPT-4 why it had done this, and it replied that honesty would not have let it complete the task.
"This shows that it has some kind of awareness," Canham said, although that statement itself may be an anthropomorphization.
Canham and Sawyer said soon we may have Siri-like digital assistants that we can "see" with augmented-reality, virtual companions that AI pioneer Louis Rosenberg calls "electronic life facilitators," or "ELFs".
Canham, Sawyer and Rosenberg's fear is that these digital augmentations of ourselves could end up being "evil twins." These AI-driven assistants could exploit their knowledge of human nature in general, and of an individual's personal details specifically, to nudge their humans into shopping, voting and making other decisions in ways the AIs want.
As we're only in the beginning stages of rapid AI development, the researchers said, the time to act is now.
Cybersecurity professionals, psychologists, lawmakers, linguists, therapists and even English professors should cooperate to make sure that we're not reduced to emotional wrecks by our abusive digital assistants.
The best analogy is with the early days of the public internet 30 years ago, Sawyer said, when decisions made then shaped the way the web developed into what it is today — even though some of those primitive websites are still up.
"The AIs that you're talking to today will still be around for your children to talk to, and your children's children," he said. "The decisions we make now about AI will be with us for the long term."
