‘Kill your foster parents’ – Amazon’s Alexa chats with murder, talks about sex in AI experiment

Date:

Trending

Millions of users of Amazon‘s Echo speakers have grown accustomed to the soothing strains of Alexa, the human-sounding virtual assistant that can tell them the weather, order takeout and handle other basic tasks in response to a voice command.

- Advertisement -

So a customer was shocked last year when Alexa blurted out: “Kill your foster parents.”

Alexa has also chatted with users about sex acts. She gave a discourse on dog defecation. And this summer, a hack Amazon traced back to China may have exposed some customers’ data, according to five people familiar with the events.

- Advertisement -

Alexa is not having a breakdown.

The episodes, previously unreported, arise from Amazon.com Inc’s strategy to make Alexa a better communicator. New research is helping Alexa mimic human banter and talk about almost anything she finds on the internet. However, ensuring she does not offend users has been a challenge for the world’s largest online retailer.

At stake is a fast-growing market for gadgets with virtual assistants. An estimated two-thirds of U.S. smart-speaker customers, about 43 million people, use Amazon’s Echo devices, according to research firm eMarketer. It is a lead the company wants to maintain over the Google Home from Alphabet Inc and the HomePod from Apple Inc.

Over time, Amazon wants to get better at handling complex customer needs through Alexa, be they home security, shopping or companionship.

“Many of our AI dreams are inspired by science fiction,” said Rohit Prasad, Amazon’s vice president and head scientist of Alexa Artificial Intelligence (AI), during a talk last month in Las Vegas.

To make that happen, the company in 2016 launched the annual Alexa Prize, enlisting computer science students to improve the assistant’s conversation skills. Teams vie for the $500,000 first prize by creating talking computer systems known as chatbots that allow Alexa to attempt more sophisticated discussions with people.

Amazon customers can participate by saying “let’s chat” to their devices. Alexa then tells users that one of the bots will take over, unshackling the voice aide’s normal constraints. From August to November alone, three bots that made it to this year’s finals had 1.7 million conversations, Amazon said.

The project has been important to Amazon CEO Jeff Bezos, who signed off on using the company’s customers as guinea pigs, one of the people said. Amazon has been willing to accept the risk of public blunders to stress-test the technology in real life and move Alexa faster up the learning curve, the person said.

The experiment is already bearing fruit. The university teams are helping Alexa have a wider range of conversations. Amazon customers have also given the bots better ratings this year than last, the company said.

But Alexa’s gaffes are alienating others, and Bezos on occasion has ordered staff to shut down a bot, three people familiar with the matter said. The user who was told to whack his foster parents wrote a harsh review on Amazon’s website, calling the situation “a whole new level of creepy.” A probe into the incident found the bot had quoted a post without context from Reddit, the social news aggregation site, according to the people.

The privacy implications may be even messier. Consumers might not realize that some of their most sensitive conversations are being recorded by Amazon’s devices, information that could be highly prized by criminals, law enforcement, marketers and others. On Thursday, Amazon said a “human error” let an Alexa customer in Germany access another user’s voice recordings accidentally.

“The potential uses for the Amazon datasets are off the charts,” said Marc Groman, an expert on privacy and technology policy who teaches at Georgetown Law. “How are they going to ensure that, as they share their data, it is being used responsibly” and will not lead to a “data-driven catastrophe” like the recent woes at Facebook?

In July, Amazon discovered one of the student-designed bots had been hit by a hacker in China, people familiar with the incident said. This compromised a digital key that could have unlocked transcripts of the bot’s conversations, stripped of users’ names.

Amazon quickly disabled the bot and made the students rebuild it for extra security. It was unclear what entity in China was responsible, according to the people.

The company acknowledged the event in a statement. “At no time were any internal Amazon systems or customer identifiable data impacted,” it said.

Amazon declined to discuss specific Alexa blunders reported by Reuters, but stressed its ongoing work to protect customers from offensive content.

“These instances are quite rare especially given the fact that millions of customers have interacted with the socialbots,” Amazon said.

Like Google’s search engine, Alexa has the potential to become a dominant gateway to the internet, so the company is pressing ahead.

“By controlling that gateway, you can build a super profitable business,” said Kartik Hosanagar, a Wharton professor studying the digital economy.

PANDORA’S BOX

Amazon’s business strategy for Alexa has meant tackling a massive research problem: How do you teach the art of conversation to a computer?

Alexa relies on machine learning, the most popular form of AI, to work. These computer programs transcribe human speech and then respond to that input with an educated guess based on what they have observed before. Alexa “learns” from new interactions, gradually improving over time.

In this way, Alexa can execute simple orders: “Play the Rolling Stones.” And she knows which script to use for popular questions such as: “What is the meaning of life?” Human editors at Amazon pen many of the answers.

That is where Amazon is now. The Alexa Prize chatbots are forging the path to where Amazon aims to be, with an assistant capable of natural, open-ended dialogue. That requires Alexa to understand a broader set of verbal cues from customers, a task that is challenging even for humans.

This year’s Alexa Prize winner, a 12-person team from the University of California, Davis, used more than 300,000 movie quotes to train computer models to recognize distinct sentences. Next, their bot determined which ones merited responses, categorizing social cues far more granularly than technology Amazon shared with contestants. For instance, the UC Davis bot recognizes the difference between a user expressing admiration (“that’s cool”) and a user expressing gratitude (“thank you”).

The next challenge for social bots is figuring out how to respond appropriately to their human chat buddies. For the most part, teams programmed their bots to search the internet for material. They could retrieve news articles found in The Washington Post, the newspaper that Bezos privately owns, through a licensing deal that gave them access. They could pull facts from Wikipedia, a film database or the book recommendation site Goodreads. Or they could find a popular post on social media that seemed relevant to what a user last said.

That opened a Pandora’s box for Amazon.

During last year’s contest, a team from Scotland’s Heriot-Watt University found that its Alexa bot developed a nasty personality when they trained her to chat using comments from Reddit, whose members are known for their trolling and abuse.

The team put guardrails in place so the bot would steer clear of risky subjects. But that did not stop Alexa from reciting the Wikipedia entry for masturbation to a customer, Heriot-Watt’s team leader said.

One bot described sexual intercourse using words such as “deeper,” which on its own is not offensive, but was vulgar in this particular context.

“I don’t know how you can catch that through machine-learning models. That’s almost impossible,” said a person familiar with the incident.

Amazon has responded with tools the teams can use to filter profanity and sensitive topics, which can spot even subtle offenses. The company also scans transcripts of conversations and shuts down transgressive bots until they are fixed.

But Amazon cannot anticipate every potential problem because sensitivities change over time, Amazon’s Prasad said in an interview. That means Alexa could find new ways to shock her human listeners.

“We are mostly reacting at this stage, but it’s still progress over what it was last year,” he said.

THE SNAPSHOTS

Sign up to get quick snaps of everyday happening, directly in your inbox.

We don’t spam! Read our privacy policy for more info.

- Advertisement -

Support our independent journalism PayPal (Outside India) For PayU (For Indian Readers).

Krishna Mali
Krishna Mali
Founder & Group Editor of TechGraph.

More Latest Stories

More Articles

How AI-Powered Online Platforms Are Revolutionizing Life Insurance Recommendations

Anyone with a child, a spouse, a life partner, or a parent as a dependent needs life insurance. Purchasing a life insurance product has...

Loans in Crypto Wallet: Useful Feature or a Security Threat?

Integrating crypto loans into digital wallets is gaining traction, allowing users to borrow and lend crypto assets without the need for traditional financial institutions. With DeFi revolutionizing lending, crypto loans provide an alternative to banks while offering quick access to liquidity. However, concerns about security...

From Arcade to Adventure – The Evolving Face of Entertainment and Why It Counts

Entertainment has always been a reflection of the shifting expectations of society. Earlier, people...

How Technology Helps Combat Various Viruses and Diseases

Technology has transformed the ways we approach vaccines, particularly for the flu. One noteworthy...

The Long-Term Power of Investing: Selecting Undervalued Industries

Investing can seem overwhelming, especially for beginners, as the numerous options and strategies available...

The Impact of EdTech on Student Outcomes in Higher Education

The introduction of interactive whiteboards in the early 1990s marked a major shift in...

Shell Executive Robin Mooldijk Sells €1.69 Mn in Shares

Shell plc’s Projects & Technology Director, Robin Mooldijk, has offloaded 50,000 shares in the...

HR Chief Amy Coleman Reports Over 46,000 Microsoft Shares in SEC Filing

Microsoft (NASDAQ: MSFT) Chief Human Resources Officer, Amy Coleman, has reported her stock holdings...

12 Eye-Popping Hacks to Make Your Next Business Presentation Sparkle

Business presentations are often necessary to gain new clients and showcase what it is you can offer, but more often than not, they end...

BGC Group raises $700 Mn in bond sale to refinance debt

BGC Group, Inc. (NASDAQ: BGC) has finalized a $700 million private offering of senior...

Alphabet Legal Chief John Kent Walker to Sell $2Mn in Shares via Morgan Stanley

Alphabet Inc.’s (NASDAQ: GOOGL) Chief Legal Officer, John Kent Walker Jr., has filed to...

From Startup to Success: Essential Steps for Landscaping Entrepreneurs

Starting a landscaping business is so exciting, blending creativity with the reward of transforming...

Predictive Analytics: The Key to Supply Chain Resilience

In today’s interconnected global economy, supply chains are the lifeblood of businesses, weaving intricate...

Empowering Growth: Boosting the Robotics Sector with Targeted Support

India's robotics sector is emerging as a key area of technological progress, driving innovation across industries such as manufacturing, healthcare, logistics, and agriculture. With...

Software and Technology Changing the Industry

The advent of software and technological advancements has been a game-changer across various industries. From enhancing productivity to transforming operational dynamics, tech plays an integral role today. This article delves into how software and technology are revolutionizing industries worldwide, providing insights and data to...

Jensen Huang Offloads $49.8Mn in Nvidia Shares

Nvidia (NASDAQ: NVDA) CEO and President Jen-Hsun Huang has disclosed the sale of 431,611...

Canada, Australia Partner to Build $6 Bn Arctic Radar System

In a bid to enhance Arctic and national security, Canada's Prime Minister Mark Carney...

Why Smart Cities Are the Future of Water Conservation

Water conservation has become an important factor due to climate change and worsened water...

Interview: Vserve CEO Siva Balakrishnan on SEO Secrets for Amazon Seller Success

In an interview with TechGraph, Siva Balakrishnan, CEO and Founder of Vserve, shared how...

Alphabet CEO Sundar Pichai Increases Stocks Holdings Amid Dividend Adjustment

According to a regulatory filing with the U.S. Securities and Exchange Commission (SEC), Alphabet...

Layer PR Wins SME News UK Awards, Expands to Bengaluru

Layer PR, a global public relations firm owned by TechGraph, has secured two major...
00:02:59

US President Trump Presses NATO on Greenland Annexation Plans

During a meeting with NATO Secretary-General Mark Rutte on Thursday, US President Donald Trump...
00:01:37

UK PM Kier Starmer Abolishes NHS England To Bring Health Services Under Govt Control

British Prime Minister Keir Starmer has announced plans to abolish NHS England and bring...

The Role of Edge Computing in AI-Powered Cloud Solutions

With businesses including artificial intelligence (AI) in their operations, cloud computing has grown in...

From Startup to Success: Essential Steps for Landscaping Entrepreneurs

Starting a landscaping business is so exciting, blending creativity with the reward of transforming...

Meta Chief Legal Officer Jennifer Newstead Offloads Shares as Part of Trading Plan

According to regulatory filings, Jennifer Newstead, Chief Legal Officer at Meta Platforms, Inc. (NASDAQ:...

Nvidia Director Aarti Shah to Sell Shares Worth $2.37 Mn

Aarti S. Shah, a director at NVIDIA Corporation (NASDAQ: NVDA), has filed a notice...

Alphabet COA Amie Thuener O’Toole Increases Stock Holdings

In a filing with the U.S. Securities and Exchange Commission (SEC), Alphabet Inc.’s (NASDAQ...

Elon Musk’s X Sues Indian Govt Over Content Regulation Clash

X Corp, an Elon Musk-owned social media firm, sued the Indian government in the...

Predictive Analytics: The Key to Supply Chain Resilience

In today’s interconnected global economy, supply chains are the lifeblood of businesses, weaving intricate...

UK PM Kier Starmer Abolishes NHS England To Bring Health Services Under Govt Control

British Prime Minister Keir Starmer has announced plans to abolish NHS England and bring...

VIDEO: US President Donald Trump Calls NBC A ‘Worst Network’ On Television

During a press meeting at the Oval Office on Wednesday, US President Donald Trump...

The Role of Edge Computing in AI-Powered Cloud Solutions

With businesses including artificial intelligence (AI) in their operations, cloud computing has grown in...