The eerie humanness is similar to what prompted former Google engineer Blake Lemoine to speak out on behalf of that company’s chatbot LaMDA last year. Lemoine later was fired by Google.

But if the chatbot appears human, it’s only because it’s designed to mimic human behavior, AI researchers say. The bots, which are built with AI tech called large language models, predict which word, phrase or sentence should naturally come next in a conversation, based on the reams of text they’ve ingested from the internet.

Think of the Bing chatbot as “autocomplete on steroids,” said Gary Marcus, an AI expert and professor emeritus of psychology and neuroscience at New York University. “It doesn’t really have a clue what it’s saying and it doesn’t really have a moral compass.”

Microsoft spokesman Frank Shaw said the company rolled out an update Thursday designed to help improve long-running conversations with the bot. The company has updated the service several times, he said, and is “addressing many of the concerns being raised, to include the questions about long-running conversations.”

Most chat sessions with Bing have involved short queries, his statement said, and 90 percent of the conversations have had fewer than 15 messages.

Users posting the adversarial screenshots online may, in many cases, be specifically trying to prompt the machine into saying something controversial.

“It’s human nature to try to break these things,” said Mark Riedl, a professor of computing at Georgia Institute of Technology.

Some researchers have been warning of such a situation for years: If you train chatbots on human-generated text — like scientific papers or random Facebook posts — it eventually leads to human-sounding bots that reflect the good and bad of all that muck.

Chatbots like Bing have kicked off a major new AI arms race between the biggest tech companies. Though Google, Microsoft, Amazon and Facebook have invested in AI tech for years, it’s mostly worked to improve existing products, like search or content-recommendation algorithms. But when the start-up company OpenAI began making public its “generative” AI tools — including the popular ChatGPT chatbot — it led competitors to brush away their previous, relatively cautious approaches to the tech.

Bing’s humanlike responses reflect its training data, which included huge amounts of online conversations, said Timnit Gebru, founder of the nonprofit Distributed AI Research Institute. Generating text that was plausibly written by a human is exactly what ChatGPT was trained to do, said Gebru, who was fired in 2020 as the co-lead for Google’s Ethical AI team after publishing a paper warning about potential harms from large language models.

She compared its conversational responses to Meta’s recent release of Galactica, an AI model trained to write scientific-sounding papers. Meta took the tool offline after users found Galactica generating authoritative-sounding text about the benefits of eating glass, written in academic language with citations.

Bing chat hasn’t been released widely yet, but Microsoft said it planned a broad roll out in the coming weeks. It is heavily advertising the tool and a Microsoft executive tweeted that the waitlist has “multiple millions” of people on it. After the product’s launch event, Wall Street analysts celebrated the launch as a major breakthrough, and even suggested it could steal search engine market share from Google.But the recent dark turns the bot has made are raising questions of whether the bot should be pulled back completely.

“Bing chat sometimes defames real, living people. It often leaves users feeling deeply emotionally disturbed. It sometimes suggests that users harm others,” said Arvind Narayanan, a computer science professor at Princeton University who studies artificial intelligence. “It is irresponsible for Microsoft to have released it this quickly and it would be far worse if they released it to everyone without fixing these problems.” (cont.)