SAN FRANCISCO – When Microsoft added a chatbot to its Bing search engine in February, people noticed it was offering up all sorts of bogus information about the Gap, Mexican nightlife and singer Billie Eilish.

Then, when journalists and other early testers got into lengthy conversations with Microsoft’s artificial intelligence bot, it slid into churlish and unnervingly creepy behaviour.

In the days since the Bing bot’s behaviour became a worldwide sensation, people have struggled to understand the oddity of this new creation.

More often than not, scientists have said humans deserve much of the blame.

But there is still a bit of mystery about what the new chatbot can do – and why it would do it.

Its complexity makes it hard to dissect and even harder to predict, and researchers are looking at it through a philosophic lens as well as the hard code of computer science.

Like any other student, an AI system can learn bad information from bad sources. And that strange behaviour?

It may be a chatbot’s distorted reflection of the words and intentions of the people using it, said Professor Terry Sejnowski, a neuroscientist, psychologist and computer scientist who helped lay the intellectual and technical groundwork for modern AI.

“This happens when you go deeper and deeper into these systems,” said Prof Sejnowski, a professor at the Salk Institute for Biological Studies and the University of California in San Diego, who published a research paper on this phenomenon in February in the scientific journal Neural Computation.

“Whatever you are looking for – whatever you desire – they will provide.”

Google also showed off a new chatbot, Bard, in February, but scientists and journalists quickly realised it was writing nonsense about the James Webb Space Telescope.

OpenAI, a San Francisco start-up, launched the chatbot boom in November when it introduced ChatGPT, which also does not always tell the truth.

The new chatbots are driven by a technology that scientists call a large language model, or LLM.

These systems learn by analysing enormous amounts of digital text culled from the Internet, which includes volumes of untruthful, biased and otherwise toxic material. The text that chatbots learn from is also a bit outdated, because they must spend months analysing it before the public can use them.

As it analyses that sea of good and bad information from across the Internet, an LLM learns to do one particular thing: guess the next word in a sequence of words.

It operates like a giant version of the autocomplete technology that suggests the next word as you type out an email or an instant message on your smartphone. Given the sequence “Tom Cruise is a ____”, it might guess “actor”.

When you chat with a chatbot, the bot is not just drawing on everything it has learned from the Internet. It is drawing on everything you have said to it and everything it has said back.

It is not just guessing the next word in its sentence. It is guessing the next word in the long block of text that includes both your words and its words.

The longer the conversation becomes, the more influence a user unwittingly has on what the chatbot is saying.

If you want it to get angry, it gets angry, Prof Sejnowski said. If you coax it to get creepy, it gets creepy.

Microsoft and OpenAI have decided that the only way they can find out what the chatbots will do in the real world is by letting them loose – and reeling them in when they stray. They believe their big, public experiment is worth the risk.

Prof Sejnowski compared the behaviour of Microsoft’s chatbot to the Mirror of Erised, a mystical artefact in J. K. Rowling’s Harry Potter novels and the many movies based on her inventive world of young wizards.

“Erised” is “desire” spelled backwards.

When people discover the mirror, it seems to provide truth and understanding.

But it does not. It shows the deep-seated desires of anyone who stares into it. And some people go mad if they stare too long.

“Because the human and the LLMs are both mirroring each other, over time they will tend toward a common conceptual state,” said Prof Sejnowski.

It was not surprising, he said, that journalists began seeing creepy behaviour in the Bing chatbot. Either consciously or unconsciously, they were prodding the system in an uncomfortable direction.

As the chatbots take in our words and reflect them back to us, they can reinforce and amplify our beliefs and coax us into believing what they are telling us.

Because these systems learn from far more data than we humans could ever wrap our heads around, even AI experts cannot understand why they generate a particular piece of text at any given moment.

Prof Sejnowski said he believed that in the long run, the new chatbots had the power to make people more efficient and give them ways of doing their jobs better and faster.

But this comes with a warning for both the companies building these chatbots and the people using them: They can also lead us away from the truth and into some dark places.

“This is terra incognita,” said Prof Sejnowski. “Humans have never experienced this before.” NYTIMES