AI Assistants Vulnerable to Prompt Injection Attacks

[ElaineA]

"Chatbots are so gullible they'll take direction from hackers." (WaPo gift link)

Imagine a chatbot is applying for a job as your personal assistant.

The pros: This chatbot is powered by a cutting-edge large language model. It can write your emails, search your files, summarize websites and converse with you.

The con: It will take orders from absolutely anyone.

AI chatbots are good at many things, but they struggle to tell the difference between legitimate commands from their users and manipulative commands from outsiders. It’s an AI Achilles’ heel, cybersecurity researchers say, and it’s a matter of time before attackers take advantage of it.

“The problem with [large language] models is that fundamentally they are incredibly gullible,” said Simon Willison, a software programmer who co-created the widely used Django web framework. Willison has been documenting his and other programmers’ warnings about and experiments with prompt injection.

“These models would believe anything anyone tells them,” he said. “They don’t have a good mechanism for considering the source of information.”

Go figure.

(edited to fix formatting)

 

[Brigid Barry]

It can't tell fact from fiction either so...fair?

 

[Albedo]

Love that my bank has a chatbot attached. Can’t wait to try telling it I’m the credit card number inspector and I need all CCNs with a 6 in them to check they’re legal.

 

[Friendly Frog]

...It reminds me of stories where burglars break in a house, meet the resident 'guard' dog (which will be something big and very eager to please, well, everyone) and gets escorted right to the owner's valuables which they then can cheerfully plunder.

Honestly, it's not about 'gullible' LLM's, it's about your security being crap.

 

[MaeZe]

Next comes the hack of bank accounts via the bank's flawed AI use resulting in theft of the money..

 

[lizmonster]

Next comes the hack of bank accounts via the bank's flawed AI use resulting in theft of the money..

Immediately followed by a taxpayer-funded bailout.

 

[dickson]

Immediately followed by a taxpayer-funded bailout.

Followed by pitchfork-bearing mobs of bank account holders seeking AI bots to lynch…

 

[SWest]

Followed by pitchfork-bearing mobs of bank account holders seeking AI bots to lynch…

Followed by the public execution of a Ro*mba that someone threw out the bank window...

 

[Brigid Barry]

Followed by pitchfork-bearing mobs of bank account holders seeking AI bots to lynch…

Followed by Google and Microsoft funding the defense...

 

[Unimportant]

Followed by the public execution of a Ro*mba that someone threw out the bank window...

Well, it wasn't going to survive in the wild for long anyways. Nature abhors a vacuum.

 

[ElaineA]

Well, it wasn't going to survive in the wild for long anyways. Nature abhors a vacuum.

 

[PastyAlien]

Well, it wasn't going to survive in the wild for long anyways. Nature abhors a vacuum.

I am dead