this post was submitted on 17 Jan 2025
75 points (100.0% liked)

TechTakes

1550 readers
167 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 2 years ago
MODERATORS
top 19 comments
sorted by: hot top controversial new old
[–] [email protected] 14 points 12 hours ago* (last edited 12 hours ago) (2 children)

So, let's go over this shall we.

Humans were always the weakest link in the security chain. Why? Because humans aren't logical and can be tricked with words and ideas.

So we've developed this new type of computer program that "thinks" and speaks naturally like a human, right? Responding naturally to human conversation with it.

The issue is (as I've said before) that we've essentially created a computer program that is just as fallible as humans.

In other words, no shit simple prompt engineering works. There's no way to "secure" a human brain from dripping out things it shouldn't on accident, and by extension, there's no way to "secure" an LLM "brain" because they operate in a somewhat similar manner (or at least appear to). Prompt engineering is just social engineering for computers. We've created a computer that can be tricked with words and ideas, just like a human.

Humans were the weakest link in security and we just made computers as weak of a link as humans. Who really thought making computers as bad at everything as humans was a good idea?

[–] [email protected] 5 points 11 hours ago

The issue is (as I’ve said before) that we’ve essentially created a computer program that is just as fallible as humans.

Id say it is worse, as we have more physical presence. We can think it rains, look outside and realize somebody is spraying water on the windows and we were wrong. The LLM can only react to input, and after a correction will apologize, and then you have a high chance it will still talk about how it rains.

We can also actually count and actually understand things, and not just predict what the next most likely word is.

But yes, I don't get from a security perspective people include LLMs in things, also with the whole data flows back into the LLM thing for training a lot of the LLM providers are prob doing.

[–] sbv 2 points 11 hours ago

we've essentially created a computer program that is just as fallible as humans.

Humans are a little better in this respect: we know when something should be secure and can try to hide it. Like you say, it takes social engineering to trick secrets out of us.

Language models don't try to keep secrets.

[–] [email protected] 20 points 14 hours ago (3 children)
[–] [email protected] 2 points 3 hours ago

Dumbest shit I've seen this year.

[–] [email protected] 9 points 13 hours ago (1 children)
[–] [email protected] 11 points 12 hours ago (1 children)

look, if we just rename it a bit harder, I’m sure we’ll get the promptfondler uptake that marketing had forecast. this can’t be an itanium situation, dave in sales assured me there’s a use!

[–] [email protected] 9 points 12 hours ago (1 children)

intel frantically taking notes

[–] [email protected] 6 points 12 hours ago

bit late

they’re gonna have to break out the emergency use tardis reserve.. and they’re gonna need to use a few of them

[–] heavydust 6 points 12 hours ago (2 children)

I'm waiting for the Microsoft 365 Copilot Office Live Premium Pro Clippy Edition.

[–] [email protected] 3 points 10 hours ago (1 children)

Wait, am I supposed to use Microsoft 365 Copilot Office Live Premium Pro Clippy Edition (new) or Microsoft 365 Copilot Office Live Premium Pro Clippy Edition? They're both installed...

[–] [email protected] 2 points 10 hours ago

not to be forgotten: the long, long years of windows malware pulling exactly this shit

[–] [email protected] 3 points 11 hours ago (1 children)

I like it. Can you get Visual, Zune, and Azure in there somehow?

[–] [email protected] 4 points 10 hours ago* (last edited 10 hours ago) (1 children)

"hello everyone my name is Bill Ballmer and welcome to today's Visual tour of Microsoft 365 Coilot Office Live Premium Pro Clippy Edition! this is of course being broadcast to all your favourite Zune stations using the Power of Azure, and wow COM-fans I'm just about as hyped up for this demo as I can be. Let's get right into it, shall we?"

[–] [email protected] 3 points 10 hours ago

(sorrynotsorry)

[–] [email protected] 18 points 13 hours ago* (last edited 10 hours ago) (1 children)

*...but don't anyone worry about that! We're going to take control of a nuclear power plant and keep pushing them to you whether you want them or not! We've never let the sanctity of users' data stop us before, and we're not about to start now!"

[–] [email protected] 8 points 13 hours ago

You’re not wrong. MS announced adding CoPilot or whatever the fuck it is called this week to the Office 365 subscriptions for an additional fee, while hiding the subscriptions without under unclear cancellations and shit.

[–] [email protected] 3 points 9 hours ago

I guess I will continue not using them

[–] pugsnroses77 8 points 13 hours ago