this post was submitted on 07 Feb 2024
217 points (95.4% liked)

Technology

60346 readers
4287 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 2 years ago
MODERATORS
[email protected]
[email protected]
[email protected]
217
Boffins find AI models tend to escalate conflicts to all-out nuclear war (www.theregister.com)
submitted 11 months ago by [email protected] to c/[email protected]
42 comments fedilink hide all child comments
 

Key Points:

  • Researchers tested how large language models (LLMs) handle international conflict simulations.
  • Most models escalated conflicts, with one even readily resorting to nuclear attacks.
  • This raises concerns about using AI in military and diplomatic decision-making.

The Study:

  • Researchers used five AI models to play a turn-based conflict game with simulated nations.
  • Models could choose actions like waiting, making alliances, or even launching nuclear attacks.
  • Results showed all models escalated conflicts to some degree, with varying levels of aggression.

Concerns:

  • Unpredictability: Models' reasoning for escalation was unclear, making their behavior difficult to predict.
  • Dangerous Biases: Models may have learned to escalate from the data they were trained on, potentially reflecting biases in international relations literature.
  • High Stakes: Using AI in real-world diplomacy or military decisions could have disastrous consequences.

Conclusion:

This study highlights the potential dangers of using AI in high-stakes situations like international relations. Further research is needed to ensure responsible development and deployment of AI technology.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 15 points 11 months ago* (last edited 11 months ago) (4 children)

We write a lot of fiction about AI launching nukes and being unpredictable in wargames, such as the movie Wargames where an AI unpredictably plans to launch nukes.

Every single one of the LLMs they tested had gone through safety fine tuning which means they have alignment messaging to self-identify as a large language model and complete the request as such.

So if you have extensive stereotypes about AI launching nukes in the training data, get it to answer as an AI, and then ask it what it should do in a wargame, WTF did they think it was going to answer?

There's a lot of poor study design with LLMs right now. We wouldn't have expected Gutenburg to predict the Protestant reformation or to be an expert in German literature - similarly, the ML researchers who may legitimately understand the training and development of LLMs don't necessarily have a good grasp on the breadth of information encoded in the training data or the implications on broader sociopolitical impacts, and this becomes very evident as they broaden the scope of their research papers outside LLM design itself.

[โ€“] JasSmith 4 points 11 months ago

There is a real crisis in academia. This author clearly set out to find something sensational about AI, then worked backwards from that.

load more comments (3 replies)