this post was submitted on 29 Nov 2023
1 points (66.7% liked)

Hardware

33 readers
1 users here now

A place for quality hardware news, reviews, and intelligent discussion.

founded 11 months ago
MODERATORS
[email protected]
1
Would it be possible to train a modern LLM on hardware from 1985? (alien.top)
submitted 9 months ago by [email protected] to c/[email protected]
14 comments fedilink hide all child comments
 

Assuming the training software could be run on the hardware and that we could distribute the load as if it was 2023, would it be possible to train a modern LLM on hardware from 1985?

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 9 months ago (5 children)

It wouldn’t. Training the neural nets for LLMs are all about brute force, and it’s only been possible in the last few years to train these models without spending billions. Even going back to 2010 I think it’d be largely infeasible.

The good news is if we fast forward even just a few years, training will become relatively cheap compared to today.

[–] [email protected] 1 points 9 months ago

They didn't ask if it it could be done without spending billions, or whether it would be feasible, i.e., practical, just whether it would be possible.

load more comments (4 replies)