Hardware

47 readers

1 users here now

A place for quality hardware news, reviews, and intelligent discussion.

founded 1 year ago

MODERATORS

[email protected]

Would it be possible to train a modern LLM on hardware from 1985? (alien.top)

submitted 11 months ago by [email protected] to c/[email protected]

14 comments fedilink hide all child comments

Assuming the training software could be run on the hardware and that we could distribute the load as if it was 2023, would it be possible to train a modern LLM on hardware from 1985?

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 1 points 11 months ago (2 children)

It wouldn’t. Training the neural nets for LLMs are all about brute force, and it’s only been possible in the last few years to train these models without spending billions. Even going back to 2010 I think it’d be largely infeasible.

The good news is if we fast forward even just a few years, training will become relatively cheap compared to today.

[–] [email protected] 1 points 11 months ago (2 children)

So my collection of about 20 Commodore 64 isn't enough? Do I need another 20? /s

[–] [email protected] 1 points 11 months ago (1 children)

I think you'd need roughly a billion of them. Minus the 20 you have, of course.

[–] [email protected] 1 points 11 months ago

Estimate of up to 30 million C64 were produced so I guess not enough if I managed to get every single one in the world

[–] [email protected] 1 points 11 months ago

You would need at least 44 more

[–] [email protected] 1 points 11 months ago

They didn't ask if it it could be done without spending billions, or whether it would be feasible, i.e., practical, just whether it would be possible.