201
AI language model runs on a Windows 98 system with Pentium II and 128MB RAM
(www.tomshardware.com)
this post was submitted on 30 Dec 2024
201 points (92.4% liked)
Technology
64653 readers
4244 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
but the hardware is not capable. it's running a miniscule custom 260k LLM and the "claim to fame" is that it wasn't slow. great? we already know tiny models are fast, they're just not as accurate and perform worse than larger models, all they did was make an even smaller than normal model. this is akin to getting Doom to run on anything with a CPU, while cool and impressive, it doesn't do much for anyone other than being an exercise in doing something because you can.
We can all imagine it, it wouldn't run very well, as shown in the article...
except i'm not wrong. the model they ran is 4 orders of magnitude smaller than even the smallest "mini" models that are generally available, see TinyLlama1.1B [1] or Phi-3 3.8B mini [2] to compare against. Most "mini" models range from 1 to about 10 Billion parameters, which makes running them incredibly inefficient on older devices.
but I can imagine it. in fact, I could have told you it would have needed a significantly smaller model in order to run at an adequate pace on older hardware. it's not at all a mystery, its a known factor. i think it's absolutely cool that they did it, but lets not pretend its more than what it is - a modern version of running Doom on non-standard hardware.
[1] https://huggingface.co/TinyLlama/TinyLlama-1.1B-step-50K-105b
[2] https://ollama.com/library/phi3:3.8b-mini-128k-instruct-q5_0
[3] https://www.thirtythreeforty.net/posts/2019/12/my-business-card-runs-linux/