this post was submitted on 05 Oct 2024
494 points (89.6% liked)
solarpunk memes
3273 readers
310 users here now
For when you need a laugh!
The definition of a "meme" here is intentionally pretty loose. Images, screenshots, and the like are welcome!
But, keep it lighthearted and/or within our server's ideals.
Posts and comments that are hateful, trolling, inciting, and/or overly negative will be removed at the moderators' discretion.
Please follow all slrpnk.net rules and community guidelines
Have fun!
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
You can run it on your own machine. It won't work on a phone right now, but I guarantee chip manufacturers are working on a custom SOC right now which will be able to run a rudimentary local model.
You can already run 3B llms on cheap phones using MLCChat, it's just hella slow
Both apple and Google have integrated machine learning optimisations, specifically for running ML algorithms, into their processors.
As long as you have something optimized to run the model, it will work fairly well.
They don't want to have independent ML chips, they want it baked into every processor.
That's fine, Qualcomm has followed suit, and Samsung is doing the same.
I'm sure Intel and AMD are not far behind. They may already be doing this, I just haven't kept up on the latest information from them.
Eventually all processors will have it, whether you want it or not.
I'm not saying this is a good thing, I'm saying this as a matter of fact.
It will run on a phone right now. Llama3.2 on Pixel 8
Only drawback is that it requires a lot of RAM so I needed to close all other applications, but that could be fixed easily on the next phone. Other than that it was quite fast and only took ~3gb of storage!