AI

4117 readers

9 users here now

Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.

founded 3 years ago

GPT4All - A free-to-use, locally running, privacy-aware chatbot. No GPU or internet required. (lemmy.world)

submitted 8 months ago by [email protected] to c/[email protected]

16 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 6 points 8 months ago (3 children)

Is running using a GPU a bad thing? I'm new to this...

[–] [email protected] 13 points 8 months ago

No, a GPU would be ideal but not everyone has one, especially one with enough VRAM. I have an AMD card with 12gb of VRAM and I can run 7B - 13B models but even the 7B models (which seems to be the lowest that is still good) use a little more than 8gb of VRAM and most people probably have an Nvidia card with 8gb or less. 13B models get very close to using the full 12gb.

[–] [email protected] 9 points 8 months ago

Not everyone has a dedicated GPU, I would guess. GPUs are good at doing tensor calculations, but they're not the only way.

[–] [email protected] 2 points 8 months ago

Its better if you have a good GPU, but will not run without a card from the last few years. It can run on CPU but it's much slower.