People Twitter

5283 readers

203 users here now

People tweeting stuff. We allow tweets from anyone.

RULES:

Mark NSFW content.
No doxxing people.
Must be a tweet or similar
No bullying or international politcs
Be excellent to each other.

founded 1 year ago

MODERATORS

[email protected]

1918

The dream (lemmy.world)

submitted 11 months ago by [email protected] to c/[email protected]

251 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 11 months ago (1 children)

I don't know of an LLM that works decently on personal hardware

Ollama with ollama-webui. Models like solar-10.7b and mistral-7b work nicely on local hardware. Solar 10.7b should work well on a card with 8GB of vram.

permalink
fedilink
source
parent
hideshow 2 child comments

[–] [email protected] 1 points 10 months ago

If you have really low specs use the recently open sourced Microsoft Phi model.

permalink
fedilink
source
parent