LocalLLaMA

2290 readers

2 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago

MODERATORS

[email protected]

Mozilla's Llamafile 0.8.2 Scores Big With New AVX2 Performance Optimizations (www.phoronix.com)

submitted 7 months ago by [email protected] to c/[email protected]

3 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 1 points 6 months ago (2 children)

is llamafile better than ollama/llama.cpp ?

[–] xcjs 2 points 5 months ago (1 children)

It's just a different use case to create a single-file large language model engine that automatically chooses the "best" parameters to run under. It uses llama.cpp under the hood.

The intent is to make it as easy as double clicking a binary to get up and running.

[–] xcjs 1 points 5 months ago* (last edited 5 months ago)

I just wanted to update this to mention that there are a lot of custom low level performance improvements for CPU based inferencing in Llamafile: https://justine.lol/matmul/