LocalLLaMA

2274 readers

3 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago

MODERATORS

SkySyrup

pax

noneabove1182

GGUF progressing nicely, ggerganov is back on it tomorrow! (github.com)

submitted 1 year ago by noneabove1182 to c/localllama

4 comments fedilink hide all child comments

Been a lot of good work done the past week by several pivotal members, and now the boss is back and focused on it, going to be a very breaking change but I'm really excited where this will lead us!

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 1 year ago (1 children)

What is GGUF? Is it a model format? Why is this important?

[–] noneabove1182 3 points 1 year ago (1 children)

it's a standardizing of a universal GGML format which would mean going forward no more breaking changes when new formats are worked on, and also includes the same functionality of llama.cpp for all GGML types (falcon, mpt, starcoder etc)

[–] Kerfuffle 2 points 1 year ago (1 children)

which would mean going forward no more breaking changes when new formats are worked on

I definitely wouldn't count on that.

But it does make it much easier to do some changes, like adding/changing model specific fields which previously would have required a format change. Stuff like changing or dropping support for existing quantizations would also break stuff independent of the model format itself.

[–] noneabove1182 2 points 1 year ago

oh yeah definitely didn't mean "no more breaking changes", just that we've had several from ggml file format changes, and so THAT portion of the breaking is going away