LocalLLaMA

2274 readers

3 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago

MODERATORS

SkySyrup

pax

noneabove1182

GGUF PR has officially been merged into master! (github.com)

submitted 1 year ago by noneabove1182 to c/localllama

7 comments fedilink hide all child comments

They still consider it a beta but there we go! It's happening :D

you are viewing a single comment's thread
view the rest of the comments

[–] noneabove1182 1 points 1 year ago

The significance is we have a new file format standard, bad news is it breaks compatibility with the old format so you'll have to update to use newer quants and you can't use your old ones

The good news is this is the last time that'll happen (it's happened a few times so far) as this one is meant to be a lot more extensible and flexible, storing a ton of extra metadata for extra compatibility

The great news is that this paves the way for better model support as we've seen already with support for falcon being merged: https://github.com/ggerganov/llama.cpp/commit/cf658adc832badaaa2ca119fe86070e5a830f8f6