LocalLLaMA

2178 readers

1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

SkySyrup

pax

noneabove1182

Distilling step-by-step: Outperforming larger language models with less training data and smaller model sizes (blog.research.google)

submitted 11 months ago by [email protected] to c/localllama

3 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] noneabove1182 3 points 11 months ago (1 children)

Woah this is pretty interesting stuff, I wonder how practical it is to do, I don't see a repo offering a script or anything so may be quite involved but looks promising. Anything to reduce size while maintaining performance is huge at this time

[–] [email protected] 2 points 11 months ago (1 children)

The code is available here:

https://github.com/google-research/distilling-step-by-step

[–] noneabove1182 1 points 11 months ago

Somehow this is even more confusing because that code hasn't been touched in 3 months, maybe just took them that long to validate? Will have to read through it, thanks!