this post was submitted on 22 Jul 2023
9 points (90.9% liked)

LocalLLaMA

2292 readers
1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago
MODERATORS
SkySyrup
pax
noneabove1182
9
chargoddard's frankensteined 22B llama2 (huggingface.co)
submitted 1 year ago by noneabove1182 to c/localllama
2 comments fedilink hide all child comments
 

This is Llama 2 13b with some additional attention heads from original-flavor Llama 33b frankensteined on.

Fine-tuned on ~10M tokens from RedPajama to settle in the transplants a little.

Not intended for use as-is - this model is meant to serve as a base for further tuning, hopefully with a greater capacity for learning than 13b.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 3 points 1 year ago (1 children)

Some mad scientist action, I love it.

[โ€“] noneabove1182 2 points 1 year ago

it apparently has larger implications than we might think, there was a big discussion in thebloke's discord