this post was submitted on 21 Jul 2023
6 points (87.5% liked)

LocalLLaMA

2292 readers
1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago
MODERATORS
SkySyrup
pax
noneabove1182
6
Llama-2, Mo’ Lora (proof of concept MOE of LoRAs) (crumbly.medium.com)
submitted 1 year ago by noneabove1182 to c/localllama
2 comments fedilink hide all child comments
 

https://twitter.com/aicrumb/status/1681846805959528448?t=sG6Xn4p0hodDoB-g7gmuJQ

https://colab.research.google.com/#fileId=https%3A//huggingface.co/datasets/crumb/Wizard-EvolInstruct70k-k4/blob/main/MoLora_7b_(PROOF_OF_CONCEPT).ipynb

Very interesting concept, excited to see where it goes

you are viewing a single comment's thread
view the rest of the comments
[–] noneabove1182 2 points 1 year ago

Probably just because of the MOE (mixture of experts) lol