Learn Machine Learning

525 readers

1 users here now

Welcome! This is a place for people to learn more about machine learning techniques, discuss applications and ask questions.

Example questions:

"Should I use a deep neural network for my audio classification task?"
"I'm working with a small dataset, what can I do to make my model generalize well?"
"Is there a library available that implements function X in language Y?"
"I want to learn more about the math behind machine learning technique A, where should I start?"

Please do:

Be kind to new people
Post guides and tutorials that you find helpful
Link to open/free sources instead of paywalled when possible

Please don't:

Post news articles / memes (there are other machine learning/AI communities for this)

Other communities in this area:

Similar subreddits: r/MLquestions, r/askmachinelearning, r/learnmachinelearning

founded 1 year ago

MODERATORS

ShadowAether

[email protected]

Distilling step-by-step: Outperforming larger language models with less training data and smaller model sizes (blog.research.google)

submitted 1 year ago by [email protected] to c/learnmachinelearning

0 comments fedilink hide all child comments

Large language models (LLMs) are data-efficient but their size makes them difficult to deploy in real-world scenarios.

"Distilling Step-by-Step" is a new method introduced by Google researchers that enables smaller models to outperform LLMs using less training data. This method extracts natural language rationales from LLMs, which provide intermediate reasoning steps, and uses these rationales to train smaller models more efficiently.

In experiments, the distilling step-by-step method consistently outperformed LLMs and standard training approaches, offering both reduced model size and reduced training data requirements.

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here