basics

A shortcut to building good small models: use a big model's answers as the textbook to teach a smaller one.

What is it?

Q: What is it?

Distillation. Distillation transfers a large 'teacher' model's abilities into a small 'student.' It powers the cheap-but-capable model race — and fuels disputes over whether training on another company's outputs is fair game.

Fermentation
Distillation ✓
Aging

Distillation
Distillation transfers a large 'teacher' model's abilities into a small 'student.' It powers the cheap-but-capable model race — and fuels disputes over whether training on another company's outputs is fair game.

▶ Quration AIQ

basics

What's one of the industry's main responses?What is this empirical rule — the justification for billion-dollar bets — called?What's the acronym for the goal beyond task-specific AI — machine intelligence as general as a human's?What are these called?What's the umbrella term?What are these pieces called?When people describe an AI model as having '70 billion parameters,' what are they counting?What is this phenomenon called?What was it called?What is this human-feedback training step called?What is this limit called?Llama, the flagship of 'open-weight' models anyone can download and run on their own servers, was released by which company?

Quration — Quration AIQ