The sentence to pause on is at 1:20: “even though the model itself is deterministic, a given prompt typically gives a different answer each time it's run.” Same weights and same prompt give the same probabilities every time. The variety is added afterwards, on purpose: the software sometimes picks a less likely word because the result reads more naturally. That's a setting in the code serving the model, not the model changing its mind.
So two different answers to the same question aren't two opinions. They're two draws from one set of odds. Worth remembering before anyone screenshots one of them as “what the AI thinks.”
35sfrom 1:13
Clip transcript
- 0:00In doing this, the output tends to look a lot more natural if
- 0:03you allow it to select less likely words along the way at random.
- 0:07So what this means is even though the model itself is deterministic,
- 0:10a given prompt typically gives a different answer each time it's run.
- 0:15Models learn how to make these predictions by processing an enormous amount of text,
- 0:19typically pulled from the internet.
- 0:21For a standard human to read the amount of text that was used to train GPT-3,
- 0:26for example, if they read non-stop 24-7, it would take over 2600 years.
- 0:31Larger models since then train on much, much more.
