MLST — AI benchmarks are broken! [Prof Melanie Mitchell]

MLST — AI benchmarks are broken! [Prof Melanie Mitchell]

I really love this part of the MLST interview in which Prof Mitchell says the key LLM question is: what kind of “understanding,” if any, is really going on?

1. They don’t and can’t truly “understand” — it’s just word statistics.
2. They do form rich, concept-like mental models.
3. Or their huge correlations amount to a new, non-human kind of understanding.

https://youtu.be/fS-NN6VRzT8?si=0SJl24g9cIW1IVwm