"Rapid Acquisition of Unforeseen Abilities by Large Language Models"

Researchers have observed that large language models (LLMs) exhibit sudden leaps in ability on certain tasks, which has been described as "breakthrough" behavior. However, a new paper from Stanford University suggests that these abilities may not be as unpredictable as previously thought, attributing the sudden appearance to the way researchers measure LLM performance. The study argues that the emergence of abilities is more gradual and predictable than previously believed, and that the choice of metric used to evaluate LLMs plays a significant role in determining their perceived capabilities.