GPT-3: Matching Human Reasoning and Problem Solving

Researchers from the University of California, Los Angeles (UCLA) have found that OpenAI's GPT-3 language model outperformed undergraduates in tests of reasoning by analogy, commonly found in standardized tests like the SAT. GPT-3 demonstrated the ability to complete comparisons and understand rules behind transformations of letters and numbers. However, the software had some glitches and struggled to consistently recognize the presented problems without specific prompts or sentence phrasing. The study suggests that large language models like GPT-3 can master reasoning by analogy and develop abstract notions that can be flexibly generalized between different domains.