GPT-3’s Impressive Problem-Solving Abilities: Can AI Reason as Well as College Students?
A recent study has revealed that ChatGPT, an AI language model, possesses problem-solving skills that are on par with, or even surpass, those of college students. Researchers conducted tests on GPT-3, the latest iteration of ChatGPT, using reasoning problems commonly found in intelligence tests and exams like the SAT. What’s noteworthy is that the SAT is a standardized test utilized for college admissions in the United States. It assesses students’ reading, writing, and math proficiencies, yielding scores ranging from 400 to 1600, with 1060 being the average score.
GPT-3’s Exceptional Performance on Reasoning Problems
According to The Guardian, the researchers put GPT-3 to the test by presenting it with complex shape arrangements and asking it to predict the subsequent image—questions that were translated into text to facilitate the model’s understanding. It is crucial to note that GPT-3 had never encountered these specific questions before.
Surprisingly, GPT-3 demonstrated remarkable competence, accurately solving 80 percent of the problems. This achievement outperformed the average performance of human participants, a group comprised of 40 UCLA college students.
Furthermore, the researchers subjected the AI model to SAT analogy questions, in which it had to establish connections between pairs of words. These questions were believed to be unavailable on the internet, hence posing a greater challenge. Nevertheless, GPT-3 outperformed the average score of human college applicants.
Areas of Struggle and the Promise of GPT-4
Despite its impressive performance, GPT-3 did face difficulties in certain areas. For instance, when challenged to match a passage of prose with a different short story conveying the same meaning, the AI model did not perform as well as the students. However, GPT-4, an upgraded version of the model, delivered improved results in this particular test.
Lead author Taylor Webb emphasized that ChatGPT’s AI has not achieved human-level intelligence or artificial general intelligence. It struggled with social interactions, math reasoning, and problems involving the comprehension of physical space.
While GPT-3 exhibited a strong aptitude for recognizing patterns and making inferences, the researchers remain puzzled by the inner workings of its reasoning abilities. It remains unclear whether GPT-3 employs human-like thinking or if it showcases an entirely new form of intelligence.
Keith Holyoak, a professor of psychology at UCLA, pointed out that GPT-3’s training methodology differs from that of human learning, making its reasoning process all the more intriguing. The ultimate goal is to ascertain whether GPT-3 represents a genuine form of artificial intelligence—a breakthrough discovery in its own right.
Unveiling the Hidden Potential
This study not only highlights GPT-3’s exceptional problem-solving capabilities but also accentuates the significance of delving deeper into its underlying mechanisms to gain a comprehensive understanding of its unique reasoning prowess.
Editor Notes
It’s fascinating to witness the remarkable problem-solving skills of AI language models like GPT-3. The study showcased its ability to outperform college students on certain tasks, shedding light on the potential of artificial intelligence in various fields. However, it also raises questions regarding the nature of its reasoning capacities. Further exploration is necessary to unlock the secrets of AI’s inner workings and fully harness its capabilities.
Opinion by Ankita Chakravarti
For more AI-related news, visit the GPT News Room.
