ChatGPT Study Habits Can Make You Score 17% Worse on Exams

The uncomfortable part of the current research on ChatGPT risks for students is not that ChatGPT gives bad answers. The sharper problem is that it can give good enough help at exactly the wrong time. In one study of roughly 1,000 Turkish high school students, students who used ChatGPT during math practice solved 48% more practice problems correctly, then scored 17% worse on a later test than students who studied without it.[1]

That is the pattern exam-prep students should care about: better practice, worse transfer. It is the student who leaves a study session calm because every explanation made sense, then sits for the real test and cannot decide between two answer choices. It is the MCAT student who can follow an AI-generated biochemistry walkthrough but cannot rebuild the reasoning under timed pressure. It is the SAT or GRE student whose review log looks cleaner while the score refuses to move.

The Practice Score Can Lie

Bastani et al. is the study worth slowing down for because it captures the exact trap in AI-assisted studying. The students were not simply asked whether they liked ChatGPT. They used it while working through math practice, and their practice performance improved sharply. During that practice period, ChatGPT users solved 48% more problems correctly.[1]

If the story ended there, the conclusion would be easy: the tool helped. But the later test changed the meaning of the practice result. When students had to perform without the same support, the ChatGPT group scored 17% worse.[1] The practice session had rewarded them immediately, then punished them later, when the support disappeared.

That matters because standardized exams are transfer tests. They do not ask whether an explanation sounded clear five minutes ago. They ask whether the student can retrieve a method, recognize the structure of a new problem, reject a tempting wrong answer, and keep reasoning when the first path fails. A tool can raise practice accuracy while still weakening those exact skills if it enters before the student has struggled with the problem.

The tutoring version in the Bastani study is especially important. It did not rescue the result.[1] That makes the finding harder to dismiss as a simple problem of students copying answers from a generic chatbot. The issue was not only that ChatGPT could hand over a solution. Even a version meant to behave more like a tutor did not produce the later test gains students needed.

The study was posted as a preprint on SSRN, so it should not be treated as the final word on every course, age group, or exam type.[1] But for exam preparation, the direction of the evidence is hard to ignore. The harmful outcome appeared in the place that counts: independent performance after practice.

Why a Clear Explanation Can Become a Score Leak

The likely mechanism is cognitive offloading: moving mental work out of the student's head and into the tool. Offloading is not automatically bad. Calculators, calendars, flashcard schedulers, and answer keys all offload something. The question is whether the tool is removing setup friction or removing the mental action that builds the skill.

For exam prep, the dangerous offloaded actions are retrieval, error correction, discrimination, and self-generated reasoning. Those are not decorative parts of studying. They are the work that makes knowledge available under pressure. If ChatGPT summarizes the passage before the student has identified the author's argument, the student has skipped a reading task. If it explains the missed algebra problem before the student has diagnosed the wrong turn, the student has skipped error correction. If it generates a perfect essay outline before the student has tried to structure one, the student has skipped planning.

This is why the phrase "I understood it when ChatGPT explained it" is not enough. Understanding while reading an explanation is a weaker signal than producing the next step without one. Many students can nod through a clean solution. Fewer can recreate the method when the numbers, passage, answer choices, or time pressure change.

A study session can feel productive because it removes confusion quickly. But confusion is also a diagnostic. It tells the student where retrieval failed, where a concept is unstable, or where two similar question types are being mixed up. When AI smooths that moment too early, the student loses the chance to see the weakness while there is still time to fix it.

The Brain-Engagement Evidence Is Vivid, but It Needs Restraint

The MIT Media Lab EEG study gives a memorable picture of what reduced mental work may look like. In a 2025 preprint with 54 participants, Kosmyna et al. reported that ChatGPT users showed the lowest brain engagement across all 32 monitored regions and underperformed at neural, linguistic, and behavioral levels.[2] The paper also reported that essays from ChatGPT users were described as "soulless."[2]

That study should be handled carefully. It is a preprint, not yet peer-reviewed, and the sample size was small.[2] It also studied an essay-writing task, not an SAT math section or an MCAT passage set. So it should not be used to claim that every ChatGPT study session literally shuts down the same parts of every student's brain.

Still, it fits the practical concern. If a student uses ChatGPT to generate structure, phrasing, examples, and corrections before doing the hard part, the student has fewer reasons to retrieve, compare, revise, and own the work. For test prep, the concern is not whether the final paragraph or solution looks better. The concern is whether the student's mind had to do enough of the job to repeat the performance alone.

Other Studies Point in the Same Direction

The broader evidence does not all come from the same method, which is useful. Abbas et al. studied 494 university students in Pakistan and found that ChatGPT use correlated with self-reported memory loss, increased procrastination, and GPA decline.[3] That is not the same as proving ChatGPT caused each student's lower GPA. It does, however, describe a risk pattern many exam students will recognize: the tool that promises efficiency can also make starting, remembering, and finishing harder.

Li et al.'s 2026 systematic review of 67 studies gives the strongest wider frame. The review found that ChatGPT use in unstructured contexts produced "asymmetrical patterns favoring creativity over critical thinking and joint declines across domains," with cognitive offloading identified as a primary pathway.[4] In plain exam-prep terms, AI may help students produce more ideas or nicer wording while weakening the slower habits that let them test, challenge, and defend those ideas.

The word "unstructured" matters. A student using AI after an attempt is not doing the same thing as a student asking for a solution before thinking. A teacher designing a guarded exercise is not the same as a tired test-taker pasting every missed problem into a chatbot at midnight. The evidence is strongest against the second pattern: convenience before effort.

What This Means for SAT, ACT, GRE, and MCAT Prep

The exam-prep problem is not that ChatGPT exists in the study room. The problem is where it enters the sequence. If it appears before retrieval, it can steal the rep. If it appears after effort, it can sometimes improve the review.

For a student behind schedule, the higher-risk column is tempting because it reduces panic. It turns a blank page into a worked example. It turns a messy passage into a neat summary. It turns uncertainty into fluency. The cost is that the student may never find out whether the skill was actually available without help.

Official-style practice still matters because it exposes transfer failure. Spaced repetition still matters because it forces recall across time. Missed-question review still matters because it makes the student name the cause of the miss instead of admiring the correct solution. Those methods can feel less smooth than an AI explanation, but that roughness is part of the signal.

This is also where AI-study advice often gets too vague. "Use AI responsibly" does not tell a student what to do at 11:40 p.m. with 30 missed GRE quant questions. A better rule is stricter and easier to enforce: retrieve first, attempt first, explain first. Then use ChatGPT as a checker, comparer, or second explanation.

Students who want a broader discussion of chatbot limits in learning can compare this with StudyMethod's piece on AI replacing teachers and the illusion-of-mastery problem. For students already using AI heavily, the guide to why tokenmaxxing backfires is closer to the daily habit problem: too many generated words, too little independent work.

The Boundary That Protects the Score

ChatGPT is most dangerous in exam prep when it gives answers, explanations, summaries, outlines, or solved steps before the student has tried to retrieve or reason independently. That is the point where a useful tool becomes a cognitive prosthetic. It holds up the performance during practice, then disappears during the exam.

Used after effort, it can still have a place. It can compare two explanations, point out a missing assumption, generate another example after the student has solved one, or help review an error log the student already wrote. The sequence is the difference. The student's brain has to go first.

The score risk is not that studying with ChatGPT always feels bad. It is that it often feels good too early. A tool that makes studying easier can also remove the struggle that exam scores are built from.

References

1. Generative AI Can Harm Learning, SSRN, 2024.
2. Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task, arXiv, 2025.
3. Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students, International Journal of Educational Technology in Higher Education, 2024.
4. ChatGPT in Education: A Systematic Review, Computers and Education: AI, 2026.