Part Two

Learn First, Then Use AI

Based on the paper "Learn First, Then Use AI: Testing an Expert Consensus on Generative AI and Learning Against the Empirical Evidence" by Paul Penfold (Proceedings of Future Science Archives, 2026).

Part One argued that choices decide the future. This part looks at one of the most important choices a teacher or school makes: when in the learning process a learner should turn to AI. It tests what thirty experts said against the studies published so far, and ends with a rule short enough to remember.

Chapter 7 The wrong question: does AI help or harm learning?

Within three years of the public release of ChatGPT, generative AI went from novelty to everyday tool for students. It drafts essays, solves problems, summarises readings and explains ideas on demand. Schools have responded unevenly. Some ban it, some embrace it, and many leave the decision to individual teachers.

Both extremes rest on the same assumption: that generative AI has one overall effect on learning. But learning isn't one thing. It includes paying attention, encoding new information, practising, getting feedback, reasoning, collaborating and transferring ideas to new problems. A tool that helps one of these may damage another.

A recent expert consensus report takes this seriously. Kendeou, Greene, Nixon and colleagues (2026) [60] asked 30 experts in learning and education to rate how generative AI is likely to affect 65 distinct learning processes. Their answer is that the useful question isn't whether AI helps learning. It's which process it touches, and when.

Expert consensus isn't the same as evidence, though. The report itself rates the strength of evidence behind most processes as low. So the question for this part is a simple one: do the studies published so far support the experts' picture? I map the main studies onto the sequence the report describes, then draw out what follows for practice. The newsletter edition that first set out these ideas is listed at the end of the part (Penfold, 2026) [118].

Chapter 8 What thirty experts found

How the report worked

The report used a preregistered Delphi method, in which experts rate items over several rounds and see the group's responses between rounds. Thirty experts identified 65 learning processes across eight categories, from motivation and memory to critical thinking, collaboration and transfer. Each process was rated on a 1 to 5 scale for how likely general-purpose generative AI is to enhance it, and how likely it is to disrupt it. A process counted as reaching consensus when its mean rating was 3.0 or above and at least 75% of experts rated it 3, 4 or 5, for enhancement or for vulnerability. Consensus was reached on 42 of the 65 processes.

Two limits of scope matter. The experts were asked about general-purpose chatbots such as ChatGPT, Claude and Gemini, used by learners on their own. They weren't asked about tools designed for teaching, or about AI built into other software. And the report concerns K–20 education in the United States.

A sequence with risks at both ends

The findings are easiest to read as a three-stage sequence.

• Acquiring something new. Encoding, information acquisition and retrieval reached consensus on vulnerability, not enhancement. AI can tempt learners to skip the effort of taking in and recalling new information.
• Consolidating it. Learning and studying strategies stand out as the most positive category. All seven consensus processes there are enhancement signals: learning from feedback, self-testing, learning from examples, tutoring, step-by-step support, learning from questioning and deliberate practice.
• Applying it in new ways. The experts are least optimistic here. Of 14 higher-order processes reaching consensus, including critical thinking, metacognition and self-regulated learning, 10 were rated vulnerable with no matching benefit.

Three further findings sit alongside this sequence. All five collaboration processes that reached consensus (social connection, socio-emotional development, social learning, collaborative learning and sense of belonging) did so on vulnerability only. Younger learners were judged more vulnerable than older ones. And in domain learning, only writing reached consensus, with high risk. The lack of consensus on mathematics, STEM reasoning and reading reflects real disagreement among the experts, not agreement that AI has no effect.

What the experts recommended

The experts proposed 11 recommendations, rated for impact and feasibility. Three scored high on both: teach generative AI literacy, use AI for adaptive support such as tutoring, hints and feedback, and bring AI in for brainstorming only once learners have set out their own thinking. Those judged hardest to carry out were gating AI use on demonstrated competence, redesigning assessment, and inclusion and equity.

Chapter 9 What learning science would predict

The experts' pattern isn't a surprise to anyone who knows the learning science. It's what established research would predict. Four ideas do most of the work.

Desirable difficulties. Conditions that make learning feel harder in the moment, such as spacing, interleaving and generating answers rather than reading them, often produce stronger long-term retention and transfer (Bjork, 1994) [13].

Retrieval practice. Actively recalling information strengthens memory far more than re-reading it (Roediger & Karpicke, 2006) [164]. An answer requested from AI is a retrieval opportunity lost.

The generation effect. Information that learners produce themselves is remembered better than information they are given (Slamecka & Graf, 1978) [170].

Cognitive load theory. Working memory is limited, and novices learn best with guidance that reduces irrelevant load while keeping the essential processing (Sweller, 1988) [177]. A well-designed tutor can reduce unhelpful load. An answer engine can remove the essential processing altogether.

Put these together and the shape of the consensus follows. AI will hurt acquisition when it replaces effortful encoding and retrieval. It will help consolidation when it adds feedback, questioning and practice. And it will threaten higher-order thinking when it does the reasoning for the learner.

Chapter 10 Four studies tested against the consensus

This chapter is the main place in the book where the best-known studies on AI and learning are set out in full. Later parts refer back to it.

Acquisition: answer-giving harms learning

The clearest evidence comes from a field experiment with nearly a thousand high-school mathematics students in Turkey (Bastani et al., 2025) [11]. Students practised with one of two GPT-4-based tools, or without AI. "GPT Base" behaved like a standard chatbot. "GPT Tutor" was prompted to give hints rather than answers.

During practice, both AI groups did much better than the control group. But when AI was removed for an exam, students who had used GPT Base did significantly worse than students who had never used it, largely because they had used it to get answers. The tutor version largely removed this harm, though it didn't produce a clear gain on the exam.

This matches the consensus on encoding and retrieval closely: unrestricted AI improves performance in the moment while weakening what is learned. It also adds a nuance the experts couldn't test, since they rated general-purpose chatbots only. Design changes the outcome.

Consolidation: tutor-style AI can produce large gains

A randomised controlled trial in an undergraduate physics course at Harvard compared an AI tutor with an active-learning class taught by experienced instructors (Kestin et al., 2025) [62]. The tutor was built on research-based teaching practice. It took problems step by step, encouraged students to think before revealing answers and managed cognitive load. Students learned significantly more in less time with the tutor, and reported more engagement and motivation.

This is strong support for the consensus on learning and studying strategies, where tutoring, feedback and learning from questioning were all rated as enhanced. It's worth noting that this tutor was purpose-built rather than a general chatbot, and that students met it inside a structured course. The evidence suggests the enhancement the experts expect is most likely when AI is designed around how people learn.

A caution about pooled evidence

Early attempts to pool the evidence need care. At least one widely cited 2025 meta-analysis reporting large positive effects of ChatGPT on learning performance and higher-order thinking has since been retracted. That underlines the report's call for rigour over speed.

Writing: offloading leaves a cognitive debt

Writing was the only domain to reach consensus, and the experts saw high risk. An EEG study from the MIT Media Lab compared participants who wrote essays with an LLM, with a search engine or with no tools (Kosmyna et al., 2025) [64]. The LLM group showed the weakest patterns of brain connectivity, struggled more to quote their own essays and reported a weaker sense of ownership. Participants who started with the LLM and later wrote without it didn't simply return to the engagement seen in the no-tools group. The authors call this "cognitive debt". The study is small and was released as a preprint, so it should be read with caution, but it fits the experts' concern.

Higher-order thinking: offloading is linked with weaker critical thinking

Survey research points the same way. Gerlich (2025) [48] found a significant negative relationship between frequent AI use and critical-thinking scores, mediated by cognitive offloading, with younger participants showing more dependence. This echoes the experts' view that younger learners are more vulnerable. A survey of knowledge workers by researchers at Microsoft and Carnegie Mellon found that greater confidence in generative AI was associated with less critical thinking, and greater self-confidence with more (Lee et al., 2025) [67]. Both studies are correlational and can't show cause and effect, but they are consistent with the experts' one-sided judgement of risk.

Chapter 11 Where the evidence and the consensus part

The fit between the experts and the studies is good, but it isn't perfect, and the gaps matter.

Design versus general-purpose tools. The consensus concerns learners using general chatbots on their own. The strongest positive evidence comes from purpose-built tutors. So the experts' pessimism about acquisition may be less warranted for well-designed tools, and their optimism about consolidation may be overstated for unstructured chatbot use.

Higher-order thinking. Some short, structured interventions report gains on immediate higher-order tasks. The experts' concern is different: whether learners develop the habit and capacity to think independently over time. Studies so far rarely measure that.

Collaboration. The experts' consensus on vulnerability in five social processes is a significant warning, but I found little direct experimental evidence either way. That's a clear gap.

Rigour. The report rates evidence strength as low across the 42 consensus processes and calls for rigour over speed. The studies reviewed here vary in size, duration and design, and several are preprints or correlational. Agreement between expert judgement and early evidence makes the pattern more credible, but it isn't settled yet.

Chapter 12 Learn first, then use AI: the rule and what it means for schools

Read together, the consensus and the evidence point to a principle about sequence more than a simple yes or no.

Stage 1: Learn without AI doing the thinking. When learners first meet new ideas, they need to encode, retrieve, generate and make mistakes. Answer-giving AI short-circuits this. If AI is used at all, it should act as a tutor that asks questions and gives hints rather than solutions.

Stage 2: Use AI to consolidate. Once something has been learned, AI can help strengthen it: building quizzes, giving feedback, probing for explanations, offering worked examples and supporting deliberate practice. This is where both the experts and the evidence are most positive.

Stage 3: Put your own thinking first when applying ideas. When learners reason, evaluate, create or transfer ideas to new contexts, they should set out their own thinking before turning to AI. AI can then act as a critical partner that challenges, questions and offers alternatives, but not as a substitute for judgement.

This is consistent with UNESCO's guidance on generative AI in education and research, which calls for a human-centred approach, age-appropriate use and protection of human agency (UNESCO, 2023) [184].

For teachers

• Be clear about where in the learning cycle each task sits: first meeting an idea, consolidating it or applying it.
• Prefer tutor-style tools that ask questions and give graduated hints over open answer engines, especially for new material.
• Assess the thinking, not just the product: drafts, reasoning, oral explanation and reflection on how AI was used.
• Keep retrieval practice, spaced practice and some work done without AI.
• Ask students to set out their own ideas before using AI to brainstorm or critique.
• Protect tasks that depend on students working with each other.

For school leaders

• Write AI guidance that says more than "allowed" or "banned", and that distinguishes stages of learning.
• Teach generative AI literacy explicitly: how it works, where it falls short and how to check its output.
• Give younger learners more protection from offloading.
• Invest in professional development for designing AI-aware tasks and assessment.
• Pilot locally and measure learning, not just engagement or task completion.

For researchers

• Run longer, well-powered trials that measure durable learning and transfer after AI is removed.
• Compare general-purpose chatbots with purpose-built tools under the same conditions.
• Study the effects on collaboration, belonging and socio-emotional development, where expert concern is high and evidence is thin.
• Test the sequencing principle directly by varying when AI is introduced.

What this part can't tell you

The review behind this part is a narrative one, not a systematic one. I chose the studies to test the consensus framework rather than gathering them exhaustively. The consensus report concerns US education and general-purpose chatbots, and its conclusions are expert judgements. The empirical studies differ in population, tool, subject and outcome measure, and the tools they used are already being replaced by newer models. Treat the conclusions as working hypotheses to be tested in real classrooms.

The rule in one paragraph

A group of 30 experts concluded that generative AI threatens the start of learning, can support its middle, and does not yet reliably support its end. The early empirical evidence largely agrees. Answer-giving AI harms acquisition, tutor-style AI can strengthen consolidation, and offloading thinking is linked with weaker critical thinking. The practical rule is plain: learn first, then use AI to strengthen what you know, and put your own thinking first when you apply it. Schools that design for this sequence are more likely to get the benefits of AI without paying for them in lost learning.

Related newsletter articles

The ideas in this part first appeared in shorter form in my newsletter. The Minnesota consensus edition is "Learn first, then use the AI: what 30 experts say about generative AI and learning" (Penfold, 2026) [118]. Related editions include "AI is making students perform better and learn less" (Penfold, 2026) [100], "The Lazy Brain: Why Convenience Is the Enemy of Intelligence" (Penfold, 2026) [139], "The Confidence Trap: Why Flawless Logic Isn't Always Truth" (Penfold, 2026) [129], "In a World of Confident Machines, Critical Thinking Is the Premium Skill" (Penfold, 2026) [116] and "Schools Are Buying AI on Hope. Here's the Evidence They're Skipping" (Penfold, 2026) [123].