Daniel Willingham--Science & Education
Hypothesis non fingo
  • Home
  • About
  • Books
  • Articles
  • Op-eds
  • Videos
  • Learning Styles FAQ
  • Daniel Willingham: Science and Education Blog

Better studying = less studying. Wait, what?

7/8/2013

 
Readers of this blog probably know about "the testing effect," later rechristened "retrieval practice." It refers to the fact that trying to remember something can actually help cement things in memory more effectively than further study.

A prototypical experiment looks like this (rows = subject groups; columns = phases of the experiment).
Picture
The critical comparison is the test in Phase three of the experiment; those who take a test during Phase 2 do better than those who study more.. There are lots of experiments replicating the effect and accounting for alternative explanations (e.g., motivation. See Agarwal, Bain & Chamberlain, 2012 for a review).

A consistent finding is that the benefit to memory is larger if the test is harder. But of course if the test is harder, then people might be more likely to make mistakes on the test in Phase 2. And if you make mistakes, perhaps you will later remember those incorrect responses.

But data show that, even if you get the answer wrong during Phase 2 you'll still see a testing benefit so long as you get corrective feedback. (Kornell, Hays & Bjork, 2009).

A tentative interpretation is that you get the benefit because the right answer is lurking in the background of your memory and is somewhat strengthened, even though you didn't produce it.

So that implies that the testing effect won't work if you simply don't know the answer at all. Suppose, for example, that I present you with an English vocabulary word you don't know and either (1) provide a definition that you read (2) ask you to make up a definition or (3) ask you to choose from among a couple of candidate definitions. In conditions 2 & 3 you obviously must simply guess. (And if you get it wrong I'll give you corrective feedback.) Will we see a testing effect?

That's what Rosalind Potts & David Shanks set out to find, and across four experiments the evidence is quite consistent. Yes, there is a testing effect. Subjects better remember the new definitions of English words when they first guess at what the meaning is--no matter how wild the guess.

Guessing by picking from amongst meanings provided by the experimenter provides no advantage over simply reading the definition. So there is something about the generation in particular that seems crucial.
Picture
Results of four experiments in Potts & Shanks, performance on final test. Error bars = standard errors.
What's behind this effect? Potts & Shanks think it might be attention. They suggest that you might pay more attention to the definition the experimenter provides when you've generated your own guess because you're more invested in the problem. Selecting one of the experimenter-provided definitions is too easy to provide this feeling of investment.

This account is speculation, obviously, and the authors don't pretend it's anything else. I wish that they were equally circumspect in their guess at the prospects for applying this finding in the classroom. Sure, it's an important piece of the overall puzzle, but I can't agree that "this line of research is relevant to any real world situation where novel information is to be learned, for example when learning concepts in science, economics, politics, philosophy, literary theory, or art."

The authors in fact cite two other studies that found no advantage for generating over reading, but Potts & Shanks think they have an account for what made those studies not very realistic (relative to classrooms) and what makes their conditions more realistic. They may yet be proven right, but college students in a lab studying word definitions is still a far cry from "any real world situation where novel information is to be learned."

The today-the-classroom-tomorrow-the-world rhetoric is over the top, but it's an interesting finding that may, indeed, prove applicable in the future.

References:

Agarwal, P. K., Bain, P. M. & Chamberlain, R. W. (2012). The value of applied research: Retrieval practice improves classroom learning and recommendations from a teacher, a principal, and a scientist. Educational Psychology Review, 24,  437-448.

Kornell, N., Hays, M. J., & Bjork, R. A. (2009). Unsuccessful retrieval
attempts enhance subsequent learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 989–998

Potts, R., & Shanks, D. R. (2013, July 1). The Benefit of Generating Errors During Learning. Journal of Experimental Psychology: General. Advance online publication. doi:10.1037/a0033194






Steve Peha link
7/8/2013 01:41:56 am

Dan,

I've had a lot of success in classrooms with kids of all ages using what I think the study here is referring to here as "generating over reading". When kids are reading a book that is above their reading level, they often run across words they don't know. They're natural inclination seems to be to skip them entirely. This causes problems on many levels, especially in non-fiction content area reading where the unfamiliar words are often the very ones I want most for kids to learn. (By contrast, I have seen no positive effect of giving kids the meanings of what I think will be the unfamiliar words before we start reading.)

The positive effect of generating over a text usually occurs for me when I teach kids a quick little routine I call Question-Infer-Clarify. When they come across an unknown word, they pose a question about it, they infer an answer (just a guess), and then they re-read the sentence, popping in the guess to see if the guess can clarify the meaning. Sometimes it works, but most of the time it doesn't.

However, when kids do this across a page or two, with maybe 3-5 words they don't know, they do seem to have a better overall understanding of the text. At this point, my offering corrective feedback seems to work better, too.

I don't know why this works exactly. I suspect it works because of a side effect: rather than skipping over a word, the Question-Infer-Clarify approach forces a re-reading of a sentence with a challenging word. They end up with a better understanding of sentences with important words in them and maybe that helps with overall understanding.

As readers move through a text with Question-Infer-Clariry, their comprehension seems to hang together a little better than it does when they skip words or make random guesses. And, of course, once in a while they get the meaning correctly or at least their guess is close enough to have a gist of what the text is all about.

This isn't the same, I don't think, as the laboratory condition mentioned in the study above, but it seems similar: kids are generating potentially correct definitions of words as they encounter words they don't know—and at least attempting to self-generate corrective feedback.

I don't know how to measure this effectively in a classroom situation. I also suspect that reading unfamiliar words in context, especially rereading them, helps a little. I see what I think is a positive effect of this most readily when I am working with non-native English speakers.

bergkamp10
7/9/2013 03:33:54 am

This is interesting. The technique I suggest to students for revision follows this model in some form:

1) Study (read material, solve some problems, answer some questions) for 35 mins.

2) Take a break (10 mins)

3) Test your self on what you just studied (5 mins). Analyse and study again as necessary (20 mins)

4) Test yourself again (5 mins)

5) Take a break and repeat.

It is not a perfect fit but I think this model works well at it provides for a feedback loop. It seems to me that the model at the start of this post could be revised to ensure the feedback is included. Maybe it would look like this:

Learn - Test - Evaluate/Review - Test

I do have questions however. This research seems to be limited to very specific knowledge based memory - where would skill development come into this issue?

Erica Kleinknecht link
7/11/2013 10:52:05 am

I haven't systematically investigated the testing effect to address generation vs selection issues, but I did recently apply the concept in an Intro to Psych class, where students did both, and in both cases they received immediate feedback. Once per week students took a 10 - 15 item MC test, then swapped and scored a neighbor's exam and I fielded questions if any arose. As well, I built in "think-pair-share" review moments where I presented an open-ended short answer question that they were to write a response to on a note card, then swap and discuss. I (with the help of my TA) kept track of students' performance though they weren't graded on their competency. No surprise that their quiz scores significantly positively correlated with their actual exam scores (I only gave two mid-terms). My impressions were such that the regular testing in class prompted students to both study in more regular intervals (ah, distributed practice) and to pay better attention in class. Many students remarked in course evals that they appreciated the regular testing. As well, even with only two exams -- something that students dread on the face of it - I found that my grade distribution was much better than usual. The only students who failed were the students who didn't fully engage with the program (poor attendance, lack of participation, failure to register for the on-line materials yoked with the text, and the like).

Whereas my Intro class can't be used to help solve the problem you discuss - what is the testing effect mechanism?- it does serve as an example of how testing effects can work in a classroom setting.

If I had to hazard a guess as to what is going on with the effect, I'd say that it stimulates more active engagement which enables (or affords) more internal repetition. If students are rotely studying, when they close their book, it's out-of-sight, out-of-mind. But when they engage in regular testing with feedback where they have to think about the material, that effort leaves them with something to think about later, that is, they rehearse it more and better connect the material to what they already know - kind of like an "encoding specificity effect" where they act of studying leaves a richer episodic trail in mind that eventually takes less effort to reconstruct.

mike.goldstein
7/20/2013 09:04:08 am

Someone smart once said "Memory is the residue of thought." I wonder if that's involved in this somehow........


Comments are closed.

    Enter your email address:

    Delivered by FeedBurner

    RSS Feed


    Purpose

    The goal of this blog is to provide pointers to scientific findings that are applicable to education that I think ought to receive more attention.

    Archives

    April 2022
    July 2020
    May 2020
    March 2020
    February 2020
    December 2019
    October 2019
    April 2019
    March 2019
    January 2019
    October 2018
    September 2018
    August 2018
    June 2018
    March 2018
    February 2018
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    April 2017
    March 2017
    February 2017
    November 2016
    September 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016
    December 2015
    July 2015
    April 2015
    March 2015
    January 2015
    September 2014
    August 2014
    July 2014
    June 2014
    May 2014
    April 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    September 2012
    August 2012
    July 2012
    June 2012
    May 2012
    April 2012
    March 2012
    February 2012

    Categories

    All
    21st Century Skills
    Academic Achievement
    Academic Achievement
    Achievement Gap
    Adhd
    Aera
    Animal Subjects
    Attention
    Book Review
    Charter Schools
    Child Development
    Classroom Time
    College
    Consciousness
    Curriculum
    Data Trustworthiness
    Education Schools
    Emotion
    Equality
    Exercise
    Expertise
    Forfun
    Gaming
    Gender
    Grades
    Higher Ed
    Homework
    Instructional Materials
    Intelligence
    International Comparisons
    Interventions
    Low Achievement
    Math
    Memory
    Meta Analysis
    Meta-analysis
    Metacognition
    Morality
    Motor Skill
    Multitasking
    Music
    Neuroscience
    Obituaries
    Parents
    Perception
    Phonological Awareness
    Plagiarism
    Politics
    Poverty
    Preschool
    Principals
    Prior Knowledge
    Problem-solving
    Reading
    Research
    Science
    Self-concept
    Self Control
    Self-control
    Sleep
    Socioeconomic Status
    Spatial Skills
    Standardized Tests
    Stereotypes
    Stress
    Teacher Evaluation
    Teaching
    Technology
    Value-added
    Vocabulary
    Working Memory