Daniel Willingham--Science & Education
Hypothesis non fingo
  • Home
  • About
  • Books
  • Articles
  • Op-eds
  • Videos
  • Learning Styles FAQ
  • Daniel Willingham: Science and Education Blog

Adaptive practice, personalized learning, and what will "obviously" work in education.

6/5/2017

 
We cannot remind ourselves often enough that the predictions for education drawn from the learning sciences that obviously, 100%, HAVE to work…often don’t.

The latest example comes from a recent paper reporting a randomized control trial of adaptive vs. static practice in Dutch schools.

It seems self-evident that adaptive practice will be superior to static. In static practice, each student receives the same set of practice materials of graded difficulty: easy, medium, hard, with difficulty defined by the performance of a large cohort of students. In the adaptive algorithm, the proportion of questions correctly answered is factored into the probability of seeing a particular type of question: if a student is getting all of the easy questions right, what’s the point of posing more? Why not move on to more challenging content?

Adaptive practice is one of the reasons offered that personalized learning ought to lead to greater achievement.

The experiment tested 1,051 Dutch 7th-9th graders studying either Dutch, Biology, History, or Economics over the course of one academic year. Assignment to static or adaptive practice was assigned by classroom, and students were blind to condition. All students received the same instruction (a hybrid of a digital environment and traditional paper textbook) and all homework was the same. The independent variable was implemented only through extra practice; students were asked to practice at least 15 minutes per week, but any more practice than that was taken at their own initiative.  
We might expect that, because students are practicing at their own initiative, they will use the adaptive program less, given that the problems will likely be more difficult. The proportion of students who used the practice module did not differ between conditions, hovering around 90% in both cases. Students in the adaptive condition did indeed work more difficult problems and they also practiced a little bit longer per session…but they worked fewer problems than students in the static condition. Presumably, more difficult problems required more time per problem.

Nevertheless, they showed no advantage on a summative test. In fact, better prepared students (those who had passed the summative test before the experiment began) were slightly negatively impacted by the adaptive regimen compared to the static. (There was no effect for students who had failed the last summative test.)

Post-experimental interviews showed that students did not know whether their practice had been adaptive or static, and showed no difference in students’ attitudes towards the practice.

Why was there not a positive effect of adaptive testing?

One possibility is low dosage. The intervention was only 15 minutes per week and although students could have practiced more, few did. At the same time, the intervention lasted an entire school year, the N was fairly large, and an effect was observed (in the unexpected direction) for the better prepared students.

Another possibility is that the program was effective in getting challenging problems to students, but ineffective in providing instruction. Students in the adaptive condition saw more difficult problems, but they got a lot of them wrong. Perhaps they needed more support and instruction at that point, so the potential benefit of stretching their knowledge and skills was not realized.

Another possibility is that the adaptive group would have shown a benefit on a different outcome measure. As the authors note, the summative test was more like the static practice than the adapative practice. Perhaps the adapative group would have shown a benefit in their readiness and ability to learn in the next unit.

​This result obviously does not show that adaptive practice is a bad idea, or cannot be made to work well. It simply adds to the list of ideas that sound like they are more or less foolproof that turn out not to be: think spiral curriculum, or electronic textbooks. Thinking and learning is simply too complicated for us to confidently predict how a change in one variable will affect the entire cognitive and conative systems.
 

 
T. Grant
6/5/2017 10:47:59 am

There are other possibilities for the outcomes too, such as not getting a few easy questions now and then to boost confidence (and to help battle fatigue). The "adaptive" questions were chosen for the students - so students were not able to match their current energy level for the present task and did not have power or choice over their learning as Glasser says is essential (so the curriculum was only extrinsically adaptive, not intrinsically adaptive).

Also, many components of the spiral curriculum (such as spacing effects) have shown to be effective.

I agree that our practice should be based on evidence and not what think "should" work. Nevertheless, like a three legged stool, some practices required that many conditions be present simultaneously to be effective. Therefore, testing those concepts in isolation is almost guaranteed to fail. It is a case of the whole being more than the sum of its parts.

Don Crawford link
6/5/2017 08:13:31 pm

Let's see. Practice on "harder problems" but without instruction. How would that ever work? Practice on harder problems but testing on easier ones? Hmmm. Wonder why that didn't work? What about pretesting to see if the students know a given problem type, then giving instruction and then practice on that problem type and then testing on it? That might work. Seems that ought to be the starting point of educational experiments.

Paul Hoss
6/6/2017 02:46:17 pm

"...personalized learning ought to lead to greater achievement," if performed correctly. Personalized learning actually makes the student the center of the classroom, not the teacher; as in the failed pedagogical paradigm of (at least) the past century. Brighter students are allowed to progress at a faster pace while other students are afforded the time necessary to "master" the material being studied. As Georgetown's James Carnevale has opined, this is the Goldilocks approach, where the pace of instruction is "just right' for each student.


Comments are closed.

    Enter your email address:

    Delivered by FeedBurner

    RSS Feed


    Purpose

    The goal of this blog is to provide pointers to scientific findings that are applicable to education that I think ought to receive more attention.

    Archives

    July 2020
    May 2020
    March 2020
    February 2020
    December 2019
    October 2019
    April 2019
    March 2019
    January 2019
    October 2018
    September 2018
    August 2018
    June 2018
    March 2018
    February 2018
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    April 2017
    March 2017
    February 2017
    November 2016
    September 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016
    December 2015
    July 2015
    April 2015
    March 2015
    January 2015
    September 2014
    August 2014
    July 2014
    June 2014
    May 2014
    April 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    September 2012
    August 2012
    July 2012
    June 2012
    May 2012
    April 2012
    March 2012
    February 2012

    Categories

    All
    21st Century Skills
    Academic Achievement
    Academic Achievement
    Achievement Gap
    Adhd
    Aera
    Animal Subjects
    Attention
    Book Review
    Charter Schools
    Child Development
    Classroom Time
    College
    Consciousness
    Curriculum
    Data Trustworthiness
    Education Schools
    Emotion
    Equality
    Exercise
    Expertise
    Forfun
    Gaming
    Gender
    Grades
    Higher Ed
    Homework
    Instructional Materials
    Intelligence
    International Comparisons
    Interventions
    Low Achievement
    Math
    Memory
    Meta Analysis
    Meta-analysis
    Metacognition
    Morality
    Motor Skill
    Multitasking
    Music
    Neuroscience
    Obituaries
    Parents
    Perception
    Phonological Awareness
    Plagiarism
    Politics
    Poverty
    Preschool
    Principals
    Prior Knowledge
    Problem-solving
    Reading
    Research
    Science
    Self-concept
    Self Control
    Self-control
    Sleep
    Socioeconomic Status
    Spatial Skills
    Standardized Tests
    Stereotypes
    Stress
    Teacher Evaluation
    Teaching
    Technology
    Value-added
    Vocabulary
    Working Memory