Daniel Willingham--Science & Education
Hypothesis non fingo
  • Home
  • About
  • Books
  • Articles
  • Op-eds
  • Videos
  • Learning Styles FAQ
  • Daniel Willingham: Science and Education Blog

Draft bill of research rights for educators

8/20/2014

 
This column originally appeared on RealClearEducation.com on July 10, 2014.

When I talk to educators about research, their most common complaint (by a long shot) is that they are asked to implement new interventions (a curriculum, a pedagogical technique, a software product, whatever), and are offered no reason to do so other than a breezy “all the research supports it.” The phrase is used as a blunt instrument to silence questions. As a scientist I find this infuriating because it abuses what ought to be a serious claim—research backs this—and in so doing devalues research. It’s an ongoing problem (see Jessica & Tim Lahey’s treatment here) that’s long concerened me.

In fact, the phrase “research supports it” invites questions. It implies that we can, in a small way, predict the future. It claims “if we do X, Y will happen.” If I take this medication, my ear infection will go away. If we adopt this new curriculum, kids will be more successful in learning math. Saying “research supports it” implies that you know not only what the intervention is, but you have at least a rough idea of what outcome you expect, the likelihood that it will happen, and when it will happen.

I offer the following list of rights for educators who are asked to change what they are doing in the name of research, whether it’s a mandate handed down from administrator to teacher or from lawmaker to administrator.

1.       The right to know what is supposed to improve. What problem is being solved? For example, when I’ve been to schools or districts implementing a one-to-one tablet/laptop policy, I’ve always asked what it’s meant to do. The modal response is a blank look followed by the phrase “we don’t want our kids left behind.” Behind in what? In what way are kids elsewhere with devices zooming ahead?

2.       The right to know the means by which improvement will be measured. How will we know things are getting better? If you’re trying to improve students’ understanding of math, for example, are you confident that you have a metric that captures that construct? Are you sure scores on that metric will be comparable in the future to those you’re looking at now? How big an increase will be deemed a success?

3.       The right to know the approximate time by which this improvement is expected. A commitment to an intervention shouldn’t be open-ended. At some point we must evaluate how it’s going.

4.       The right to know what will be done if the goal is or is not met. Naturally, conditions may change, but let’s have a plan. If we don’t meet our target, will we quit? Keep trying for a while? Tweak it?

5.       The right to know what evidence exists that the intervention will work as expected. Is the evidence from actual classrooms or is it laboratory science (plus some guesswork)? If classrooms, were they like ours? In how many classrooms was it tried?

6.       The right to have your experience and expertise acknowledged. If the intervention sounds to you and your colleagues like it cannot work, this issue should be addressed in detail, not waved away with the phrase “all the research supports it.” The fact that it sounds fishy to experienced people doesn’t mean it can’t work, but whoever is pitching it should have a deep enough understanding of the mechanisms behind the intervention to be able to say why it sounds fishy, and why that’s not a problem.

This list is not meant to dictate criteria that must be met before an intervention should be tried, but rather what information ought to be on the table. In other words, the information provided in each category need not unequivocally support the intervention for it to be legitimate. For example, I can imagine an administrator admitting that the research support for an intervention is absent, yet mounting a case for why it should be tried anyway.

This list should also be considered a work in progress. I invite your additions or emendations.

John Hetts
8/20/2014 06:16:35 am

I generally strongly agree with one caveat. I think there needs to be a commitment to applying the same rigor in understanding/defending current practice. I understand why a change sometimes is held to a higher standard, but part of the reason those pushing for changes/technology adoption with "because science" find that type of reasoning effective is that often very little has been done to document and understand the existing local educational practices and their effects. Having more of that mindset and practice in place locally would help enormously and would also provide the structure against which proposed changes could be compared and rigorously evaluated and against which strong existing practices could be defended against Johnny-come-lately nonsense.
For example, what I find just as problematic (and in need of the same rigorous approach) is the complete absence of this mindset for the evaluation of all kinds of existing programs, curriculums, pedagogical techniques, interventions. Some of these are longstanding and not questioned by anyone (everyone feels good about them/agrees they must work) like DARE, Red Ribbon week, or assemblies that bring in outside performers - jump-ropers seem to be back in – or, worse, motivational speakers with a nominal message like Stay in School or Don't Do Drugs. The rote way we still teach most early mathematics might be in this category as might be the widescale but not very well thought through deployment of token economies/classroom reward systems. These existing practices/pedagody/interventions often absorb enormous resources (time, effort, and money) while deploying widely discredited techniques (i.e, still use messaging that conveys to kids the magnitude of problems by emphasizing how many kids do negative behaviors X or Y (conveying normative information to students about how many people are doing this) and the negative consequences (focusing on fear of punishment often or other negative consequences that most kids expect not to happen to them) rather than emphasizing the much more useful frame of most kids like you never X. And many of them do next to nothing to demonstrate, test, or track their effectiveness.
In addition, having an ongoing default structure in place to think carefully and evaluate the effects of existing local practices and intervention would really help with point 6 - it's one thing to value and acknowledge expertise but it's easier for everyone to do that when that expertise and experience is backed by evidence too (and experience becomes much easier to defend when it’s not being appropriately acknolwedged). If we want to be able to defend our classroom practices, we need to be able to provide evidence of their effectiveness beyond our say so as experts. Paul Meehl and Robyn Dawes did a lot to suggest that expert and experienced judgment didn't always perform better than straightforward rules for following evidence. And the one thing that can ultimately trump the hucksters, shills, and grifters who try to wave us off with "because science" is actual science.

Mewtow link
8/21/2014 03:52:14 am

7 - The right to know why it will work and what are the pedagogical theory behind the tweak/tool/practice/etc.

I think it's important for several reasons :

- first, the intervention will be accepted more easily by teachers (i think) ;
- second, i think it's important to know how implement correctly the intervention ;
- third, it help to know when to use correctly the intervention/tool/etc.
- fourth, it will help teachers to detect interventions that can't work.

Dodiscimus link
8/21/2014 04:47:30 am

In reply to John Hetts comment above. The difference between justifying a new initiative, and justifying the existing one, is that changing to the new one involves extra work. Therefore there needs to be justification for that extra work because something has to give and whether it is time for lesson preparation, marking, calling a parent, or just the teachers' home lives (it's usually the latter) it requires more justification than a lack of evidence in support of what is already in place. Dan - that seems like a pretty solid draft. I think it will find favour with others in the UK but for us, getting through to people that matter in the DfE is quite important although change of personnel may reduce the rate of new initiatives for a while.

Frank Gue
8/21/2014 05:24:30 am

Much educational "reform" is done without a clear objective, as commented by others.

An accepted (supported by research!) definition of "an objective" is that it is:

* Specific (stated with a verb and a noun)
* Numeric
* Scheduled
* Do-able (feasible technically AND POLITICALLY)
* Resourced
* Measurable

John Hetts
8/21/2014 05:27:01 am

@Dodiscimus - Totally agree that the bar for new initiatives should be higher. Making sure we understand what we're doing and its effectiveness does a lot to help raise that bar.

Philip Kerr
8/21/2014 07:32:22 am

The right to know who has paid for the research and what vested interests they may have in the results.

Mike G
8/21/2014 10:06:04 am

With 2 caveats...

a. I wouldn't limit to "Changes in the name of research" but instead to "Changing teacher practice"

b. "Right" overstates my beliefs here

....I would argue that change in a large organization, like a state or district, should come with a real-life example of the "new thing" happening at an actual single school.

Example

"The new 7th grade curriculum will be very different than the 6th grade curriculum. We tried it at XYZ School. You'll hear from 2 of the teachers affected. You'll see video of what happened. We'll describe all the mistakes we made along the way, the concerns of the 2 teachers, and how we responded."

Having tried that in Kenya a few times now, I think it worked! I.e., the power of a "grounded example" with authentic, seemingly typical teachers may be > a detailed recount of the underlying research.


Comments are closed.

    Enter your email address:

    Delivered by FeedBurner

    RSS Feed


    Purpose

    The goal of this blog is to provide pointers to scientific findings that are applicable to education that I think ought to receive more attention.

    Archives

    April 2022
    July 2020
    May 2020
    March 2020
    February 2020
    December 2019
    October 2019
    April 2019
    March 2019
    January 2019
    October 2018
    September 2018
    August 2018
    June 2018
    March 2018
    February 2018
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    April 2017
    March 2017
    February 2017
    November 2016
    September 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016
    December 2015
    July 2015
    April 2015
    March 2015
    January 2015
    September 2014
    August 2014
    July 2014
    June 2014
    May 2014
    April 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    September 2012
    August 2012
    July 2012
    June 2012
    May 2012
    April 2012
    March 2012
    February 2012

    Categories

    All
    21st Century Skills
    Academic Achievement
    Academic Achievement
    Achievement Gap
    Adhd
    Aera
    Animal Subjects
    Attention
    Book Review
    Charter Schools
    Child Development
    Classroom Time
    College
    Consciousness
    Curriculum
    Data Trustworthiness
    Education Schools
    Emotion
    Equality
    Exercise
    Expertise
    Forfun
    Gaming
    Gender
    Grades
    Higher Ed
    Homework
    Instructional Materials
    Intelligence
    International Comparisons
    Interventions
    Low Achievement
    Math
    Memory
    Meta Analysis
    Meta-analysis
    Metacognition
    Morality
    Motor Skill
    Multitasking
    Music
    Neuroscience
    Obituaries
    Parents
    Perception
    Phonological Awareness
    Plagiarism
    Politics
    Poverty
    Preschool
    Principals
    Prior Knowledge
    Problem-solving
    Reading
    Research
    Science
    Self-concept
    Self Control
    Self-control
    Sleep
    Socioeconomic Status
    Spatial Skills
    Standardized Tests
    Stereotypes
    Stress
    Teacher Evaluation
    Teaching
    Technology
    Value-added
    Vocabulary
    Working Memory