My new book, When Can You Trust the Experts: How to Tell Good Science from Bad in Education is now available. (There’s a link for a free download of Chapter 1 on this page.) EDIT (7:17 pm 7/18) I just saw that it's listed as "Recommended," with a micro-review on the Scientific American website.
I wrote the book out of frustration with a particular problem: the word “research” has become meaningless in education. Every product is claimed to be research-based. But we all know that can’t be the case. How are teachers and administrators supposed to know which claims are valid?
It’s notable that this problem exists in many other fields. However good your training, research doesn’t stand still. So how does a pediatrician who has been in practice 10 years know that what she learned in medical school as the optimal treatment for, say, croup, is still the best treatment?
The answer is NOT that the pediatrician is expected to comb through medical journals each night to keep up to date. Who has the time? Instead, reliable summaries are published (usually annually) that bring doctors up to date.
But there is no institution in education that publishes such summaries, and indeed, there’s not an institution that would have the credibility to do so. The Institute of Educational Sciences tried with the What Works Clearinghouse, but it has not been a success, mostly because the criteria it used to evaluate products was so stringent, and also because (at least initially) it was dogged by suspicions that its conclusions would not be politically neutral.
Teachers and administrators are on their own.
But doing a good job of evaluating research carries two requirements: you need a certain level of expertise in evaluating research (which requires knowing the existing research literature) AND you need a good deal of time.
The purpose of When Can You Trust the Experts is to offer a short cut, a work-around, so that you can size up the likelihood that a claim like “all the research supports it” is actually true.
The first half of the book focuses on what cues we use to tell us “this is probably true,” and how they can be misleading. You need to understand why we believe what we do in order to make it less likely that you’ll believe something that is false,
I also describe at some length how when can know when science might help with a particular problem and when it can’t. I think science is terrific for answering certain types of questions, but there are many questions it can’t answer.
The second half of the book describes the short cut, which consists of four steps:
I should also mention that When Can You Trust the Experts makes a splendid gift for weddings, anniversaries, birthdays, christenings, etc.
Free download of Chapter 1 on this page.
I wrote the book out of frustration with a particular problem: the word “research” has become meaningless in education. Every product is claimed to be research-based. But we all know that can’t be the case. How are teachers and administrators supposed to know which claims are valid?
It’s notable that this problem exists in many other fields. However good your training, research doesn’t stand still. So how does a pediatrician who has been in practice 10 years know that what she learned in medical school as the optimal treatment for, say, croup, is still the best treatment?
The answer is NOT that the pediatrician is expected to comb through medical journals each night to keep up to date. Who has the time? Instead, reliable summaries are published (usually annually) that bring doctors up to date.
But there is no institution in education that publishes such summaries, and indeed, there’s not an institution that would have the credibility to do so. The Institute of Educational Sciences tried with the What Works Clearinghouse, but it has not been a success, mostly because the criteria it used to evaluate products was so stringent, and also because (at least initially) it was dogged by suspicions that its conclusions would not be politically neutral.
Teachers and administrators are on their own.
But doing a good job of evaluating research carries two requirements: you need a certain level of expertise in evaluating research (which requires knowing the existing research literature) AND you need a good deal of time.
The purpose of When Can You Trust the Experts is to offer a short cut, a work-around, so that you can size up the likelihood that a claim like “all the research supports it” is actually true.
The first half of the book focuses on what cues we use to tell us “this is probably true,” and how they can be misleading. You need to understand why we believe what we do in order to make it less likely that you’ll believe something that is false,
I also describe at some length how when can know when science might help with a particular problem and when it can’t. I think science is terrific for answering certain types of questions, but there are many questions it can’t answer.
The second half of the book describes the short cut, which consists of four steps:
- Strip it. Clear away the verbiage and look at the actual claim. What exactly is the claim suggesting a teacher or parent should do, and what outcome is promised?
- Trace it. Who created this idea, and what have others said about it? It's common to believe something because an authority confirms it, and this is often a reasonable thing to do. I think people rely heavily on credentials when evaluating education research, but I argue that it’s a weak indicator of truth.
- Analyze it. Why are you being asked to believe the claim is true? What evidence is offered, and how does the claim square with your own experience?
- Should I do it? You're not going to adopt every educational program that is scientifically backed, and it may make sense to adopt one that has not been scientifically evaluated.
I should also mention that When Can You Trust the Experts makes a splendid gift for weddings, anniversaries, birthdays, christenings, etc.
Free download of Chapter 1 on this page.
