Daniel Willingham--Science & Education
Hypothesis non fingo
  • Home
  • About
  • Books
  • Articles
  • Op-eds
  • Videos
  • Learning Styles FAQ
  • Daniel Willingham: Science and Education Blog

What science teachers need to know

10/14/2013

 
The results of this experiment probably won't surprise you. What surprised me was the fact that we didn't already have data like this in hand.

The researchers (Sadler et al., 2013) tested 181 7th and 8th grade science teachers for their knowledge of physical science in fall, mid-year, and years end. They also tested their students (about 9,500) with the exact same instrument.

Each was a twenty-item multiple choice test. For 12 of the items, the wrong answers tapped a common misconception that previous research showed middle-schoolers often hold. For example, one common misconception is that burning produces no invisible gases. This question tapped that idea:
Picture
But the researchers didn't just ask the teachers to pick the right answer. They also asked teachers to pick the answer that they thought their students would pick.

What makes this study interesting is that it tests teacher subject-matter knowledge directly (instead of using a proxy like courses taken, or degrees) and that it directly measures one aspect of pedagogical content knowledge, namely, student misconceptions. The dependent measure of interest is student gain scores in content knowledge over the course of the year.

The results?

Teachers content knowledge was good, but not perfect. They got about 84% of the questions right.

Their knowledge of student misconceptions was not as good. Teachers correctly identified just 43% of those. (And their students had, as in previous studies, selected those incorrect items in high numbers.)

And what type of teacher knowledge matters to student learning? It turns out to interact with past student achievement, as measured by standard math and reading tests.
Picture
The graph shows gains in student knowledge, separated by items for which teachers have (or lack) various types of knowledge. Filled circles are for students who scored well on a math and reading test (high achievers), and open circles are students who scored poorly (low achievers)

Look first at learning for concepts without  a common misconception. If teachers have subject matter knowledge (SMK in the graph) students learn the concept better. In fact, low-achieving students learned nothing about a concept if teachers didn't know the concept themselves. High-achieving students did. The researchers speculate they may have learned the content from a textbook or other source.

For the strong misconception items, the low-achieving students learned very little, whatever the teacher knowledge. For high-achieving students, knowledge mattered, and they were most likely to learn when their teacher had both subject-matter knowledge and knew the misconceptions their students likely held (KoSM in the graph).

So the overall message is not that surprising. Students learn more when their teachers know the content, and when they can anticipate student misconceptions.

Somewhat more surprising (and saddening), low-achieving students are especially vulnerable when teachers lack knowledge. High-achieving students are more resilient.

There are limitations to this study, the most notable being that the sample is far from random (teachers were volunteers), and that the test was zero-stakes for all.

The strength was the direct measure of both types of knowledge, and that the researchers could examine the relationship of knowledge to performance at the level of individual items. One hopes we'll see more studies using this type of design.

Reference:
Sadler, P. M., Sonnert, G., Coyle, H.P., Cook-Smith, N., & Miller, J.L. (2013) Student learning in middle school science classrooms. American Educational Research Journal, 50, 1020-1049.
Douglas Hainline
10/14/2013 12:38:56 pm

We have to go to war with the army we have. So how can we help all teachers, those with, and those without, subject matter knowledge, to get better at teaching science?

Why doesn't some benevolent millionaire fund the making of a series of videos, each of which addresses a common misconception? (Many of them exist already, as a search of YouTube will reveal.) Then we campaign for knowledge of how to access these free videos, and associated supporting material (simple printed explanations, quizes, interactive simulations, games), to be made widely available.

This fellow (http://www.conceptualphysics.com/pghewitt.shtml) has already done what was needed for physics ... perhaps his work could simple be bought and made available for free.

Paul Bruno link
10/14/2013 02:05:14 pm

Is there an easy-access list of the questions they used and the most common incorrect/misconception answer for each? I didn't see one on a quick scan of the interwebs.

don link
10/15/2013 02:59:22 am

This proves, as I've always said, at least in science, that content is king. The teacher needs to be the resident expert in the subject he/she is teaching. Understand it, able to explain it, model it, and knows how to set up experiments and demonstrations of the concepts. The teacher also needs to understand the misconceptions of the science subject being taught. I have witnessed this countless times in my courses, mainly from the questions students ask, the comments on their essays, and lab answers. All the yahoo about skills, 21st century stuff, etc. etc. are nice, but beside the point. Know your science!

Douglas Hainline
10/15/2013 05:38:52 am

Don: of course you are right. In an ideal world, even Primary [Elementary] level teachers would be able to explain Newton's Laws, Natural Selection, the Periodic Table ... but, most of them can't, and unfortunately a lot of Middle Years teachers who are responsible for science cannot either. We have to address this reality.

In the long term, raising requirements for teachers (which will mean paying them more) may be the way to go, but for the immediate situation, I think that 'automating' science teaching through making a set of excellent videos available, with printed supporting material, would go a long way to solving the problem.

Videos like this one: http://www.youtube.com/watch?v=IM630Z8lho8


Douglas Hainline
10/15/2013 05:39:46 am

Don: of course you are right. In an ideal world, even Primary [Elementary] level teachers would be able to explain Newton's Laws, Natural Selection, the Periodic Table ... but, most of them can't, and unfortunately a lot of Middle Years teachers who are responsible for science cannot either. We have to address this reality.

In the long term, raising requirements for teachers (which will mean paying them more) may be the way to go, but for the immediate situation, I think that 'automating' science teaching through making a set of excellent videos available, with printed supporting material, would go a long way to solving the problem.

Videos like this one: http://www.youtube.com/watch?v=IM630Z8lho8


Logan link
10/15/2013 06:03:04 am

Hello Dr. Willingham,

Great article discussion. One of the best tools I've used in the classroom to discover strong misconceptions and address them is live anonymous polling with ''clickers'' or online voting. Raised hands polling is slower and doesn't seem to have nearly as high of participation.

Jason Olson link
10/15/2013 07:59:52 am

AAAS has an assessment-specific site set up that has cataloged a wide variety of misconceptions for all science areas, and it can be found here: http://assessment.aaas.org/topics
The other document I refer back to frequently for chemistry-specific misconceptions is Vanessa Kind's "Beyond Appearances," found here: http://www.rsc.org/images/Misconceptions_update_tcm18-188603.pdf

Paul Bruno link
10/15/2013 04:48:30 pm

Thanks, Jason. I wasn't aware of that first link, and it seems pretty cool.

Jason Olson link
10/16/2013 01:19:13 am

It definitely is. Not sure how much you explored, but if you register you can access lists of questions that target specific misconceptions and create tests for your students to take.

Douglas Hainline
10/16/2013 02:41:35 am

There ought to be -- and perhaps there already is? -- a single portal-style, frequently-updated website where all links like the above are assembled, categorized, and described. As it is, there seem to be hundreds of useful sites for science and maths education, of varying format and quality and target audience, but it's a major operation in itself to find them.

Patricia DiPadova
10/18/2013 01:51:22 pm

This seems to explain why low performing schools can still have a few high achieving students while failing a large part of the student body.

Mike G
10/20/2013 05:51:51 am

How does this square with various findings (recently Mathematica study of TFA, TNTP) that does NOT seem to show high correlation with teacher knowledge and student achievement?

Any chance you can review the bidding here...I'm confused at this point.

Dan Willingham
10/25/2013 01:39:36 pm

correlation is always higher in math & sci, and also depends heavily on how you measure students and teachers

Beliavsky
10/23/2013 01:49:17 am

The graph shows that the incremental learning produced by a teacher with high subject matter knowledge and knowledge of student misconceptions is higher for high ability than low ability students. To put it more bluntly, low-ability students cannot learn much science even from good teachers. This contradicts the fashionable idea that the worst schools (with the worst students) should get the best teachers.

Jason Olson link
10/25/2013 04:23:39 am

The first issue I have is automatically labeling a student who has low math and reading scores as "low-ability". It has been shown that standardized test scores are affected by more than just a student's raw ability in that subject area, such as native language, race, income level, etc.

I also disagree with your conclusion that "low-ability students cannot learn much science even from good teachers." It is pretty clear from the first set of data that when they are learning subject matter that does not involve strong misconceptions that all students are able to learn better from a teacher with more knowledge about the subject area. What the second set of data is showing is just how difficult misconceptions (especially strong ones) are to change in students, even over the course of an entire year with an excellent teacher. There is also a great deal of other research that has shown this same difficulty in changing misconceptions over shorter periods of time (one year or less).

Douglas Hainline
10/23/2013 02:51:42 pm

The best teachers tend to flee from the worst schools to the better ones. For he that hath, to him shall be given: and he that hath not, from him shall be taken even that which he hath.

We need an educational system which can adapt to the needs and capacities of individual children, those with IQs of 75 and those with IQs of 150. Perhaps modern technology will allow us to craft such a system some day.

Douglas Hainline
10/25/2013 10:38:31 am

The concepts of "ability" or "IQ" may, or may not, be objectively real. (I think there is a sense in which they are, but that's irrelevant.)

But phrases like "low ability" , "bottom set", etc. should be absolutely forbidden in public use. That is, no student should hear this phrase, or, worse yet, hear it applied to himself.

These phrases are often used by people of high ability, which probably includes all of the people who read this blog, and often carry an undertone of scorn and disdain.

If I were to take up piano, I am sure that serious musicians would classify me, justly, as "low ability". Fortunately, the way one studies piano (at my age at least) -- via individual practice, tested when my teacher and I think I am ready -- means that I can progress at my own (slow) rate, or maybe not progress at all beyond a certain musical grade, without being stigmatized as a musical moron.

We need to design education systems that accomplish this for all subjects.

Beliavsky
10/28/2013 06:09:54 am

'But phrases like "low ability" , "bottom set", etc. should be absolutely forbidden in public use. That is, no student should hear this phrase, or, worse yet, hear it applied to himself.'

People need to be realistic about their IQ in order to make educational and career plans. Half of the U.S. population has an IQ below 100, and that half is not mentally equipped to study at the college level, but some politicians are encouraging everyone to attend college.

Douglas Hainline
10/25/2013 10:39:22 am

The concepts of "ability" or "IQ" may, or may not, be objectively real. (I think there is a sense in which they are, but that's irrelevant.)

But phrases like "low ability" , "bottom set", etc. should be absolutely forbidden in public use. That is, no student should hear this phrase, or, worse yet, hear it applied to himself.

These phrases are often used by people of high ability, which probably includes all of the people who read this blog, and often carry an undertone of scorn and disdain.

If I were to take up piano, I am sure that serious musicians would classify me, justly, as "low ability". Fortunately, the way one studies piano (at my age at least) -- via individual practice, tested when my teacher and I think I am ready -- means that I can progress at my own (slow) rate, or maybe not progress at all beyond a certain musical grade, without being stigmatized as a musical moron.

We need to design education systems that accomplish this for all subjects.

Sam Platts
10/26/2013 07:59:18 am

I say the moon is the cause of earthquakes. Anybody want to challenge me?

JIM COURTON
10/30/2013 08:18:03 pm

ITS GOOD FOR NEWBIE


Comments are closed.

    Enter your email address:

    Delivered by FeedBurner

    RSS Feed


    Purpose

    The goal of this blog is to provide pointers to scientific findings that are applicable to education that I think ought to receive more attention.

    Archives

    April 2022
    July 2020
    May 2020
    March 2020
    February 2020
    December 2019
    October 2019
    April 2019
    March 2019
    January 2019
    October 2018
    September 2018
    August 2018
    June 2018
    March 2018
    February 2018
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    April 2017
    March 2017
    February 2017
    November 2016
    September 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016
    December 2015
    July 2015
    April 2015
    March 2015
    January 2015
    September 2014
    August 2014
    July 2014
    June 2014
    May 2014
    April 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    September 2012
    August 2012
    July 2012
    June 2012
    May 2012
    April 2012
    March 2012
    February 2012

    Categories

    All
    21st Century Skills
    Academic Achievement
    Academic Achievement
    Achievement Gap
    Adhd
    Aera
    Animal Subjects
    Attention
    Book Review
    Charter Schools
    Child Development
    Classroom Time
    College
    Consciousness
    Curriculum
    Data Trustworthiness
    Education Schools
    Emotion
    Equality
    Exercise
    Expertise
    Forfun
    Gaming
    Gender
    Grades
    Higher Ed
    Homework
    Instructional Materials
    Intelligence
    International Comparisons
    Interventions
    Low Achievement
    Math
    Memory
    Meta Analysis
    Meta-analysis
    Metacognition
    Morality
    Motor Skill
    Multitasking
    Music
    Neuroscience
    Obituaries
    Parents
    Perception
    Phonological Awareness
    Plagiarism
    Politics
    Poverty
    Preschool
    Principals
    Prior Knowledge
    Problem-solving
    Reading
    Research
    Science
    Self-concept
    Self Control
    Self-control
    Sleep
    Socioeconomic Status
    Spatial Skills
    Standardized Tests
    Stereotypes
    Stress
    Teacher Evaluation
    Teaching
    Technology
    Value-added
    Vocabulary
    Working Memory