Daniel Willingham--Science & Education
Hypothesis non fingo
  • Home
  • About
  • Books
  • Articles
  • Op-eds
  • Videos
  • Learning Styles FAQ
  • Daniel Willingham: Science and Education Blog

American kids "in the middle" on PISA science? How big is that middle again?

4/23/2013

 
My Facebook feed today has lots of links to this article. The upshot: a new Pew study showing that Americans think that US 15 year olds rank "near the bottom" on international science tests, whereas the truth is that they "rank in the middle among developed countries."

I guess "the middle" covers a lot of terrain, but the way I look at the data, this assertion doesn't hold.

The international comparison in question is the 2009 PISA. Here are the rankings. (Click for larger image)
Picture
Most everyone would agree that it's not appropriate to compare scores of US kids to those of poorer countries with little infrastructure and funding to support education.

That's why the article specifies the ranking of the US among "developed countries," and by the author's reckoning, kids from 12 developed countries scored better, and kids from 9 developed countries scored worse. That would put US kids at the 41st percentile.

The US is ranked 30th on the list. Just eyeballing it, it's hard to see how 17 of the countries scoring better could be considered "not developed."

On measures of "developed" status would be the International Monetary Fund's definition of "advanced economies" which includes: Australia, Austria, Belgium, Canada, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hong Kong, Iceland, Ireland, Israel, Italy, Japan, Luxembourg, Malta, Netherlands, New Zealand, Norway, Portugal, San Marino, Singapore, Slovakia, Slovenia, South Korea, Spain, Sweden, Switzerland, Taiwan, United Kingdom, United States (Click image for larger image
Picture
By this definition of "advanced" US kids are 23rd out of 32 countries, or the 28th percentile.

It's true that "near the bottom" is too grim an assessment. But I can't see a way to put the 2009 PISA data together such that American kids are scoring about average.
Scott McLeod link
4/23/2013 07:49:41 am

Does it matter what our international rankings are? Has anyone yet shown a significant, statistical (not just anecdotal) relationship between international rankings and economic and/or life success? America has ranked fairly low on these kinds of comparisons for decades, but that doesn't seem to affect our economic progress or societal welfare any...

Are there any data out there that indicate why we should care?

Dan Willingham link
4/23/2013 08:19:49 am

Rick Hanushek (Stanford) has made this argument forcefully, as have several other economists. (Not that this interpretation goes undisputed.)
The article itself uses these data to question whether we should really be emphasizing science so much in reform efforts.

Sandra Stotsky
4/23/2013 11:59:48 am

I should think that our scores on the TIMSS science tests (which measure curriculum) are more important than the scores on PISA tests which measure the skills of 15-year olds (as determined by what the average young adult knows).

Daniel Willingham
4/23/2013 12:25:02 pm

TIMMS rank is much better. Unclear why: PISA is supposed to be more "applied" but that's confounded with age.

P.J. Patel link
4/23/2013 12:17:58 pm

"But I can't see a way to put the 2009 PISA data together such that American kids are scoring about average."

Disaggregating the scores by race will do just that.

See http://imgur.com/a/KVpvI (view charts in full screen mode).



Daniel Willingham
4/23/2013 12:23:06 pm

sooo. . . you mean if we pay attention only to the white and asian kids, "Americans" are about average?

P.J. Patel
4/23/2013 02:42:03 pm

Not quite, but such an analysis offers a more targeted solution to raising achievement. Rather than viewing students as uniformly uneducated, this view has implications for future interventions ( close the achievement without disturbing the schooling of those who are doing well). Current reformer methods ignore that US underachievement is moderated by race and so fail to offer an effective, nuanced solution.

Tim
4/23/2013 01:34:22 pm

I'd be curious how students are selected to be tested. Do these represent public school students only? If so, I'd be curious how high-income enrollment rates in public schools vary among developed countries. From my anecdotal seat in Houston, I feel like public schools are becoming the refuge of those who have no other option.

If not... I got nothin'.

Tom Sundstrom link
4/24/2013 02:24:47 am

What about alignment of the PISA (and TIMSS) content? If these test outcomes are important to us, then student's opportunity to learn the assessed knowledge and skills is essential to achieving higher outcomes. Has there been any content alignment analysis done between state standards and these international assessments? Are the Common Core State Standards aligned?


Comments are closed.

    Enter your email address:

    Delivered by FeedBurner

    RSS Feed


    Purpose

    The goal of this blog is to provide pointers to scientific findings that are applicable to education that I think ought to receive more attention.

    Archives

    April 2022
    July 2020
    May 2020
    March 2020
    February 2020
    December 2019
    October 2019
    April 2019
    March 2019
    January 2019
    October 2018
    September 2018
    August 2018
    June 2018
    March 2018
    February 2018
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    April 2017
    March 2017
    February 2017
    November 2016
    September 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016
    December 2015
    July 2015
    April 2015
    March 2015
    January 2015
    September 2014
    August 2014
    July 2014
    June 2014
    May 2014
    April 2014
    March 2014
    February 2014
    January 2014
    December 2013
    November 2013
    October 2013
    September 2013
    August 2013
    July 2013
    June 2013
    May 2013
    April 2013
    March 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    September 2012
    August 2012
    July 2012
    June 2012
    May 2012
    April 2012
    March 2012
    February 2012

    Categories

    All
    21st Century Skills
    Academic Achievement
    Academic Achievement
    Achievement Gap
    Adhd
    Aera
    Animal Subjects
    Attention
    Book Review
    Charter Schools
    Child Development
    Classroom Time
    College
    Consciousness
    Curriculum
    Data Trustworthiness
    Education Schools
    Emotion
    Equality
    Exercise
    Expertise
    Forfun
    Gaming
    Gender
    Grades
    Higher Ed
    Homework
    Instructional Materials
    Intelligence
    International Comparisons
    Interventions
    Low Achievement
    Math
    Memory
    Meta Analysis
    Meta-analysis
    Metacognition
    Morality
    Motor Skill
    Multitasking
    Music
    Neuroscience
    Obituaries
    Parents
    Perception
    Phonological Awareness
    Plagiarism
    Politics
    Poverty
    Preschool
    Principals
    Prior Knowledge
    Problem-solving
    Reading
    Research
    Science
    Self-concept
    Self Control
    Self-control
    Sleep
    Socioeconomic Status
    Spatial Skills
    Standardized Tests
    Stereotypes
    Stress
    Teacher Evaluation
    Teaching
    Technology
    Value-added
    Vocabulary
    Working Memory