The PIRLS results are better than you may realize.

Last week, the results of the 2011 Progress in International Reading Literacy Study (PIRLS) were published. This test compared reading ability in 4th grade children.

U.S. fourth-graders ranked 6th among 45 participating countries. Even better, US kids scored significantly better than the last time the test was administered in 2006.

There's a small but decisive factor that is often forgotten in these discussions: differences in orthography across languages.
Picture
Lots of factors go into learning to read. The most obvious is learning to decode--learning the relationship between letters and (in most languages) sounds. Decode is an apt term. The correspondence of letters and sound is a code that must be cracked.

In some languages the correspondence is relatively straightforward, meaning that a given letter or combination of letters reliably corresponds to a given sound. Such languages are said to have a shallow orthography. Examples include Finnish, Italian, and Spanish.

In other languages, the correspondence is less consistent. English is one such language. Consider the letter sequence "ough." How should that be pronounced? It depends on whether it's part of the word "cough," "through," "although," or "plough." In these languages, there are more multi-letter sound units, more context-depenent rules and more out and out quirks.

Another factor is syllabic structure. Syllables in languages with simple structures typically (or exclusively) have the form CV (i.e., a consonant, then a vowel as in "ba") or VC (as in "ab.") Slightly more complex forms include CVC ("bat") and CCV ("pla"). As the number of permissible combinations of vowels and consonants that may form a single syllable increases, so does the complexity. In English, it's not uncommon to see forms like CCCVCC (.e.g., "splint.")

Here's a figure (Seymour et al., 2003) showing the relative orthographic depth of 13 languages, as well as the complexity of their syllabic structure.

Picture
From Seymour et al (2003)
Orthographic depth correlates with incidence of dyslexia (e.g., Wolf et al, 1994) and with word and nonword reading in typically developing children (Seymour et al. 2003). Syllabic complexity correlates with word decoding (Seymour et al, 2003).

This highlights two points, in my mind.

First, when people trumpet the fact that Finland doesn't begin reading instruction until age 7 we should bear in mind that the task confronting Finnish children is easier than that confronting English-speaking children. The late start might be just fine for Finnish children; it's not obvious it would work well for English-speakers.

Of course, a shallow orthography doesn't guarantee excellent reading performance, at least as measured by the PIRLS. Children in Greece, Italy, and Spain had mediocre scores, on average. Good instruction is obviously still important.

But good instruction is more difficult in languages with deep orthography, and that's the second point. The conclusion from the PIRLS should not just be "Early elementary teachers in the US are doing a good job with reading." It should be "Early elementary teachers in the US are doing a good job with reading despite teaching reading in a language that is difficult to learn."


References

Seymour, P. H. K., Aro, M., & Erskine, J. M. (2003). Foundation literacy acquisition in European orthographies. British Journal of Psychology, 94, 143-174.

Wolf, M., Pfeil, C., Lotz, R., & Biddle, K. (1994). Towarsd a more universal understanding of the developmental dyslexias: The contribution of orthographic factors. In Berninger, V. W. (Ed), The varieties of orthographic knowledge, 1: Theoretical and developmental issues.Neuropsychology and cognition, Vol. 8., (pp. 137-171). New York, NY, US: Kluwer
 
 
One finding (from Seymour, Aro & Erskine, 2003) illustrated in one figure (Figure 5.3 from Stan Dehaene's marvelous book,Reading in the Brain.).  The figure shows errors in word reading at the end of first grade, by country.
Are we to conclude that the differences are due to educational practice? The vaunted Finnish system shows smashing results even at this early age, whereas the degenerate British system can't get it right?

Countrywide differences in instruction could play a role, but Dehaene emphasize that the countries in which children make a lot of errors--Portugal, France, Denmark, and especially Britain--just happen to have deeper orthographies.

A shallow orthography means that there is a straightforward correspondence between letters and phonemes. English, in contrast, has one of the deepest (most complex) orthographies among the alphabetic languages: for example, the letter combination "gh" if pronounced differently in in "ghost," "eight," and "enough."

In short, children learning to read English have a difficult task in front of them--and so too, therefore, do teachers.

Is there a lesson to be drawn here?

To me, the difficult orthography of English highlights the importance of careful sequencing in the learning of grapheme-phoneme pairs, along with a limited number of sight words--sequencing that exploits the regularities that exist, and bring children as swiftly as possible to the point that they can read texts and so feel a sense of accomplishment.

In Italy, for example, the order in which grapheme-phoneme pairs are taught would matter much less because there simply is not that much to learn. Several months of instruction is sufficient for most children to reach a point that they can decode most texts.

The deep orthography of English also sheds light on why American schools spends as much time on English-language arts (ELA) as they do: something like two-third of instructional time in the first grade (NICHD Early Child Care Research Network, 2002).

One might draw the conclusion that the difficulty of the task in reading requires enormous amounts of time. Another point of view--one I share--is that this practice places too much emphasis on ELA at the expense of other content, and runs a high risk of discouraging kids who might become passionate about science, or history, or geography, but won't because the early elementary years contain so little content beyond ELA and mathematics.

I think it would worth our accepting slower progress in reading in exchange for broader subject-matter coverage in early grades--coverage that will actually pay dividends for reading comprehension in later grades.


Dehaene, S. (2009). Reading in the Brain. New York: Viking.

National Institute of Child Health and Human Development Early Child Care Research Network (2002). The Relation of Global First-Grade Classroom Environment to Structural Classroom Features and Teacher and Student Behaviors. The Elementary School Journal, 102, 367-387

Seymour, P. H. K., maro, M., & Erskine, J. M. (2003). Foundation literacy acquisition in European orthographies. British Journal of Psychology, 94, 143-174.

 
 
I was just at a reading conference and gave a talk on reading comprehension strategies.

I’ve written about them before (article here). The next paragraph provides just a brief summary of what I’ve written. The figure below shows the strategies themselves, if you’re not familiar with them (click the image for a larger version).

The short version of my conclusion is that they don’t really improve the comprehension process per se. Rather, they help kids who have become good decoders to realize that the point of reading is communication. And that if they can successfully say written words aloud but cannot understand what they’ve read, that’s a problem. Evidence for this point of view include data that kids don’t benefit much from reading comprehension instruction after 7th grade, likely because they’ve all drawn this conclusion, and that increased practice with reading comprehension strategies doesn’t bring any improved benefit. It’s a one-time increment.

How much time is devoted to reading comprehension strategy instruction? I can’t find good (or poor) data on this question, and I doubt it exists. There is so much variation among districts (and probably even classrooms) on this issue, it’s hard to draw a conclusion with much confidence. Any time I talk about reading, a lot of teachers, coaches, and administrators tell me that enormous amounts of time go to reading comprehension strategy instruction in their district—but I’m sure the people who make sure to mention this to me are not a random sample. 

Whatever the proportion of time, much of it is wasted, at least if educators think it’s improving comprehension, because the one-time boost to comprehension can be had for perhaps five or ten sessions of 20 or 30 minutes each.

Some reading comprehension strategies might be useful for other reasons. For example, a teacher might want her class create a graphic organizer as a way of understanding how an author builds narrative arc

The wasted time obviously represents a significant opportunity cost. But has anyone ever considered that implementing these strategies make reading REALLY BORING? Everyone agrees that one of our long-term goals in reading instruction is to get kids to love reading. We hope that more kids will spend more time reading and less time playing video games, watching TV, etc.
How can you get lost in a narrative world if you think you’re supposed to be posing questions to yourself all the time? How can a child get really absorbed in a book about ants or meteorology if she thinks that reading means pausing every now and then to anticipate what will happen next, or to question the author’s purpose?

To me, reading comprehension strategies seem to take a process that could bring joy, and turn it into work.
 
 
A lot of people have been tweeting or posting to Facebook a link to a graph which was posted to the Atlantic Monthly website on April 6, with this provocative headline:
And here's the chart you are urged to show, derived from Gallup poll numbers over the years.
We're invited to conclude that because the percentage of readers increased after the advent of the Internet, the Internet did not have a negative impact on book reading.

All of the postings I've seen have apparently taken this conclusion at face value, so it seems like it's worth going through why this conclusion is not justified.

First, lots of stuff happened between 1949 and 2005. For example, household income increased for middle- and high-income families. It could be that the internet has negatively affected book reading, but a number of other factors have increased it, so overall we see an increase. The idea that other factors are having a big impact on reading seems legitimate, given the big increase in reading from 1957 to 1990, a year in which very few people had internet access. So perhaps those factors are continuing to boost reading, despite the negative impact of the internet.

The type of analysis we're being asked to perform implicitly is a variety of time series analysis. It's useful in situations where one can't conduct an experiment with a control group. For example, I might track classroom behavior daily for one month using a scale that runs from 1-10. I find that it ranges from 4 to 6 every day. The teacher implements a new classroom management scheme, and from that day forward, classroom behavior ranges from 7 to 9 every day.

Interpreting the new classroom management scheme as the cause of the change is still not certain--some outside-of-class-factor could have just happened to have occurred on that same day that prompted the change in classroom behavior. But the fact that the change in behavior happened over such a narrow time window makes us more confident that such a coincidence is unlikely.

And of course it helps my confidence that it's the same class. In the chart above, we're looking at events that happened over years, with different people who we hope had similar characteristics.

To really get at the effect of the internet on reading habits, you need more finely controlled data. A number of studies were done in the 1950s, examining the effect of television on reading habits. The best of these (see Coffin, 1955) measured reading habits in a city that did not have a television station but was poised to get one, and then measured reading habit again after people in the city had access to television. (The results showed an impact on reading, especially light fiction. Newspaper reading was mostly unaffected, as television news was still in its infancy. Radio listening took a big hit.)

Second, we might note that the most recent year on the chart is 2005. According to the Pew Internet and American Life Project, only 55% of Americans had internet access at that point, and only 30% had broadband. So perhaps the negative impact wouldn't be observed until more people had internet access.

This brings us to more serious studies of whether use of the Internet displaces other activities. The studies I know of (e.g., Robinson, 2010) conclude that Internet use does not displace other activities, but rather enhances them. The data are actually a bit weird--IT use is associated with more of everything: more leisure reading, more newspaper reading, more visits to museums, playing music, volunteering, and participating in sports.

The obvious explanation would be that heavy IT users have higher incomes and more leisure time, but the relationships held after the author controlled for education, age, race, gender, and income--though most of these effects were much attenuated.

This research is really just getting going, and I don't think we're very close to understanding problem.

In sum, the question of what people do with their leisure time and how one activity influences another is complicated. One chart will not settle it.




Coffin, T. E. (1955) Television's impact on society. American Psychologist, 10, 630-641.
Robinson, J. P. (2011) Arts and leisure participation among IT users: Further evidence of time enhancement over time displacement. Social Science Computer Review, 29,  470-480.
 
 
David Daniel and I have a letter in latest issue of Science. It's behind a paywall, so I thought I'd provide a summary of the substance.

David and I note that there is, in some quarters, a rush to replace conventional paper textbooks with electronic textbooks. It is especially noteworthy that members of the Obama administration are eager to speed this transition. (See this report.)

On the face of it, this transition is obvious: most people seem to like reading on their Nook, Kindle or iPad--certainly sales of the devices and of ebooks are booming. And electronic textbooks offer obvious advantages that traditional textbooks don't, most notably easy updates, and embedded features such as hyperlinks, video, and collaboration software.

But David and I urged more caution.

We should note that there are not many studies out there regarding the use of electronic textbooks, but those that exist show mixed results. A consistent finding is that, given the choice, students prefer traditional textbooks. That's true regardless of their experience with ebooks, so it's not because students are unfamiliar with them (Woody, Daniel & Baker, 2010). Further, some data indicate that reading electronic textbooks, although it leads to comparable comprehension, takes longer (e.g., Dillon, 1992; Woody et al, 2010).

Why don't students like electronic textbooks if they like ebooks? The two differ. Ebooks typically often have a narrative structure,  they are usually pretty easy to read, and we read them for pleasure. Textbooks in contrast, have a hierarchical structure, the material is difficult and unfamiliar, and we read them for learning and retention. Students likely interact with textbooks differently than books they read for pleasure.

That may be why the data for electronic books are more promising for early grades. Elementary reading books tend of have a narrative structure, and students are not asked to study from the books as older kids are.

Further, many publishers are not showing a lot of foresight in how they integrate video and other features in the electronic textbooks. A decade of research (much of it by Rich Mayer and his collaborators and students) show that multimedia learning is more complex than one would think. Videos, illustrative simulations, hyperlinked definitions--all these can aid comprehension OR hurt comprehension, depending on sometimes subtle differences in how they are placed in the text, the specifics of the visuals, the individual abilities of readers, and so on.

None of this is to say that electronic textbooks are a bad thing, or indeed to deny that they ought to replace traditional textbooks. But two points ought to be kept in mind.
(1) The great success of ebooks as simply the porting over of traditional books into another format may not translate to electronic textbooks. Textbooks have different content, different structure, and they are read for different purposes.
(2) Electronic textbooks stand a much higher chance of success if publishers will exploit the rich research literature on multimedia learning, but most are not doing so.

For these two reasons, it's too early to pick the flag and shout "Hurrah!" on electronic textbooks.


A. Dillon, Ergonomics 35, 1297 (1992).
W. D. Woody, D. B. Daniel, C. Baker , Comput. Educ. 55, 945 (2010)
 
 
Today's New York Times has an article speculating that when you read on an ereader or tablet, your attention is likely to be diverted to other applications.

If you hit a dull patch in the book, can you resist the pull of YouTube, Twitter, or your email? Even if you're engaged in the book, Google may beckon to clarify a point in the book ("Essex? Where's that?") and next thing you know, 25 minutes have elapsed in surfing. Perhaps interesting, perhaps productive, but not what you sat down intending to do.

Many people I've spoken with have the impression that this sort of distraction is predictable, and that it is a greater problem when reading on a tablet computer, even compared to reading a print book with a computer nearby.

The data on this question are still thin, but I do know of one relevant study (Woody et al, 2011). Nearly 300 college students took part, each reading a chapter from an introductory psychology textbook in one of five formats: print textbook, printed text pages, printed manuscript in MS Word, electronic pdf file, or electronic textbook. Some students read in a laboratory, some at home, and everyone took a quiz on the chapter material after reading it.

The results showed that media format did not affect quiz grades. But students who read electronic media versions were more likely to respond to instant messages and email while reading, and were more likely to use social networking sites (Facebook/Myspace) while reading.

It's only one experiment, but this feels like an instance where the intuitions of the majority of people will end up according with data. Whether the extra level of distraction is really a problem remains to be seen; and it may well be that users (or software designers) come up with strategies to solve the problem, if it proves significant.

Woody, W. D., Daniel, D. B., & Stewart, J. M. (2011).  Students’ Preferences and Performance  Using E-Textbooks and Print Textbooks.  In F. Columbus (Ed.), Computers in Education.  New York: Nova Publishing.


 
 
There’s a new article (Melby-Lervåg, Lyster, & Hulme, 2012) out in Psychological Bulletin (commonly considered the premier outlet for large literature reviews), which summarizes a great deal of data on phonological skills and reading.

The broad conclusions will startle no one who follows this research literature—we know phonological skills are important--but the article is notable for making a couple of finer-grain distinctions.

It is sensible that a child’s ability to understand that words are composed of sounds should be important in learning to read. After all, letters and groups of letters correspond to sound, they don’t signify meaning directly. If that weren’t true you wouldn’t be able to read nonsense words like “flotupe.” Letters signify sounds, and decoding means that kids must match letters (T) and groups of letters (TH) to sound. That means they must first appreciate that words are composed of sounds.

There has been some debate, however, as to the size of the sound unit that matters: the phoneme, or rimes and onsets. The onset refers to the consonant string that precedes a vowel sound, and the rime refers to the vowel and any consonants that follow. Hence, in the word TRIP, the onset would be /TR/ and the rime would be /IP/.

Other researchers have suggested that children must appreciate still smaller sound units—phonemes—as preparation for reading. Phonemes are individual speech sounds that cannot be further subdivided. For example, the rime /IP/ is composed of two phonemes: /I/ and /P/. Perhaps it helps if kids perceive that /IP/ is really two sounds. . .but perhaps that’s not necessary. (And indeed, perceiving that the letter P goes with the individual speech sound /P/ is no small feat, because /P/ is nearly impossible to say on its own. It’s really just a plosion of air, so it sound like you’re imitating a champagne cork popping. Parents typically add a vowel, usually saying “puh” for P.

Then again, maybe whether kids perceive onsets/rimes or phonemes is less important than their having a sizable verbal short term memory in which to manipulate and consider these speech sounds as they are learning to read.

Melby-Lervåg et al. included 235 studies in their analysis and concluded that existing research suggests that all three—rime awareness, phoneme awareness, and size of verbal short-term memory—predict word reading but the largest effect is observed with phonemic awareness.  (In fact, the predictive value of verbal short-term memory is quite small.)

The second important conclusion from this review concerns causality. All of these studies in the meta-analysis are correlational. Hence, one interpretation is that phonemic awareness is related to word reading skills because phonemic awareness is necessary for that skill. However, an equally viable interpretation is that reading changes phonemic awareness, so the association is observed because more skilled readers have undergone greater change.

Melby-Lervåg et al. separately consider longitudinal studies that measure phonemic awareness when children are still quite young and have little reading experience. Then reading ability (and especially, the rate at which this ability grows) of these same children is then measured later. Phonemic awareness remains an excellent predictor of reading skill in these studies with a mean correlation of .43; reading could not have caused phonemic awareness, because phonemic awareness was measured before kids could read. (Rime awareness was also a significant predictor in this sort of study, but not as strong, mean correlation = .29). Coupled with other data (not included in the meta-analysis) showing a positive effect for phonemic awareness training, the evidence for a causal role of phonemic awareness in learning to read is growing quite strong.

Melby-Lervåg, M., Lyster, S-A H., & Hulme, C. (2012). Phonological skills and their role in learning to read: A meta-analytic review. Psychological Bulletin, 138,  322-352.

 
 
Title tells all.

 
 
One strategy for thinking about interventions to boost kids success in school is to conduct the following sort of study. Step one:  measure lots of factors early in life, i.e., before kids start school. Step two: measure academic success after kids have been in school awhile (say, fourth grade). Then see which factors you measured early in life are associated with school success measured later.

Some factors are well-known, e.g., socio-economic status of the parents, and so you’d statistically remove those “usual suspects” first.

In 2007 Duncan and colleagues introduced a new method of analyzing this type of data, and they applied it to six sizable international data sets that followed kids from as early as birth to 3rd grade, focusing especially on reading and math achievement.  They concluded that early measures of math and reading, and measures of attention were significant predictors of later math and reading skills, but early social skills were not. Curiously, early math scores predicted later reading scores as well as early reading scores did.

Their conclusions, while not startling, attracted a lot of attention because the new method was deemed quite useful, and because it was applied meticulously to several large-scale datasets.

In 2010, another article was published using the same methodology, but with a startling result.

David Grissmer and his colleagues noted that three of the data sets had early measures of fine motor skills. They found that, after they statistically accounted for all of the factors that Duncan et al had examined, fine motor skills was and additional, strong predictor of student achievement.

I have to note that what the tests called “fine motor skills” strikes me as a bit odd.  Cognitive psychologists think of that as being tasks like buttoning a button, or picking something up with tweezers—i.e., requiring precise movements, usually of the fingers. But in these data sets it was tested with tasks like copying simple designs, or drawing a human figure. These are not solely motor tasks.

The fuzziness of exactly what the tasks mean may cloud the interpretation, but it doesn’t cloud the size of the effect—these tasks are a robust predictor of later math and reading achievement.

There’s plenty of speculation as to why this effect might work. Perhaps the measure of “fine motor skills” is really another way of measuring some aspect of attention. Perhaps it’s another way of measuring how well kids can understand and use space. Or the effect may be more direct; it’s commonly thought that the motor and cognitive domains are intertwined, and so practicing motor  tasks may aid cognition.

The big question: does this mean that practice of fine motor skills will boost academic achievement? Those studies are ongoing, and I hope to report on the results here before long.

Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C.,Klebanov, P., . . . Japel, C. (2007). School readiness and later achievement. Developmental Psychology, 43, 1428–1446.

Grissmer, D., Grimm, K., J., Aiyer, S. M., Murrah, W. M., & Steele, J. S. (2010). Fine Motor Skills and Attention: Primary Developmental Predictors of Later Achievement. Developmental Psychology, 46, 1008-1017.

 
 
     It's a funny thing about the use of new technologies in schools: it's not only seen as inevitable, it's often described as necessary because today's students are digital natives.
     But for at least some technologies, the evidence supporting that contention is weak.
     An article in Computers & Education by Woody, Daniel, & Baker (2010) replicated other studies in showing that college students preferred studying from traditional textbooks rather than etextbooks, and also reported no correlation between previous experience with ebooks and how much students liked etextbooks.
     Some technology boosters have suggested that previous findings of student indifference to etextbooks is due to their novelty--once students get used to them, the argument goes, they will like them. Woody et al. suggest that their finding casts doubt on this explanation.

Woody, W. D., Daniel, D. B. & Baker, C. A. (2010). E-books or textbooks: Students prefer textbooks. Computers & Education, 55, 945-948.