It is known that quality preschool can improve academic outcomes for kids (Barnett, 2011). Getting a handle on measuring “quality” is challenging, but if states and the federal government are to support preschool, such measurement is vital.

More than half of US states have adopted Quality Rating and Improvement Systems (QRISs) in an attempt to quantify preschool quality at the level of individual programs.

States adopt different measures to go into the QRIS, but they are uniform in that they use input measures, not child outcome measures.

A study out in Science this week (Sabol et al, 2013) sought to evaluate whether QRISs work; do they identify quality preschools?

The study used two national data sets from the early 2000s to test whether the type of metrics most often included in QRISs are related to schooling outcomes. So the researchers ask “if you use the kinds of measures QRISs use to evaluate preschools and combine them as the QRISs do, are you probably measuring good learning outcomes for kids?”

The four characteristics of programs were  (1) qualifications and experience of teachers; (2) teacher-student ratio; (3) family partnerships and (4) how conducive the environment is to learning (as measured by the  Early Childhood Education Rating System, which evaluates both the physical classroom and interactions between teachers, students and parents).

The outcome measures included math, prereading, language, and social skills. These were measured at the end of the year, accounting for beginning-of-year-score, child, and family characteristics.

The figure shows difference between high-scoring and low-scoring preschools on the four metrics. The four qualities of preschools do not differentiate high vs. low quality on the outcomes for kids.

Next, in separate analyses for each state, the researchers aggregated the four qualities; they combined and weighted the metrics as each state does. That didn’t make much difference.

Why are QRISs such a flop? I suspect that the problem is not just that they are “input” measures. The problem is that most are quite distal from where the action takes place—the classroom. Measuring things like years of experience and parental partnership is inexpensive and easy, and that’s nice. Someone at the school can submit this sort of data online.

Classroom measures, in contrast, are expensive. Someone with training has to actually observe what’s going on. That's part of what goes into the "environment" measure, and it does look like that measure showed the most promise.

And indeed, a measure wholly focused on classroom interaction does much better. Sabol et al conducted another analysis, using the Classroom Assessment Scoring System (CLASS), (brainchild of the study’s third author, Bob Pianta), which evaluates interactions between teacher and child. As you can see , the CLASS does quite well.  (It’s labeled “interactions” in this graph.)

The phrase “high quality preschool” has been repeated to the point that it’s become almost meaningless. It’s not, and it’s not hopeless to characterize a high-quality classroom. Further, we don’t have to test every child to spot one. Sabol et al (2013) show that qualities of teachers (e.g. experience) and programs (e.g., parental involvement) may not mean much, but qualities of teacher-student interactions might be what we need.

References

Barnett, W. S. Effectiveness of Early Educational Intervention. Science, 333, 975-978.
Sabol, T. J., Hong, S. L. S., Pianta, R. C., & Burchinal, M. R. (2013). Can rating pre-k programs predict children's learning? Science, 341, 845-846.

Spring, 2013 at the Harvard Initiative for Learning and Teaching.

There are some studies in psychology where you pretty much know what the results will be before you collect the data. But you gotta do 'em to be sure you're right.

One example is a recent study (Sana et al, 2013) on the effects of laptop multitasking on classroom learning. (Thanks to Twitter users @rboulle and @CyniqueDeGauche for tipping me off to this study.)

The authors had college-aged subjects come into a laboratory to listen to a 45 minute lecture on meteorology, meant to simulate the sort of experience they would have in a college classroom. Half of the subjects were given a list of secondary tasks to perform, meant to represent the sort of thing that a bored student in a lecture might investigate during part of the lecture that seemed slow. For example, one question was "What is on Channel 3 tonight at 10 p.m.?" All the questions were designed to be answerable with a simple search using websites that virtually students are familiar with (Google, YouTube, et al.)

The number of questions--twelve--seemed pretty high to me. The authors said that pilot testing indicated students could answer all twelve in about 15 minutes. Thus, students would be multitasking for one third of the 45 minute lecture. Researchers argued that other data indicate this percentage estimate is not unreasonable, although it makes me want to cry.

A forty item comprehension test administered 20 minutes after the lecture showed a cost to multitasking.

Experiment 2 examined what happens when you are not multitasking  yourself, but someone near you is doing so. Again, you kind of know what's going to happen. Motion in your peripheral vision is distracting, a phenomenon that web page designers have capitalized on for years, much to our annoyance.

And sure enough, a peer multitasking in your view is distracting.

There is a fundamental tension here, and I don't know how to resolve it. On the one hand, I like it when students have their laptops in class. Many of them are more comfortable taking notes this way than longhand. In the middle of a lecture I might ask someone to look something up that I don't know off the top of my head.

On the other hand, the potential for distraction is terrible. I've walked in the back of the classroom of many of my colleagues and seen that perhaps 50% of the students are on the Web.

Students think that they can snap attention back to class "when it gets interesting again." I don't have much confidence they can. Student judgments of their own learning are often not that well calibrated, and that seems to be especially true of multitasking. They think it's cost free.

Tellingly, researchers asked subjects in Experiment 2 to provide ratings as to whether they were distracted by other people multitasking and whether other people multitasking affected their own (the observers') learning. Average answers? "Somewhat distracting" and "Barely" hindered my learning.

What can be done?

Some educators simply ban laptops. Some banish laptop users to the back rows. I don't like either of these solution much because they impose a penalty on anyone who wants to use a laptop.

I asked our IT group if the Wifi could be turned on and off in my classroom. Nope.

Some argue that students are learning how to manage distraction, although there's not much evidence that students are learning this lesson. Certainly, I don't know of anyone actively teaching them this lesson.

Got ideas? I'd love to hear them.

How should textbooks be designed? A new paper by Jennifer Kaminski and Vladimir Sloutsky shows that that can be real subtly in the answer.

The researchers examined early elementary materials meant to teach kids how to read graphs. They were specifically interested in comparing boring, monochromatic, abstract, bar graphs versus colorful, fun graphs that use a graphic. (Please excuse the black & white reproduction.)

We all know that textbook publishers are eager to make books more visually appealing. And in this case, what's the harm? The graph with the objects seems like a natural scaffold to learn the concept.

kaminski & Sloutsky found that some children shown the graph with embedded objects adopted a counting strategy to read a graph, even if they were taught to focus on the bar height and the axis. The authors surmise that the counting routine is so well-learned that when the child is presented with the vivid graphic with salient objects to count, it's simply very easy to go down that mental path. And of course the child does read the graph correctly.

The problem is not just the child hasn't learned a good strategy to read the graph, or is distracted--the child has learned a bad strategy. So when kids who adopted the counting strategy see graphs like this . . . 

. . . some of them count the stripes or count the dots to "read" the graph.

The effect fades as kids get older--first graders are better than kindergarteners in ignoring extraneous information when reading graphs.

On the one hand you could see this as small potatoes--kids will get over it, they will learn how to read graphs. But on the other hand, why knowingly put a stumbling block in front of kids trying to learn math? And more important, how many other small stumbling blocks are there that we don't know about? 

A blog posting over at Schools Matter @ The Chalk Face has gathered a lot of interest--78 comments, many of them outraged.

The New York State Education Dept. has a website that is meant to help teachers prepare for the Common Core Standards. Author Chris Cerrone posted a bit of a 1st grade curriculum module on early civilizations. Here it is:

Cerrone asked primary grade educators to weigh in: "what do you think of the vocabulary contained in this unit of study?"

The responses in the 78 comments were nearly uniformly negative. As you might expect from that volume of commentary, the criticisms were wide-ranging, much of it directed more generally at standardized testing and the idea of the CCSS themselves.

But a lot of the commentary concerned cognitive development, and I want to focus there. This comment was typical (click for larger image).

There is an important idea at the heart of this criticism: developmental stages. This commenter specifically invokes Piaget, but you don't have to be a Piagetian to think that stages are a good way to think about children's thinking. Stage theories hold that children's thinking is relatively stable, but then undergoes a big shift in a relatively brief time (say, a few months) whereupon it stabilizes again.

So lessons would be developmentally inappropriate if they demanded a type of thinking that the child was simply incapable of, given his developmental stage.

I have argued in some detail that stage theories have two major problems: first, data from the last twenty years or so make development look like it's continuous, rather than occurring in discrete stages. Second, children's cognition is fairly variable day to day, even when the same child tries the same task.

I have argued elsewhere that trying to take a psychological finding and using it to draw strong conclusions about instruction--including what children are, in principle, ready for--is fraught with problems. How much the more is that true when using a psychological theory rather than an experimental finding.

So if Piaget will not be our guide as to what 1st graders are ready for, what should be?

The experience of early elementary educators, of course, and some of the people commenting on the blog posting are or were first grade teachers. And almost unanimously, they thought this material was inappropriate for first graders. (Some thought kids this age shouldn't be learning about other religions at this age. No argument there, that's a matter of ones values. I'm only talking about what kids can cognitively handle.)

But if we adopt a proof-of-the-pudding-is-in-the-eating criterion, lessons on ancient civilizations are fine because they are in use and children are learning. The material shown above is part of the Core Knowledge sequence, around for more than a decade and used by over a thousand schools. (NB: I'm on the Board of the Core Knowledge Foundation.)

And Core Knowledge is not alone. Another curriculum has had first-graders learn about ancient civilizations not for a decade, but for about a century: Montessori. (NB again: my children experienced these lessons at their school, and my wife teaches them--she's an early elementary Montessori teacher.)

Montessori schools teach the same "Five Great Lessons" at the beginning of first, second, and third grades. They are

1. The history of the universe and earth
2. The coming of life
3. The origins of human beings
4. The history of signs and writing
5. The story of numbers and mathematics

Naturally, these lessons are presented in ways that make sense to young children, but they are far from devoid of content. Montessori educators see them as the foundation and the wellspring of interest for everything to come: biology, geology, mathematics, reading, writing, chemistry and so on.


If it seems impossible or highly unlikely to you that 6 year olds could really get anything out of such lessons, I'll ask you to consider this. Our understanding of any new concept is always incomplete.

For example, how do children learn that some people they hear about (Peter Pan) are made up and never lived, whereas others (the Pharaohs) were real? Not by an inevitable process of neurological maturation that makes their brain "ready" for this information, whereupon  they master it quickly. They learn it bit by bit, in fits and starts, sometimes seeming to get it, other times not.

And you can't always wait until children are "ready." Think about mathematics. Children are born understanding numerosity, but they understand it on a logarithmic scale--the difference between five and ten is larger than the difference between 70 and 75. To understand elementary mathematics they must learn to think of numbers of a linear scale. In this case, teachers have to undo Nature. And if you wait until the child is "developmentally ready" to understand numbers this way, you'll never teach them mathematics. It will never happen.

In sum, I don't think developmental psychology is a good guide to what children should learn; it provides some help in thinking about how children learn. The best guide to "what" is what children know now, and where you want their learning to head.