It is known that quality preschool can improve academic outcomes for kids (Barnett, 2011). Getting a handle on measuring “quality” is challenging, but if states and the federal government are to support preschool, such measurement is vital.
More than half of US states have adopted Quality Rating and Improvement Systems (QRISs) in an attempt to quantify preschool quality at the level of individual programs.
States adopt different measures to go into the QRIS, but they are uniform in that they use input measures, not child outcome measures.
A study out in Science this week (Sabol et al, 2013) sought to evaluate whether QRISs work; do they identify quality preschools?
The study used two national data sets from the early 2000s to test whether the type of metrics most often included in QRISs are related to schooling outcomes. So the researchers ask “if you use the kinds of measures QRISs use to evaluate preschools and combine them as the QRISs do, are you probably measuring good learning outcomes for kids?”
The four characteristics of programs were (1) qualifications and experience of teachers; (2) teacher-student ratio; (3) family partnerships and (4) how conducive the environment is to learning (as measured by the Early Childhood Education Rating System, which evaluates both the physical classroom and interactions between teachers, students and parents).
The outcome measures included math, prereading, language, and social skills. These were measured at the end of the year, accounting for beginning-of-year-score, child, and family characteristics.
The figure shows difference between high-scoring and low-scoring preschools on the four metrics. The four qualities of preschools do not differentiate high vs. low quality on the outcomes for kids.
More than half of US states have adopted Quality Rating and Improvement Systems (QRISs) in an attempt to quantify preschool quality at the level of individual programs.
States adopt different measures to go into the QRIS, but they are uniform in that they use input measures, not child outcome measures.
A study out in Science this week (Sabol et al, 2013) sought to evaluate whether QRISs work; do they identify quality preschools?
The study used two national data sets from the early 2000s to test whether the type of metrics most often included in QRISs are related to schooling outcomes. So the researchers ask “if you use the kinds of measures QRISs use to evaluate preschools and combine them as the QRISs do, are you probably measuring good learning outcomes for kids?”
The four characteristics of programs were (1) qualifications and experience of teachers; (2) teacher-student ratio; (3) family partnerships and (4) how conducive the environment is to learning (as measured by the Early Childhood Education Rating System, which evaluates both the physical classroom and interactions between teachers, students and parents).
The outcome measures included math, prereading, language, and social skills. These were measured at the end of the year, accounting for beginning-of-year-score, child, and family characteristics.
The figure shows difference between high-scoring and low-scoring preschools on the four metrics. The four qualities of preschools do not differentiate high vs. low quality on the outcomes for kids.
Next, in separate analyses for each state, the researchers aggregated the four qualities; they combined and weighted the metrics as each state does. That didn’t make much difference.
Why are QRISs such a flop? I suspect that the problem is not just that they are “input” measures. The problem is that most are quite distal from where the action takes place—the classroom. Measuring things like years of experience and parental partnership is inexpensive and easy, and that’s nice. Someone at the school can submit this sort of data online.
Classroom measures, in contrast, are expensive. Someone with training has to actually observe what’s going on. That's part of what goes into the "environment" measure, and it does look like that measure showed the most promise.
And indeed, a measure wholly focused on classroom interaction does much better. Sabol et al conducted another analysis, using the Classroom Assessment Scoring System (CLASS), (brainchild of the study’s third author, Bob Pianta), which evaluates interactions between teacher and child. As you can see , the CLASS does quite well. (It’s labeled “interactions” in this graph.)
Classroom measures, in contrast, are expensive. Someone with training has to actually observe what’s going on. That's part of what goes into the "environment" measure, and it does look like that measure showed the most promise.
And indeed, a measure wholly focused on classroom interaction does much better. Sabol et al conducted another analysis, using the Classroom Assessment Scoring System (CLASS), (brainchild of the study’s third author, Bob Pianta), which evaluates interactions between teacher and child. As you can see , the CLASS does quite well. (It’s labeled “interactions” in this graph.)
The phrase “high quality preschool” has been repeated to the point that it’s become almost meaningless. It’s not, and it’s not hopeless to characterize a high-quality classroom. Further, we don’t have to test every child to spot one. Sabol et al (2013) show that qualities of teachers (e.g. experience) and programs (e.g., parental involvement) may not mean much, but qualities of teacher-student interactions might be what we need.
References
Barnett, W. S. Effectiveness of Early Educational Intervention. Science, 333, 975-978.
Sabol, T. J., Hong, S. L. S., Pianta, R. C., & Burchinal, M. R. (2013). Can rating pre-k programs predict children's learning? Science, 341, 845-846.
References
Barnett, W. S. Effectiveness of Early Educational Intervention. Science, 333, 975-978.
Sabol, T. J., Hong, S. L. S., Pianta, R. C., & Burchinal, M. R. (2013). Can rating pre-k programs predict children's learning? Science, 341, 845-846.
