More than half of US states have adopted Quality Rating and Improvement Systems (QRISs) in an attempt to quantify preschool quality at the level of individual programs.
States adopt different measures to go into the QRIS, but they are uniform in that they use input measures, not child outcome measures.
A study out in Science this week (Sabol et al, 2013) sought to evaluate whether QRISs work; do they identify quality preschools?
The study used two national data sets from the early 2000s to test whether the type of metrics most often included in QRISs are related to schooling outcomes. So the researchers ask “if you use the kinds of measures QRISs use to evaluate preschools and combine them as the QRISs do, are you probably measuring good learning outcomes for kids?”
The four characteristics of programs were (1) qualifications and experience of teachers; (2) teacher-student ratio; (3) family partnerships and (4) how conducive the environment is to learning (as measured by the Early Childhood Education Rating System, which evaluates both the physical classroom and interactions between teachers, students and parents).
The outcome measures included math, prereading, language, and social skills. These were measured at the end of the year, accounting for beginning-of-year-score, child, and family characteristics.
The figure shows difference between high-scoring and low-scoring preschools on the four metrics. The four qualities of preschools do not differentiate high vs. low quality on the outcomes for kids.
Classroom measures, in contrast, are expensive. Someone with training has to actually observe what’s going on. That's part of what goes into the "environment" measure, and it does look like that measure showed the most promise.
And indeed, a measure wholly focused on classroom interaction does much better. Sabol et al conducted another analysis, using the Classroom Assessment Scoring System (CLASS), (brainchild of the study’s third author, Bob Pianta), which evaluates interactions between teacher and child. As you can see , the CLASS does quite well. (It’s labeled “interactions” in this graph.)
Barnett, W. S. Effectiveness of Early Educational Intervention. Science, 333, 975-978.
Sabol, T. J., Hong, S. L. S., Pianta, R. C., & Burchinal, M. R. (2013). Can rating pre-k programs predict children's learning? Science, 341, 845-846.