Dave and I pointed out that the means do show gains, but state programs vary in their effectiveness. It’s not the case that any old preschool is worth doing, and that’s why everyone always says that preschool must be “high quality.” But exactly how to ensure high quality is not so obvious.

A recent study (Weiland & Yoshikawa, 2013) reported research that was notable in this respect: important decisions and procedures concerning the programs were made by the people and in the way such decisions will likely to be made as state preK programs expand or are initiated. The district was Boston Public Schools, and they offer preK for any child of age—there is no restriction based on income. The district:
1. picked the curriculum.
2. figured out how to implement the curriculum at scale without any input from its developers.
3. developed its own coaching program for teachers, meant to ensure that the curricula were implemented effectively.
The second and third points are especially important, as the greatest challenge in education research has been bringing what look like useful ideas to scale. It’s not certain why that’s so, but one good guess is that as you scale up, the people actually implementing the curriculum have little or no contact with the person who developed it. So it’s harder to tell exactly how it’s supposed to go.
Naturally, schools and classrooms will want to tweak the program here and there to make it a better fit for their school or classroom. They will use their judgment as to which changes won’t affect the overall integrity of the program, but the voice of the developer of the curriculum is probably important in this conversation.

Weiland and Yoshikawa measured the progress of 2,018 children in 238 classrooms during the 2008/09 school year. They found moderate to large gains in language, pre-reading, and math skills. There was even a small effect in executive function skills, although the two curricula did not target these directly. Interestingly (and in contrast to other findings) they found no interaction with household income; poor and wealthy children showed the same benefit. There were some interactions with ethnicity: children from Hispanic homes showed larger benefits than others on some measures.
There are questions that could be raised. The comparison children were those who had just missed the age cut-off to attend the preschool. So those children are, obviously, younger, and might be expected to show less development during those 9 months than older children. Another objection concerns what those control kids were doing during the year. The researchers did have data on this question, and reported that many were in setting that typically do not offer much opportunities for cognitive growth, e.g., center-based care (although the researchers argued that Massachusetts imposes stricter regulations for quality on such settings than most states do.)
Despite these caveats, this study represents the kind of thing Dave and I had in mind when we said the Department of Education should make communicating research findings to states a priority. Boston faced exactly the problem that many districts will face, they solved it using their own limited resources as districts will have to, and by all appearances, it’s been a success.
References:
Gormley, W. T., Gayer, T., Phillips, D., & Dawson, B. (2005). The effects of universal pre-K on cognitive development. Developmental Psychology, 41, 872–884.
Magnuson, K., Ruhm, C., & Waldfogel, J. (2007). Does prekindergarten improve school preparation and per-formance? Economics of Education Review, 26,33–51.
Weiland, C. & Yoshikawa, H. (2013). Impacts of a prekindergarten program on children’s mathematics, language, literacy, executive function, and emotional skills. Child Development, 84, 2112-2130.