This column originally appeared at RealClearEducation on May 27, 2014.
Is college worth the high cost? After all, everyone knows a college degree doesn’t guarantee a good job—English majors all work at McDonalds, right? This lore is gaining acceptance (see discussion of the “college bubble” here and here, for example) but a new review in Science (Autor, 2014) marshals data suggesting that it’s dead wrong. 

College graduates not only continue to make more than non-graduates, the education-wage premium is increasing. It has risen in most advanced economies, and nowhere more than the US. In fact the difference between the median incomes of high school graduates and college graduates in 1979 was about $17,400 (in 2012 dollars). In 2012 it had doubled to around $35,000. (Autor doesn’t cite these data, but a 2013 OECD report also suggested that unemployment rose much less for college graduates during the recent recession.)

The income difference is not an artifact of the wild increases among the top 1% of earners. As Autor emphasizes, wage inequality has increased across the spectrum of incomes and is well represented among the “other 99%.” He cites various studies suggesting that 60% or so of the increasing difference in US wages is due to the increase in the education-wage premium. Other factors contributing to the difference include the decline (in real dollars) of income for high school graduates, the declining influence of trade unions, the rise in automation and subsequent loss of jobs calling for minimal cognitive skills, and competition for those jobs in the developing world.

Autor makes the case that these seemingly disparate factors can come together into a relatively simple model of supply and demand. Since the 1980’s the number of people getting a college education has increased, but the rate of increase has slowed, relative to earlier decades. Hence, the supply of people with the cognitive skill set afforded by education was expanding, but at a slow pace. Meanwhile, the percentage of jobs requiring that skill set was increasing at a much faster pace, as low skill jobs were exported or automated. The market responded to supply and demand: wages for college graduates skyrocketed because there were not enough of them; wages for high school graduates declined (in real dollars) because there was a glut of such workers, relative to the need.

As Richard Murnane (who often collaborates with Autor) has argued for decades, there is every reason to think that these employment trends will continue; jobs that require modest cognitive skill yet pay living wages will not return to the United States economy.

So yes, college is worth it—or better, obtaining cognitive skill that is demonstrable to employers is worth it. The promise of obtaining them outside of the usual educational route (via MOOCs, for example) is still a work in progress.

For educators and education researchers, this conclusion doesn’t change much of what we’ve been trying to do, as we already believed in the importance of education for individual economic promise. Autor’s analysis offers data should we care to take on skeptics, and adds urgency (if any was needed) to the work.

This column was originally published on RealClearEducation.com on May 13, 2014.

Last week I discussed the case Carl Wieman made that education research and physics have more in common than people commonly appreciate. I mentioned in passing Wieman’s suggestion that random control trials—often referred to as the “gold standard” of evidence—are not the be-all and end-all of research, and that qualitative research contributes too. But I didn’t say how it contributes, and neither did Wieman (at least in the paper I discussed). Here, I offer a suggestion.

Qualitative research is usually contrasted with quantitative research. In quantitative research the dependent variable—that is, the outcome—is measured as a quantity, usually using a measurement that is purportedly objective. In qualitative research, the outcome is not measured as a quantity, but as a quality.

For example, suppose I’m curious to know what students think about their school’s use of technology. In a quantitative study, I might develop a questionnaire that solicits student’s opinions about, say, the math and reading software they’ve been using, and also asks students to compare them to the paper versions they used last year. I’d probably try to get most or all of the students in the school to respond. I would end up with ratings which I can treat as quantitative data.  

Or I could do a qualitative study, in which I use focus groups to solicit their opinions. The result of this research is not numeric ratings, but transcripts of what people said. I’d try to find common themes in what was said. Because focus groups are time-consuming, I’d probably talk to just a handful of students.

People who criticize qualitative research treat it as a weak version of quantitative methods. For example, a critic would point out that you can’t trust the results of the focus groups, because I might have, by chance, interviewed students with idiosyncratic views. Another problem is that focus groups are not very objective; responses are a product of the dynamic between the interviewer and the subjects, and among the subjects themselves.

These complaints are valid, or would be if we hoped to treat the focus group results the way we treat the outcomes of other experiments. I suggest we should not.

Here’s a simple graphic I used in a book to describe how science works. We begin with observations about the world. These can come from our casual, everyday observations or from previous experiments. We then try to synthesize those observations into a theory; we try to find simplifying rules that describe the many observations we’ve made. This theory can be used to generate a prediction, something we haven’t yet observed, but ought to. Then we test the prediction, and that test leads to a new observation, a new fact about the world. And we continue around the circle, always seeking to refine the theory.

The criticism of qualitative research is that it does not provide a very reliable test of a theory. But I think it’s better viewed as providing a new observation of the world.

An advantage of qualitative over quantitative data is the flexibility in how its collected. If I want to know what students think of the new technology instruction and I create a quantitative scale to measure it, I’m really guessing that I know the important dimensions of student opinion. I may try to capture their views on effectiveness and ease-of-use, for example, but what students really care about is the fact that so many websites are blocked.

Qualitative studies allow the subject to tell you what he thinks is important. For example in this qualitative study, students suggested that the schools policy on blocking websites affected their relationships with teachers—they felt untrusted. Maybe that’s the kind of thing a researcher would have been looking for, but I doubt it.

In addition, qualitative data tend to be richer. I’m letting the subject describe in his or her own words what she thinks, rather than asking her to select from reactions that I’ve framed.

Naturally, either type of research—qualitative or quantitative--can be poorly conducted. Each has its own rules of the road. It’s important to judge the quality of research by its own set of best-practice rules, and to judge it by how well it fulfils the purpose to which it is suited.

That’s why I believe qualitative research has an undeserved bad reputation. It is judged by standards of quantitative research, and deemed unable to serve purposes that ought to be served by quantitative research. But I agree with Wieman that qualitative research, well-conducted, makes a valuable contribution, one to which quantitative research is ill-suited.

I have a new article (with Sian Beilock) on math anxiety in the new American Educator. Free download here:
http://www.aft.org/pdfs/americaneducator/summer2014/Beilock.pdf

This post first appeared at RealClearEducation.com on May 6, 2014

As someone who spends most of their time thinking about the application of scientific findings to education, I encounter plenty of opinions about the scientific status of such efforts. Almost always, a comparison is made between the rigor of “hard” sciences and education. What varies is the accompanying judgment: derision for education researchers or despondency about the difficulty of the enterprise. In a recent article, physicist Carl Wieman (2014) offers a different perspective on the issue, suggesting that the difference between research in education and in physics is smaller than you might guess.

In education, Wieman is best known for refining and popularizing techniques to have college students better engage in large lecture courses. He started his career as a physicist, producing work that culminated in a Nobel prize. So he has some credentials in talking about the work of “hard” scientists.

Wieman begins by making clear what he takes to be the outcome of good science: predictive power. Can you use the results of your research to predict with some accuracy what will happen in a new situation? A common mistake is to believe that in education one ought to be able to predict outcomes for individual students; not necessarily so, any more than a physicist must be able to predict the behavior of each atom. Prediction in aggregate—a liter of gas or a school of children—is still an advance.

Wieman’s other points follow from his strong emphasis on prediction.

First, it follows that “rigorous” methods are any that contribute to better prediction. You don’t state in the absolute that randomized controlled trials are better than qualitative research. They provide different types of information in a larger effort to allow prediction.

Second, the emphasis on prediction frames the way one thinks about the messiness of research. Education research is often portrayed as inherently messy because there are so many variables at play. Physics research, in contrast, is portrayed as better controlled and more precise because there are many fewer variables that matter. Wieman argues this view is a misconception.

Physics seems tidy because you’ve probably only studied physics in textbooks, where everything is worked out: in other words, where no extraneous variables are discussed as possibly mattering. When the work that you study was first being conducted, it was plenty messy: false leads were pursued, ideas that (in retrospect) are self-contradictory were taken seriously, and so on. The same is true today: the frontiers of physics research is messy. “Messy” means that you don’t have a very good idea of which variables are important in gaining the predictive power that characterizes good science.

Third, Wieman suggests that bad research is the same in physics and education. Research is bad when the researcher has failed to account for factors that, based on prior research, he or she should have known to include. There is plenty of bad research in the hard sciences. People aren’t stupid; it’s just that science is hard.

I agree with Wieman that differences in “hardness” are mostly illusory. (That’s why I’ve been putting the term in quotation marks.) The fundamentals of the scientific method don’t differ much wherever they are applied. I also agree that people (usually people uninvolved in research) are too quick to conclude that, compared to other fields, a higher proportion of education research is low-quality. Come to a meeting of the Society for Neuroscience and I’ll show you plenty of studies that were poorly conceived, poorly controlled, or were simply wheel-spinning and will be ignored.

Wieman does ignore a difference between physics and education that I take to have important consequences: physics (and other basic sciences) strive to describe the world as it is, and so strive to be value-neutral. Education is an applied science; it is in the business of changing the world, making it more like it ought to be. As such, education is inevitably saturated with values.

Education policy would, I think, benefit, with a greater focus on the true differences between education and other varieties or research, and a reduced focus on the phantom differences in rigor.

Reference:

Wieman, C. E. (2014). The similarities between research in education and research in the hard sciences. Educational Researcher, 43, 12-14.