I finally got around to reading Paul Tough's How Children Succeed. If you haven't read it yet, I recommend that you do.

You probably know by now the main message: what really counts for academic success is conscientiousness (or its close cousins, grit, or character or non-cognitive skills).

Tough intersperses explanations of the science behind these concepts with stories of students that he's met and followed. The stories add texture and clarity, and Tough is among a very small number of reporters who gets complex science right consistently. He takes you through attachment theory, the HPA axis, and executive control functions, all without losing his footing nor prompting glazing in the reader's eyes.

Tough also devotes considerable space to a fascinating inside look at how charter school mavens have thought about self-control, how their thinking has changed over time, and how their views square with the science.

The only flaws I see in the book concern a couple of big-picture conclusions that Tough draws.

First, there's what Tough calls the cognitive hypothesis--that academic success is driven primarily (perhaps even solely) by cognitive skills. The book suggests that this premise may be in error. What really counts is self-control.

But of course, you do need cognitive skills for academic success. In fact, Tough describes in detail the story of a boy who is very gritty indeed when it comes to chess, and who scales great heights in that world. But he's not doing all that well in school, and a teacher who tries to tutor him is appalled by what he does not know.

Self-control predicts academic success because it makes you more likely to do the work to develop cognitive skills. I'm sure Tough understands this point, but a reader could easily miss it.

Second, Tough closes the book with some thoughts on education reform. This section, though brief, struck me as unnecessary and in fact ill-advised. The whole book is about individual children and what makes them tick. Jumping to another level of analysis--policy--can only make this speculation seem hasty.

But these two minor problems are mere quibbles. If you have heard about "non-cognitive skill," or "self-control" or "grit" and wonder whether there's anything to it, you'd be hard put to find a better summary than How Children Succeed.

My colleague, Tim Wilson, has long advocated that the psychology department at the University of Virginia stop interviewing potential graduate students or job applicants.

We conduct unstructured interviews, as most departments do, meaning the candidate meets with an individual for twenty or thirty minutes and chats.

You do end feeling as though you have a richer impression of the person than that gleaned from the stark facts on a resume. But there's no evidence that interviews prompt better decisions (e.g., Huffcutt & Arthur, 1994).

A new study (Dana, Dawes, & Peterson, 2013) gives us some understanding of why.

The information on a resume is limited but mostly valuable: it reliably predicts future job performance. The information in an interview is abundant--too abundant actually. Some of it will have to be ignored. So the question is whether people ignore irrelevant information and pick out the useful. The hypothesis that they don't is called dilution. The useful information is diluted by noise.

Dana and colleagues also examined a second possible mechanism. Given people's general propensity for sense-making, they thought that interviewers might have a tendency to try to weave all information into a coherent story, rather than to discard what was quirky or incoherent.

Three experiments supported both hypothesized mechanisms.

The general method was this. 76 students at Carnegie Mellon University served as interviewers. They were shown the academic record of a fellow student who they would then interview.  (The same five students served as interviewees throughout the experiment.)

The interviewers were to try to gain information through the interview to help them predict the grade point average of the interviewee in the next semester. The actual GPA was available so the dependent measure in the experiment was the accuracy of interviewers' predictions.

The interviewers were constrained to asking yes-or-no questions. The interviewee either answered accurately or randomly. (There was an algorithm to produce random "yeses" or "nos" on the fly.) Would interviewers do a better job with valid information than random information?

It's possible that limiting the interview to yes or no questions made the interview artificial so a third condition without that constraint was added, for comparison. This was called the natural condition.

The results? There was evidence for both dilution and for sense-making.

Dilution because interviewers were worse at predicting GPA than they would have been if they had used previous GPA alone. So the added information from the interview diluted the useful statistical information.

Sense-making because ratings made after the interview showed that interviewers generally agreed with the statement "From the interview, I got information that was valuable in making a GPA prediction."

There were no differences among the accurate, random, and natural conditions on these measures.

It's possible that the effect is due, at least in part, to the fact that interviewers themselves pose the questions. That makes them feel that answers confirm their theories about the interviewee.

So in a second experiment researchers had subjects watch a video of one the interviews conducted for the first experiment, and use that as the basis of their GPA prediction. All of the results replicated.

Keep in mind, what's new in this experiment is not the finding that unstructured interviews are not valid. That's been long known. What's new is some evidence as to the mechanisms: dilution and sense-making.

And sense-making in particular gives us insight into why my colleagues in the psychology department have never taken Tim Wilson's suggestion seriously.

Reference:
Dana, J., Dawes, R., & Peterson, N. (2013) Belief in the unstructured interview: The persistence of an illusion. Judgement and Decision Making, 8,  512-520.

Huffcutt, A. I. & Arthur, W. Jr. (1994). Hunter and Hunter (1984) revisited: Interview validity for entry-level jobs. Journal of Applied Psychology, 79,  184-190.

The results of this experiment probably won't surprise you. What surprised me was the fact that we didn't already have data like this in hand.

The researchers (Sadler et al., 2013) tested 181 7th and 8th grade science teachers for their knowledge of physical science in fall, mid-year, and years end. They also tested their students (about 9,500) with the exact same instrument.

Each was a twenty-item multiple choice test. For 12 of the items, the wrong answers tapped a common misconception that previous research showed middle-schoolers often hold. For example, one common misconception is that burning produces no invisible gases. This question tapped that idea:

But the researchers didn't just ask the teachers to pick the right answer. They also asked teachers to pick the answer that they thought their students would pick.

What makes this study interesting is that it tests teacher subject-matter knowledge directly (instead of using a proxy like courses taken, or degrees) and that it directly measures one aspect of pedagogical content knowledge, namely, student misconceptions. The dependent measure of interest is student gain scores in content knowledge over the course of the year.

The results?

Teachers content knowledge was good, but not perfect. They got about 84% of the questions right.

Their knowledge of student misconceptions was not as good. Teachers correctly identified just 43% of those. (And their students had, as in previous studies, selected those incorrect items in high numbers.)

And what type of teacher knowledge matters to student learning? It turns out to interact with past student achievement, as measured by standard math and reading tests.

The graph shows gains in student knowledge, separated by items for which teachers have (or lack) various types of knowledge. Filled circles are for students who scored well on a math and reading test (high achievers), and open circles are students who scored poorly (low achievers)

Look first at learning for concepts without  a common misconception. If teachers have subject matter knowledge (SMK in the graph) students learn the concept better. In fact, low-achieving students learned nothing about a concept if teachers didn't know the concept themselves. High-achieving students did. The researchers speculate they may have learned the content from a textbook or other source.

For the strong misconception items, the low-achieving students learned very little, whatever the teacher knowledge. For high-achieving students, knowledge mattered, and they were most likely to learn when their teacher had both subject-matter knowledge and knew the misconceptions their students likely held (KoSM in the graph).

So the overall message is not that surprising. Students learn more when their teachers know the content, and when they can anticipate student misconceptions.

Somewhat more surprising (and saddening), low-achieving students are especially vulnerable when teachers lack knowledge. High-achieving students are more resilient.

There are limitations to this study, the most notable being that the sample is far from random (teachers were volunteers), and that the test was zero-stakes for all.

The strength was the direct measure of both types of knowledge, and that the researchers could examine the relationship of knowledge to performance at the level of individual items. One hopes we'll see more studies using this type of design.

Reference:
Sadler, P. M., Sonnert, G., Coyle, H.P., Cook-Smith, N., & Miller, J.L. (2013) Student learning in middle school science classrooms. American Educational Research Journal, 50, 1020-1049.

We all know that most Americans don't read much. A recent poll showed that a common reason they don't read is "lack of time." Fifty-one percent suggested that was a major factor that kept them from reading more books.

It's tempting to quote Sir John Lubbock: "In truth, people can generally make time for what they choose to do; it is not really the time but the will that is lacking." That's the line of thinking taken in this Atlantic blog, noting that many of us spend plenty of time watching television.

This line of argument is true enough, but probably won't help much. So without scolding, here are some ideas on how to think about reading and time differently.

1) Don't assume that that you have to have a long block of time to read. Bit and pieces add up. If you think "I need at least thirty minutes of uninterrupted time to get into the book," well, try fitting reading into the bits and pieces of time in your day. You're ready to go out and your spouse isn't? There's five minutes. Long line in the grocery store? There's five minutes. 

2) Be prepared. To make use of these times, keep books in places where you find yourself with a few minutes. Bathroom. (Let's not deny it.) Kitchen (if you eat alone). Car (also useful when you're not driving, but at your destination. B. F. Skinner noted he read Thoreau's Walden, which he kept in his glove box, in snippets when waiting for late-comers.) Get audio books for your commute.
3) The best preparation is on your phone. It's not my favorite way to read, but you always have your phone with you. Get Kindle for your iPhone or Android. Reading emergencies--e.g., my kid was supposed to play but isn't and now I'm stuck watching other people's kids play pee-wee soccer--reading emergencies happen.
3) Don't assume that you can only read one book at a time. If you've got books distributed in different spots, won't you get mixed up? Probably not. But if you are really worried about that, start with books that have lots of short stuff: Uncle John's bathroom reader in the car, Chekhov short stories in your purse, etc.
4) You don't you have to finish what you start. For a long time I assumed that if I started a book I was in some way obligated to finish it. Or maybe that if I didn't, I had wasted my time in starting it. This attitude makes no sense. Don't fail to start a book because you're afraid it might turn out too challenging or emotionally hard, or whatever. If you don't like the book, abandon it.
5) No, seriously, I'm too busy. When was the last time you were bored? If you really can't remember, then okay, you're too busy. If you can name a time, then you could have been reading instead of being bored.