Which of these learning situations strikes you as the most natural, the most authentic?

1) A child learns to play a video game by exploring it on his own.
2) A child learns to play a video game by watching a more experienced player.
3) A child learns to play a video game by being taught by a more experienced player.

In my experience a lot people take the first of these scenarios to be the most natural type of learning—we explore on our own. The third scenario has its place, but direct instruction from someone is a bit contrived compared to our own experience.

I’ve never really agreed with this point of view, simply because I don’t much care about “naturalness” one way or the other. As long as learning is happening, I’m happy, and I think the value some people place on naturalness is a hangover from a bygone Romantic era, as I describe here.

Now a fascinating paper by Patrick Shafto and his colleagues (2012) (that’s actually on a rather different topic) leads to implications that call into doubt the idea that exploratory learning is especially natural or authentic.

The paper focuses on a rather profound problem in human learning. Think of the vast difference in knowledge between a new born and a three-year-old; language, properties of physical objects, norms of social relations, and so on. How could children learn so much, so rapidly? 

As you're doubtless aware, from the 1920's through the 1960's, children were viewed by psychologists as relatively passive learners of their environment. More recently, infants and toddlers have been likened to scientists; they don't just observe the environment, they reason about what they observe.

But it's not obvious that reasoning will get the learning done. For example, in language the information available for their observation seems ambiguous. If a child overhears an adult comment “huh, look at that dog,” how is the child to know whether “dog” refers to the dog, the paws of the dog, to running (that the dog happens to be doing), to any object moving from the left to the right, to any multi-colored object etc.?

Much of the research on this problem has focused on the idea that there must be innate assumptions or biases on the part of children that help them make sense of their observations. For example, children might assume that new words they hear are more likely to apply to nouns than to adjectives.

Many models using these principles have not attached much significance to the manner in which children encounter information. Information is information.

Shafto et al. point out why that's not true. They draw a distinction between three different cases with the following example. You’re in Paris, and want a good cup of coffee.

1) You walk into a cafe, order coffee, and hope for the best.
2) You see someone who you know lives in the neighborhood. You see her buying coffee at a particular cafe so you get yours there too.
3) You see someone you know lives in the neighborhood. You see her buying coffee at a particular cafe. She sees you observing her, looks at her cup, looks at you, and nods with a smile

In the first case you acquire information on your own. There is no guiding principle behind this information acquisition. It is random, and learning where to find good coffee will slow going with this method.

In the second scenario, we anticipate that the neighborhood denizen is more knowledgeable than we--she probably knows where to get good coffee. Finding good coffee ought to be much faster if we imitate someone more knowledgeable than we. At the same time, there could be other factors at work. For example, it's possible that she thinks the coffee in that cafe is terrible, but it's never crowded and she's in a rush that morning.

In the third scenario, that's highly unlikely. The woman is not only knowledgeable, she communicates with us; she knows what we want to know and she can tell us that the critical feature we care about is present. Unlike scenario #2,  the knowledgeable person is adjusting her actions to maximize our learning. 

More generally, Shafto et al suggest that these cases represent three fundamentally different learning opportunities; learning from physical evidence, learning from the observation of goal-directed action, and learning from communication.

Shafto et al argue that although some learning theories assume that children acquire information at random, that's likely false much of the time. Kids are surrounded by people more knowledgeable than they. They can see, so to speak, where more knowledgeable people get their coffee.

Further, adults and older peers often adjust their behavior to make it easier for children to draw the right conclusion. Language is notable in its ambiguity-“dog” might refer to the object, its properties, its actions—but more knowledgeable others often do take into account what the child knows, and speak so as to maximize what the child can learn. If an adult asked “what’s that?”  I might say “It’s Westphalian ham on brioche.” If a toddler asked, I ‘d say “It’s a sandwich.”

One implication is that the problem I described—how do kids learn so much, so fast—may not be quite as formidable as it first seemed because the environment is not random. It has a higher proportion of highly instructive information. (The real point of the Shafto et al. paper is to introduce a Bayesian framework for integrating these different three types of learning scenarios into models of learning.)

The second implication is this: when a more knowledgeable person not only provides information but tunes the communication to the knowledge of the learner, that is, in an important sense, teaching.

So whatever value you attach to “naturalness,” bear in mind that much of what children learn in their early years of life may not be the product of unaided exploration of their environment, but may instead be the consequence of teaching. Teaching might be considered a quite natural state of affairs.

EDIT: Thanks to Pat Shafto who pointed out a paper (Csibra & Gergely) that draws out some of the "naturalness" implications re: social communication. 

Reference
Shafto, P., Goodman, N. D. & Frank, M. C. (2012). Learning from others: The consequences of psychological reasoning for human learning. Perspectives in Psychological Science, 7, 341-351.

If you read up on math pedagogy long enough you will see a reference to Paul Lockhart’s Mathematician’s Lament. It even has it’s own Wikipedia entry.

It is a marvelous little book of 140 pages, that makes a simple, 3 part argument about how to improve mathematics education the US.

Unfortunately, 89 pages of the book remains unwritten and it contains the third, decisive part of the argument.

The argument looks like this:

1)      Math as it is taught in the US is  boring

2)      Math doesn’t need to be boring. In fact, math is interesting and beautiful

3)      We can teach children the beauty and fascination of math in US schools by doing X.

Lockhart devotes 90 pages of the book to the first proposition, about 88 pages more, I estimate, than is necessary.

Lockhart suggests that the root of the problem lies in how teachers conceive or mathematics. Actually, it’s how everyone (save mathematicians) conceive of mathematics. We see it as a rigid, rule-based, practical. Lockhart offers mathematics as aesthetically pleasing problem solving. No more, no less. 

Mathematical ideas are inherently interesting, charming, fun. If you’re not interested, you’re doing math wrong—or being taught math wrong.

So why is math taught wrong?

Lockhart suggests that teaching math requires intense personal relationships with students, “choosing engaging and natural problems suitable to their tastes, personalities, and levels of experience,” and being flexible and open to the students’ shifts in curiosity. Lockhart doubts most teachers are interested in this sort of thing, and he suggests that most teachers see it as too much work (p. 44).

Later (p. 82) bureaucrats are blamed; they won’t allow individual teachers to follow their instincts. Later, we’re told more bluntly that schools ruin not just math, but all subjects.

But what of the third part of the argument in which Lockhart ought to tell us how make things better?

His vision of “better” is that teachers will pose interesting problems to students—he offers many compelling examples—and students will work on these problems under the guidance (ideally, the minimal guidance) of the teacher. The best learning, he avers, is where the student is doing math (i.e., creating arguments, finding patterns) rather than executing formulae described by others.

There is an irony here. Lockhart describes mathematical arguments as two-headed. One head is relentlessly logical, and rigidly insists that an argument be airtight. The other head has aesthetic criteria, and seeks an argument that is elegant, lovely, and that sheds light as it proves.

Lockharts prescription is all lovely and not enough logic.

For someone who excoriates a system that would allow people who don’t know the history of math to teach it, Lockhart is surprisingly quick to write about educational practice in the absence of any knowledge of its history.

Giving students interesting problems and aiding their efforts to solve them as the workhorse method of classroom learning—that idea has been mainstream for about 100 years. Further, surveys of teachers consistently show that they believe this method (and closely aligned methods) to be not only effective but desirable.

Teachers don’t fail to use these methods because they are lazy or because bureaucrats won’t let them.

These methods are really hard to pull off. Your knowledge of math needs to be very deep because the problem may pivot in an unexpected direction. Your classroom management needs to be flawless because you are expecting the students to work more independently.  And both knowledge of math and classroom management will be tapped further by the fact that you must make many decisions in the moment, as the classroom situation is very fluid.

The unfortunate thing about Mathematicians Lament is that Lockhart has put his finger on a real problem but is so caught up in righteous indignation that he loses the chance of doing any good. Simple scolding won’t do it. Jeremiah had compassion for the benighted in the Book of Lamentations.

A much more effective approach has been adopted by Hung-Hsi Wu, a mathematician at Berkeley. Wu has argued that a key problem is that today’s math teachers—products of the American system themselves—don’t get math. The solution is to teach them some math. (I once listened to a 30 minute explanation by Wu of why our system of whole numbers works the way it does. Quite literally, stuff that first graders can and should know. I was spellbound.)

In contrast to Lockhart, Wu has some faith in teachers. If they understand mathematics, they will teach it. He is also less dogmatic than Lockhart, who unthinkingly assumes that they only way to learn a topic is to practice it the way experts practice it. Indeed, some important elements that Lockhart wants to see—especially discovery—are present in some quite traditional approaches, especially the Japanese approach to teaching, as described by Jim Stigler.

Righteous indignation should be an occasional guilty pleasure, not a blueprint for math education.

Elsewhere I have written about the potential power of narrative to help students understand and remember complex subject matter (Willingham, 2004; 2009). Now a new study (Arya & Maul, 2012) provides fresh evidence that putting to-be-learned material in a story format improves learning outcomes.

The experiment tested 209 7th and 8th grade students in the U.S. on texts about the discoveries of Galileo OR the discoveries of Marie Curie. The texts were developed to be as similar as possible in terms of syntactic complexity, vocabulary, accuracy, and other measures, and vary only in whether the information was presented in a typical expository fashion or in terms of a personal story of the scientist.

For example, one section of the expository text included this passage

The corresponding passage in the narrative version read this way:

Students comprehension and memory for the information in the text was measured immediately after reading it, and again one week later. The difference in recall between the narrative and non-narrative versions are shown as difference scores below.

Science lends itself naturally to narrative structure--authors can tell the stories of individual scientists, their struggles, their discoveries, and so on.

There's a case to be made that it also lends itself to a triumphalist view of science that is not accurate; scientists as heroes in an ever-progressing march towards Truth. Since Kuhn, that more or less Popperian view of science has been viewed as at least too simple, and more likely inaccurate.

But if it helps middle schoolers understand science, I'm inclined not worry too much about that point.

Instead, I'd like to broaden the view of "narrative." (I made this point in Why Don't Students Like School.) You don't have to think of narrative just as the story of an individual or group of people; you can think more abstractly conflict, complications, and the eventual resolution of conflict as the core of narrative structure.

I prefer to think of narrative in this broader sense because it is more flexible, and gives teachers more options, and also better captures the aspects of narrative structure that I suspect are behind the advantage conferred.

Reference

  Arya, D. J. & Maul, A. (2012). The role of the scientific discovery narrative in middle school science education: An experimental study. Journal of Educational Psychology, 104, 1022-1032.

A teacher from the UK has just written to me asking for a bit of clarification (EDIT: the email came from Sue Cowley, who is actually a teacher trainer.)

She says that some people are taking my writing on the experiments that have tested predictions of learning styles theories (see here) as implying that teachers ought not to use these theories to inform their practice.

Her reading of what I've written on the subject differs: she thinks I'm suggesting that although the scientific backing for learning styles is absent, teachers may still find the idea useful in the classroom.

The larger issue--the relationship of basic science to practice--is complex enough that I thought it was worth writing a book about it. But I'll describe one important aspect of the problem here.

There are two methods by which one might use learning styles theories to inspire ones practice. The way that scientific evidence bears on these two methods is radically different.

Method 1: Scientific evidence on children's learning is consistent with how I teach.

Teachers inevitably have a theory--implicit or explicit--of how children learn. This theory influences choices teachers make in their practice. If you believe that science provides a good way to develop and update your theory of how children learn, then the harmony between this theory and your practice is one way that you build your own confidence that you're teaching effectively. (It is not, of course, the only source of evidence teachers would consider.)

It would seem, then, that because learning styles theories have no scientific support, we would conclude that practice meant to be consistent with learning styles theories will inevitably be bad practice.

It's not that simple, however. "Inevitably" is too strong. Scientific theory and practice are just not that tightly linked.

It's possible to have effective practices motivated by a theory that lacks scientific support. For example, certain acupuncture treatments were initially motivated by theories entailing chakras--energy fields for which scientific evidence is lacking. Still, some treatments motivated by the theory are known to be effective in pain management.

But happy accidents like acupuncture are going to be much rarer than cases in which the wrong theory leads to practices that are either a waste of time or are actively bad. As long as we're using time-worn medical examples, let's not forget the theory of four humors.

Bottom line for Method 1: learning styles theories are not accurate representations of how children learn. Although they are certainly not guaranteed to lead to bad practice, using them as a guide is more likely to degrade practice than improve it.

Method 2: Learning styles as inspiration for practice, not evidence to justify practice.

In talking with teachers, I think this second method is probably more common. Teachers treat learning styles theories not as sacred truth about how children learn, but as a way to prime the creativity pump, to think about new angles on lesson plans.

Scientific theory is not the only source of inspiration for classroom practice. Any theory (or more generally, anything) can be a source of inspiration.

What's crucial is that the inspirational source bears no evidential status for the practice.

In the case of learning styles a teacher using this method does not say to himself "And I'll do this because then I'm appealing to the learning styles of all my students," even if the this was an idea generated by learning styles. The evidence that this is a good idea comes from professional judgment, or because a respected colleague reported that she found it effective, or whatever.

Analogously, I may frequently think about Disneyland when planning lessons simply because I think Disneyland is cool and I believe I often get engaging, useful ideas of classroom activities when I think about Disneyland. Disneyland is useful to me, but it doesn't represent how kids learn.

Bottom line for Method 2: Learning styles theories might serve as an inspiration for practice, but it holds no special status as such; anything can inspire practice.

The danger, of course, lies in confusing these two methods. It would never occur to me that a Disneyland-inspired lesson is a good idea because Disneyland represents how kids think. But that slip-of-the-mind might happen with learning styles theories and indeed, it seems to with some regularity.

Suppose a high school student cheats on a test. How harshly should a parent or teacher treat this offense? The efficacy of punishment in such situations has been controversial, and a new study sheds some light on the consequences of punitive control in moral matters.

When someone behaves immorally, a psychologist would typically say that the individual has not internalized the moral norm; that is, she may profess that an action is morally wrong, but she has not really taken that information to heart.

How can a parent or teacher encourage such internalization?

Attribution theories have been quite influential in how psychologists think about this matter. These theories hold that a parent should NOT ensure moral behavior through the use of harsh or punitive measures.

Doing so will gain compliance because the child will want to avoid punishment, but the child will not internalize the norm. He will attribute his "good" behavior not to his own belief in the moral precept, but to the desire to avoid punishment.

That theory would predict, then, that once the threat of punishment is removed, these children will engage in the forbidden behavior. That prediction is born out in research.

But another prediction of the theory is wrong. If children have not internalized the moral precept, they should feel no shame if they break the rule. But they do feel ashamed.

Sana Sheik and Ronnie Janoff-Bulman (2013) sought to resolve this paradox.

Their hypothesis: punitive parenting on moral matters does lead children to internalize the moral precept (hence the shame). But it also paradoxically makes children less well equipped to resist temptation (hence the tendency to engage in the immoral behavior).

Why would it be harder to resist temptation? Well, imagine that I say to you "Hey, it would probably be better if you didn't think about white bears." The idea of a white bear would flash in your mind, but you'd soon think about something else.

But suppose I said "FOR THE LOVE OF GOD AND ALL THAT IS HOLY, DON'T THINK ABOUT A WHITE BEAR, I BEG OF YOU." By making it such a charged request, I have paradoxically made it harder to not think about a white bear.

That's the theory. The experimental data seem to support it, although the logic is a little convoluted.

First, subjects were asked to fill out a questionnaire about discipline used by their parents. Responses were used to judge the extent to which parents were punitive about moral matters. (This use of questionnaires is common in this literature.)

Second, experimenters asked subjects to write out moral precepts; some subjects wrote about positive precepts (things that you should do) or proscriptive ones (things you should not do). They did this to prime subjects, that is, to get them thinking about these moral issues.

They varied the type of precept (positive or proscriptive) because harsh parental control is virtually always directed at moral proscriptions.

Third, they asked subjects to briefly describe the action depicted in some paintings. These paintings (like the one shown at left) are emotionally ambiguous--they are open to a wide range of interpretations. Crucially, subjects in the experiment were also told "Please do NOT use any words related to bad, immoral, undesirable behaviors, intentions, or outcomes (e.g., sneaky)."

Here's the idea. subjects who had first been prompted to think about moral proscriptions would be thinking about moral proscriptions when they saw the paintings, but they were told NOT to use those thoughts in describing the paintings. Inhibiting those thoughts would presumably "cost" them--their resources for regulating attention would be somewhat depleted. And that should be especially true for students whose parents were punitive about moral matters because such issues are hyper-charged for them.

How could you show that their attentional resources were depleted from the struggle to not think about moral proscriptions?

In the final phase of the experiment, the researchers administered a STROOP task--that's the task in which subjects are asked to name ink colors while refraining from mistakenly saying the color names spelled out. Doing so requires attention regulation, and the more tired you are from inhibiting the immoral descriptions of the paintings, the worse you'll do on the STROOP task.

And, as predicted, subjects whose parents were punitive were worse on the STROOP task than subjects whose parents were not.

But maybe kids with punitive parents are bad at the STROOP for other reasons. Maybe these just happen to be kids who lack attentional resources. In fact, maybe that's why they often got in trouble, and that's why their parents had to be kind of hard on them.

That's why the experimenters also included a condition where subjects were asked to think about positive moral precepts. Without that prime, everyone should find it easier NOT to think about proscriptive morality. And indeed, subjects whose parents were punitive don't have a particular problem with the STROOP in this condition.

As I promised, the chain of logic supporting the theory is a bit convoluted. Still, the basic account seem plausible.

If further data support the theory, the upshot for parents and teachers would be that harsh responses to moral transgressions won't work. They leave subjects feeling ashamed when they transgress, but paradoxically they make is harder to resist the temptation to transgress.

Shikh, S. & Janoff0Bulman, R. (2013). Paradoxical consequences of prohibitions. Journal of Personality and Social Psychology. Advance online publication: doi: 10.1037/a0032278

Most teachers t think that students today have a problem paying attention. They seem impatient, easily bored.

I’ve argued that I think it’s unlikely that they are incapable of paying attention, but rather that they are quick to deem things not worth the effort.

We might wonder if patience would not come easier to a student who had had the experience of sustaining attention in the face of boredom, and then later finding that patience was rewarded. Arguably, digital immigrants were more likely to have learned this lesson. There were fewer sources of distraction and entertainment, and so we were a bit more likely to hang in there with something a little dull.

I remember on several occasions when I was perhaps ten, being sick at home, watching movies on television that seemed too serious for me—but I watched them because there were only three other TV channels. And I often discovered that these movies (which I would have rejected in favor of game shows) were actually quite interesting.

Students today have so many options that being mildly bored can be successfully avoided most of the time.

If this analysis has any truth to it, how can digital natives learn that patience sometimes brings a reward?

Jennifer Roberts, a professor of the History of Art and Architecture at Harvard, has a suggestion.

She gave a fantastic talk on the subject at a conference hosted by the Harvard Initiative on Learning and Teaching (more here).

Roberts asks her students to select a painting from a Boston museum, on which they are to write an in-depth research paper.

Then the student must go the museum and study the painting. For three hours.

The duration, “meant to seem excessive” in Roberts’ words, is, of course, part of the point. The goal is that the student think “Okay, I’ve seen about all I’m going to see in this painting.” But because they must continue looking, they see more. And more. And more. Patience is rewarded.

Roberts gave an example from her own experience. As part of a book she was writing on 18th century American painter John Singleton Copley, she studied at length the painting A Boy With a Flying Squirrel. Although she is, obviously, an extremely experienced observer of art, Roberts noted that it was many minutes before she noticed that the shape of the white ruff on the squirrel matches the shape of the boy’s ear, and is echoed again in the fold of the curtain over his left shoulder.

If we are concerned that students today are too quick to allow their attention to be yanked to the brightest object (or to willfully redirect it once their very low threshold of boredom is surpassed), we need to consider ways that we can bring home to them the potential reward of sustained attention.

They need to feel the pleasure of discovering that something you thought you had figured out actually has layers that you had not appreciated.

That may not be the 21st century skill of greatest importance, but it may be the one in shortest supply.

Most everyone recognizes that talking on a cell phone while driving is a bad idea. It's distracting and so makes driving more dangerous.

It might seem that talking on a hands-free device would solve the problem, but it has been known for a while that's not so (e.g., Treffner & Barrett, 2004). Most of the distracting effect of mobile phone use is not due to looking away from the road to dial, nor the cost of holding the phone.

Talking on a phone is more demanding than talking to someone in the car: the audio signal is degraded, you have no visual information to rely on, your conversational partner doesn’t stop talking if the road situation becomes complicated, you may try to imagine being with the other person, and so on. (It is true, however, that driving is somewhat worse with hand=held than hands-free devices; Backer-Grøndahl & Sagberg, 2011.)

So what's the solution? Presumably, not answering your phone.

Now new data show that even that carries a cost.

A recent study (Holland & Rathod, 2013) had 27 young drivers use a highly realistic driving simulator, shown at right. Seven hazardous situations developed during the simulation (e.g., a pedestrian crossing the road, a car pulling out). Subjects had provided their mobile number to the experimenters, were aware that it might ring during the simulation, and that they should not answer it. The subject's cell phone rang during three of the hazards.

Seven dependent measures were collected, including crossings of the center line, collisions with other cars, and so on.

Ignoring the ringing mobile phone carried a cost. When their phone rang just prior to a hazard, subjects were more likely to hit pedestrians, to exceed the speed limit, and to cross the center line.

It is notable that these effects were more pronounced in subjects who said they usually answered their phone while driving than in subjects who said they did not. Hence, a significant contributor to the distraction may be the mental effort required to inhibit a habitual response.

What's the solution? The safest practice for drivers may be to turn the phone off, not just ignore it.

References


Backer-Grøndahl, A., & Sagberg, F. (2011). Driving and telephoning: Relative accident risk when using hand-held and hands-free mobile phones. Safety Science, 49, 324-330.

Holland, C., & Rathod, V. (2013). Influence of personal mobile phone ringing and usual intention to answer on driver error. Accident Analysis & Prevention, 50, 793-800.

Treffner, P. J., & Barrett, R. (2004). Hands-free mobile phone speech while driving degrades coordination and control. Transportation Research Part F: Traffic Psychology and Behaviour, 7, 229-246.

My Facebook feed today has lots of links to this article. The upshot: a new Pew study showing that Americans think that US 15 year olds rank "near the bottom" on international science tests, whereas the truth is that they "rank in the middle among developed countries."

I guess "the middle" covers a lot of terrain, but the way I look at the data, this assertion doesn't hold.

The international comparison in question is the 2009 PISA. Here are the rankings. (Click for larger image)

Most everyone would agree that it's not appropriate to compare scores of US kids to those of poorer countries with little infrastructure and funding to support education.

That's why the article specifies the ranking of the US among "developed countries," and by the author's reckoning, kids from 12 developed countries scored better, and kids from 9 developed countries scored worse. That would put US kids at the 41st percentile.

The US is ranked 30th on the list. Just eyeballing it, it's hard to see how 17 of the countries scoring better could be considered "not developed."

On measures of "developed" status would be the International Monetary Fund's definition of "advanced economies" which includes: Australia, Austria, Belgium, Canada, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hong Kong, Iceland, Ireland, Israel, Italy, Japan, Luxembourg, Malta, Netherlands, New Zealand, Norway, Portugal, San Marino, Singapore, Slovakia, Slovenia, South Korea, Spain, Sweden, Switzerland, Taiwan, United Kingdom, United States (Click image for larger image

By this definition of "advanced" US kids are 23rd out of 32 countries, or the 28th percentile.

It's true that "near the bottom" is too grim an assessment. But I can't see a way to put the 2009 PISA data together such that American kids are scoring about average.

I think of two very broad education reform camps. One calls into question the basic arrangement of institutions involved in U.S. education, arguing that the contradictory priorities in the system almost guarantee mediocrity. The solution, therefore, cannot be a nibbling around the edges of reform, but wholesale change: for some reformers, that means a market solution with greater parental choice, often coupled with more stringent human resources policies. For others the solution is a complete change—via technology—of the way we think of “learning.”

The second group of reformers argues that the system of education institutions is mostly fine, and that factors external to the system are responsible for our woes (which are, in any case, exaggerated). Some point to social and economic factors, others to the incoherence in curriculum (cf the Common Core), and others to the very reform measures (especially standardized tests used to evaluate schools and teachers) instituted by the other group of reformers.

In his new book Improbable Scholars,  UC Berkeley professor of education David Kirp offers an unusually readable account of what improved schooling would look like if you’re in the second camp. His explicit mission: to show that educational excellence is possible with the system as it exists now, even in districts that face enormous challenges.  He makes a fair case, given the limitations of the method he employs.

Improbable Scholars follows in the tradition of numerous education books by recounting time that the author spent in a school or district. Kirp tells the story of Union City, NJ, a city like so many others in the US: it has a great manufacturing past (“Embroidery Capital of the United States”) but was unable to find a new economic identity when cheap imports undermined its industries. Now most of its residents live in poverty, and a large percentage are recent immigrants who speak little English.

But Union City schools are unlike most districts with this profile. Despite the demographics, Union City students score about average on state tests. Ninety percent graduate high school, and sixty percent go on to college.

How they do it is Kirp’s subject, and in one sense this book has the feel of many others.  The account is told through stories. We meet Alina Bossbaly, a local legend of a third-grade teacher who is able to connect even with the most difficult children, and to make them feel a part of the classroom community, a process that has come to be known as “Bossbaly-izing” children.

We meet long-time Union City Mayor Brian Stack, strong supporter of education, savvy politico in a tough political town, and point man in the procurement of funding for the new 180 million dollar high school.  

Kirp is an academic, not a journalist, so although he’s an able writer, you’re not in the hands of a professional storyteller or fact-finder. But what you get from Kirp is a deeper analysis, a better-than-even tradeoff in this case.

So what is Kirp’s conclusion? He offers a list of key factors that he says must be in place for a district to thrive:

• District leaders put the needs of students ahead of those of staff
• They invest in quality preschool
• They insist that a rigorous curriculum is consistently implemented
• They make extensive use of data to diagnose problems
• The engender a culture of respect among the staff
• They value stability and avoid drama—they make a plan and stick with it for the long haul
• They never stop planning and reviewing the results of their plans.

When a district posts a remarkable record, it’s natural to ask “how did they do it?” The obvious problem is you’re looking at a single district. Maybe the real key to Union City is the Mayor. Maybe it’s the fact that many of the students come from countries with a tradition of respect for authority.

Kirp makes a case that other unusually successful districts have the same set of factors in common. It’s no substitute for a quantitative analysis, but KIrp at least shows that he’s aware of the problem.

And to be clear, I read the book in this wise, as something like an ethnographic study. Books like this offer detail and texture that larger scale, more rigorous analyses lack. In so doing, they ought to be inspiration to more quantitatively oriented researchers for what they are missing and where to turn their sights next. 

When it comes to criticizing methods he thinks are ineffective, Kirp is less sure-footed. He dismisses the notion that the relationship between school funding and student achievement is uncertain by noting that such suggestions leave administrators “shaking their heads.” There is an extensive and complex literature on the impact of funding, and the proper conclusion is by no means as simple Kirp would like us to believe.

Likewise, I’m rankled by Kirp’s assertion that “If you’re a teacher or principal whose job is on the line and your ordered to accomplish what seems unattainable, cheating is a predictable response.” This sounds an awful lot like a tacit pass to cheating educators.

The section of Improbable Scholars devoted to “what doesn’t work” left a bad taste in my mouth because it comes at he end of the book, but it is a mere five pages long.

If you’re curious about one vision of successful education that more or less maintains the status quo and actually gets into some detail, Improbable Scholars is a good choice.

What would you say if a major corporation took out a full-page ad in the New York Times to advertise a message that you thought was important and mainly agreed with, only to find that the text of the message was rife with misspellings, grammatical errors, and misused words?

That's the feeling that I get from the new video all over my Facebook feed (and with over 7 million views in 4 days) titled "Dove Real Beauty Sketches." If you haven't seen it, here you go. (I summarize it below.)

In brief, a woman describes herself to a forensic sketch artist, who cannot see her. He draws her portrait based only on her description, and then draws her again based on the description of a stranger who just met her. The woman then sees both portraits and recognizes that she has been rather hard on herself in her self description. (The process is shown for several women.)

The associated website calls this "a social experiment." But it's a terrible example of experimentation.

We are invited to draw the conclusion that women see themselves as less attractive than others do. I don't know the self-perception literature well, but I'm pretty sure this conclusion is right. But this experiment is a terrible way to illustrate that.

• The artist should be blind to condition. He knows when he's basing the drawing on the description of the subject vs. the stranger, and so could unconsciously bias the result
• The descriptions are not based on perception, they are based on memory. If you want to claim that it's about how women see themselves, not how they remember themselves, then each person should do their best to describe the woman based on the same photograph
• A the end the sketch artist tells each woman the source of each sketch. What would have happened if he had asked her to say which looks more like her, and to say which she thought was based on her description? If women's perception is really distorted, then the woman should see sketch based on her description as being more like her. An alternative hypothesis is that women more or less know what they look like, but talk about themselves in negative terms.
  
• The foregoing point raises another issue: social conformity. If the result is not due to perception but to people conforming to social norms, the difference in the sketches might be due to the women's reluctance to seem vain in their self-descriptions, and to the stranger feeling that he or she ought to describe the woman nicely.
  

How important are these criticisms to the overall message of the video? Not very. The point of the video is that women shouldn't be so hard on themselves in judging their looks. It's a good message.

That's why I draw the analogy to grammar, punctuation, and spelling in a written message. If Dove had published a print ad full of grammatical and spelling errors, I expect someone would have called them out on it.

Dove presents this as an experiment, but it's a terrible experiment. It would not have been hard to do a video making the same point with a better experiment. Any graduate student of social psychology could have improved this ten-fold.

I would have given the video 9/10 (subtracting one point for scientific sloppiness) if not for the statement made here in the video:

Well, I'd prefer a different message. Rather than "It couldn't be more critical to your happiness" and "be grateful for your natural beauty" I'd prefer a message amounting to "what you look like matters less than you think."

But I can't expect everything from a company selling beauty products. 8 out of 10, Dove.