A few weeks ago I published an op-ed in the NY Daily News about how purchasing decisions are made in educational technology. In this blog I want to elaborate on a couple of points I made.

First, regarding the overall point of the op-ed: most people got it, but a few took it to be a criticism of the use of technology in schools, and wrote me irate emails, tweets, or blog posts. The op-ed was a criticism of how tech purchasing decisions are made, not a blanket criticism of ed tech. "Ed tech" is such a broad category (like "explicit instruction" or "progressive pedagogy") it seems improbable you're going to be able to draw conclusions about effectiveness that will apply in most cases. Sure, you can do meta-analyses (Hattie has the effect size of tech interventions at around 0.3, I believe) but there will be so many exceptions and caveats, it may not be worth it. 

My point in the op-ed was that data to guide purchasing decisions seldom exist. Decisions must be based on intuition, and intuition has in the past been a poor guide. 

Second, op-eds do not provide space to get much into evidence, so I provide some citations here, specifically for the three reasonable-sounding intuitions that I said were wrong.

Intuition 1: reading on screen and paper will be equivalent. I claimed that reading comprehension consistently takes a small hit on screen vs. paper. I just published a piece over at Larry Felazzo's blog that cites lots of the data on this point.  

Intuition 2: information is more readily accessible, so students don't need to have as much information in their memory. I claimed that the Internet has not changed the need to have information in ones head. There's a great deal to say about this claim, and other reasons I listed ended up edited out of the piece; I've described these reasons in magazine pieces, books (here and here), blog posts, and a video. One claim that remained in the op-ed was that people can look stuff up, but they don't. Once the number of words you don't know hits two percent, the odds go way up that readers find the text difficult and will quit.

The 2% figure comes from Carver (1994), who called texts with 0% unknown words "easy," 1% unknown words "appropriate" and 2% or more "difficult." Schmitt et al (2011), studying second language learners, argued that there is (unsurprisingly) a linear relationship between known vocabulary and comprehension, and that 98% is a reasonable middle ground for what constitutes appropriate challenge.

I touched only briefly on the fact that "looking things up" is trickier than it sounds, and usually requires a fair bit of knowledge to get right, a point made forcefully by George Miller in 1987. As he points out, meaning comes from context, and dictionaries can't provide much. So children look up a word like "relegate," take away an abbreviated, context-free understanding of the definition, like "send away," and so misuse the word, e.g., "I relegated the letter to my pen pal."

It's not that "looking things up" isn't useful. It's that people not doing the looking-up underestimate the extent to which lookers-up will view it as mental work, and overestimate the likelihood that an accurate definition will be learned.

Intuition 3: kids don't need to worry about handwriting, because keyboarding is so important. Finally, I mentioned evidence that handwriting may improve cognition: recent papers include Cameron et al 2012, Grissmer et al 2010; Dinehart & Manfra, 2013. These are correlational findings, but the effect size is pretty big, and the weird thing is that what seems to be the best predictor of academic ability along these lines is how well children can copy a figure; look at a complex model, and then try to reproduce it on paper. That's pretty much exactly what handwriting practice is. The effect may not be causal, but before we jettison a year of practicing something that might aid cognition substantially, we should be sure it's disposable.

My real motivation in writing this op-ed was frequent encounters with teachers, who feel that ed-tech interventions are held to a lower standard of promise when administrators evaluate them. Teachers who are doubtful about a new intervention are brushed off as fuddy-duddies, or frightened of change.

I think there's something to this. I don't think it's that ed tech enthusiasts are dazzled by the shiny and new. But I do think this business about intuition is a problem. It often seems obvious how the human cognitive system will interact with a new situation, but these intuitions can easily mislead. 

Texting while driving is banned in 46 states.

It makes you wonder what's on the minds of the legislators in the other 4 states, like the old Trident ads that claimed "4 out 5 dentists recommend sugarless gum"--what the hell is that fifth dentist thinking? But I digress.

Compliance is a problem, however. The ad campaign implores: "It can wait." That's meant to keep us from peeking at the screen when we hear that ping.

One might take this advice as an indication that driving will not be disrupted by an incoming text message so long as you don't answer it. That might be the case if the distracting part of the message were the notification itself, but that's probably not the problem. The notification is, after all, quite brief. The bigger problem may be persistent thoughts after the notification, especially 

wondering who texted you and why, and also what psychologists call prospective memory--reminding yourself that you have an unchecked message and that you ought to check it when you stop driving.

This account seems especially plausible in light of the data showing that talking on a hands-free cell phone is just has distracting as talking on a hand-held device (Horrey & Wickens, 2006). The overt activity (holding the phone, checking the message) may be less important than the unobservable mental activity. A recent study offered support for this hypothesis, examining the consequences to attention of refraining from checking a message or call. 

Cary Stothart and colleagues asked subjects to perform a simple, but attention-demanding lab task. A digit appeared every 900 ms, and subjects were to press a button as quickly as possible when it did...except if the digit was a 3.

Subjects performed a few practice trials to be sure they understood the task, and then performed two blocks of 360 trials each. 

There were three conditions. Some subjects received four text messages on their phone during the second block of trials, some received phone calls, and the third group served as a control. 

A unique and clever feature of the study is that subjects were unaware of its purpose. They were asked for their cell phone number along with other information as part of the study; the program controlling the attention task contained a script that executed the calls or texts. The program also assigned subjects to condition randomly, so the experimenter did not know in advance which condition the subject would be in. 

An experimenter remained in the room, seated behind the subject, to monitor whether the subject actually checked his or her phone, and to monitor whether the phone was off or silenced. (Each was seldom the case.)

The data showed, as expected, that unanswered calls and text notifications led to performance decrements. One measure is the probability of pressing the button when a 3 appears, shown below.

It's typical to make more errors in the second block of trials--it's just hard to maintain focused attention for a long time--and control subjects show that pattern. But the increase is greater for the subjects receiving phone or text notifications. 

Another way to measure attention in this task is the frequency of very fast response times. If a subject doesn't really evaluate the digit, he or she can fall into a rhythm of pushing the button more or less in sync with the appearance of the digit, because its timing is predictable. The frequency of this sort of response increased in block 2 for the call and the text conditions. 

An important feature of the data was that the distraction effect was as great when the phone was ringing as when it was not. That supports the interpretation that the detriment is not so much the distraction of the notification, but thoughts about who might be contacting you.

The authors did not include a condition that would really help support their interpretation. They ought to have people perform this task in pairs. If my cell phone rings it ought to distract me, but have little impact on you, because you won't wonder who is trying to make contact.

How detrimental this distraction effect would be to driving is hard to say; obviously some driving conditions (a straight, empty interstate) are less demanding than others (Boston rush-hour in a snowstorm). Still, the impact on overall accident rates is predictable. Even a small distraction effect, multiplied by the number of driving hours in this and other countries where smartphones are the norm, means more accidents.

If we really want people to refrain from texting while driving, "turn it off," is better advice than "it can wait."

I doubt, however, that even well-intentioned drivers will remember to do so. The solution would be greater use of existing apps that block texting while driving. 

Humans show remarkable cultural diversity. Different groups—let’s call them tribes—use different technologies, economic organizations, political organizations, they hold different religious beliefs, and so on. Explanations of this diversity typically fall into one of two categories:

1. Humans inhabit almost every corner of the globe. Diversity of tribes’ behavior is a product of environmental diversity and the fact that humans are so good at problem-solving. Different environments prompt different behaviors, but even when environments are similar people are so ingenious they come up with different solutions to the same environmental challenges.
2. Diversity of behavior is due to cultural traditions. Sure, there are some environmental constraints on what I do—people living in the desert won’t fish—but diversity is so high because small changes are preserved with fidelity. People keep doing what the tribe has always done.

 
The first account predicts that local environmental conditions will determine tribe behaviors. Two predictions may be drawn from the second of these accounts: (1) tribes that live spatially near one another will be more behaviorally similar and; (2) behaviors will persist across generations.
 
A recent study sought to test both predictions using an enormous dataset of 172 tribes in western North America.  The dataset records 297 behavioral variables (e.g., what people eat, their religious practices, family organization, and so on) and 133 variables concerning the environment (available flora & fauna, characteristics of soil, altitude, precipitation, etc.). All data represent practices and conditions at the time the tribe first encountered Europeans.
 
Spatial distance between tribes is simple enough to measure.  The researchers used language phylogeny as a proxy of cultural phylogeny. Analyses of the similarity of languages yields an “evolutionary tree” of languages, so the distance of any two languages on the tree can be measured with a “most recent common ancestor” approach.
 
The question of interest is which of three variables predicts whether two tribes show similar behaviors. If behavior is mostly a matter of smart individuals adapting to the local ecology, then tribes inhabiting similar terrain should behave similarly. But if learning is mostly social, then tribes that are physically and/or culturally close should be more likely to behave similarly.
 
The results showed that cultural history and ecology both affect everything…but cultural history generally has the stronger effect. Within cultural history, phylogeny mattered more than spatial distance.
 
This analysis is about group behaviors, things that most people in a tribe do. But the result showing the importance of social learning may hold a lesson for those of us who think about the education of individuals. It’s easy to be a little dazzled by the brilliance of the human mind, and to see most of cognitive development as the intrepid mind of the individual, exploring the environment like a little scientist.
 
That’s certainly the emphasis we get from many psychologists. Bandura’s social learning duly noted, the towering figure of Piaget puts the child’s individual discoveries at the center of learning.
 
When it comes to schooling, I sometimes sense a similar reverence for learning that is the product of an individual mind at work, over the mere copying of someone else’s solution. It’s true that you only get true invention/innovation from original thought. But it’s a whole lot quicker and more reliable to copy what others have done. That is probably why social learning seems to be the workhorse of cultural learning.

One of my graduate school mentors noted that if he was walking when presented with a really difficult cognitive challenge, he would stop, as though walking drew, however slightly, on his attention. Rousseau, in contrast, claimed “I can only meditate when I am walking.”
​
The advent of the treadmill desk makes the question of walking and cognition more urgent. Okay, there may be health benefits, but if walking is not fully automatic, it siphons away some of your thinking capacity; it demands multitasking, so why put one in the workplace? 

Researchers have caught up to business trends, and a couple of recent studies indicate that dedicated office walkers can relax—treadmills don’t seem to compromise cognition. Probably.

In one, researchers administered well-normed measures of working memory and executive function (digit span forwards and backwards, digit-symbol coding, letter-number substitution, and others) to 45 college undergraduates. Each completed the tasks while sitting, standing, and walking (in random order), with 1 week elapsing between sessions. Participants could set the walking speed as they preferred, between 1 and 3 km/hour. Performance on the working memory/executive function tasks was statistically indistinguishable in the three conditions.

That study had people walk (or not) and measured the impact on working memory. Another approach is for researchers to tax working memory (or not) and observe the impact on walking. Other researchers used that method, having subjects either just walk (again, at a self-selected pace), or walk while performing a working memory task, or walk while reading. Researchers recorded several aspects of gait, focusing on variability. Again, they found no evidence of interference.

The neurophysiology of walking would seem consistent with these results. Humans have central pattern generators in the spinal cord—neural circuits that, even in the absence of input of the brain, can generate patterns of flexion and extension in muscles that look like walking. Thus, if the spinal cord can handle walking on its own, it’s easy to see why walking is not compromised when the brain is doing something else.

But central pattern generators set up pretty crude motor output; locomotion (like all movement) requires close monitoring of perceptual feedback (from vision, from balance) which is used to fine-tune walking movements (For a review, see Clark, 2015). We notice the need for perceptual information when we walk on ice, and for motor tuning when we pick our way through a rocky beach, but the fine-tuning goes on in a less obtrusive way in everyday situations. To get a feel for that, find yourself a nice long hallway, pick a spot about 50 feet away, and walk towards it with your eyes closed. If walking were a completely automatic program that could run without visual input from the brain, this would be no problem, yet most sighted people feel uneasy just a few steps in.

So if walking actually can’t run with total automaticity, why does treadmill walking show no attentional cost?

It may be that there is a cost, but it’s so small that it’s not detected in these experiments. And that may mean it’s not worth worrying about. Follow-up experiments with greater statistical power to detect small effects would be needed to address that possibility. Three other caveats are worth considering before we all buy treadmill desks.

First, the studies to date have been of relatively brief duration—less than an hour. It’s possible that subjects can with some effort of concentration, walk without cognitive cost for a short period of time, but a few hours would reveal a deficit; tiredness might make walking a little sloppy, and thus more attention-demanding.

Second, at least one study has shown a movement task (key tapping) was compromised when people walk (Oblinger et al, 2011). That’s not an effect of attention, but of trying to do two motor actions at once, like rubbing your stomach while patting your head. Hence, although office activities like typing or data entry have not been tested on treadmills, I’d be willing to bet they would be compromised. 

Third, my graduate mentor and Rousseau may have been talking about different types of thought. My mentor referred to answering a question, whereas Rousseau may have meant more creative thought. Walking may not be helpful when the environment presents pressing problems in need of timely answers. But a meandering gait may promote meandering thought, which in turn promotes creativity. The latter has not be tested on office treadmills either.