There was a brief, lively thread on Twitter over the weekend concerning the definition of learning. To tip my hand here at the outset, I think this debate—on Twitter and elsewhere--is a good example of the injunction that scientists ought not to worry overmuch about definitions. That might seem backwards—how can you study learning if you aren’t even clear about what learning is? But from another perspective don’t we expect that a good definition of learning might be the result of research, rather than a prerequisite? 

The Twitter thread began when Old Andrew asked whether John Hattie’s definition (shown below) was not “really terrible."

I'll first consider this definition (and one or two others) as our instincts would dictate they be considered. Then I'll suggest that's a bad way to think about definitions, and offer an alternative. 

Hattie's definition has two undesirable features. First, it entails a goal (transfer) and therefore implies that anything that doesn’t entail the goal is not learning. This would be….weird. As Dylan Wiliam pointed out, it seems to imply that memorizing one’s social security number is not an example of learning. 

The second concern with Hattie’s definition is that it entails a particular theoretical viewpoint; learning is first shallow, and then later deep. It seems odd to include a theoretical perspective in a definition. Learning is the thing to be accounted for, and ought to be independent of any particular theory. If I’m trying to account for frog physiology, I’m trying to account for the frog and it's properties, which have a reality independent of my theory. 

The same issue applies to Kirschner, Sweller and Clark's definition, "Learning is a change in long-term memory." The definition is fine in the context of particular theories that specify what long term memory is, and how it changes. Absent that, it invites those questions: “what is long term memory? What prompts it to change?” My definition of learning seems to have no reality independent of the theory, and my description of the thing to be explained changes with the theory.

It’s also worth noting that Kirscher et al’s definition does not specify that the change in long term memory must be long-lasting…so does that mean that a change lasting a few hours (as observed in repetition or semantic priming) qualifies? Nor does their definition specify that the change must lead to positive consequences…does a change in long term memory that results from Alzheimer’s disease qualify as learning? How about a temporary change that’s a consequence of transcranial magnetic stimulation? 

I think interest in defining learning has always been low, and always for the same reason: it’s a circular game. You offer a definition of learning, then I come up with a counter-example that fits your definition, but doesn’t sit well with most people’s intuitions about what “learning” means, you revise your definition, then I pick on it again and so on. That's what I've done in the last few paragraphs, and It’s not obvious what’s gained. 

The fading of positivism in the 1950s reduced the perceived urgency (and for most, the perceived possibility) of precise definitions. The last well-regarded definition of learning was probably Greg Kimble's in his revision of Hilgard & Marquis’s Conditioning and Learning, written in 1961: “Learning is a relatively permanent change in a behavioral potentiality that occurs as a result of reinforced practice,” a formulation with its own problems.

Any residual interest in defining learning really faded in the 1980s when the scope of learning phenomena in humans was understood to be larger than anticipated, and even the project of delineating categories of learning turned out to be much more complicated than researchers had hoped. (My own take (with Kelly Goedert) on that categorization problem is here, published in 2001, about five years after people lost interest in the issue.)

I think the current status of “learning” is that it’s defined (usually narrowly) in the context of specific theories or in the context of specific goals or projects. I think the Kirschner et al were offering a definition in the context of their theory. I think Hattie was offering a definition of learning for his vision of the purpose of schooling. I can't speak for these authors, but I suspect neither hoped to devise a definition that would serve a broader purpose, i.e., a definition that claims reality independent of any particular theory, set of goals, or assumptions. (Edit, 6-28-17. I heard from John Hattie and he affirmed that he was not proposing a definition of learning for all theories/all contexts, but rather was talking about a useful way to think about learning in a school context.)

​This is as it should be, and neither definition should be confused with an attempt at a all-purpose, all-perspectives, this-is-the-frog definition.

Educators have been concerned about students' ability to vet information they find on the internet. Better put, the problem lies not just in students’ ability to vet information, but the extent to which they see the need to do so. This issue took on new urgency during the 2016 Presidential campaign, amidst charges that individuals and groups were willfully spreading false stories about candidates they opposed—stories that were accepted by readers and spread on social media.

​A new study indicates that people are somewhat more prone to be credulous when they think they are part of a group of readers.

​Subjects read purported headlines from a news organization. The headlines covered a variety of topics, and they were evenly split for veracity. Subjects were asked to mark them as True or False, or to “flag” the fact, which indicated they would find out the truth or falsehood of that statement at the end of the series. Subjects got a small monetary reward when they were right and a same-size penalty when they were wrong. Raising the flag paid nothing in one experiment and a small reward in another.

​The critical variable of interest is whether subjects thought they performed this task alone or with others, in which case they saw 100 or so other names of people they were told were doing the task at the same time. They were told their performance was not influenced by these other people; they just happened to be doing the task at the same time.

​Across 8 experiments, there was a small but consistent bias to check facts less often if subjects thought other people performed the task too. Of the 8, the most interesting experiment was one where researchers changed the presentation format to make the headlines look like part of a Facebook feed. In that case, it didn’t matter whether they told subjects they were alone; the format was enough to make it feel social, as shown on the graph below. 

The authors considered (and tested) several hypotheses about what’s going on.

1) Flagging something doesn’t signal “I want to know the answer.” It signals “I’m not sure.” So being in a group, you’re less sure of your answers because you unconscious think “in this big group, someone else really knows the answer, so I won’t pretend to know.” But the researchers collected confidence ratings when people said “true” or “false” and those didn’t differ between the answering alone and answerd with others conditions.

2)  Social loafing. When you’re in a crowd, you figure others are doing at least some of the work (in this case, checking facts) so you don’t feel the need. To check on this, the researchers tried to make subjedts feel like they stood out in the group by making the subjects name on the screen red. This had no impact on the effect.

3) The presence of others makes it a social context, and taking people at their word is a social norm. But you would predict this would also make subjects say that more statements are true and they didn’t do this.

The researchers of a safety-in-numbers interpretation. When people are in a crowd, they feels safe; “there are lots of people here, so there’s no need to be vigilant, to gather more information. If there were a problem, someone would notice it.” The researchers offer some evidence for this: they found that people performed worse on a proof-reading task when they believed they were in a group.

This theory is certainly plausible. It calls to mind Jerry Clore's theory about the impact of emotion on cognition. Clore suggests that negative emotions (anger, sadness) put you in a mode to collect more information—things are not going well (hence the negative emotion) so you need more information, and to focus on details, to figure out what’s going wrong. When you’re happy, in contrast, you’re content to do the sort of thing you usually do in the current circumstance, i.e., to rely on memory and don’t worry so much about collecting new information. Everything is fine, so you keep doing what you’re doing.

That’s all very logical, but it certainly seems to create a difficult situation. If true, the we are all of us slightly more ready to believe anything we read on social media. No, logging into Instagram does not turn you into a credulous zombie—spend five minutes on Twitter, for example, and you’ll see as much disbelief (and anger) as you care for in a day. But the social nature of social media may be one of many contributing factors.

​It highlights the fact that we are still working out the kinks in new media, trying to take advantage of the enormous increase in the our ability to create and to consume content while identifying and thwarting the drawbacks attendant to those increases. 

We cannot remind ourselves often enough that the predictions for education drawn from the learning sciences that obviously, 100%, HAVE to work…often don’t.

The latest example comes from a recent paper reporting a randomized control trial of adaptive vs. static practice in Dutch schools.

It seems self-evident that adaptive practice will be superior to static. In static practice, each student receives the same set of practice materials of graded difficulty: easy, medium, hard, with difficulty defined by the performance of a large cohort of students. In the adaptive algorithm, the proportion of questions correctly answered is factored into the probability of seeing a particular type of question: if a student is getting all of the easy questions right, what’s the point of posing more? Why not move on to more challenging content?

Adaptive practice is one of the reasons offered that personalized learning ought to lead to greater achievement.

The experiment tested 1,051 Dutch 7th-9th graders studying either Dutch, Biology, History, or Economics over the course of one academic year. Assignment to static or adaptive practice was assigned by classroom, and students were blind to condition. All students received the same instruction (a hybrid of a digital environment and traditional paper textbook) and all homework was the same. The independent variable was implemented only through extra practice; students were asked to practice at least 15 minutes per week, but any more practice than that was taken at their own initiative.  
We might expect that, because students are practicing at their own initiative, they will use the adaptive program less, given that the problems will likely be more difficult. The proportion of students who used the practice module did not differ between conditions, hovering around 90% in both cases. Students in the adaptive condition did indeed work more difficult problems and they also practiced a little bit longer per session…but they worked fewer problems than students in the static condition. Presumably, more difficult problems required more time per problem.

Nevertheless, they showed no advantage on a summative test. In fact, better prepared students (those who had passed the summative test before the experiment began) were slightly negatively impacted by the adaptive regimen compared to the static. (There was no effect for students who had failed the last summative test.)

Post-experimental interviews showed that students did not know whether their practice had been adaptive or static, and showed no difference in students’ attitudes towards the practice.

Why was there not a positive effect of adaptive testing?

One possibility is low dosage. The intervention was only 15 minutes per week and although students could have practiced more, few did. At the same time, the intervention lasted an entire school year, the N was fairly large, and an effect was observed (in the unexpected direction) for the better prepared students.

Another possibility is that the program was effective in getting challenging problems to students, but ineffective in providing instruction. Students in the adaptive condition saw more difficult problems, but they got a lot of them wrong. Perhaps they needed more support and instruction at that point, so the potential benefit of stretching their knowledge and skills was not realized.

Another possibility is that the adaptive group would have shown a benefit on a different outcome measure. As the authors note, the summative test was more like the static practice than the adapative practice. Perhaps the adapative group would have shown a benefit in their readiness and ability to learn in the next unit.

​This result obviously does not show that adaptive practice is a bad idea, or cannot be made to work well. It simply adds to the list of ideas that sound like they are more or less foolproof that turn out not to be: think spiral curriculum, or electronic textbooks. Thinking and learning is simply too complicated for us to confidently predict how a change in one variable will affect the entire cognitive and conative systems.
 

 

 
There’s a new blog post over at The 74 commenting on a “finding” that I’ve seen reported in other places (e.g., Inc and Forbes).
There are two parts to the claim.

1. “[Valedictorians] do well, but they don’t go on to change the world or lead the world.” Elsewhere these behaviors are characterized as those of "disruptors."
2. “School rewards people who follow the rules, not people who shake things up,”

This blog post would make a good final exam question for an undergraduate course in experimental methods. (If you like, head on over and see if you can find the problems in the claims.)

Problem #1: The evidence offered for the claim that valedictorians do not become “disruptors” is that a study of 81 valedictorians showed few or none became disruptors. To draw the conclusion “valedictorians don’t become disruptors” you need to show that fewer valedictorians become disruptors relative to other achievers e.g., non-valedictorians in the top quartile, or better, compare valedictorians to all students sorted by grade quartile. That few valedictorians become disruptors is expected--the baserate is low (i.e., very few people in any group would be expected to be disruptors). 

The second bit of evidence offered is that a study analyzing 700 millionaires found that their average college GPA was 2.9. First, It’s not obvious that status as a millionaire means you’re a disruptor. Second, if the criterion for disruption is income, well, it’s well-known that GPA predicts income.

Problem #2: The author not only assumes a relationship between two variables (status as a valedictorian & status as a disruptor) based on inadequate evidence, but also claims to understand the causal relationship; both are caused by a third variable, conformity. It’s great fun to propose causal mechanisms when you haven’t measured the relevant construct, but absent other evidence, it ought to be thought of in just those terms: fun, merriment, whimsy. If the relationship actually exists, we can have equal fun proposing other causal relationships; disruptors are bad at assessing risks but valedictorians are good at assessing risks; gaining status as a valedictorian makes people buy into societal norms; disruptors don’t do very well in school because they aren’t very smart—that’s why they take big risks.

See, isn’t this fun?

Maybe the book is better. If so, this is a case of careless reporting. Either way, it’s a case of careless thinking.