Greetings, Superfriends, and welcome to this week’s show.

Not too long ago, some of you guys who are also enrolled in my accelerated learning course sent me an interesting article from Scientific American entitled “What Works, and What Doesn’t.” In the study, 5 researchers from Duke, Kent State, the University of Virginia, and the University of Wisconsin evaluated a number of different learning strategies to determine which ones actually worked.

Needless to say, I was eager to speak with the researchers and compare notes, share some of my findings, and learn more than the short article had to tell. Well… today, we have the privilege of doing that. We’re joined by Daniel T. Willingham, a professor of psychology from the University of Virginia.

Dr. Daniel T. Willingham earned his Ph.D in cognitive psychology from Harvard, and he spent about 8 years focusing on the brain basis of learning and memory. Today, his research is focused on the cognitive psychology behind K-16 education. He writes for a number of magazine columns and blogs, and has even published a number of books, from Raising Kids Who Read to Why Don’t Students Like School.

Given the topic of my TED talk, which was recorded just two days before this recording, I was very eager to engage with Dr. Willingham, compare our notes, and learn more about him and his research.

We talk about cognitive psychology, techniques that work, techniques that don’t, and some of the challenges facing the outdated education system in most countries. There are a ton of take-homes in this episode, and it is applicable not just to kids or teens or parents, but also to anyone who wants to learn more effectively, at any age.

This episode is brought to you by the all new SuperLearner Academy!

In this episode, we discuss:

• Why aren’t we teaching kids how to learn instead of just what to learn?
• What would a retooling of education to focus on mnemonics look like?
• Does everyone learn in their own unique way, or are there more commonalities than we think?
• How Dr. Willingham and his colleagues conducted their study
• Which study techniques are proven to work, and which ones are not? (with some surprises)
• Four actionable changes you can make TODAY to make your learning and studying more effective
• From a cognitive psychology perspective, what differentiates things that work and things that don’t?
• Why you should do a large part of your studying without your notes or textbooks
• What is “the retrieval practice effect,” why is it quite surprising, and why should you care?
• What are Dr. Willingham’s thoughts on Memory Palaces and other mnemonic techniques?
• The very strong stigma against “memorization” in academia, and why it might be misguided
• What is the relationship between creativity, experience, critical thinking, and memory?
• An example of how experience can out-rank memory in a real-world scenario
• What does the future of education look like, and what will be the roles of memory and technology?
• Will brain-sensing technology be allowed in classrooms?
• What books has Dr. Willingham most recommended to others?
• How could we improve teacher training?
• What is Dr. Daniel T. Willingham working on next?

Resources Mentioned in This Episode:

• The many books by Daniel T. Willingham
• What Works, What Doesn’t – The research paper Dr. Daniel T. Willingham contributed to
• The article in The Scientific American about Dr. Willingham’s work
• Malcolm Knowles and his 5 requirements of adult andragogy
• Our previous episode with Harry Lorayne, the godfather of modern memory training
• A study done in Connecticut to test how students place trust in the internet and the NorthWest Tree Octopus
• The Gates Foundation and their efforts to measure engagement in classrooms
• What if Everything You Knew About Education Was Wrong? by David Didow
• How We Learn by Benedict Carey
• Make It Stick by Peter C. Brown et al (check out our podcast interview with him here)
• Daniel’s website, DanielWillingham.com

Favorite Quotes from GUEST:

Transcript:

Introduction: Welcome to the Becoming SuperHuman Podcast. Where we interview extraordinary people to bring you the skills and strategies to overcome the impossible. And now here's your host. Jonathan Levi.

Jonathan Levi: This episode is brought to you by my all-new SuperLearner Academy. Okay. Let me tell you a bit about this new project that we've been working on. In essence, what we've done is created Masterclass versions of all of our best-selling courses, including our completely redone for 2016 course Become a SuperLearner.

And what we offer in these Masterclasses is not only additional worksheets, not only additional access to instructors through discussions but also tons of new content, tons of new lectures, audio bonuses and exclusive access, and discounted access to future courses. So you can pick up a Masterclass version of Become a Speed Demon or Become a SuperLearner, which you've heard so much about on this podcast, but you can also pick up a bundle version and save hundreds of dollars by purchasing all the masterclasses together at once.

You can also check out our audiobooks and digital books and buy them all together at one bundled price. So check it out today and you can get a free trial of each of the courses at becomeasuperlearner.com.

Greetings, super friends and welcome to this week show. Before we get started, I just want to read you guys a quick review that I really appreciated. It's by Heather Ambler. And the title is “This Podcast Is Like Rocket Fuel”. Heather writes I'm a podcast junkie and this one is my go-to for motivation, inspiration, and information. To help me achieve my goals. I tap play when I start my walk in the morning before work, and then by the time I get home, I'm fired up and confident about getting everything done that will propel me toward realizing my biggest dreams.

She goes on to say that there are excellent guests and she looks forward to every episode. Heather, thank you so much for your review. It really brightened my day. And if you guys want to hear your review read it at the top of next week's episode. Just submit it on iTunes or Stitcher, and maybe your review will be picked.

Onto today's episode, not too long ago, some of you guys who are also enrolled in my accelerated learning course sent me an interesting article from Scientific American entitled “What Works, and What Doesn’t.” In the study, 5 researchers from Duke, Kent State, the University of Virginia, and the University of Wisconsin evaluated a number of different learning strategies to determine which ones actually worked.

Needless to say, I was eager to speak with the researchers and compare notes, share some of my findings, and learn more than the short article had to tell. Well… today, we have the privilege of doing that. We’re joined by Daniel T. Willingham, a professor of psychology from the University of Virginia.

Dr. Daniel T. Willingham earned his Ph.D in cognitive psychology from Harvard, and he spent about 8 years focusing on the brain basis of learning and memory. Today, his research is focused on the cognitive psychology behind K-16 education. He writes for a number of magazine columns and blogs, and has even published a number of books, from Raising Kids Who Read to Why Don’t Students Like School.

Given the topic of my TED talk, which was recorded just two days before this recording, I was very eager to engage with Dr. Willingham, compare our notes, and learn more about him and his research.

We talk about cognitive psychology, techniques that work, techniques that don’t, and some of the challenges facing the outdated education system in most countries. There are a ton of take-homes in this episode, and it is applicable not just to kids or teens or parents, but also to anyone who wants to learn more effectively, at any age.

And so without further ado, ladies and gents Dr. Daniel T. Willingham.

Dr. Daniel Willingham and welcome to the show. We are so happy to have you today.

Dr. Daniel T. Willingham: Glad to be here.

Jonathan Levi: Awesome. So I have to admit, I read your research because actually a number of my students sent it to me and I was super excited to see that someone's looking into this because I just gave a little bit of a rant really on the TEDx stage about how we never teach kids actually how to learn. So I was so excited to find out that you and your colleagues are as invested in this as I am.

Dr. Daniel T. Willingham: Yeah. I've had the same concerns that you have. This is something that I've been asking informally. I've been asking my students here at the University of Virginia for years. I teach a big course, 350 students every fall an introduction to cognitive psychology.

And I like to ask them early in this semester has any time in your K-12 education, has anyone talked to you about how learning works and what that implies for how you want to study? And usually, no more than about 10% of my students will raise their hands in response to that question. So, yeah, this is something that's been on my radar for a long time as well.

Jonathan Levi: It's crazy. And as I discovered in the research, even the 10% who are being taught to learn are probably not being taught the right techniques or at least not enough of them. 

Dr. Daniel T. Willingham: That may be true. 

Jonathan Levi: Why do you think it is that we never teach? I mean, it seems so logical, you know, you teach a mechanic how to use tools before he actually uses them.

Why doesn't anyone ever teach kids to learn do you think? 

Dr. Daniel T. Willingham: That's a great question. I'm not sure of the answer. It may be one of those things that is a problem because there's not a super obvious time that it ought to happen. So in other words, the extent to which we ask kids to regulate their own studies, and teach themselves at home.

That's sort of a slowly increasing curve, right? So starting in the middle of elementary school, kids will be asked to read something at home, and that it will be discussed the next day, but the idea of formally studying it isn't really there. And then slowly quizzes start to creep in and so forth.

So this is purely a guess on my part, but yeah, the real answer is, I don't know why it's not formally taught more often. 

Jonathan Levi: I'm totally with you. I mean, one of the points that I made in my talk was, you know, it would be great if we had even one class on learning practicum, but really at least I believe the solution that needs to be there is a series of courses that follow children through their intellectual development because as a child grows, more and more sophisticated techniques will be applicable I think.

Dr. Daniel T. Willingham: I think that's right. And I think that this idea of sort of having this be an ongoing process, I think is a great idea. This may give us some hint as to why this isn't done more often because at least in the States our education system is so decentralized.

It's hard to know what's happening in other grades and other classrooms because it's not uniform. It's pretty fluid that can change from year to year. And of course, administrators and teachers are talking to one another, but to really have this down, you really need to be organized within a school. So you need to know, okay.

Kids have very little experience doing this new task in fifth grade, that they're about to be set. You need to be confident they haven't encountered in fourth grade. You also need to know what they have been asked to do in fourth grade. And so that you have a good idea of what's new. And then somebody within the fifth-grade curriculum that by fifth grade, they may have a couple of teachers already.

Somebody needs to step forward and say, okay, I'm going to handle this part of it. And then, of course, you've got transfer students coming in their background is going to be varied. So that may be part of it. But getting back to your main point, I agree. I mean, ideally, teachers would have a clear idea of what new skills were being introduced that we're asking students to be independent.

We're asking them to do independent work and being sure that those skills are explicitly taught. 

Jonathan Levi: Sure. I think also part of it is that I think in the education system, we have this idea that everyone learns in their own unique way. What are your thoughts on that? 

Dr. Daniel T. Willingham: Yeah, that's really an interesting observation as well.

That may be another reason that some educators feel that. This is sort of a hopeless cause. And before I answer, I also want to be clear. I mean, I think when I was talking a moment ago, I made it sound as though teachers are never instructing students in skills. And of course, I don't mean that. And I don't think that's true at all.

I'm talking specifically about self-study skills, how to approach a text for the particular purpose of studying it, and how to study for a quiz. So very explicit memory functions that we implicitly ask kids to engage by giving them quizzes and tests, but they don't always study for but are not always instructed them.

So getting back to your point about the part of the issue may be that some educators think, well, there's so much variation in how kids learn that it may be almost hopeless. That may be a reason, indeed, that educators think there's not a whole lot that teachers can really do here because every child is different.

And of course, the learning literature would indicate that that's not true. There may be some variation, but there's an awful lot that kids have in common also. I mean, every child in a classroom has a human brain and the right basic cognitive architecture is going to be the same across all of those kids. So things like how vision works, how memory works, and how attention operates.

There may be a little bit of variation here and there, but first of all, what scientists know about the variation is not that large compared to what we know about what they have in common. Right. And there, we really do have some things that teachers ought to know about and that ultimately students thought to know about.

Jonathan Levi: Absolutely. I always like to say, we all evolved from hunter-gatherers, you know, our minds all crave novelty. We all have the same vision skills. And I think any difference in the way that we learn is nurture, not nature. Some kids are read to more and so they process information kind of linguistically a little bit more.

I mean, but I don't have the background to claim that it's just a theory, of course. 

Dr. Daniel T. Willingham: Well, certainly they have a one way in which kids, very, a lot that's very noticeable is in their preparation when they walk into a classroom. So as you said, if a child in a pre-K classroom has been read to a lot, they're going to have a broader vocabulary.

They're going to know things literally about how books operate, where you open it. And that text is read from left to right and so forth. So there's a lot of variation in the background knowledge that children bring into the classroom. And then that will typically get exacerbated and more exaggerated as kids get older.

But that's different than saying they have different mechanisms by which they learn. That is where the data are much less clear. And so I'm sort of you know, suggesting that that may be true. You know the fact that the scientists can't describe them doesn't mean they don't exist. It just means that if those variations exist, scientists can't really help teachers make sense of it.

Jonathan Levi: Sure. Dr. Willingham, I want to change gears a little bit and ask, I realized, I mentioned your study, I've read it, but maybe our audience hasn't. So perhaps I could have you summarize the Scientific American study and a little bit of the findings as well.

Dr. Daniel T. Willingham: Yeah. I mean, the Scientific American article was abstracted from a much longer, more technical article where my colleagues and I pulled together literature from the last 50 years or so and tried to abstract out some basic principles that would be useful in educational settings.

And we looked at two types of study techniques and memory techniques. We looked at ones that we thought that students were typically using to see whether or not those were good techniques. And then we also looked at ones that irrespective of whether or not we thought they were commonly used. We looked at ones that seem to have a lot of bang for the buck.

That would be a good idea for students to use them. And what we found is that the techniques that students most often use are really not that effective. So what do you find students using. So let's start with studying what students do an awful lot of, and many of these studies were done with college students.

But what I'm going to say is I think pretty broadly true. I'll try and remember if I'm not comfortable extending these results down to earlier grades. I'll try and remember to say that. Sure. But in terms of studying, students do a lot of highlighting. And highlighting is not a great study technique. They do this when they're initially reading.

And the problem with highlighting is that highlighting is students' way of indicating to themselves that this is important. This is worthy of future study. Now the problem with that is if you are new to some subject matter, how do you know what it is that's important. If you already have a fair amount of expertise.

So one of your listeners is supposed, you know, a tax attorney and they're reading a new document that comes their way about their profession. Well, they've got very deep background knowledge and they're going to be very good at spotting something in a new document that they're reading for the first time that's important.

And there'll also be good at spotting stuff that's not very important that they already know. A student who's reading a chapter from a biology textbook has no way of making that judgment. So highlighting is not a great idea for studying. Another thing that students do that isn't not terrific is now after they've read the material, now they know they've got an upcoming quiz or test.

What they tend to do is reread. Rereading is not very good because what this does is it leads to a feeling of familiarity. It feels like as you're reading something the third and fourth time you feel like, yes, yes. I understand it. This makes perfect sense and so forth. What I like to say to my students is if you're rereading it's a little bit like listening to me, explain the same thing three or four times.

And after a while, you're sort of like, yeah, yeah, I get it. Like, you don't need to keep repeating yourself. Right. It all sounds familiar. But being able to recognize something as familiar is not the same as being able to describe it yourself. Requires a different level of understanding. And of course, that's exactly what students are going to have to do when they are in a test or quiz situation.

They're not gonna be shown their textbook or their notes and be asked, is this your textbook? If that were the test that students would do very, very well just on the basis of rereading. But they're going to have to actually produce the knowledge. Right. So those are two things that are very commonly done that are not a good idea.

So what should students be doing? What we talked about a number of techniques that are effective. I'll just give you a couple. When students are studying, instead of highlighting what they can do is annotate. Or take notes on what they're reading. That's a better strategy because it requires you to put things in your own words.

It also requires you if you're going to annotate, you're more likely to try and look at chunks of text and try to come up with some sort of a summary. Now, summarizing is a little dangerous because you may actually not really summarize it correctly, but you can supplement this with a second technique

that's known to be effective, which is self-questioning. As students are going along especially when they spot something in the text where sort of a flat statement is made, it looks like a conclusion is being drawn. Students can stop and ask themselves. Why did the author say that? Why does author think that he or she is justified in drawing that conclusion? And that sort of forces you to go back in your mind and

think, what evidence is there in the text that leads logically to this conclusion. And studies show that these sorts of techniques are helpful when you're first reading a text, but reading a text for the first time helps to make sure that you're understanding it. And then when students get to the point of actually trying to study, there are a couple of things they can do.

One is something they already probably know they should be doing, but they're not doing, which is cramming is not a very good idea. You really want your studying to be distributed in time. So if you say, okay, I've got a French test, that's coming up on Friday and ballparking it, I think I'm probably going to study safe 10 hours for that French test.

Well, you could study 10 hours the night before the French test and probably sacrifice some sleep, or you could take that same 10 hours and distributed over the five days that are coming up until the test and study two hours each night. In terms of memory, you're much better off distributing your study and studying two hours a night for five nights running. The truth is I think most the students know that's a good idea.

They know that cramming might lead to okay performance on that Friday when you're taking the French test, but then you're going to forget it all by Monday. The problem is students are not that great at organizing their time. So yeah. What I always tell teachers is, you know, you tell them not to cram and you can tell them about the research literature, showing that cramming, in the long run, is not good. It probably won't have any impact, but maybe it will. Who knows? So you may as well at least tell them and let them know. 

Jonathan Levi: Yeah. As you were talking, I was really thinking about the way that this works in terms of neuroscience. That, and correct me if I'm wrong, but that we have such a huge portion of the brain, the hippocampi devoted to determining what we actually need to maintain and trying to eliminate waste in this organ that otherwise consumes such a disproportionate amount of the energy and oxygen.

But I'm wondering from a cognitive psychology perspective, why is it that the techniques that don't work, don't work and the ones that work, work. 

Dr. Daniel T. Willingham: Well, yeah, from a cognitive perspective well, the highlighting, I think I gave you a hint about why it doesn't work, which is you don't necessarily highlight the stuff that's useful.

Rereading doesn't work because it leads to this feeling of familiarity. So you think that you know it well enough to actually be able to describe it on your own, but you really don't. What you're having is it's a little bit like familiarity, the way I'm using it actually does have a technical sense in cognitive psychology.

It's very close to the way you would talk about familiarity in everyday language that you see someone out on the street and you say that person's familiar. What you mean by that is I know I've encountered that person before, but I can't tell you anything else about them. Uh, it's like that little piece of knowledge is unconnected.

And so likewise as you're rereading, you're getting this sense of familiarity. I've seen this before, I've seen this before, but the knowledge is not really all that well integrated. So you're not going to be able to describe it on your own. So, you know, if I'm giving you a, you mentioned the hippocampus, if you're reading in a textbook about the anatomic conductivity of the hippocampus familiarity will be like, yeah, yeah, yeah

it's connected to this, it's connected to that. I got it. I got it. I got it. But then it doesn't mean if I confront you and say, tell me what the hippocampus is connected to that you're actually going to be able to give me the right answer. You'll be frustrated because that will seem familiar. You'll say like, yes, yes.

I know I read that, but you're not very likely to be able to describe it. 

Jonathan Levi: That is so interesting. I think that's a really important distinction right there. It's the difference between actual functional, accessible, usable knowledge and just that familiarity.

Dr. Daniel T. Willingham: Exactly. So let's move on to what students should do.

Yeah. If they're not just going to reread, what should they do. Very simple thing to tell them is at least part of your studying should be in a situation where you cannot see your textbook and you cannot see your notes. In other words, you create questions yourself or someone else creates questions that they're lobbing at you where you have to produce the right answer.

So this is useful in a couple of different ways. One is that it gets around this problem of the familiarity where it sort of feels like, you know it because you're reading the correct answer over and over again. Here you have to actually produce the answer. So if you don't know the answer, it's going to be very obvious to you, right?

Someone says, tell me what the hippocampus is connected to. Now, you're just going to say, Oh, well, clearly I don't know this. Cause I can't say what the answer is. So that's one benefit of studying this way. The other is, and this was, we went over the, I mentioned this in the Scientific American article.

What's a phenomenon called retrieval practice. And this phenomenon has been around in memory textbooks for a long time, but no one really looked at it that carefully or thought about exploiting it until about 10 years ago. And there's been a real flurry of research. The basic idea is suppose that you've read a chapter in a, let's make it a brain anatomy textbook.

You've read that chapter one time and you know, you don't really know it that well. So how are you going to ensure that you know it better? Well, one thing you could do is you can read the chapter second time and read and study it just the way you've already read it and studied at one time. Another thing you could do is instead of reading it a second time, you could take a quiz on that chapter.

Take a little test on it and see how you do. The retrieval practice effect is the fact that taking a test actually will lead to better later memory than studying it a second time. Right. So this is very surprising to students. It doesn't feel like it should work. It feels like studying a second time will lead to better memory and studying it a second time does lead to better memory in the very short run.

So if you take a final exam or something, you know, within an hour, say of that second study session, you are better off compared to if you had taken a quiz. But if you wait until the next day memory is better having taken the quiz. So the reason it's called retrieval practice is retrieval refers to trying to draw something out of memory.

And it's this strange phenomenon that if something is sort of in memory, but it's not really from, and you're trying to kind of cement it into memory practice and retrieving it, going into your memory store and rummaging around and trying to find things and bring them out. That's actually better for cementing them in there.

Then having another round of studying. 

Jonathan Levi: Yeah. I mean, that's totally logical. It's in line with the Malcolm Knowles stuff of adults needing, pressing need and immediate application of the knowledge. You kind of have to trick your brain and say, look, this stuff's going to come up. You're going to be expected to regurgitate it and interact with it.

Not just be familiar with it. 

Dr. Daniel T. Willingham: Well, that's actually an interesting connection. I hadn't thought about that. Yeah. I definitely see where you're coming from with that. 

Jonathan Levi: Yeah. I mean, I kind of connect it all. I think I don't know how much Knowles actually spent thinking about the hippocampus and ways to kind of game it.

But if you look at his requirements, it's kind of like, hey, adults need a respectful environment. Well, yeah. I mean that makes sense. Adults need to connect to their existing knowledge. Well, I mean, yeah, you need to connect it to some existing neurons to have a memory. I think he was way ahead of his time in that regard.

Let me ask this. I have to admit, I opened up the study and I was very eager to find things about memory palaces or creating visual memories and stuff like that. And I was surprised to only find keyword mnemonics, which isn't actually a term that I'm familiar with. But you mentioned that the Scientific American article was a very condensed version.

So I'm wondering if you guys had the opportunity to explore the whole idea of memory palaces or the loci method. 

Dr. Daniel T. Willingham: You know, it's well-known that those are effective, but the truth is, I think they're application to classroom situations, which is what we're really focused on. We weren't even focused on other types of training.

We're really pretty focused on classrooms. I think the opportunities to use the method of loci and memory palaces is relatively limited. Those are effective mnemonics when you're dealing with material, that's really unconnected list of random words would be a prime example. So you could imagine that sort of coming up when you're learning vocabulary, especially foreign language vocabulary, where the association seemed really arbitrary.

That's where pneumonics are really at their best. Absolutely. Right is where you've got material. It's very hard to make any sense of it. Most of the time when you're trying to learn things, the material is sensible. And so using the fact that it is sensible, that's the way you want to go. You want to make meaning, and as you were just saying, connect it to things that you already know.

That's the way the memory is going to be long-lasting. Mnemonics to me, shouldn't be off the table in educational situations. But in one sense, they're kind of a last resort. My go-to strategy is to make this material meaningful to me and connect the things I already know. If I'm looking at something that's really not very meaningful and nevertheless, I think it's worth memorizing. And I think that does happen in educational situations then yeah, I'm going to use the mnemonic. 

Jonathan Levi: Yeah, I couldn't agree more. I mean, for learning history, the memory palace is probably not your ideal situation, but for foreign language vocabulary, or even, you know, with some pretty kind of creative modification for learning grammar and grammatical rules, I've found it really, really useful.

Dr. Daniel T. Willingham: Yeah. And I mean, I think when I've written about this, I think teachers think about this very clearly because they're not eager to give their students meaningless material to memorize. They recognize that that's hard on the students and it seems like it's not enriching, but it happens. It especially happens when you're the students are entering sort of a new content area and the teacher say to themselves, look, they've got to have certain vocabulary terms and certain ideas under their belts before we can really get a toe hold and move forward in this content domain.

Jonathan Levi: Sure. I think also part of it, we had Harry Lorraine on the show a couple of months ago, and he talked about how in the 1960s, he tried to get into academia and get into schools and talk about memory techniques. And there's the stigma around memory and memorization. While we have to memorize new vocabulary words one way or another, I think a lot of educators hear the word memorization and think, no, no, no, no, no, no, no, no.

We teach students to be creative and we teach them to interact with the information. We're not a fan of memorization. Right. It's semantics really. 

Dr. Daniel T. Willingham: Well, I think it isn't, it isn't, I mean, I think you've struck on a very important point and it's especially relevant today because there's a sense that wow if 20 or 30 years ago, teachers maybe were not eager to encourage memorization in their students.

I think that's even more true today because teachers figure we've got Google. What's the point of memorizing stuff. Totally. You can access information instantly. What I've argued is that memory is important to exactly the types of critical thinking skills we want in our students. Nobody wants students who can just spout out facts without really understanding them, or even just spout off facts, even if they understand them. 

We want students who can think, who can analyze, and can think creatively. These are not independent skills. It's not the case that you learn how to think well and then you have the knowledge, which is sort of a database that gets that where the content of the database gets plugged into these independent thinking skills.

The thinking skills are intertwined with the knowledge. So if you're learning to think like a scientist, you're really not learning to think like a scientist in general. You're learning how to think about the particular domain knowledge that you already understand. 

Jonathan Levi: Sure. I'm gonna use one of the techniques that you mentioned and replay back what you said in kind of my own language, which is creative thinking and generating solutions comes from actually memorizing, or at least remembering hundreds of different situations, you know, and hundreds of different examples and scenarios and being able to combine those.

Am I getting at the right point? 

Dr. Daniel T. Willingham: I think that's right. This is probably a whole other discussion about the relationship between memory and critical thinking. I think there are some instances where we can get even more fine-grained about it and talk about specific ways that it's not just that let me abstract further what you just said.

What I heard you say is creativity and critical thinking comes from generalizing across many, many individual situations. I think that's true. Again, I think we could get more specific about ways that knowledge contributes, not just to the development of skills, but also how they're deployed. So if we've got time, can I give you one example of this? Please.

Yeah, because I feel like I'm getting very abstract and it would be easier to understand what the particular example. So here's a standard example that I use, think about scientific thinking. And the idea of having an appropriate control group. Okay. So scientific thinking everybody would say, right. I mean, you know, the scientific method is certainly part of scientific, critical thinking.

Being able to evaluate whether an experiment is any good. That's an example of scientific thinking. All right well, let's think about the idea of a control group. You can memorize the definition of a control group. So let's make it in behavioral science since that's my area and that's what I understand.

We would say a control group should be just like the experimental or comparison group, except for one critical variable. That's the one that we're testing. So for example, I might decide that I want to know whether students can listen to a chapter out of a textbook on their iPod and understand it as well as they do when they are reading the text on paper. 

So I've got an experimental group listening to the chapter and I've got my control group, which is sort of business as usual reading a chapter. So how do I know whether or not that's a good control group? Well, the way that you typically define a control group is, as I said, they're exactly the same, except on this critical variable, but that's really not true in practice.

In practice, it's a little more complicated. So for example, if I got a hundred students and I split them into two groups and I have 50 listening to the iPod and 50 reading, and then I discover that actually the readers that's composed of 70% women. Whereas the listeners are 50% women. Now, do I have an appropriate control group here?

Right. Well, the answer is actually kind of complicated because what I'm saying is I'm going to be looking at these two groups to make sure that they're evenly matched on important variables. And I've just asked the question. Is gender an important variable to know whether or not gender is an important variable in a memory experiment like this,

you need to know the memory literature. Right, because gender's really obvious to me, but what if you know, handedness influences memory, or what if your sign of the Zodiac influences memory or, hey, how, about how much they knew about the topic of the chapter before the experiment started? That's probably pretty important.

All of these are things that if you're behavioral scientists, you'll actually think about measuring, to assure yourself. These two groups are really equivalent before I begin my experiment. So this is I think a great example of something where you can talk about the skill, the skill of conducting a scientific experiment, thinking critically about science.

Part of that is having a control group. You can have a nice little textbook definition of a control group, and somebody can memorize that, spout it back on a test, and seem to know what a control group is. But in practice, actually deploying that knowledge depends on background knowledge. You can't construct a good control group in a memory experiment unless you already know something about the memory literature. 

Jonathan Levi: Right. This actually raises another question I wanted to ask. I mean, you mentioned, you know, in modern schools, teachers know that we have Google and a lot of our memory has been offloaded into books or the internet. And so I'm wondering what you think the future of education looks like. I mean, if we need all of these examples, but at the same time, the information is always at our fingertips.

How much, and where do you think technology plays a part in the education system in the future? 

Dr. Daniel T. Willingham: Well, I think that again, I think a lot of knowledge needs to be in the head, not just in Google. Everyone's had the experience of Googling something and getting a million hits and having no idea of what to make of it.

And you've probably also had the experience of Googling something, getting a million hits. Not knowing what to do, asking a friend, who's got some experience in the area and they immediately look and they're like, okay, this site is probably good. This one's probably terrible and so forth. Right? So background knowledge is enormously important in being able to evaluate what it is that you find.

This is something that educators have been struggling with because they're very well aware that first of all, students are way too trusting of what they find on the internet. They're not very discriminating. Maybe you've even heard there was a fabulous study that was conducted in Connecticut, I think a while ago now, where they constructed a fake website that describes the Northwest Tree Octopus, looked very much like a legitimate site and it sort of imitated the sort of science textbook language and described this octopus that lived in trees in the Northwest.

And they asked, these were seventh graders and they specifically asked schools, send us your kids who are the savviest when it comes to evaluating stuff online. And all of the kids fell for it. And even when they were told this is a fake site and, you know, octopi don't live in trees, they couldn't. Kids were fighting back.

First of all, the kids who did believe them, couldn't really figure out how to evaluate that it was false and what the clues were there was false. Some kids actually fought back and said, no, no, look at it. It's right. Yeah. Yeah. And that stuff was done a while ago. There've been many studies following up on that.

The ways that people have tried to educate kids have so far not been very successful. They have been successful in alerting students to the fact that there's a problem. They recognize I can't trust everything that I find on the web, but it hasn't really given them an all-purpose toolbox for evaluating whether or not what they find on the web is legitimate. 

And truthful I think that's very difficult if you know something about octopi then it's easy to evaluate that that website is false, right? Because it's telling you something that's ridiculous. If you don't know anything about octopi, but then you're trying to count on things like, well, was it written by a scientist? And those cues are just not very reliable.

So sorry. This was all about sort of the role of Google in education and the extent to which Google is making knowledge in the head obsolete. I would argue pretty strongly is not making knowledge in the head obsolete. Knowledge is still important. And we already talked about how knowledge is important for critical thinking as well, in terms of the future of technology and education, you know, who knows, there are a lot of people with a lot of deep pockets gambling on the idea that we're going to have technological innovations that will make the way in the classroom and play a really significant role in education. This is not a new idea that technology is better than it ever was, but people have been trying to exploit computer technology for instruction for a long time.

And in one sense, there's a lot of promise, but that promise has been very hard to live up to again. I mean the first time this was really sort of, they hadn't non-computer solutions back in the fifties and sixties, so-called teaching machines, which were mechanical machines.

And then in the seventies, you know, the larger school districts could actually afford to buy mainframes and have dummy terminals in labs in schools. And so they were trying to do instruction with those. Now, of course, we've got much better computer technology and the opportunities for animation and video, and it's easier to make it interesting for students, at least on the surface, but some of the problems remained. The big problems are, writing really high-quality content that is going to be engaging and educational for students. That's proven very difficult. 

Jonathan Levi:  Right. I think where it gets actually really interesting, at least for me is the idea that we're very, very close to consumer-grade technology that can actually measure if you understood something.

I mean, I was proceeded on the TEDx stage by a guy who has like a $60 device he's building that can actually determine basically if the light bulb has turned on when something is explained. And of course, I'm going to go visit him next week and see if my SuperLearner brain is any different from a normal brain.

But I think, wow. Even without automating the teaching aspect, can you imagine what education would look like if the teacher literally knew in real-time, how many students just by them wearing a hat or a visor or some kind of device knew which students weren't understanding? I mean, that would be incredible.

Dr. Daniel T. Willingham: It would be, it's also something that you could expect the educational community might be a little leery of. The reason being that the one sort of nefarious use to which that could be put as evaluation of teachers. So if an administrator sees only 25% of your students hats are lighting up every morning,

I'm very concerned, and so forth. That feels a little invasive. It feels a little big brother-ish. The reason that that immediately came to mind was there was sort of a kerfuffle a few years ago where the Gates Foundation was funding an effort that was not about student understanding but was about student engagement.

Whether or not students are paying attention and they were trying to develop a, something that kids could wear as a bracelet that would measure autonomic nervous system activity. The idea was that there would be sort of a signature for whether or not kids were paying attention. Now, the weird thing here is they could be paying attention to anything.

It would really only determine, whether kids are kind of spacing out or whether they were focusing attention on something. Right. Even passing notes. Yeah. Passing notes or thinking about the kid in front of you or what you're going to do later or whatever. The idea was I mean, this is a, a significant problem in education research.

You come up with a new curriculum or a new pedagogical technique or whatever one of your predictions is, kids will find this interesting. How do you test whether or not your prediction is right? That's actually a pretty big problem. And, and that was why the Gates Foundation was interested in this as a tool for researchers to see whether or not their efforts to make things more engaging with work.

But it was picked up on several blogs. Um, a lot of people got very anxious that the real purpose of this was to check up on teachers. They weren't even contemplating like using it for the teacher to check up on students so that you knew like, Oh, Dan's spacing out. I better bring him back into the fold here.

They weren't even getting to that. People were jumping over that and thinking, this is really going to be eventually something that's going to be used to evaluate teachers. So all of this is sort of a long story, but it's a good illustration. I think of how it's complicated. It's something that seems like it's an undeniable boon.

Like you suggest there could be in the near future, a low-cost device where we can tell whether or not kids understand is not so straightforward that that's going to be embraced enthusiastically. 

Jonathan Levi: Sure. Yeah. I think anywhere you have these entrenched interests and the concept of unions and tenure, you have a lot of resistance to change, particularly change that can be used to evaluate employee performance.

Dr. Daniel T. Willingham: Yeah. I mean, these are complicated issues and I'm not a policy guy, so I try not to weigh in because I'm speaking from ignorance, but yeah. At the least it's complicated. 

Jonathan Levi: Absolutely. Well, let me change gears on that. I know we're just about to run out of time. I wanted to ask what book besides your own and in the bio, I talked quite a bit about your own books, but which book have you recommended the most to others?

Dr. Daniel T. Willingham: That's an interesting question. Yeah. I guess I've recommended different books for different purposes for the general sort of thoughtful application of cognitive principles to classroom situations, I quite like a book by a British writer named David Didau. That was the title What If Everything You Knew About Education Was Wrong?

I love that. It's a pretty provocative title and it sounds like the point of the book is going to be too kind of tear down myths and that's useful, but you also want someone who's going to build up. And he actually does quite a bit of that. He's not just telling you stuff that's wrong. He's telling you stuff that he thinks is right.

So I quite liked that book. Specific to memory, two books have come out in the last year that were really terrific. One is How We Learn by Benedict Carey. And One is Make it Stick, which has multiple authors. Oh yeah. And Mark McDaniel, a very prominent memory researcher, absolutely Washington University in St. Louis. Those are three books that I've been happy to recommend to people. 

Jonathan Levi: Well recently. Awesome. Yeah, we actually had Peter C. Brown on the show, a few months ago. So yeah, definitely with you on that recommendation. Let me ask, one more question before we part, which is what are you working on next?

Dr. Daniel T. Willingham: A few things. I mean, in the short term, I'm working on an article on creativity and also working on an article that is about the relationship of basic science and application. And it's looking at the question of whether or not teachers as part of their preparation should learn educational psychology.

So in most teacher preparation programs, teachers do learn educational psychology, either as a standalone course or in a broader course, usually called foundations of education. What I end up arguing in the paper is there's probably more clutter in those courses than there needs to be, those courses could be more focused on trying to clarify, this is why I'm looking at the relationship between basic science and application to draw some dividing lines about when that knowledge is useful to teachers and when it's not. And then in the longer term, I'm also working on a book about critical thinking. That was why I agreed to write the paper about creativity.

That's sort of in preparation for a larger work on critical thinking. 

Jonathan Levi: Amazing. You have a very, very full plate. 

Dr. Daniel T. Willingham: I do. 

Jonathan Levi: That's fun for me. So let me ask this then. First of all, it's been a pleasure having you on the show. Thank you so much.

Dr. Daniel T. Willingham: My pleasure. 

Jonathan Levi: Thank you. And second of all, where can people learn more, get engaged, subscribe to hear about all the new stuff you're coming up with.

Dr. Daniel T. Willingham: Well, I've got a website, danielwillingham.com. Which has reprint anything that I'm allowed to put up for free on the web. You can find it there in terms of articles and so forth. I'm not blogging right now I blogged for different institutions that various times.

I'm not blogging right now but my personal blog was up there when I was not blogging for any individual institutes. If I start again, my blog will probably be reprinted there. 

Jonathan Levi: Awesome. So we will link that up in the show notes, Dr. Willingham. I want to thank you once again. It's been really an absolute pleasure to get to speak with you. Thanks so much.

Dr. Daniel T. Willingham: Pleasure's all mine.

Jonathan Levi: All right. Well you take care.

Dr. Daniel T. Willingham: Do the same. 

Jonathan Levi: All right. Superfriends. That's it. For this week's episode, we hope you really, really enjoyed it and learn a ton of applicable stuff that can help you go out there and overcome the impossible.

If so, please do us a favor and leave us a review on iTunes or Stitcher, or however you found this podcast. In addition to that, we are. Always looking for great guest posts on the blog or awesome guests right here on the podcast. So if you know somebody or you are somebody, or you have thought of somebody who would be a great fit for the show or for our blog, please reach out to us either on Twitter or by email or email is info@becomingasuperhuman.com. Thanks so much. 

Closing: Thanks for tuning in to the Becoming Superhuman Podcast. For more great skills and strategies, or for links to any of the resources mentioned in this episode, visit www.becomingasuperhuman.com/podcast. We'll see you next time.